electronic machines

Abstract

To provide a downsized electronic device with an eye tracking function.SOLUTION: An electronic device with an eye tracking function includes a display device, an infrared light source, and an optical system. The display device includes a display element and a light-receiving element; the infrared light source has a function of emitting infrared light; the light-receiving element has a function of detecting the infrared light reflected by an eyeball; and the optical system includes a first optical element positioned on an optical path through which an image from the display element enters the eyeball and a second optical element positioned on an optical path through which the reflected infrared light enters the light-receiving element. The light-receiving element is integrated with the display device and thus, the electronic device can have a reduced size.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] One aspect of the present invention relates to an electronic device, and more particularly to an electronic device equipped with an eye-tracking function.

[0002] It should be noted that one aspect of the present invention is not limited to the above-mentioned technical field. Examples of technical fields of one aspect of the present invention disclosed herein include semiconductor devices, display devices, light-emitting devices, energy storage devices, memory devices, electronic devices, lighting devices, input devices, input / output devices, methods for driving them, or methods for manufacturing them. A semiconductor device refers to any device that can function by utilizing semiconductor properties. [Background technology]

[0003] In recent years, electronic devices for virtual reality (VR) or augmented reality (AR) have attracted attention. Furthermore, electronic devices for VR or AR equipped with eye-tracking capabilities are being developed. Electronic devices for VR or AR with eye-tracking capabilities can be applied to, for example, consumer behavior analysis, image processing, avatar creation, and eye-tracking-based manipulation.

[0004] For example, Patent Document 1 discloses a head-mounted display that performs eye-tracking using an image of light from an infrared light source reflected by the cornea. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Special Publication No. 2019-512726 [Overview of the project] [Problems that the invention aims to solve]

[0006] The aforementioned eye-tracking head-mounted displays have separate optical sensors from the electronic display, which resulted in larger head-mounted displays. Furthermore, the optical system positioned between the display and the eyeball is shared for both display viewing and optical sensor detection, leading to challenges such as difficulty in focusing the display or difficulty in optical sensor detection.

[0007] In view of the above issues, one aspect of the present invention aims to provide a miniaturized electronic device equipped with eye-tracking functionality. Also in view of the above issues, one aspect of the present invention aims to provide an electronic device equipped with eye-tracking functionality having a novel optical system.

[0008] Alternatively, one aspect of the present invention aims to provide an electronic device with eye-tracking capabilities that consumes little power. Alternatively, one aspect of the present invention aims to provide an electronic device with eye-tracking capabilities that is equipped with a high-definition display device.

[0009] Furthermore, the description of these problems does not preclude the existence of other problems. These problems are considered independent of each other, and the present invention does not need to solve all of them. Moreover, it is possible to extract other problems from the description in this specification, etc. [Means for solving the problem]

[0010] One aspect of the present invention is an electronic device having an eye-tracking function, wherein the electronic device comprises a display device, an infrared light source, and an optical system, the display device comprising a display element and a light-receiving element, the infrared light source having the function of emitting infrared light, the light-receiving element having the function of detecting reflected infrared light from the eyeball, and the optical system comprising a first optical element located in the optical path through which an image from the display element enters the eyeball, and a second optical element located in the optical path through which reflected infrared light enters the light-receiving element.

[0011] Another aspect of the present invention is an electronic device having an eye-tracking function, wherein the electronic device comprises a display device, an infrared light source, and an optical system, the display device comprising a display element and a light-receiving element, the infrared light source having the function of emitting infrared light, the light-receiving element having the function of detecting reflected infrared light from the eyeball, and the optical system comprising a first optical element located in the optical path through which an image from the display element enters the eyeball, and a second optical element located in the optical path through which reflected infrared light enters the light-receiving element, the second optical element comprising at least a mirror.

[0012] Another aspect of the present invention is an electronic device having an eye-tracking function, wherein the electronic device comprises a display device, an infrared light source, and an optical system, the display device comprising a display element and a light-receiving element, the infrared light source having the function of emitting infrared light, the light-receiving element having the function of detecting reflected infrared light in the right eyeball or the left eyeball, and the optical system comprising a first optical element located in the optical path through which an image from the display element enters the right eyeball or the left eyeball, and a second optical element located in the optical path through which reflected infrared light enters the light-receiving element, wherein the position of the first optical element for detecting reflected infrared light in the right eyeball and the position of the second optical element for detecting reflected infrared light in the left eyeball are different.

[0013] Another aspect of the present invention is an electronic device having an eye-tracking function, the electronic device comprising a display device, an infrared light source, an optical system, and a drive mechanism, wherein the display device comprises a display element and a light-receiving element, the infrared light source has the function of emitting infrared light, the light-receiving element has the function of detecting reflected infrared light from the right or left eyeball, the optical system comprises a first optical element located in the optical path through which an image from the display element enters the right or left eyeball, and a second optical element located in the optical path through which reflected infrared light enters the light-receiving element, and the drive mechanism has the function of moving or rotating either the first optical element or the second optical element or both.

[0014] In another aspect of the present invention, the display device may include an image processing unit, and the image processing unit may have a function of specifying the position of the line of sight based on the data acquired by the light receiving element.

[0015] In another aspect of the present invention, the electronic device may include a control unit, and the control unit may have a function of executing image processing.

[0016] In another aspect of the present invention, the image processing may include a process of emphasizing an image of a region including the line of sight.

[0017] In another aspect of the present invention, the infrared light source may include a light emitting element that emits infrared light, and the light emitting element that emits infrared light may be provided in the display device.

Advantages of the Invention

[0018] According to the present invention, it is possible to provide a downsized electronic device equipped with a line-of-sight tracking function. Or, according to the present invention, it is possible to provide an electronic device equipped with a line-of-sight tracking function having a new optical system. Or, according to the present invention, it is possible to provide an electronic device equipped with a line-of-sight tracking function with low power consumption. Or, according to the present invention, it is possible to provide an electronic device equipped with a line-of-sight tracking function and having a high-definition display device.

[0019] Note that the description of these effects does not prevent the existence of other effects. Also, these effects are considered to be independent of each other, and the present invention does not necessarily have to exhibit all of these effects. Furthermore, it is possible to extract other effects from the description in this specification and the like.

Brief Description of the Drawings

[0020] [Figure 1] FIG. 1 is a diagram related to a configuration example of an electronic device. [Figure 2] FIGS. 2(A) and 2(B) are diagrams related to the optical system of an electronic device. [Figure 3] FIGS. 3(A) and 3(B) are diagrams related to the optical system of an electronic device. [Figure 4]Figures 4(A) and 4(B) are diagrams relating to image processing in electronic devices. [Figure 5] Figures 5(A) through 5(B2) are diagrams relating to display devices in electronic equipment. [Figure 6] Figures 6(A) to 6(B2) are diagrams relating to display devices in electronic equipment. [Figure 7] Figures 7(A) and 7(B) are diagrams relating to display devices in electronic equipment. [Figure 8] Figures 8(A) and 8(B) are diagrams relating to display devices in electronic equipment. [Figure 9] Figures 9(A) and 9(B) are diagrams relating to display devices in electronic equipment. [Figure 10] Figures 10(A) and 10(B) are diagrams relating to display devices in electronic equipment. [Figure 11] Figure 11 is a flowchart illustrating an example of the operation of an electronic device. [Figure 12] Figures 12(A) to 12(C) are diagrams illustrating examples of electronic device configurations. [Figure 13] Figures 13(A) to 13(E) are diagrams illustrating examples of electronic device configurations. [Figure 14] Figures 14(A) and 14(B) are diagrams illustrating examples of electronic device configurations. [Figure 15] Figure 15 is a diagram illustrating an example of the configuration of a neural network's computational circuit. [Figure 16] Figure 16 shows an example of the configuration of a neural network's computational circuit. [Figure 17] Figure 17 shows an example of the configuration of a neural network's computational circuit. [Figure 18] Figures 18(A) and 18(B) are diagrams illustrating examples of the configuration of a display device. [Figure 19] Figures 19(A) and 19(B) are diagrams illustrating examples of the configuration of a display device. [Figure 20] Figures 20(A) to 20(E) are diagrams illustrating examples of display device configurations. [Figure 21] Figures 21(A) and 21(B) are diagrams illustrating examples of the configuration of a display device. [Figure 22] Figures 22(A) and 22(B) are diagrams illustrating examples of display device configurations. [Figure 23] Figures 23(A) and 23(B) illustrate examples of the configuration of a display device. [Figure 24] Figures 24(A) to 24(D) are diagrams illustrating examples of the configuration of a light-emitting element. [Figure 25] Figure 25 is a diagram illustrating an example of a display device configuration. [Figure 26] Figure 26 is a diagram illustrating an example of a display device configuration. [Modes for carrying out the invention]

[0021] The embodiments will be described below with reference to the drawings. However, it will be readily apparent to those skilled in the art that the embodiments can be implemented in many different ways, and their form and details can be modified in various ways without departing from the spirit and scope. Accordingly, the present invention is not to be construed as being limited to the following descriptions of embodiments, etc.

[0022] In the configuration of the invention described below, the same reference numerals are used in common across different drawings for parts that are identical or have similar functions, and repeated explanations are omitted. Furthermore, when referring to similar functions, the hatch patterns are the same, and reference numerals may not be assigned.

[0023] In this specification, ordinal numbers such as "the first," "the second," etc., are used to avoid confusion of constituent elements and do not imply any numerical limitation.

[0024] In this specification, for example, when we refer to "B on A" or "B below A," it is not necessarily required that A and B have areas in contact.

[0025] In this specification, the terms "film" and "layer" are interchangeable. For example, the terms "conductive layer" and "insulating layer" may be interchangeable with the terms "conductive film" and "insulating film," respectively.

[0026] In this specification, the term "light-emitting layer" refers to a layer provided between a pair of electrodes of a light-emitting element and containing at least a light-emitting substance.

[0027] In this specification, a display panel, which is one form of a display device, has the function of displaying (outputting) images or the like on its display surface. Therefore, a display panel is one form of an output device.

[0028] In this specification, the terms "source" and "drain" of a transistor are interchangeable depending on the transistor's polarity and the potential applied to each terminal. Generally, in an n-channel transistor, the terminal to which a low potential is applied is called the source, and the terminal to which a high potential is applied is called the drain. In a p-channel transistor, the terminal to which a low potential is applied is called the drain, and the terminal to which a high potential is applied is called the source. In practice, the terms "source" and "drain" may be interchangeable according to the above potential relationship, but in this specification, when explaining the connection relationship of a transistor, the terms "source" and "drain" are fixed for convenience.

[0029] In this specification and other documents, the source and drain of a transistor may be described using the terms first electrode and second electrode.

[0030] In this specification, the source of a transistor refers to the source region, which is part of the semiconductor layer that functions as the active layer, or the source electrode connected to the semiconductor layer. Similarly, the drain of a transistor refers to the drain region, which is part of the semiconductor layer, or the drain electrode connected to the semiconductor layer. The gate of a transistor refers to the gate electrode.

[0031] In this specification, the state in which transistors are connected in series means, for example, a state in which only one of the sources or drains of the first transistor is connected to only one of the sources or drains of the second transistor. The state in which transistors are connected in parallel means a state in which one of the sources or drains of the first transistor is connected to one of the sources or drains of the second transistor, and the other of the sources or drains of the first transistor is connected to the other of the sources or drains of the second transistor.

[0032] In this specification, "connection" may refer to an electrical connection, and includes a state in which current, voltage, or potential can be supplied or transmitted. Therefore, it also includes states in which devices are connected to each other via elements such as wiring, resistors, diodes, and transistors. Furthermore, electrical connection also includes states in which devices are directly connected to each other without the use of elements such as wiring, resistors, diodes, and transistors.

[0033] In this specification, a conductive layer may have multiple functions, such as wiring or electrodes. When a wiring is described as being connected to an electrode in this specification, this includes the case where there is a single conductive layer that has both of the above functions.

[0034] In this specification, a light-emitting element may be referred to as a light-emitting device.

[0035] In this specification, a device in which a light-emitting layer is formed using a metal mask (MM) may be referred to as a light-emitting device having a metal mask structure (MM structure). Depending on the miniaturization of the apertures, the metal mask may be referred to as a fine metal mask (FMM, high-resolution metal mask). Furthermore, in this specification, a device in which a light-emitting layer is fabricated without using a metal mask or fine metal mask may be referred to as a light-emitting device having a metal maskless structure (MML structure).

[0036] In this specification, a structure that creates separate light-emitting layers for each color of light-emitting element (for example, red (R), green (G), and blue (B)) may be referred to as an SBS (Side By Side) structure. Furthermore, in this specification, a light-emitting element capable of emitting white light may be referred to as a white light-emitting element. A white light-emitting element can be combined with a colored layer to create a full-color display device. An example of a colored layer is a color filter.

[0037] Furthermore, light-emitting devices can be broadly classified into single structures and tandem structures. A single structure has one light-emitting unit between a pair of electrodes. Preferably, the light-emitting unit includes one or more light-emitting layers. In a single structure, to obtain white light emission, one light-emitting unit should have two or more light-emitting layers, and the light emitted from each light-emitting layer should be complementary in color. For example, by making the light-emitting color of the first light-emitting layer and the light-emitting color of the second light-emitting layer complementary in color, a configuration that emits white light as a whole can be obtained. Similarly, in the case of a light-emitting device having three or more light-emitting layers, a configuration that emits white light can be obtained by satisfying the complementary color relationship.

[0038] A tandem structure preferably has two or more light-emitting units between a pair of electrodes, and each light-emitting unit preferably includes one or more light-emitting layers. In a tandem structure, it is preferable to provide intermediate layers, such as charge-generating layers, between multiple light-emitting units. In a tandem structure, to obtain white light emission, the structure should be such that the light from the light-emitting layers of two or more light-emitting units is combined to produce white light emission. In addition, the structure that produces white light emission should satisfy the complementary color relationship, similar to the single structure.

[0039] Furthermore, when comparing the aforementioned white light-emitting elements (single or tandem structure) with SBS structure light-emitting elements, SBS structure light-emitting elements can consume less power than white light-emitting elements. If you want to keep power consumption low, it is preferable to use SBS structure light-emitting elements. On the other hand, white light-emitting elements are preferable because their manufacturing process is simpler than that of SBS structure light-emitting elements, which can lead to lower manufacturing costs or higher manufacturing yields.

[0040] (Embodiment 1) This embodiment describes an example of the configuration of an electronic device according to one aspect of the present invention.

[0041] <Electronic devices and display devices> Figure 1 shows an example configuration of an electronic device 150 according to one aspect of the present invention. The electronic device 150 is preferably worn on the user's body, such as the head, like a head-mounted display, and the user can view the image on the display device 100 of the electronic device 150 while it is worn on the head. In this specification, the display device includes a display module, and a display module refers to a display panel substrate with connectors such as FPC (Flexible Printed Circuit) or TCP (Tape Carrier Package) attached, or a substrate on which ICs are mounted using the COG (Chip On Glass) method or the like.

[0042] The number of display devices 100 may be one or two or more. If there is only one display device 100, it can be used with a smartphone or the like. Furthermore, if there is only one display device 100, it can be shared between the right eye (referred to as "right eye") and the left eye (referred to as "left eye"). Figure 1 shows a case where the display device 100 has a display device 100R for the right eye and a display device 100L for the left eye. The electronic device 150 having two display devices may also have a microlens array sheet or the like to make the space between the display devices 100R and 100L less noticeable.

[0043] Each display device 100R and 100L has a display panel, the display panel has at least a pixel section, and the pixel section has at least a light-emitting element. Specifically, the pixel section may have two or more light-emitting elements capable of emitting different colors. Different colors are mutually different colors selected from the visible light spectrum. For example, in order to enable full-color display, it is preferable to use light-emitting elements that emit at least red (R), green (G), and blue (B).

[0044] The light-emitting element is preferably an organic electroluminescent element (sometimes referred to as an organic field-emitting element). An organic electroluminescent element has a laminated structure including a pair of electrodes and a light-emitting layer located between the pair of electrodes. By using different light-emitting materials (also referred to as different light-emitting materials) in the light-emitting layer, light-emitting elements exhibiting different emission colors can be obtained. The light-emitting element can have a single structure or a tandem structure. Furthermore, the light-emitting element can have an SBS structure. The specific configurations of the single structure, tandem structure, and SBS structure will be described later.

[0045] By using an MML structure in the light-emitting element, a high-resolution display device can be provided, and furthermore, an electronic device equipped with such a display device can be provided. The specific configuration of the MML structure will be described later.

[0046] In this specification and other documents, elements used for displaying light-emitting elements, etc., may be referred to as display elements.

[0047] The display devices 100R and 100L may each have, in addition to the pixel section, a drive circuit section, an image processing section, a frame memory, and a light-receiving element. One or more of these components may be integrally formed with the display device, or they may be formed separately using an IC or the like. For example, if the light-receiving element is integrally formed with the display device, the number of components such as the IC can be reduced, and the electronic device can be further miniaturized. The configuration of a light-receiving element that can be integrally formed with the display device will be described later.

[0048] Figure 1 illustrates a case where the display device 100R has a drive circuit unit 120R, an image processing unit 121R, a frame memory 122R, and a light-receiving element 159R, and the display device 100L has a drive circuit unit 120L, an image processing unit 121L, a frame memory 122L, and a light-receiving element 159L.

[0049] However, the display device 100L does not need to have the same configuration as the display device 100R. The components to be installed in each display device can be selected. For example, the light-receiving element 159 may be provided only in the display device corresponding to the eyeball to be tracked, depending on the application. That is, a configuration in which the light-receiving element 159 is provided only in the display device 100R for the right eye, or only in the display device 100L for the left eye, may be applied.

[0050] Furthermore, when using a single display device 100, a portion of the display device 100 can be designated as a display area for the right eye, and the other portion as a display area for the left eye. In this case, one or more components selected from the drive circuit unit 120, image processing unit 121, and frame memory 122 of the display device 100 can be shared between the display area for the right eye and the display area for the left eye, making it difficult to clearly distinguish between components for the right eye and components for the left eye.

[0051] <Drive Circuit Section> The drive circuit section has at least one or more drivers selected from source drivers and gate drivers. Furthermore, drive circuit section 120R and drive circuit section 120L each have at least one or more drivers selected from source drivers and gate drivers.

[0052] <Image Processing> The image processing unit has the function of processing image data. Therefore, it is preferable that the image processing unit includes an arithmetic circuit that has the function of performing sum-of-accumulate operations. Furthermore, it is preferable that the arithmetic circuit has a transistor containing a metal oxide in the channel formation region.

[0053] The image processing unit also has the function of converting analog data acquired by the light-receiving element 159 into digital data. Furthermore, the image processing unit has the function of identifying the user's gaze position from the acquired data, generating digital data related to the gaze, and enabling gaze tracking. Additionally, the image processing units 121R and 121L each have the function of converting analog data acquired by the light-receiving element 159 into digital data, and further have the function of identifying the user's gaze position from the acquired data, generating digital data related to the gaze, and enabling gaze tracking.

[0054] <Frame Memory> The frame memory has the function of storing image data, for example, it can store the video signal of each pixel in a memory cell. The frame memory also has the function of retaining image data even when the power supply to the frame memory is cut off. Frame memory 122R and frame memory 122L also have the function of storing image data, for example, they can store the video signal of each pixel in a memory cell, and furthermore they have the function of retaining image data even when the power supply to the frame memory is cut off.

[0055] <Photodetector> In one embodiment of the present invention, the electronic device 150, as shown in Figure 1, has a display device 100R with a light-receiving element 159R and a display device 100L with a light-receiving element 159L. In Figure 1, an example is shown where two light-receiving elements 159 are provided, one for the right eye and one for the left eye, but the number of light-receiving elements 159 may be one or three or more. Furthermore, if one or more light-receiving elements 159 are shared between the right eye and the left eye, it may not be possible to clearly distinguish between the optical elements for the right eye and the optical elements for the left eye.

[0056] In this specification, when describing the configurations of the photodetector 159R and the photodetector 159L, unless it is necessary to distinguish between them, the photodetector 159 will be used. The photodetector 159 may also be referred to as a photodetector or a photoelectric conversion device. The photodetector 159 can convert received light into an electrical signal. For example, an electronic device 150 having a photodetector 159 has the function of detecting infrared light and converting it into an electrical signal. The electronic device 150 can also generate an image by processing the electrical signal obtained from detecting infrared light. The image may be referred to as an infrared image, and the function of generating an image may be referred to as an imaging function. To efficiently perform the imaging function, it is preferable to arrange multiple photodetectors 159 on the display device 100. The imaging capability of the electronic device 150 is improved by having multiple photodetectors 159.

[0057] In this specification, infrared light may be referred to as infrared light, and light with a wavelength of 780 nm to 3000 nm may be referred to as infrared light, while light with a wavelength of 800 nm to 2500 nm may be referred to as near-infrared light. These wavelengths differ from those in the visible light region. The photodetector 159 is not limited to the above configuration and may have a function to detect visible light in addition to infrared light.

[0058] For example, the light-receiving element 159R is integrally formed with the display device 100R, as shown in Figure 1. Similarly, the light-receiving element 159L is integrally formed with the display device 100L, as shown in Figure 1. By integrally forming the light-receiving element 159 with the display device 100, the number of components can be reduced, and the electronic device 150 can be made smaller. Reducing the number of components also allows for weight reduction of the electronic device 150. Furthermore, reducing the number of components allows for space saving within the electronic device 150, which in turn allows for, for example, a larger battery 158.

[0059] The light-receiving element 159R may be formed separately from the display device 100R, and the light-receiving element 159L may also be formed separately from the display device 100L.

[0060] <Infrared light source> An electronic device 150 according to one aspect of the present invention has an infrared light source 160 in addition to the above. The infrared light source 160 has the function of emitting light with a wavelength of 780 nm to 3000 nm, preferably 800 nm to 2500 nm, and can use, for example, a light-emitting diode (sometimes referred to as an LED). Light of such wavelengths is preferable because it does not obstruct the visibility of the image. It is often said that infrared light is emitted from the infrared light source 160, but near-infrared light may also be emitted, and for eye-tracking, a wavelength of 830 nm to 870 nm is preferred. The infrared light source 160 is preferably formed separately from the display device 100, as shown in Figure 1. Furthermore, in order to perform eye-tracking with high accuracy, it is more preferable that the infrared light source 160 be placed near the eyeball, and an infrared light source 160 formed separately from the display device 100 is easier to place near the eyeball.

[0061] Although Figure 1 shows one infrared light source 160, two may be provided, one for the right eye and one for the left eye. Furthermore, there may be three or more infrared light sources 160. Also, if one or more infrared light sources 160 are shared between the right and left eyes, it may not be possible to clearly distinguish between the optical elements for the right eye and the optical elements for the left eye.

[0062] Infrared light from the infrared light source 160 can be detected by the photodetector 159. Specifically, infrared light reflected by the eyeball, or at least the cornea, can be detected by the photodetector 159. Infrared light reflected by the cornea is sometimes referred to as reflected light. Even if the user is wearing contact lenses or the like, infrared light can still be reflected by the cornea.

[0063] The infrared light source 160 may be an LED or a light-emitting element equipped with the function of emitting infrared light, and such light-emitting element may be integrally formed with the display device 100. For example, a light-emitting element (IR) equipped with the function of emitting infrared light can be formed in the pixel portion of the display device 100 as the infrared light source 160. Multiple light-emitting elements (IR) may be formed in the pixel portion.

[0064] When an infrared-emitting element (IR) is incorporated into the display device, the electronic device 150 can be miniaturized and its weight reduced. Furthermore, space savings are achieved within the electronic device 150, allowing for, for example, a larger battery 158.

[0065] By having the above-described display device 100, light-receiving element 159, and infrared light source 160, etc., the electronic device 150 according to one aspect of the present invention can be equipped with an eye-tracking function. Eye-tracking methods include the Pupil Centre Corneal Reflection method and the Bright / Dark Pupil Effect method, and either method may be applied to the electronic device 150 according to one aspect of the present invention. The electronic device 150 according to one aspect of the present invention may also appropriately combine the Pupil Centre Corneal Reflection method and the Bright / Dark Pupil Effect method, etc.

[0066] Electronic devices with eye-tracking capabilities can determine the number of blinks. In other words, one embodiment of the present invention, the electronic device 150, can have a blink detection function. Furthermore, since the number of blinks increases with fatigue, fatigue can be determined from the number of blinks. Therefore, one embodiment of the present invention, the electronic device 150, can have a fatigue detection function.

[0067] Furthermore, by equipping an electronic device according to one embodiment of the present invention with a sensor unit, it is possible to measure one or more selected parameters from the user's blinking frequency, body temperature, pulse rate, and blood oxygen saturation, and to detect the user's fatigue level and health condition. The electronic device 150 according to one embodiment of the present invention can detect the user's fatigue level and health condition and display warnings or the like on the display device 100.

[0068] Furthermore, since the electronic device 150 in one aspect of the present invention utilizes infrared light, it is sometimes described as having an infrared light-based eye-tracking function. Because infrared light is invisible, eye-tracking can be performed without obstructing the image of the display device 100 in one aspect of the present invention, which is preferable.

[0069] The electronic device 150 shown in Figure 1 includes, in addition to the display device 100, an optical system 151 (optical systems 151R and 151L), a motion detection unit 153, an audio system 154, a camera 155, a control unit 156, a communication unit 157, and a battery 158. In one aspect of the present invention, the electronic device 150 has a novel optical system 151 that constructs the optical path of the reflected light, thereby providing an electronic device with a novel configuration.

[0070] <Optical system 1> This section describes the optical system that constructs the optical path of reflected light. The optical system 151 shown in Figure 1 is divided into two parts, optical system 151R and optical system 151L, for the right and left eyes. However, the number of optical systems may be one or three or more. Optical system 151 can sometimes be shared between the right and left eyes, making differentiation difficult in some cases. In this specification, when describing the configurations of optical system 151R and optical system 151L, optical system 151R will be used as the basis for the description when there is no need to distinguish between them.

[0071] The optical system 151 will now be described. Figure 2(A) is a top view of the electronic device 150 and illustrates an optical system 151L that can be used for the left eye. A similar optical system to that in Figure 2(A) can be used for the right eye 151R, and part or all of the optical system may be shared between the left and right eye systems.

[0072] Figure 2(A) shows the display device 100L of the electronic device 150 and the user's eyeball 108L that views the image on the display device 100L. The electronic device 150 further has an optical system 151L between the display device 100L and the eyeball 108L. The optical system 151L has a first optical element 111OP, a second optical element 112OP, and a third optical element 113OP, and visible light from the display device 100L passes through these in order to reach the eyeball 108L. The optical system 151L also has a fourth optical element 114OP, a fifth optical element 115OP, and a sixth optical element 116OP, but these are not arranged in the optical path of visible light. The optical path of visible light is shown by a solid line in Figure 2(A). Note that when the user is wearing the electronic device 150 and is allowed to view the outside world, the display device 100L is positioned so as not to overlap with the line of sight. Visible light from the display device 100L, positioned so as not to overlap with the line of sight, can be directed into the optical system 151L using a reflector (sometimes referred to as a mirror) or the like. The reflectance of visible light in the reflector should be 90% or higher, preferably 95% or higher, and more preferably 98% or higher.

[0073] Each optical element may be one or more selected from lenses, prisms, mirrors, half-mirrors, filters, and diffraction gratings.

[0074] For example, filters may be used for the first optical element 111OP and the third optical element 113OP, and IR cut filters are even more preferred. An IR cut filter has the function of transmitting visible light while cutting infrared light. In an IR cut filter, the transmittance of visible light should be 80% or more, preferably 90% or more. The wavelength of infrared light to be cut should be 780 nm or more, preferably 800 nm to 2500 nm.

[0075] Furthermore, the second optical element 112OP may be a lens, and a biconvex lens is even more preferable. The maximum thickness of the lens should be between 10 mm and 20 mm. The second optical element 112OP may have a function to adjust the focus between the display device 100R and the eyeball 108L.

[0076] Figure 2(A) also shows the state after infrared light from the infrared light source 160 is reflected by the eyeball 108L, with the optical path of the reflected light indicated by a dotted line. The reflected light is reflected by the third optical element 113OP, reflected by the fourth optical element 114OP, transmitted through the fifth optical element 115OP, reflected by the sixth optical element 116OP, and reflected by the first optical element 111OP before reaching the display device 100L. In other words, the reflected light is incident on the light-receiving element 159L of the display device 100L via an optical path that is partially different from the optical path of the visible light. Note that the optical elements are just examples, and the reflected light may reach the display device 100L using optical elements other than those described above, but it is preferable that the optical path of the reflected light be partially different from the optical path of the visible light.

[0077] Since the light-receiving element 159L is integrally formed with the display device 100L, reflected light can enter the light-receiving element 159L via the same optical path as visible light. However, if the optical path of the reflected light is the same as that of the visible light, it becomes difficult for the user to see the visible light. Specifically, the user may have difficulty focusing and may see a blurred image. Therefore, an optical system 151L is provided so that the reflected light follows a partially different optical path from that of the visible light. The optical system 151L preferably has a first group of optical elements located in the optical path through which the image from the display device 100L enters the eyeball 108L, and a second group of optical elements located in the optical path through which the reflected light enters the light-receiving element 159L. Some or all of the optical elements in the first group may be located in the optical path through which the reflected light enters the light-receiving element 159L. That is, some or all of the optical elements in the first group can be shared with the optical elements in the second group, and the number of optical elements can be reduced by this shared configuration.

[0078] To reiterate, the reflected light is reflected by, for example, the third optical element 113OP, the fourth optical element 114OP, passes through the fifth optical element 115OP, is reflected by the sixth optical element 116OP, and is reflected by the first optical element 111OP before being incident on the light-receiving element 159L of the display device 100L. It is preferable that one or more light-receiving elements 159L are arranged on the display device 100L, as this allows for the generation of infrared images like a camera.

[0079] The fifth optical element 115OP may have a function to adjust the optical distance between the fourth optical element 114OP and the sixth optical element 116OP. However, if the reflected light can be incident on the light-receiving element 159L of the display device 100L, the fifth optical element 115OP does not need to be provided.

[0080] The optical elements include a fourth optical element 114OP, a fifth optical element 115OP, and a sixth optical element 116OP, which correspond to an optical path that is partially different from the optical path of visible light. As mentioned above, the fifth optical element 115OP is not necessarily required.

[0081] As mentioned above, to transmit visible light, the first optical element 111OP and the third optical element 113OP may each be fitted with filters, and IR cut filters are even more preferable. Since the first optical element 111OP and the third optical element 113OP also have the function of reflecting infrared light, one side of the filter or IR cut filter has a material that reflects infrared light. Of course, the other side of the filter or IR cut filter transmits visible light. In other words, it is preferable that the material differs on one side and the other side of the filter or IR cut filter. To make the material differ on one side and the other side, it is preferable that the filter or IR cut filter has a multilayer structure.

[0082] The fourth optical element 114OP and the sixth optical element 116OP may each be a filter or a mirror, and an IR cut filter or an IR mirror is more preferable. The IR mirror has the function of reflecting infrared light. In the IR mirror, the reflectance of infrared light should be 90% or more, preferably 95% or more.

[0083] The fifth optical element 115OP may be a lens, and a biconvex lens is even more preferable. The maximum thickness of the lens should be between 10 mm and 20 mm.

[0084] By using this type of optical path, the focus of visible light can be adjusted independently of the reflected light path, thereby suppressing image focus shifts and other issues.

[0085] <Optical system 2> Figure 2(B) illustrates an optical system 151L for the left eye that is different from that in Figure 2(A). Figure 2(B) is a top view of the electronic device 150 and illustrates an optical system 151L that can be used for the left eye. Note that the same optical system as in Figure 2(B) can be used for the right eye optical system 151R, and part or all of the optical system may be shared between the left and right eyes.

[0086] As shown in Figure 2(B), the optical system 151L allows visible light from the display device 100L to pass through the first optical element 111OP, the seventh optical element 117OP, the second optical element 112OP, the eighth optical element 118OP, and the third optical element 113OP to reach the eyeball 108L. The optical path of the visible light is shown by a solid line in Figure 2(B).

[0087] Filters may be used for the seventh optical element 117OP and the eighth optical element 118OP, and IR cut filters are even more preferred.

[0088] Figure 2(B) also shows the state after infrared light from the infrared light source 160 is reflected by the eyeball 108L, with the optical path of the reflected light indicated by a dotted line. The reflected light enters the display device 100L via an optical path that is partially different from the optical path of the visible light.

[0089] The reflected light is reflected by the third optical element 113OP, then by the fourth optical element 114OP, then by the ninth optical element 119OP, then by the eighth optical element 118OP, passes through the second optical element 112OP, is reflected by the seventh optical element 117OP, then by the tenth optical element 130OP, then by the sixth optical element 116OP, then by the first optical element 111OP, and is incident on the light-receiving element 159L of the display device 100L.

[0090] The optical elements include a fourth optical element 114OP, a ninth optical element 119OP, a tenth optical element 130OP, and a sixth optical element 116OP, which correspond to an optical path that is partially different from the optical path of visible light.

[0091] Filters may be used for the seventh optical element 117OP and the eighth optical element 118OP, with IR cut filters being even more preferred. Since the seventh optical element 117OP and the eighth optical element 118OP each have the function of reflecting infrared light, one side of the filter or IR cut filter has a material that reflects infrared light. Of course, the other side of the filter or IR cut filter transmits visible light. In other words, it is preferable for the material on one side and the other side of the filter or IR cut filter to be different. To make the material on one side and the other side different, it is preferable for the filter or IR cut filter to have a multilayer structure.

[0092] Furthermore, filters or mirrors may be used for the ninth optical element 119OP and the tenth optical element 130OP, respectively, and IR cut filters or IR mirrors are even more preferred.

[0093] By using such an optical path, the focus of the visible light can be adjusted independently of the reflected light path, and the optical distance of the visible light path can be made appropriate, thereby suppressing image focus shifts and other issues.

[0094] <Optical system 3> Figures 3(A) and 3(B) illustrate a case where, unlike optical systems 1 and 2, each optical element has a drive mechanism 123a to 123d. Figures 3(A) and 3(B) are top views of the electronic device 150, and the optical system 151 with a drive mechanism is applied to an optical system for both eyes (eyeballs 108R and 108L). An optical system for both eyes is one in which part or all of the optical system is shared for the left eye and the right eye.

[0095] Figures 3(A) and 3(B) show a display device 100R for the right eye and a display device 100L for the left eye. However, instead of display devices 100R and 100L, a single display device may be used, with the left side of that single display device used as display device 100L and the right side as display device 100R.

[0096] In Figures 3(A) and 3(B), the optical path of visible light is shown by a solid line, and the reflected light is shown by a dotted line. In Figure 3(A), in order to track the position of the left eyeball 108L, infrared light from an infrared light source 160L is reflected by the eyeball 108L, and then, as shown by the dotted line, the reflected light is reflected by each optical element and incident on the light-receiving element 159L of the display device 100L. In this way, the line of sight can be tracked (eye tracking). This optical system is the same configuration as shown in Figure 2(A).

[0097] Next, in Figure 3(B), in order to track the position of the right eyeball 108R, the device has drive mechanisms 123a to 123d so that infrared light reflected by the right eyeball 108R is reflected by each optical element. The drive mechanism has one or more functions selected from the functions of moving and rotating each optical element. For example, each optical element is rotated using the drive mechanisms 123a to 123d. As a result, as shown by the dotted line in Figure 3(B), the reflected light is reflected by each optical element and incident on the light-receiving element 159R of the display device 100R. In this way, line of sight tracking can be performed.

[0098] By appropriately equipping each optical element with a drive mechanism, it becomes possible to perform eye tracking for both eyes.

[0099] Such optical systems 1 to 3 are preferably mounted on the electronic device 150 while fitted into a housing. The optical system fitted into the housing is sometimes referred to as a lens unit. The electronic device 150 is preferably equipped with a lens unit position adjustment mechanism so that the position of each optical element can be adjusted by the user.

[0100] <Motion detection unit> The motion detection unit 153 in Figure 1 has an inertial sensor and is equipped with the function of detecting the user's body movements. Here, an inertial sensor refers to a sensor that detects the acceleration and angular velocity of an object. If it is not necessary to detect the user's body movements, the electronic device 150 does not need to have the motion detection unit 153.

[0101] <Audio> The audio 154 in Figure 1 includes, for example, one or more of a microphone and a speaker. The electronic device 150 does not have to have the audio 154.

[0102] <Camera> The camera 155 in Figure 1 has the function of an information input device, and for example, a digital camera can be used. The electronic device 150 does not necessarily have to have the camera 155.

[0103] <Department Head> The control unit 156 in Figure 1 includes a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and memory.

[0104] <Communications Department> The communication unit 157 in Figure 1 can perform wireless communication and exchange data with other terminals or servers located on the network. The electronic device 150 does not necessarily have to have the communication unit 157.

[0105] The control unit 156 and the communication unit 157 may be provided separately, rather than being integrated with the electronic device 150. In that case, the bulk of the electronic device 150 can be reduced.

[0106] <Battery> A lithium-ion secondary battery or the like can be used for the battery 158 in Figure 1.

[0107] <Bus wiring> In Figure 1, the display device 100, motion detection unit 153, audio 154, camera 155, control unit 156, communication unit 157, and infrared light source 160 can send and receive various signals from each other via bus wiring BW.

[0108] For example, data related to gaze direction generated by the image processing unit 121R or image processing unit 121L can be sent to the control unit 156 via the bus wiring BW.

[0109] <Image Analysis> The image processing unit 121 in Figure 1 can perform image analysis and has the function of identifying the position of the user's gaze using this image analysis. In other words, by performing image analysis, gaze position data can be constructed from gaze data acquired by the light-receiving element 159. Gaze tracking can be performed using such image analysis.

[0110] The above image analysis can utilize a neural network, an arithmetic circuit with the function of performing multiply-accumulate operations, etc. In other words, the image processing unit 121 has one or two selected from a processing unit capable of executing a program including a neural network, and an arithmetic circuit with the function of performing multiply-accumulate operations.

[0111] For the image analysis described above, it is particularly preferable to use an arithmetic circuit equipped with a function to perform multiply-accumulate operations. By using such an arithmetic circuit, image analysis can be performed with low power consumption. In other words, the power consumption of the display device 100 according to one aspect of the present invention, or the electronic device 150 on which the display device is installed, can be reduced. Details of the arithmetic circuit equipped with a multiply-accumulate operation function will be described later.

[0112] A neural network may be used for the image analysis described above. In particular, deep learning is preferred as the neural network. For example, it is preferable to use a Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Autoencoder (AE), Variational Autoencoder (VAE), Random Forest, Support Vector Machine, Gradient Boosting, Generative Adversarial Network (GAN), etc.

[0113] <Image Processing> The control unit 156 preferably includes an image processing function. It is desirable that the image processing is performed to identify the area where the user's line of sight overlaps with the image viewed by the user. Specifically, by using the image analysis described above, the area overlapping with the line of sight can be identified for the display device 100.

[0114] Figure 4(A) shows the user's line of sight G and the user's field of view superimposed on the image of the display device 100. The user's field of view includes the first region S1 to the third region S3, which are defined as the first region S1 containing the line of sight G, the second region S2 outside the first region S1, and the third region S3 outside the second region S2. Note that the first region S1 to the third region S3 can be understood by substituting display device 100R or display device 100L for display device 100.

[0115] Human vision, although varying from person to person, is classified into the discriminative field of vision, effective field of vision, stable gaze field of vision, guided field of vision, and auxiliary field of vision. The discriminative field of vision is the area in which visual functions such as visual acuity and color discrimination are best. Considering a line drawn from the user's eyeball 108 to intersect with the display device 100 (hereinafter referred to as the intersection line), the discriminative field of vision refers to the area within an angle θx1 = approximately 5° in the vertical and horizontal directions centered on the intersection line. Figure 4(B) shows the angle θx1 corresponding to the horizontal angle θx1 = approximately 5°. That is, the discriminative field of vision corresponds to the first area S1 in Figure 4(A).

[0116] The effective field of view is the area in which specific information can be instantly identified solely by eye movements. It is defined as the area within a horizontal angle θx2 = approximately 30° centered on the intersection line, and within a vertical angle of approximately 20° (excluding the discrimination field of view), although this cannot be shown in Figure 4(B). Figure 4(B) shows the angle θx2 corresponding to the horizontal angle θx2 = approximately 30°. In other words, the effective field of view corresponds to the second region S2 in Figure 4(A).

[0117] The stable field of view is the area in which specific information can be identified without strain while moving the head. The guided field of view is the area in which the presence of a specific object can be recognized, but the ability to identify it is low. The auxiliary field of view is the area in which the ability to identify a specific object is extremely low, and only the presence of a stimulus can be recognized. One or more fields of view selected from the stable field of view, guided field of view, and auxiliary field of view correspond to the third region S3 in Figure 4(A).

[0118] From the above, it is clear that image quality from the discrimination field to the effective field of view is important in video. In particular, it is crucial to improve the image quality of the discrimination field. Therefore, it is advisable to perform image processing using the control unit 156 to improve the image quality of the first region S1, or the first region S1 and the second region S2, as shown in Figure 4(A). That is, it is advisable to enhance the image of the first region S1, or the first region S1 and the second region S2. Image processing includes increasing the resolution of the video by upconversion. Such upconversion can provide video that is easy to view.

[0119] In addition to image processing, audio processing may also be applied.

[0120] <Generating Image Data> For example, if software is installed on the electronic device 150, the control unit 156 can generate image data based on information from the software, information from the camera 155, and the image processing information described above. The image data is sent to the display device 100 via the bus wiring BW. The image data is stored in the frame memories 122R and 122L of the display device 100. Subsequently, the display unit of the display device 100 is controlled by source drivers and the like included in the drive circuit units 120R and 120L, enabling the display of the image data.

[0121] <Display device> As shown in Figure 5(A), the case in which the display device 100 has light-emitting elements 110R, 110G, and 110B will be described. When it is not necessary to distinguish between light-emitting elements 110R, 110G, and 110B, they will be referred to as light-emitting element 110. Note that the configuration of the light-emitting elements 110, etc., can be understood by substituting display device 100 with display device 100R or display device 100L.

[0122] Figure 5(A) illustrates a case where the display device 100 has a drive circuit unit 120, an image processing unit 121, and a frame memory 122.

[0123] Furthermore, the electronic device shown in Figure 5(A) includes an infrared light source 160 and a light-receiving element 159, wherein the infrared light source 160 is located outside the display device 100, and the light-receiving element 159 is located within the display device 100. The infrared light source 160 may also be provided within the display device 100. The light-receiving element 159 may also be provided outside the display device 100.

[0124] Figure 5(B1) shows an example arrangement of an infrared light source 160, a display device 100 having a photodetector 159, an optical system 151, and an eyeball 108. Infrared light from the infrared light source 160 enters the eyeball 108 without passing through the optical system 151. If the light passes through the optical system 151, the amount of infrared light output from the infrared light source 160 may decrease, so it is preferable for the light to enter the eyeball 108 without passing through the optical system 151. The electronic device shown in Figure 5(B1) allows for miniaturization of the optical system 151. The configuration of the optical system 151 is as described using Figures 2 and 3 above. Note that the structure of the electronic device can be understood by substituting display device 100R or display device 100L for display device 100.

[0125] As another example of arrangement, as shown in Figure 5(B2), infrared light from the infrared light source 160 can be incident on the eyeball 108 via the optical system 151. The configuration of the optical system 151 is as described above using Figures 2 and 3, and infrared light from the infrared light source 160 can be incident on the eyeball 108 via optical elements arranged in the optical path of the reflected light. In addition to the optical system described above using Figures 2 and 3, infrared light from the infrared light source 160 can also be incident on the eyeball 108 via new optical elements. One or more of the new optical elements can be selected from lenses, prisms, mirrors, half-mirrors, filters, and diffraction gratings. Note that the structure of the electronic device can be understood by reading display device 100 as display device 100R or display device 100L.

[0126] As shown in Figures 5(B1) and 5(B2), the reflected light from the eyeball 108 can pass through the optical system 151 and enter the photodetector 159. The configuration of the optical system 151 is as described above using Figures 2 and 3.

[0127] Such an electronic device 150 according to one aspect of the present invention can be equipped with an eye-tracking function and can further suppress image focus shifts.

[0128] As shown in Figure 6(A), the display device 100 has a light-emitting element 110IR as an infrared light source, and the light-emitting element 110IR may be integrally formed with the display device 100. Such a display device is referred to as an infrared light source integrated type display device. Unlike Figure 5, etc., in the display device 100 shown in Figure 6(A), the light-emitting element 110IR is formed through the same process as the light-emitting element 110, etc., which allows for miniaturization of the electronic device 150. Note that the structure of the electronic device can be understood by substituting display device 100R or display device 100L for display device 100.

[0129] Figure 6(B1) shows an example of the arrangement of a display device 100 having a light-emitting element 110IR and a photodetector 159, an optical system 151, and an eyeball 108. As shown in Figure 6(B1), infrared light from the light-emitting element 110IR can enter the eyeball 108 via the optical system 151. The configuration of the optical system 151 is as described above using Figures 2 and 3, and infrared light from the light-emitting element 110IR can enter the eyeball 108 via optical elements arranged in the optical path of the reflected light. In addition to the optical system described above using Figures 2 and 3, infrared light from the light-emitting element 110IR can also enter the eyeball 108 via new optical elements. One or more of the new optical elements can be selected from lenses, prisms, mirrors, half-mirrors, filters, and diffraction gratings. Note that the structure of the electronic device can be understood by reading display device 100 as display device 100R or display device 100L.

[0130] As an example of the above arrangement, as shown in Figure 6(B2), multiple light-emitting elements 110IR and light-receiving elements 159 may be provided on the display device 100. Providing multiple light-emitting elements 110IR and light-receiving elements 159 is preferable because it does not increase the size of the display device 100. Note that the structure of the electronic device can be understood by substituting display device 100R or display device 100L for display device 100.

[0131] As shown in Figures 6(B1) and 6(B2), the reflected light from the eyeball 108 can pass through the optical system 151 and enter each photodetector 159. The configuration of the optical system 151 is as described above using Figures 2 and 3.

[0132] In Figures 6(B1) and 6(B2), the electronic device can grasp information about the infrared light emitted from the light-emitting element 110IR. Therefore, when detecting reflected light with the photodetector 159, data processing may be performed in the control unit of the electronic device 150 based on this information. Through data processing, it becomes possible to accurately grasp information about the reflected light, even if, for example, the amount of infrared light output from the light-emitting element 110IR decreases after passing through the optical system 151.

[0133] As described above, the electronic device 150 according to one aspect of the present invention can be equipped with an eye-tracking function and can further suppress image focus shift.

[0134] This embodiment can be implemented in appropriate combination with other embodiments described herein, at least in part.

[0135] [Example of a display device configuration] Next, an example of the configuration of a display device 100 in an electronic device according to one aspect of the present invention will be described. Although the electronic device has at least an optical system in addition to the display device 100, the illustration of the optical system will be omitted in the description of this example of the display device configuration. The optical system is located between the display device 100 and the eyeball 108, and the optical system described using Figures 2 and 3 above can be used. Note that the structure of the electronic device can be understood by substituting display device 100 for display device 100R or display device 100L.

[0136] The display device 100 shown in Figure 7(A) is an infrared light source integrated display device and comprises a substrate 101, a substrate 102, and light-emitting elements 110R, 110G, 110B, and 110IR sandwiched between substrates 101 and 102. The light-emitting elements 110R, 110G, 110B, and 110IR are each formed on substrate 101 through the same process. The light-emitting elements 110R, 110G, 110B, and 110IR are also formed on a layer 106 sandwiched between substrates 101 and 102. An electrode (referred to as a common electrode) may be provided in layer 106 so as to be common to the light-emitting elements 110R, 110G, 110B, and 110IR. The light-emitting elements 110R, 110G, and 110B function as display elements, and the light-emitting element 110IR functions as an infrared light source. The light-emitting elements 110R, 110G, and 110B are located in the pixel section 109, while the light-emitting element 110IR is located outside the pixel section 109.

[0137] The light-emitting element 110R emits red light, the light-emitting element 110G emits green light, the light-emitting element 110B emits blue light, and the light-emitting element 110IR emits infrared light. The number of light-emitting elements 110IR is not particularly limited; there may be one or two or more.

[0138] The light emitted by the light-emitting element 110IR preferably includes infrared light, preferably near-infrared light. For example, infrared light with a wavelength of 700 nm or more, preferably near-infrared light having one or more peaks in the range of 800 nm to 2500 nm, can be used.

[0139] The display device 100 shown in Figure 7(A) has a light-receiving element 159 located below the substrate 101, and a support plate 103 that supports the light-receiving element 159. The light-receiving element 159 is integrally formed with the display device 100 and is positioned to overlap with the pixel portion 109. Furthermore, a protective member 105 may be provided on the substrate 102.

[0140] The light-receiving element 159 only needs to have the function of detecting infrared light, and more specifically, it is preferable that it has the light sensitivity to infrared light, preferably near-infrared light, emitted by the light-emitting element 110IR.

[0141] As shown in Figure 7(A), an image is displayed on the pixel section 109 by the light emitted from the light-emitting elements 110R, 110G, and 110B. In addition, infrared light emitted from the light-emitting element 110IR is reflected by the user's eyeball 108, and the reflected light is detected by the light-receiving element 159 to perform gaze tracking. For this reason, the substrate 102 and the protective member 105 need to transmit visible light from the light-emitting elements 110R, 110G, and 110B, as well as infrared light emitted from the light-emitting element 110IR and infrared light reflected by the eyeball 108. Therefore, it is preferable that the substrate 102 and the protective member 105 are translucent to visible light and infrared light. Furthermore, the infrared light reflected by the eyeball 108 needs to pass through the substrate 101. Therefore, it is preferable that the substrate 101 is translucent to at least infrared light.

[0142] The substrate 101 can be made of an insulator such as glass, quartz, ceramic, sapphire, or stabilized zirconia (such as yttria-stabilized zirconia), an insulating resin, a conductive resin, a semiconductor such as silicon, germanium, silicon carbide, silicon germanium, gallium arsenide, indium phosphide, or zinc oxide, a metal, an alloy, etc. However, the substrate 101 must be made of a material that is transparent to infrared light.

[0143] The substrate 102 can be made of an insulator such as glass, quartz, ceramic, sapphire, or stabilized zirconia (such as yttria-stabilized zirconia), an insulating resin, a conductive resin, a semiconductor such as silicon, germanium, silicon carbide, silicon germanium, gallium arsenide, indium phosphide, or zinc oxide, a metal, an alloy, etc. However, the substrate 102 must be made of a material that is transparent to visible light and infrared light.

[0144] Furthermore, by using flexible materials for substrates 101 and 102, the flexibility of the display device 100 can be increased, and the display device 100 can be made lighter and thinner.

[0145] Flexible materials that can be used include polyester resins such as polyethylene terephthalate (PET) and polyethylene naphthalate (PEN), polyacrylonitrile resin, acrylic resin, polyimide resin, polymethyl methacrylate resin, polycarbonate (PC) resin, polyethersulfone (PES) resin, polyamide resin (nylon, aramid, etc.), polysiloxane resin, cycloolefin resin, polystyrene resin, polyamide-imide resin, polyurethane resin, polyvinyl chloride resin, polyvinylidene chloride resin, polypropylene resin, polytetrafluoroethylene (PTFE) resin, ABS resin, and cellulose nanofiber. ABS resin refers to a copolymerized resin of acrylonitrile, butadiene, and styrene.

[0146] Furthermore, glass of a thickness sufficient to be flexible may be used for one or both of the substrates 101 and 102, and polarizing plates may also be used for one or both of the substrates 101 and 102.

[0147] Furthermore, one aspect of the present invention may be configured as shown in Figure 7(B). The display device shown in Figure 7(B) is an infrared light source integrated type display device, and differs from the display device shown in Figure 7(A) in that the light-emitting element 110IR, which serves as the infrared light source, is provided within the pixel section 109. Since both the light-emitting element 110IR and the light-receiving element 159 are located in the pixel section 109, a part of the light-emitting element 110IR may overlap with the light-receiving element 159, but even in such cases, the light-receiving element 159 can still receive reflected light. The other configurations are the same as those of the display device shown in Figure 7(A), so for details, please refer to the description of Figure 7(A) above.

[0148] As described above, in the display device 100 shown in Figures 7(A) and 7(B), the layer 106 containing the light-emitting element 110IR is superimposed on the light-receiving element 159, so the distance between the light-emitting element 110IR and the light-receiving element 159 becomes relatively small. Therefore, the detection sensitivity of reflected light from the eyeball 108 can be increased.

[0149] It should be noted that the present invention is not limited to this embodiment, and as shown in Figures 8(A) and 8(B), an infrared light source integrated display device may also be provided, in which the light-receiving element 159 is provided together with the light-emitting element 110 between the substrate 101 and the substrate 102, for example, in layer 106. Specifically, the display device 100 shown in Figure 8(A) differs from the display device 100 shown in Figure 7(A) in that the light-receiving element 159 and the light-emitting element 110 are provided between the substrate 101 and the substrate 102. Similarly, the display device 100 shown in Figure 8(B) differs from the display device 100 shown in Figure 7(B) in that the light-receiving element 159 and the light-emitting element 110 are provided between the substrate 101 and the substrate 102. In the display devices 100 shown in Figures 8(A) and 8(B), since the light-receiving element 159 is provided on the substrate 101, the substrate 101 may have low light transmittance to infrared light, or may not have light transmittance to infrared light at all. Furthermore, in the display devices 100 shown in Figures 8(A) and 8(B), the support plate 103 may be omitted.

[0150] In the display device 100 shown in Figure 7(A) and the display device 100 shown in Figure 8(A), the light-emitting element 110IR, which serves as an infrared light source, is located outside the pixel section 109. In this case, as shown in Figure 2 or Figure 3, the optical system 151 provided between the display device 100 and the eyeball 108 should be designed so that the infrared light emitted from the light-emitting element 110IR is irradiated onto the user's eyeball. The infrared light can be irradiated onto the eyeball using an optical element located in the optical path of the reflected light of the optical system 151, or a new optical element may be added. With this configuration, the infrared light emitted from the light-emitting element 110IR is reflected by the user's eyeball 108, and the reflected light is detected by the light-receiving element 159 via the optical system 151, thereby enabling gaze tracking.

[0151] In one aspect of the present invention, the light-receiving element 159 may be provided outside the pixel portion 109. Specifically, the display device 100 shown in Figure 9(A) is a display device with an integrated infrared light source, in which the light-receiving element 159 is also provided outside the pixel portion 109 together with the light-emitting element 110IR. The display device 100 shown in Figure 9(B) is a display device with an integrated infrared light source, in which the light-receiving element 159 is provided outside the pixel portion 109, and the light-emitting element 110IR is provided in the pixel portion 109.

[0152] The display device 100 shown in Figure 10(A) is an infrared light source integrated display device, in which the light-receiving element 159 is formed in layer 106 together with the light-emitting element 110IR and is provided outside the pixel portion 109. The display device 100 shown in Figure 10(B) is an infrared light source integrated display device, in which the light-receiving element 159 is formed in layer 106 together with the light-emitting element 110IR, and the light-receiving element 159 is provided outside the pixel portion 109, while the light-emitting element 110IR is provided in the pixel portion 109. Furthermore, in the display devices 100 shown in Figures 10(A) and 10(B), the support plate 103 may be omitted.

[0153] Furthermore, although the above describes a configuration in which pixels are formed using four types of light-emitting elements: a red light-emitting element 110R, a green light-emitting element 110G, a blue light-emitting element 110B, and an infrared light-emitting element 110IR, the present invention is not limited thereto. For example, the light-emitting element 110R may be configured to emit light with peaks at both red and infrared wavelengths, and the pixels may be formed using three types of light-emitting elements: light-emitting element 110R, light-emitting element 110G, and light-emitting element 110B.

[0154] Furthermore, in the display devices 100 shown in Figures 7(A), 7(B), 9(A), and 9(B), a substrate may be provided instead of the support plate 103, and an insulating layer may be provided instead of the substrate 101. In this case, the light-receiving element 159 may be provided on the substrate, or the light-receiving element 159 may be formed using the substrate. Alternatively, an insulating layer may be provided on the light-receiving element 159, and the light-emitting element 110 may be provided on the insulating layer. It is preferable that the insulating layer is transparent to at least infrared light.

[0155] In each of the display devices 100 shown, it may not be necessary to provide the protective member 105.

[0156] [Examples of electronic device operation] An example of the operation of an electronic device according to one aspect of the present invention will be described below using the flowchart shown in Figure 11. The electronic device is equipped with a display device that is integrally formed with an infrared light source.

[0157] The flowchart shown in Figure 11 has steps S210 to S213. First, in step S210, the user's eyeball is illuminated using the light-emitting element 110IR, which emits infrared light as an infrared light source. An infrared light source 160 may also be used in this step. Next, in step S211, the light reflected from the eyeball 108 is detected by a light-receiving element.

[0158] In step S212, the user's line of sight is determined based on information from the light-receiving element. The information from the light-receiving element may be processed as needed. Then, in step S213, the image displayed by the display device is updated based on the user's line of sight. As part of the update, it may be necessary to include image processing to increase the resolution in the area including the user's line of sight, as shown in Figure 4. By increasing the resolution only in that area, the load on the GPU and other components of the control unit can be reduced.

[0159] By using this invention, it is possible to track the user's gaze. By tracking the user's gaze, for example, it is possible to understand what the user is paying attention to and analyze the user's behavior. Furthermore, it is possible to reproduce the user's eye movements on an avatar. In addition, it is possible to perform operations or menu selections using gaze.

[0160] This embodiment makes it possible to provide an electronic device equipped with eye-tracking functionality.

[0161] The configuration examples illustrated in this embodiment, and the corresponding drawings, etc., can be appropriately combined with other configuration examples or drawings, etc., at least in part.

[0162] (Embodiment 2) This embodiment describes an example configuration of an electronic device according to one aspect of the present invention, and an electronic device to which a display device according to one aspect of the present invention is applied.

[0163] A display device according to one aspect of the present invention is equipped with an eye-tracking function and can enhance resolution, making it suitable for use in electronic devices for virtual reality or augmented reality. Alternatively, it can be suitable for use in electronic devices for substitute reality (SR) or mixed reality (MR). Examples of electronic devices according to one aspect of the present invention, and electronic devices to which a display device according to one aspect of the present invention is applied, include wearable devices such as head-mounted displays, glasses-type terminals, and goggle-type terminals. In this specification, "electronic device" may be read as "head-mounted display," "glasses-type terminal," "goggle-type terminal," etc.

[0164] Figures 12(A) to 12(C) show perspective views of the electronic device 750. Figures 12(B) and 12(C) are perspective views showing the internal structure of the housing 752.

[0165] The electronic device 750 includes a pair of display devices 751, a housing 752, a pair of mounting parts 754, a buffer member 755, a pair of lenses 756, etc. The pair of display devices 751 are each provided inside the housing 752 in a position visible through the lenses 756. Although not shown, the electronic device 750 shown in Figures 12(A) to 12(C) includes an optical system 151, a motion detection unit 153, an audio system 154, a camera 155, a control unit 156, a communication unit 157, and a battery 158, as described in Embodiment 1, within the housing 752. The optical system 151 is preferably located between the lenses 756 and the display devices 751. Each of the pair of display devices 751 is an infrared light source integrated display device and can correspond to the display device 100 shown in Figure 6, etc.

[0166] The electronic device 750 is an electronic device for VR. A user wearing the electronic device 750 can view images displayed on the display device 751 through the lenses 756. Furthermore, by displaying different images on a pair of display devices 751, a three-dimensional display using parallax can also be performed.

[0167] Furthermore, the rear side of the housing 752 is provided with an input terminal 757 and an output terminal 758. The input terminal 757 can be connected to a cable that supplies video signals from a video output device or power to charge a battery located inside the housing 752. The output terminal 758 functions, for example, as an audio output terminal, allowing the connection of earphones, headphones, etc.

[0168] Furthermore, it is preferable that the housing 752 has a mechanism (referred to as a position adjustment mechanism) that allows the left and right positions of the lens 756 and the display device 751 to be in the optimal position according to the user's eye position. It is also preferable that the housing 752 has a mechanism that adjusts the focus by changing the distance between the lens 756 and the display device 751.

[0169] A display device according to one embodiment of the present invention can be applied to the display device 751. Therefore, an electronic device 750 capable of displaying extremely high resolution can be made. This allows the user to experience a high level of immersion.

[0170] The cushioning member 755 is the part that comes into contact with the user's face (forehead, cheeks, etc.). By ensuring that the cushioning member 755 is in close contact with the user's face, light leakage can be prevented, thereby enhancing immersion. It is preferable to use a soft material for the cushioning member 755 so that it comes into close contact with the user's face when the user wears the electronic device 750. For example, materials such as rubber, silicone rubber, urethane, and sponge can be used. Furthermore, if the surface of a sponge or similar material is covered with cloth, leather (genuine leather or synthetic leather), etc., gaps are less likely to form between the user's face and the cushioning member 755, effectively preventing light leakage. In addition, using such materials is preferable because it feels good against the skin and does not make the user feel cold when worn in cold seasons. It is preferable that the components that come into contact with the user's skin, such as the cushioning member 755 or the mounting part 754, be removable, as this makes cleaning or replacement easier.

[0171] The electronic device of this embodiment may have an earphone 754A as shown in Figure 12(B). The earphone 754A has a communication unit (not shown) and has wireless communication functionality. The earphone 754A can output audio data through its wireless communication functionality. The earphone 754A may also have a vibration mechanism to function as a bone conduction earphone.

[0172] Furthermore, the earphone 754A can be configured to be directly connected to the mounting part 754 or connected via a wire, as shown in Figure 12(C) with the earphone 754B. The earphone 754B and the mounting part 754 may also have magnets. This allows the earphone 754B to be magnetically fixed to the mounting part 754, making storage easier and therefore preferable.

[0173] Figures 13(A) to 13(C) show the external appearance of the electronic device 8300. Figure 13(A) is a front perspective view of the electronic device 8300, Figure 13(B) is a rear perspective view of the electronic device 8300, and Figure 13(C) shows the inside of the housing 8301 of the electronic device 8300. The electronic device 8300 includes a housing 8301, a display device 8302, a band-shaped fastener 8304, and a pair of lenses 8305.

[0174] The user can view the image on the display device 8302 through the lens 8305. It is preferable to position the display device 8302 in a curved shape, as this allows the user to experience a greater sense of realism. Furthermore, by viewing different images displayed in different areas of the display device 8302 through the lens 8305, three-dimensional display using parallax can be performed. Note that the configuration is not limited to a single display device 8302; two display devices 8302 may be provided, with one display unit for each of the user's eyes.

[0175] A display device according to one aspect of the present invention can be applied to the display device 8302. The display device according to one aspect of the present invention can also achieve extremely high resolution. For example, even when the display is magnified and viewed using the lens 8305 as shown in Figure 13(C), the pixels are difficult for the user to see. In other words, the display device 8302 can be used to allow the user to see images with a high degree of realism. The optical system 151 is preferably located between the lens 8305 and the display device 8302. The display device 8302 is a display device with an integrated infrared light source and can correspond to the display device 100 shown in Figure 6, etc.

[0176] Figure 13(D) is a front perspective view showing the external appearance of the electronic device 8400. The electronic device 8400 comprises a pair of housings 8401, a mounting portion 8402, and a cushioning member 8403. A display device 8404 and a lens 8405 are provided inside each of the pair of housings 8401. By displaying different images on the pair of display devices 8404, a three-dimensional display using parallax can be achieved.

[0177] The user can view the image on the display device 8404 through the lens 8405. The lens 8405 has a focus adjustment mechanism and its position can be adjusted according to the user. The display device 8404 is preferably a square or a horizontally elongated rectangle. This can enhance the sense of presence. The optical system 151 is preferably located between the lens 8405 and the display device 8404. Each pair of display devices 8404 is an infrared light source integrated display device and can correspond to the display device 100 shown in Figure 6, etc.

[0178] The mounting portion 8402 is preferably made of plastic and elastic material so that it can be adjusted according to the size of the user's face and does not slip off. Furthermore, it is preferable that a part of the mounting portion 8402 has a vibration mechanism in order to function as a bone conduction earphone. This eliminates the need for separate audio equipment such as earphones or speakers, allowing users to enjoy video and audio simply by wearing the device. The housing 8401 may also have a function to output audio data via wireless communication.

[0179] Regarding the mounting portion 8402 and the cushioning member 8403, refer to the description of the cushioning member 755.

[0180] Figure 13(E) is a front perspective view showing the external appearance of the electronic device 8200.

[0181] The electronic device 8200 includes a mounting section 8201, a lens 8202, a main body 8203, a display device 8204, a cable 8205, and the like. The mounting section 8201 also has a built-in battery 8206.

[0182] Cable 8205 supplies power from battery 8206 to main unit 8203. Main unit 8203 is equipped with a wireless receiver and can display received video information on display device 8204. Main unit 8203 is also equipped with a camera and can use information about the user's eye or eyelid movements as an input.

[0183] Furthermore, the attachment portion 8201 may have multiple electrodes at the point where it touches the user, and may have a function to monitor the user's pulse by the current flowing through these electrodes. The attachment portion 8201 may also have various sensors such as a temperature sensor, a pressure sensor, and an acceleration sensor, and may have a function to display the user's biometric information on the display device 8204, or a function to change the image displayed on the display device 8204 in accordance with the user's head movements. The optical system 151 is preferably located between the lens 8202 and the display device 8204. The display device 8204 is an infrared light source integrated type display device and may have a configuration such as the display device 100 shown in Figure 6, etc.

[0184] Figure 14(A) shows the external appearance of the electronic device 700, and is a front perspective view. The electronic device 700 includes a pair of display devices 701, a pair of housings 702, a pair of optical members 703, a pair of mounting parts 704, etc.

[0185] The electronic device 700 can project an image displayed by the display device 701 onto the display area 706 of the optical element 703. Furthermore, because the optical element 703 is translucent, the user can view the image displayed in the display area 706 superimposed on the transmitted image seen through the optical element 703. Therefore, the electronic device 700 is an electronic device capable of AR display.

[0186] Furthermore, one of the pair of housings 702 is equipped with a camera 705 capable of capturing images of the area in front. Although not shown, the pair of housings 702 may also be equipped with a wireless receiver or a connector to which a cable can be connected. In addition, by equipping the pair of housings 702 with an accelerometer such as a gyro sensor, the orientation of the user's head can be detected and an image corresponding to that orientation can be displayed in the display area 706. It is also preferable that the housings 702 are equipped with a battery, which can be charged wirelessly or via a wired connection.

[0187] Next, using Figure 14(B), a method for projecting an image onto the display area 706 of the electronic device 700 will be explained. Inside the housing 702, a display device 701, a lens 711, and a mirror 712 are provided. In addition, the portion of the optical element 703 corresponding to the display area 706 has a reflective surface 713 that functions as a half-mirror.

[0188] Light 715 emitted from the display device 701 passes through the lens 711 and is reflected towards the optical element 703 by the mirror 712. Inside the optical element 703, the light 715 undergoes total internal reflection repeatedly at the end face of the optical element 703 and reaches the reflective surface 713, thereby projecting an image onto the reflective surface 713. As a result, the user can see both the light 715 reflected by the reflective surface 713 and the transmitted light 716 that has passed through the optical element 703 (including the reflective surface 713).

[0189] A display device without an infrared (IR) light-emitting element can be used for the display device 701. In that case, the electronic device 700 has an infrared light source. The infrared light source may be placed inside or outside the housing 702.

[0190] Figure 14(B) shows an example where the mirror 712 and the reflective surface 713 each have curved surfaces. This increases the degree of freedom in optical design and allows for a thinner optical component 703 compared to when they are flat. However, the mirror 712 and the reflective surface 713 may also be flat.

[0191] As the mirror 712, a material having a mirror surface can be used, and it is preferable that it has a high reflectivity. Furthermore, as the reflective surface 713, a half-mirror that utilizes the reflection of a metal film may be used, but using a prism or the like that that utilizes total internal reflection can increase the transmittance of the transmitted light 716.

[0192] Here, it is preferable that the housing 702 has a mechanism for adjusting the distance between the lens 711 and the display device 701, or the angle between them. This makes it possible to adjust the focus, enlarge or reduce the image, etc. For example, one or both of the lens 711 or the display device 701 may be configured to move in the optical axis direction.

[0193] Furthermore, it is preferable that the housing 702 has a mechanism that allows the angle of the mirror 712 to be adjusted. By changing the angle of the mirror 712, the position of the display area 706 on which the image is displayed can be changed. This makes it possible to position the display area 706 in an optimal position according to the user's eye position.

[0194] A display device according to one embodiment of the present invention can be applied to the display device 701. Therefore, an electronic device 700 capable of displaying with extremely high resolution can be made.

[0195] Furthermore, a display device according to one aspect of the present invention may be applied to the display section of electronic devices having a display function, in addition to the electronic devices described above. Examples of such electronic devices include electronic devices with relatively large screens such as television sets, notebook personal computers, monitors, digital signage, pachinko machines, and game machines, as well as portable information terminals such as digital cameras, digital video cameras, digital photo frames, mobile phones, portable game consoles, smartphones and tablets, and sound playback devices.

[0196] In particular, electronic devices such as portable game consoles and smartphones can sometimes be used as VR devices by attaching the housing to which the electronic device is mounted to the head using a band-like fixing part or attachment part. Therefore, a display device according to one aspect of the present invention may be applied to the display part of such electronic device.

[0197] Furthermore, since the display device according to one aspect of the present invention can increase resolution, it may be used in electronic devices having a relatively small display unit. Examples of such electronic devices include wristwatch-type and bracelet-type information terminals (wearable devices).

[0198] A display device according to one aspect of the present invention preferably has an extremely high resolution such as HD (1280 x 720 pixels), FHD (1920 x 1080 pixels), WQHD (2560 x 1440 pixels), WQXGA (2560 x 1600 pixels), 4K2K (3840 x 2160 pixels), or 8K4K (7680 x 4320 pixels). In particular, a resolution of 4K2K, 8K4K, or higher is preferred. Furthermore, the pixel density (resolution) of the display device according to one aspect of the present invention is preferably 300 ppi or more, more preferably 500 ppi or more, more preferably 1000 ppi or more, more preferably 2000 ppi or more, more preferably 3000 ppi or more, more preferably 5000 ppi or more, and even more preferably 7000 ppi or more. By using display devices with such high resolution or high detail, it becomes possible to further enhance the sense of presence and depth in portable or personal-use electronic devices for home use.

[0199] The electronic device of this embodiment may have an antenna. By receiving signals with the antenna, the display unit can display images and information. Furthermore, if the electronic device has an antenna and a secondary battery, the antenna may be used for contactless power transmission.

[0200] The electronic device of this embodiment may have sensors (including those with the function of detecting, detecting, or measuring force, displacement, position, velocity, acceleration, angular velocity, rotational speed, distance, light, liquid, magnetism, temperature, chemical substances, sound, time, hardness, electric field, current, voltage, power, radiation, flow rate, humidity, gradient, vibration, odor, or infrared radiation).

[0201] The electronic device of this embodiment can have a variety of functions. For example, it can have a function to display various information (still images, videos, text images, etc.) on the display unit, a touch panel function, a function to display a calendar, date or time, a function to execute various software (programs), a wireless communication function, a function to read programs or data recorded on a recording medium, and so on.

[0202] The configuration examples illustrated in this embodiment, and the corresponding drawings, etc., can be appropriately combined with other configuration examples or drawings, etc., at least in part.

[0203] (Embodiment 3) An arithmetic circuit according to one aspect of the present invention will now be described. This arithmetic circuit can be used, for example, as an arithmetic circuit having the function of performing multiply-accumulate operations. Note that an arithmetic circuit having the function of performing multiply-accumulate operations can be rephrased as an arithmetic circuit of a neural network.

[0204] <Example of arithmetic circuit configuration 1> First, an example of the configuration of an arithmetic circuit according to one aspect of the present invention will be described. This arithmetic circuit is applicable to the image processing unit 121 described in the above embodiment.

[0205] The arithmetic circuit 350 shown in Figure 15 includes, as an example, an array unit ALP, a circuit ILD, a circuit WLD, a circuit XLD, a circuit AFP, and circuits TW[1] to TW[n].

[0206] Circuit ILD and circuit AFP are electrically connected to wiring OL[1] to wiring OL[n] and wiring OLB[1] to wiring OLB[n], respectively, via circuits TW[1] to TW[n].

[0207] Circuits TW[1] through TW[n] function as switching circuits. Each of circuits TW[1] through TW[n] can switch between inputting the output signals of wiring OL[1] through wiring OL[n] and wiring OLB[1] through wiring OLB[n] to circuit AFP, and inputting the output signals of circuit ILD to wiring OL[1] through wiring OL[n] and wiring OLB[1] through wiring OLB[n].

[0208] Circuit WLD is electrically connected to wiring WL[1] to wiring WL[m] and wiring WX1L[1] to wiring WX1L[m]. Circuit XLD is electrically connected to wiring WX1L[1] to wiring WX1L[m].

[0209] The arithmetic circuit 350 shown in Figure 15 has circuits MP arranged in an m × n matrix of array units ALP. In Figure 15, the circuit MP located in row i and column j (where i is an integer between 1 and m, and j is an integer between 1 and n) is denoted as circuit MP[i,j]. However, Figure 15 only shows circuits MP[1,1], MP[1,n], MP[i,j], MP[m,1], and MP[m,n], and does not show the other circuits MP.

[0210] Circuit MP[i,j] is electrically connected to wiring WL[i], wiring WX1L[i], wiring OL[j], and wiring OLB[j].

[0211] Circuit MP[i,j] has the function of holding weight coefficients (also called first data). Weight coefficients are sometimes also called weight values. Specifically, circuit MP[i,j] holds information corresponding to the weight coefficients input from wiring OL[j] and wiring OLB[j].

[0212] The circuit ILD has the function of outputting information corresponding to the first data, which is a weighting coefficient, to the wiring OL[1] to wiring OL[n] and wiring OLB[1] to wiring OLB[n].

[0213] For example, information such as electric potential, resistance, and current can be used as the information corresponding to the weighting coefficient. When using current values ​​as the information corresponding to the weighting coefficient, the input current can be generated using a current-output type digital-to-analog converter (IDAC).

[0214] Furthermore, circuit MP[i,j] has the function of outputting the product of the input value (also called the second data) input from wiring WX1L[i] and the weighting coefficient (first data). As a specific example, when circuit MP[i,j] receives the second data from wiring WX1L[i], it outputs a current corresponding to the product of the first data and the second data to wiring OL[j] and wiring OLB[j]. Although Figure 15 shows an example where wiring OL[j] and wiring OLB[j] are arranged, one aspect of the present invention is not limited to this. Only one of wiring OL[j] and wiring OLB[j] may be arranged.

[0215] Circuit XLD has the function of supplying a second data, which is an input value, to wiring WX1L[1] to wiring WX1L[m].

[0216] The information corresponding to the input value can be, for example, electric potential or current value. When using current value as the information corresponding to the input value, the input current can be generated using a current-output type digital-to-analog conversion circuit.

[0217] The currents corresponding to the product of the first data and the second data output from circuit MP[1,j] or circuit MP[m,j] are added together and output to wiring OL[j] and wiring OLB[j]. In this way, the arithmetic circuit can perform a sum-of-products operation on the weight coefficients and input values.

[0218] Furthermore, circuits XLD and WLD have the function of selecting the circuit MP to which information will be written according to the first data input from circuit ILD. For example, when writing information to circuits MP[i,1] to MP[i,n] located in row i of the array ALP, circuit XLD supplies a potential (sometimes called a signal) to wiring WX1L[i] to turn on or off the first writing switching element contained in each of circuits MP[i,1] to MP[i,n], and supplies a potential to other wiring WX1L to turn off the first writing switching element contained in each of the circuits MP other than row i. Circuit WLD also supplies a potential to wiring WL[i] to turn on or off the second writing switching element contained in each of circuits MP[i,1] to MP[i,n], and supplies a potential to other wiring WL to turn off the second writing switching element contained in each of the circuits MP other than row i.

[0219] Circuit AFP has circuits ACTF[1] to ACTF[n]. Circuit ACTF[j] is electrically connected to wiring OL[j] and wiring OLB[j] via circuit TW[j] which has a switching function. Circuit ACTF[j] generates a signal corresponding to information (e.g., potential, current value, etc.) corresponding to the result of the sum-of-accumulate operation input from wiring OL[j] and wiring OLB[j], and z j (k) It can be output as follows. The AFP circuit compares information (e.g., potential, current value, etc.) corresponding to the results of the sum-of-accumulate operation input from wiring OL[1] to wiring OL[n] and wiring OLB[1] to wiring OLB[n], generates a signal according to the comparison result, and z1 (k) ~z n (k) It can be output as follows.

[0220] <Circuit MP> Next, circuit MP will be described. An example of a circuit configuration applicable to circuit MP[i,j] is shown in Figure 16. Circuit MP includes circuit MC and circuit MCr. Circuit MC also includes transistors M1 to M3 and capacitor C1. For example, the holding part HC is formed by transistor M2 and capacitor C1.

[0221] In circuit MP shown in Figure 16, circuit MCr has almost the same circuit configuration as circuit MC. Therefore, the circuit elements of circuit MCr are denoted with "r" to distinguish them from the circuit elements of circuit MC.

[0222] As an example, the transistors M1 to M3 shown in Figure 16 are n-channel type transistors with a multi-gate structure having gates above and below the channel, and each of transistors M1 to M3 has a first gate and a second gate.

[0223] Furthermore, in the transistors applied to the arithmetic circuit 350 described in this embodiment, the multi-gate structure described above may be applied, or a structure other than a multi-gate structure (for example, a single-gate structure) may be applied. Transistors M1 to M3 shown in Figure 16 have back gates, and although the connection configuration of the back gates is not shown, the electrical connection destination of the back gates can be determined at the design stage. For example, in a transistor having a back gate, the gate and the back gate may be electrically connected in order to increase the on-current of the transistor. That is, for example, the gate and the back gate of transistor M2 may be electrically connected. Also, for example, in a transistor having a back gate, wiring electrically connected to an external circuit may be provided to vary the threshold voltage of the transistor or to reduce the off-current of the transistor, and a potential may be applied to the back gate of the transistor by the external circuit. Note that this also applies to transistors described in other parts of the specification or to transistors shown in other drawings, not just Figure 16.

[0224] Furthermore, in a transistor applied to a semiconductor device according to one aspect of the present invention, the transistor may have a multi-gate structure or a single-gate structure. Also, some transistors may have a back gate configuration, while others may not have a back gate configuration. This applies not only to the circuit diagram shown in Figure 16, but also to transistors described elsewhere in the specification or those illustrated in other drawings.

[0225] Furthermore, in this specification, transistors of various structures can be used. Therefore, there are no limitations on the type of transistor used. Examples of transistors include transistors having single-crystal silicon, or transistors having non-single-crystal semiconductor films such as amorphous silicon, polycrystalline silicon, or microcrystalline (also called microcrystal, nanocrystal, or semi-amorphous) silicon. Alternatively, thin-film transistors (TFTs) made by thinning these semiconductors can be used. Using TFTs offers various advantages. For example, they can be manufactured at lower temperatures than single-crystal silicon, which can reduce manufacturing costs or allow for larger manufacturing equipment.

[0226] As an example of a transistor, one can use a transistor made of a compound semiconductor (e.g., SiGe, GaAs, etc.) or a metal oxide with semiconductor properties (also called an oxide semiconductor). A transistor using an oxide semiconductor in the channel formation region is sometimes called an OS transistor. Alternatively, thin-film transistors made by thinning these compound semiconductors or oxide semiconductors can be used. These compound semiconductors or oxide semiconductors can be used not only in the channel portion of a transistor but also in other applications. For example, these compound semiconductors or oxide semiconductors can be used as wiring, resistive elements, pixel electrodes, or translucent electrodes. Since these can be deposited or formed simultaneously with the transistor, costs can be reduced.

[0227] Examples of the above-mentioned oxide semiconductor include oxides containing at least one of indium, element M, and zinc. Examples of element M include one or more selected from aluminum, gallium, yttrium, copper, vanadium, beryllium, boron, titanium, iron, nickel, germanium, zirconium, molybdenum, lanthanum, cerium, neodymium, hafnium, tantalum, tungsten, or magnesium.

[0228] As an example of a transistor, transistors formed using inkjet or printing methods can be used. These can be manufactured at room temperature, at low vacuum levels, or on large substrates. Therefore, since manufacturing is possible without using a mask (reticle), the layout of the transistor can be easily changed. Alternatively, since manufacturing is possible without using resist, material costs are reduced and the number of processes can be decreased. Furthermore, since it is possible to apply the film only to the necessary parts, material waste is reduced and the manufacturing cost is lower compared to methods that involve depositing a film over the entire surface and then etching.

[0229] As an example of a transistor, one that contains an organic semiconductor or carbon nanotubes can be used. This allows transistors to be formed on a flexible substrate. Devices using transistors containing organic semiconductors or carbon nanotubes can be made more resistant to impact.

[0230] In circuit MP of Figure 16, the first terminal of transistor M1 is electrically connected to wiring VE. The second terminal of transistor M1 is electrically connected to the first terminal of transistor M3. The gate of transistor M1 is electrically connected to the first terminal of capacitor C1 and the first terminal of transistor M2. The second terminal of capacitor C1 is electrically connected to wiring VE. The second terminal of transistor M2 is electrically connected to wiring OL. The gate of transistor M2 is electrically connected to wiring WL. The second terminal of transistor M3 is electrically connected to wiring OL, and the gate of transistor M3 is electrically connected to wiring WX1L.

[0231] In circuit MCr, the connection configuration differs from that of circuit MC. The second terminal of transistor M3r is electrically connected to wiring OLB, not wiring OL. The first terminal of transistor M1r and the terminal of capacitor C1r are electrically connected to wiring VEr.

[0232] In the holding section HC shown in Figure 16, node n1 is defined as the electrical connection point between the gate of transistor M1, the first terminal of capacitor C1, and the first terminal of transistor M2.

[0233] The holding unit HC has the function of holding a potential corresponding to the weighting coefficient (first data). Holding this potential in the holding unit HC included in the circuit MC in Figure 16 can be done by turning on transistors M2 and M3, inputting a current of a predetermined value from wiring OL, writing a potential corresponding to that current value to capacitor C1, and then turning off transistor M2. This makes it possible to hold the potential of node n1 as a potential corresponding to the weighting coefficient (first data). At this time, a current is input from wiring OL, and a potential of a magnitude corresponding to the magnitude of that current can be held in capacitor C1. Therefore, the influence of variations in the current characteristics (threshold voltage, etc.) of transistor M1 can be reduced when inputting the first data.

[0234] The current input to the wiring OL can be input and generated using a current-output type digital-to-analog conversion circuit.

[0235] Furthermore, since transistor M2 maintains the potential of node n1 for a long time, it is preferable to use a transistor with a small off-current. For example, an OS transistor can be used as a transistor with a small off-current. Because OS transistors use an oxide semiconductor with a large bandgap in the channel formation region, the off-current of the OS transistor can be reduced.

[0236] Alternatively, a transistor with a back gate may be used as transistor M2, and a low-level potential may be applied to the back gate to shift the threshold voltage to the positive side, thereby reducing the off-current.

[0237] In this way, a highly accurate arithmetic circuit is provided. Or, a highly reliable arithmetic circuit is provided.

[0238] <Example of arithmetic circuit configuration 2> Next, we will describe another example: an arithmetic circuit MAC1 that performs a sum-of-accumulate operation. The arithmetic circuit MAC1 is applicable to the image processing unit 121 described in the above embodiment.

[0239] Figure 17 shows an example of a configuration of an arithmetic circuit that performs a sum-of-products operation on a first data point (positive or "0") and a second data point (positive or "0"). The arithmetic circuit MAC1 shown in Figure 17 is a circuit that performs a sum-of-products operation on the first data point corresponding to the potential held in each cell and the input second data point, and then performs an activation function calculation using the result of the sum-of-products operation. The first data point and the second data point can be analog data or multi-level data (discrete data), for example.

[0240] This arithmetic circuit also functions as a memory for holding the first data, and can therefore be called a memory. In particular, when analog data is used as the first data, it can be called an analog memory.

[0241] The arithmetic circuit MAC1 includes circuit WCS, circuit XCS, circuit WSD, circuit SWS1, circuit SWS2, cell array CA, and conversion circuits ITRZ[1] to ITRZ[n].

[0242] The cell array CA comprises cells IM[1,1] to IM[m,n] (where m is an integer greater than or equal to 1, and n is an integer greater than or equal to 1) and cells IMref[1] to IMref[m]. Each of cells IM[1,1] to IM[m,n] has the function of holding a potential corresponding to the amount of current corresponding to the first data, and cells IMref[1] to IMref[m] have the function of supplying a potential corresponding to the second data to the wiring XCL[1] to XCL[m], which is necessary for performing a sum-of-products operation with the held potential.

[0243] Note that while the cell array CA in Figure 17 has n+1 cells arranged in the row direction and m cells in the column direction in a matrix, the cell array CA may also be configured with 2 or more cells arranged in the row direction and 1 or more cells in the column direction in a matrix.

[0244] Each of cells IM[1,1] through IM[m,n] has, for example, a transistor F1, a transistor F2, and a capacitor C5, and each of cells IMref[1] through IMref[m] has, for example, a transistor F1m, a transistor F2m, and a capacitor C5m.

[0245] In particular, it is preferable that the size of transistor F1 contained in each of cells IM[1,1] to IM[m,n] (e.g., channel length, channel width, transistor configuration, etc.) is equal to that of each cell IM[1,1] to IM[m,n], and that the size of transistor F2 contained in each of cells IM[1,1] to IM[m,n] is equal to that of each cell. Furthermore, it is preferable that the size of transistor F1m contained in each of cells IMref[1] to IMref[m] is equal to that of each cell. Furthermore, it is preferable that the sizes of transistor F1 and transistor F1m are equal to that of each cell. Furthermore, it is preferable that the sizes of transistor F2 and transistor F2m are equal to that of each cell.

[0246] Unless otherwise specified, transistors F1 and F1m are assumed to operate in the linear region when they are ON. That is, the gate voltage, source voltage, and drain voltage of each of the transistors described above are assumed to be appropriately biased to voltages within the range in which they operate in the linear region. However, one aspect of the present invention is not limited thereto. For example, transistors F1 and F1m may operate in the saturation region when they are ON, or they may operate in a mixture of the linear region and the saturation region.

[0247] Furthermore, unless otherwise specified, transistors F2 and F2m include cases where they operate in the subthreshold region (i.e., in transistor F2 or F2m, the gate-source voltage is lower than the threshold voltage, more preferably the drain current increases exponentially with respect to the gate-source voltage). That is, the gate voltage, source voltage, and drain voltage of each of the transistors described above include cases where they are appropriately biased to voltages within the operating range of the subthreshold region. For this reason, transistors F2 and F2m also include cases where they operate such that an off-current flows between the source and drain.

[0248] Furthermore, transistor F1 and / or transistor F1m are preferably OS transistors, for example. By using OS transistors as transistors F1 and / or transistor F1m, the leakage current of transistors F1 and / or transistor F1m can be suppressed, thereby reducing the power consumption of the arithmetic circuit. Specifically, when transistors F1 and / or transistor F1m are in a non-conductive state, the leakage current from the holding node to the write word line can be made very small, which reduces the refresh operation of the holding node's potential, and thus reduces the power consumption of the arithmetic circuit. In addition, by making the leakage current from the holding node to the write word line very small, the cell can hold the holding node's potential for a long time, which increases the calculation accuracy of the arithmetic circuit.

[0249] Furthermore, by using OS transistors for transistor F2 and / or transistor F2m, they can be operated within a wide current range in the subthreshold region, thereby reducing current consumption. Also, by using OS transistors for transistor F2 and / or transistor F2m, they can be fabricated simultaneously with transistors F1 and F1m, potentially shortening the fabrication process for the arithmetic circuit. In addition, transistors F2 and / or transistor F2m can be transistors containing silicon in the channel formation region (hereinafter referred to as Si transistors), other than OS transistors. Examples of silicon that can be used include amorphous silicon (sometimes called hydrogenated amorphous silicon), microcrystalline silicon, polycrystalline silicon, and monocrystalline silicon.

[0250] Incidentally, when arithmetic circuits and the like are highly integrated onto a chip, heat may be generated on the chip due to the operation of the circuit. This heat can cause the temperature of transistors to rise, which can change the characteristics of those transistors, leading to changes in field-effect mobility or a decrease in operating frequency. OS transistors have higher heat resistance than Si transistors, so their field-effect mobility is less likely to change due to temperature changes, and a decrease in operating frequency is also less likely to occur. Furthermore, OS transistors tend to maintain the characteristic that the drain current increases exponentially with respect to the gate-source voltage, even at high temperatures. Therefore, by using OS transistors, it is easier to perform the multiply-accumulate operations described later, even in high-temperature environments. For this reason, when constructing arithmetic circuits that are resistant to heat generated by operation, it is preferable to use OS transistors.

[0251] In each of cells IM[1,1] through IM[m,n], the first terminal of transistor F1 is electrically connected to the gate of transistor F2. The first terminal of transistor F2 is electrically connected to wiring VE. The first terminal of capacitor C5 is electrically connected to the gate of transistor F2.

[0252] Furthermore, in each of cells IMref[1] through IMref[m], the first terminal of transistor F1m is electrically connected to the gate of transistor F2m. The first terminal of transistor F2m is electrically connected to wiring VE. The first terminal of capacitor C5m is electrically connected to the gate of transistor F2m.

[0253] Furthermore, the arithmetic circuit described in this embodiment is not particularly limited in terms of the polarity of the transistors included in the arithmetic circuit. For example, although transistors F1 and F2 shown in Figure 17 are n-channel type transistors, some or all of the transistors may be replaced with p-channel type transistors.

[0254] The above examples of changes in transistor structure and polarity are not limited to transistors F1 and F2. For example, the same applies to transistors F1m, F2m, transistors F3[1] to F3[n], F4[1] to F4[n] described later, and transistors described elsewhere in the specification or illustrated in other drawings.

[0255] Wiring VE is a wiring for conducting current between the first and second terminals of transistor F2 in cells IM[1,1], IM[m,1], IM[1,n], and IM[m,n], and also functions as wiring for conducting current between the first and second terminals of transistor F2 in cells IMref[1] and IMref[m]. For example, wiring VE functions as wiring that supplies a constant voltage. This constant voltage can be, for example, a low-level potential or ground potential.

[0256] In cell IM[1,1], the second terminal of transistor F1 is electrically connected to wiring WCL[1], and the gate of transistor F1 is electrically connected to wiring WSL[1]. The second terminal of transistor F2 is electrically connected to wiring WCL[1], and the second terminal of capacitor C5 is electrically connected to wiring XCL[1]. In Figure 17, the connection point between the first terminal of transistor F1, the gate of transistor F2, and the first terminal of capacitor C5 in cell IM[1,1] is defined as node NN[1,1].

[0257] In cell IM[m,1], the second terminal of transistor F1 is electrically connected to wiring WCL[1], and the gate of transistor F1 is electrically connected to wiring WSL[m]. The second terminal of transistor F2 is electrically connected to wiring WCL[1], and the second terminal of capacitor C5 is electrically connected to wiring XCL[m]. In Figure 17, the connection point between the first terminal of transistor F1, the gate of transistor F2, and the first terminal of capacitor C5 in cell IM[m,1] is defined as node NN[m,1].

[0258] In cell IM[1,n], the second terminal of transistor F1 is electrically connected to wiring WCL[n], and the gate of transistor F1 is electrically connected to wiring WSL[1]. The second terminal of transistor F2 is electrically connected to wiring WCL[n], and the second terminal of capacitor C5 is electrically connected to wiring XCL[1]. In Figure 17, the connection point between the first terminal of transistor F1, the gate of transistor F2, and the first terminal of capacitor C5 in cell IM[1,n] is defined as node NN[1,n].

[0259] In cell IM[m,n], the second terminal of transistor F1 is electrically connected to wiring WCL[n], and the gate of transistor F1 is electrically connected to wiring WSL[m]. The second terminal of transistor F2 is electrically connected to wiring WCL[n], and the second terminal of capacitor C5 is electrically connected to wiring XCL[m]. In FIG. 17, in cell IM[m,n], the connection point of the first terminal of transistor F1, the gate of transistor F2, and the first terminal of capacitor C5 is defined as node NN[m,n].

[0260] In cell IMref[1], the second terminal of transistor F1m is electrically connected to wiring XCL[1], and the gate of transistor F1m is electrically connected to wiring WSL[1]. The second terminal of transistor F2m is electrically connected to wiring XCL[1], and the second terminal of capacitor C5 is electrically connected to wiring XCL[1]. In FIG. 17, in cell IMref[1], the connection point of the first terminal of transistor F1m, the gate of transistor F2m, and the first terminal of capacitor C5 is defined as node NNref[1].

[0261] In cell IMref[m], the second terminal of transistor F1m is electrically connected to wiring XCL[m], and the gate of transistor F1m is electrically connected to wiring WSL[m]. The second terminal of transistor F2m is electrically connected to wiring XCL[m], and the second terminal of capacitor C5 is electrically connected to wiring XCL[m]. In FIG. 17, in cell IMref[m], the connection point of the first terminal of transistor F1m, the gate of transistor F2m, and the first terminal of capacitor C5 is defined as node NNref[m].

[0262] The above-described nodes NN[1,1], NN[m,1], NN[1,n], NN[m,n], NNref[1], and NNref[m] function as the holding nodes of their respective cells.

[0263] In cells IM[1,1] through IM[m,n], for example, when transistor F1 is ON, transistor F2 is configured as a diode connection. With the constant voltage supplied by wiring VE as the ground potential (GND), when transistor F1 is ON and a current of amount I flows from wiring WCL to the second terminal of transistor F2, the potential of the gate (node ​​NN) of transistor F2 is determined according to the current amount I. Ideally, the potential of the second terminal of transistor F2 is equal to that of the gate (node ​​NN) of transistor F2 because transistor F1 is ON. By turning transistor F1 OFF, the potential of the gate (node ​​NN) of transistor F2 is maintained. As a result, transistor F2 can flow a current of amount I between its source and drain, corresponding to the ground potential of its first terminal and the potential of its gate (node ​​NN). In this specification, such operation is referred to as "transistor F2 is programmed to have the amount of current flowing between its source and drain set to I."

[0264] Circuit SWS1, for example, includes transistors F3[1] to F3[n]. The first terminal of transistor F3[1] is electrically connected to wiring WCL[1], the second terminal of transistor F3[1] is electrically connected to circuit WCS, and the gate of transistor F3[1] is electrically connected to wiring SWL1. The first terminal of transistor F3[n] is electrically connected to wiring WCL[n], the second terminal of transistor F3[n] is electrically connected to circuit WCS, and the gate of transistor F3[n] is electrically connected to wiring SWL1.

[0265] The transistors F3[1] to F3[n] are preferably OS transistors applicable to, for example, transistor F1 and / or transistor F2.

[0266] Circuit SWS1 functions as a circuit that makes the connection between circuit WCS and each of the wires WCL[1] through WCL[n] either conductive or nonconductive.

[0267] Circuit SWS2 includes, for example, transistors F4[1] to F4[n]. The first terminal of transistor F4[1] is electrically connected to wiring WCL[1], the second terminal of transistor F4[1] is electrically connected to the input terminal of conversion circuit ITRZ[1], and the gate of transistor F4[1] is electrically connected to wiring SWL2. The first terminal of transistor F4[n] is electrically connected to wiring WCL[n], the second terminal of transistor F4[n] is electrically connected to the input terminal of conversion circuit ITRZ[n], and the gate of transistor F4[n] is electrically connected to wiring SWL2.

[0268] The transistors F4[1] to F4[n] are preferably OS transistors applicable to, for example, transistor F1 and / or transistor F2.

[0269] Circuit SWS2 functions as a circuit that makes the connection between wiring WCL[1] and conversion circuit ITRZ[1], and between wiring WCL[n] and conversion circuit ITRZ[n], either conductive or nonconductive.

[0270] The WCS circuit has the function of supplying data to be stored in each cell of the CA cell array.

[0271] Circuit XCS is electrically connected to wiring XCL[1] to wiring XCL[m]. Circuit XCS has the function of supplying current to each of cells IMref[1] to cell IMref[m] of the cell array CA according to the reference data or according to the second data.

[0272] Circuit WSD is electrically connected to wiring WSL[1] to wiring WSL[m]. When writing first data to cells IM[1,1] to cells IM[m,n], circuit WSD has the function of selecting the row of cell array CA to which the first data will be written by supplying a predetermined signal to wiring WSL[1] to wiring WSL[m].

[0273] Furthermore, circuit WSD is electrically connected, for example, to wiring SWL1 and wiring SWL2. Circuit WSD has the function of making circuit WCS and cell array CA conduct or non-conductive by supplying a predetermined signal to wiring SWL1, and the function of making conversion circuits ITRZ[1] to ITRZ[n] and cell array CA conduct or non-conductive by supplying a predetermined signal to wiring SWL2.

[0274] Each of the conversion circuits ITRZ[1] to ITRZ[n] has, for example, an input terminal and an output terminal. For instance, the output terminal of conversion circuit ITRZ[1] is electrically connected to wiring OL[1], and the output terminal of conversion circuit ITRZ[n] is electrically connected to wiring OL[n].

[0275] Each of the conversion circuits ITRZ[1] to ITRZ[n] has the function of converting the current input to the input terminal into a voltage and outputting the voltage from the output terminal, depending on the amount of current input to the input terminal. The voltage can be, for example, an analog voltage, a digital voltage, etc. Furthermore, each of the conversion circuits ITRZ[1] to ITRZ[n] may have a function-based arithmetic circuit. In this case, for example, the arithmetic circuit may perform a function calculation using the converted voltage and output the result of the calculation to wiring OL[1] to wiring OL[n].

[0276] In particular, when performing calculations on hierarchical neural networks, the functions mentioned above can include, for example, the sigmoid function, tanh function, softmax function, ReLU function, threshold function, etc.

[0277] A current-output type digital-to-analog conversion circuit can be used as the circuit WCS shown in Figure 17. Similarly, a current-output type digital-to-analog conversion circuit can be used as the circuit XCS shown in Figure 17.

[0278] The configuration examples illustrated in this embodiment, and the corresponding drawings, etc., can be appropriately combined with other configuration examples or drawings, etc., at least in part.

[0279] This embodiment can be implemented in appropriate combination with other embodiments described herein, at least in part.

[0280] (Embodiment 4) This embodiment describes a schematic top view and cross-sectional view of the light-emitting element and its surroundings in the display device, an example of the configuration of the light-emitting element, an example of the configuration of the light-emitting element and light-receiving element, and an example of the configuration of the display device.

[0281] <Schematic top view and cross-sectional view of the display device> Figure 18(A) is a schematic top view (also referred to as a schematic plan view) showing an example of the configuration of a display device 100 according to one aspect of the present invention, which is an infrared light source integrated display device in which light-emitting elements and light-receiving elements are arranged within pixels 80. The display device 100 has multiple red light-emitting elements 61R, green light-emitting elements 61G, blue light-emitting elements 61B, infrared light-emitting elements 61IR, and light-receiving elements 62. In the case of electronic equipment equipped with an infrared light source, the light-emitting elements 61IR in the display device 100 in this embodiment can be omitted.

[0282] In Figure 18(A), the labels R, G, B, and IR are added to the light-emitting region of each light-emitting element 61 for easier identification. Additionally, the label PD is added to the light-receiving region of each photodetector 62.

[0283] In the following description, when explaining matters common to the light-emitting element 61R, the light-emitting element 61G, the light-emitting element 61B, and the light-emitting element 61IR, the symbols appended to the reference numerals may be omitted, and they may be described simply as the light-emitting element 61. Or, when referring to the light-emitting element 61, it may refer to any one or more of the light-emitting element 61R, the light-emitting element 61G, the light-emitting element 61B, and the light-emitting element 61IR.

[0284] The display device 100 corresponds to the display device 100 described in the above embodiment. Also, the light-emitting element 61R, the light-emitting element 61G, the light-emitting element 61B, and the light-emitting element 61IR respectively correspond to the light-emitting element 110R, the light-emitting element 110G, the light-emitting element 110B, and the light-emitting element 110IR in Embodiment 1. Also, the light-receiving element 62 corresponds to the light-receiving element 159 described in the above embodiment.

[0285] The light-emitting element 61R, the light-emitting element 61G, the light-emitting element 61B, the light-emitting element 61IR, and the light-receiving element 62 are each arranged in a matrix. Specifically, as shown in Fig. 18(A), in a top view, the light-emitting element 61R, the light-emitting element 61G, the light-emitting element 61B, and the light-emitting element 61IR are arranged along the X direction, and an example is shown where the light-receiving element 62 is arranged on the lower side (the side along the Y direction). Also, Fig. 18(A) shows, as an example, a configuration in which the light-emitting elements that emit the same color light are arranged along the Y direction. In the display device 100 shown in Fig. 18(A), for example, a sub-pixel having the light-emitting element 61R arranged along the X direction, a sub-pixel having the light-emitting element 61G, a sub-pixel having the light-emitting element 61B, and a sub-pixel having the light-emitting element 61IR, and a sub-pixel having the light-receiving element 62 provided on the lower side (the side along the Y direction) of these sub-pixels can constitute the pixel 80. The light-receiving element 62 has a function of detecting infrared light.

[0286] Fig. 18(A) shows a so-called stripe arrangement in which the light-emitting elements of the same color are arranged in one direction. Note that the arrangement method of the light-emitting elements is not limited to this, and arrangement methods such as a delta arrangement, a zigzag arrangement, etc. may be applied, or a pentile arrangement may also be used.

[0287] It is preferable to use EL elements such as OLED (Organic Light Emitting Diode) or QLED (Quantum-dot Light Emitting Diode) as the light-emitting elements 61R, 61G, 61B, and 61IR. Examples of light-emitting materials for EL elements include fluorescent materials, phosphorescent materials, inorganic compounds (such as quantum dot materials), and thermally activated delayed fluorescence (TADF) materials.

[0288] For example, a pn-type or pin-type photodiode can be used as the light-receiving element 62. The light-receiving element 62 functions as a photoelectric conversion element that detects light incident on it and generates an electric charge. The amount of charge generated is determined based on the amount of incident light.

[0289] In particular, it is preferable to use an organic photodiode having a layer containing an organic compound as the light-receiving element 62. Organic photodiodes can be easily made thinner, lighter, and larger in area, and because they offer a high degree of freedom in shape and design, they can be applied to various display devices.

[0290] In one aspect of the present invention, an organic EL element is used as the light-emitting element 61, and an organic photodiode is used as the light-receiving element 62. The organic EL element and the organic photodiode can be formed on the same substrate. Therefore, an organic photodiode can be incorporated (also referred to as integrally formed) into a display device using an organic EL element.

[0291] Incidentally, in the case of an SBS structure in which light-emitting layers are created separately between light-emitting elements of different colors, it is known that the light-emitting layer is formed by a deposition method using a shadow mask such as a metal mask or FMM (Fine Metal Mask, high-resolution metal mask). This is a light-emitting device having an MM structure. However, in an MM structure, deviations from the design occur in the shape and position of the island-like light-emitting layer due to various influences such as the precision of the metal mask, the misalignment between the metal mask and the substrate, the deflection of the metal mask, and the spreading of the contour of the deposited film due to vapor scattering, making it difficult to achieve high resolution and high aperture ratio.

[0292] For example, by employing a light-emitting device with an MML structure in which the light-emitting layer is formed without using a metal mask or a shadow mask such as FMM, it is possible to realize a display device with high resolution and a large aperture ratio, which was difficult to achieve with an MM structure. Furthermore, by employing an SBS structure in which the light-emitting layer is made in different ways, it is possible to realize a display device with extremely vivid colors, high contrast, and high display quality.

[0293] For simplicity, this section describes the process of applying an MML structure to a two-color light-emitting element and processing the light-emitting layer using photolithography. First, a first light-emitting film and a first sacrificial film are laminated to cover the two pixel electrodes. The term "light-emitting film" refers to the state before processing by photolithography. Next, a resist mask is formed on the first sacrificial film at a position overlapping one of the pixel electrodes (the first pixel electrode). Subsequently, the resist mask, a portion of the first sacrificial film, and a portion of the first light-emitting film are etched. The etching is stopped when the other pixel electrode (the second pixel electrode) is exposed. As a result, an island-shaped light-emitting layer (also called the first light-emitting layer) is formed on the first pixel electrode, and a portion of the sacrificial film (also called the first sacrificial layer) remains on the first light-emitting layer.

[0294] Next, the second light-emitting film and the second sacrificial film are laminated together. Then, a resist mask is formed at the position overlapping with the first pixel electrode and the position overlapping with the second pixel electrode. Subsequently, the resist mask, a portion of the second sacrificial film, and a portion of the second light-emitting film are etched in the same manner as above. As a result, the first light-emitting layer and the first sacrificial layer are provided on the first pixel electrode, and the second light-emitting layer and the second sacrificial layer are provided on the second pixel electrode. In this way, the first and second light-emitting layers can be created separately. Finally, the first and second sacrificial layers are removed, exposing the first and second light-emitting layers, and then a common electrode is formed to create two different colored light-emitting elements.

[0295] Furthermore, by repeating the above process, it is possible to create different light-emitting layers for three or more light-emitting elements, thereby realizing a display device having three or four or more light-emitting elements.

[0296] Regarding the distance between light-emitting layers of different colors, for example, it is difficult to reduce it to less than 10 μm in an MM structure, but with an MML structure, it can be narrowed to less than 6 μm, less than 4 μm, less than 3 μm, less than 2 μm, or even less than 1 μm. Furthermore, by using exposure equipment for LSIs, for example, the spacing can be narrowed to less than 500 nm, less than 200 nm, less than 100 nm, and even less than 50 nm. This significantly reduces the area of ​​the non-light-emitting region that can exist between two light-emitting elements, making it possible to approach an aperture ratio of 100%. For example, an aperture ratio of 50% or more, 60% or more, 70% or more, 80% or more, and even less than 90% can be achieved.

[0297] Furthermore, the pattern of the light-emitting layer itself can be made extremely small compared to when a metal mask is used. Also, for example, when a metal mask is used to create different light-emitting layers, variations in thickness occur between the center and edges of the light-emitting layer pattern, resulting in a smaller effective area that can be used as a light-emitting region relative to the total area of ​​the pattern. On the other hand, in the above-described photolithography method, the light-emitting layer pattern is formed by processing a film deposited to a uniform thickness, so the thickness is uniform within the pattern, and even if the pattern is fine, almost the entire area can be used as a light-emitting region. Therefore, the above-described photolithography method can achieve both high resolution and a high aperture ratio.

[0298] Thus, the fabrication method using the above-described photolithography method makes it possible to realize a display device that integrates fine light-emitting elements. Therefore, it is not necessary to apply a special pixel arrangement method such as the PenTile method to artificially increase the resolution. Thus, it is possible to realize a display device with a resolution of 500 ppi or more, 1000 ppi or more, 2000 ppi or more, 3000 ppi or more, or even 5000 ppi or more, using a so-called stripe arrangement in which R, G, and B are each arranged in one direction.

[0299] Figure 18(A) shows the common electrode 81, which has an overlapping region with the light-emitting element 61, the light-receiving element 62, and the connecting electrode 82, as indicated by a dashed line. Figure 18(A) also shows the connecting electrode 82, which is electrically connected to the common electrode 81.

[0300] The connecting electrode 82 is located outside the display area where the light-emitting element 61 and the light-receiving element 62 are arranged, and can be provided along the outer perimeter of the display area. For example, it may be provided along one side of the outer perimeter of the display area, or it may be provided across two or more sides of the outer perimeter of the display area. That is, if the top surface shape of the display area is rectangular, the top surface shape of the connecting electrode 82 can be strip-shaped, L-shaped, U-shaped (angle bracket-shaped), or square, etc.

[0301] Here, we will describe the connecting electrodes. It is preferable to use a conductive layer formed on the same plane as the pixel electrodes for the connecting electrodes. The display device 100 has a configuration in which the conductive layer is electrically connected to a common electrode. The connecting electrodes are arranged on the outside of the pixel portion. Here, in order to prevent the upper surface of the connecting electrodes from being exposed to etching when the first light-emitting film is etched, it is preferable to provide a first sacrificial layer on the connecting electrodes as well. Similarly, when etching the second light-emitting film, it is preferable to provide a second sacrificial layer on the connecting electrodes.

[0302] Figure 18(B) is a schematic top view showing an example configuration of the display device 100, and is a modified version of the display device 100 shown in Figure 18(A). The display device 100 shown in Figure 18(B) differs from the display device 100 shown in Figure 18(A) in that the light-receiving elements 62 and light-emitting elements 61IR are arranged alternately in the X direction.

[0303] In the display device 100 shown in Figure 18(B), the light-emitting elements 61R, 61G, and 61B are arranged in different rows from the light-emitting element 61IR. Therefore, the width (length in the X direction) of the light-emitting elements 61R, 61G, and 61B can be increased, thereby increasing the brightness of the light emitted by the pixels 80.

[0304] Figure 19(A) is a schematic top view showing an example configuration of the display device 100, and is a modified version of the display device 100 shown in Figure 18(B). The display device 100 shown in Figure 19(A) differs from the display device 100 shown in Figure 18(B) in that the light-emitting elements 61 are arranged in the order G, B, R in the X direction, rather than in the order R, G, B. Also, the light-receiving element 62 is provided below the light-emitting elements 61G and 61B, and the light-emitting element 61IR is provided below the light-emitting element 61R, which is another difference from the display device 100 shown in Figure 18(B).

[0305] The area occupied by the light-receiving element 62 in the display device 100 shown in Figure 19(A) is larger than the area occupied by the light-receiving element 62 in the display device 100 shown in Figure 18(B). Therefore, the light detection sensitivity of the light-receiving element 62 can be increased. Consequently, for example, if the display device 100 has an eye-tracking function, it can track the gaze with high accuracy.

[0306] Figure 19(B) is a schematic top view showing an example configuration of the display device 100, and is a modified version of the display device 100 shown in Figure 19(A). The display device 100 shown in Figure 19(B) differs from the display device 100 shown in Figure 19(A) in that the light-receiving element 62 is provided below the light-emitting element 61G, and the light-emitting element 61IR is provided below the light-emitting elements 61B and 61R.

[0307] The area occupied by the light-receiving element 62 in the display device 100 shown in Figure 19(B) is smaller than the area occupied by the light-receiving element 62 in the display device 100 shown in Figure 19(A). By reducing the area occupied by the light-receiving element 62, the light-receiving range of each individual light-receiving element 62 can be reduced. This reduces the overlap of light-receiving ranges between different light-receiving elements 62, for example, between adjacent light-receiving elements 62. Therefore, blurring of images captured using the light-receiving element 62, which prevents clear imaging, can be suppressed. For example, if the display device 100 has an eye-tracking function, reducing the area occupied by the light-receiving element 62 is preferable because it allows for clearer imaging of the eyeball, etc., and improves the accuracy of authentication.

[0308] Figure 20(A) is a cross-sectional view corresponding to the dashed line A1-A2 in Figure 18(B), and Figure 20(B) is a cross-sectional view corresponding to the dashed line B1-B2 in Figure 18(B). Furthermore, Figure 20(C) is a cross-sectional view corresponding to the dashed line C1-C2 in Figure 18(B), and Figure 20(D) is a cross-sectional view corresponding to the dashed line D1-D2 in Figure 18(B). In Figure 20(A), etc., the light-emitting element 61R, light-emitting element 61G, light-emitting element 61B, light-emitting element 61IR, and light-receiving element 62 are provided on the substrate 83.

[0309] As the substrate 83, a substrate having at least sufficient heat resistance to withstand subsequent heat treatment can be used. When an insulating substrate is used as the substrate 83, glass substrates, quartz substrates, sapphire substrates, ceramic substrates, organic resin substrates, etc., can be used. In addition, semiconductor substrates such as single-crystal semiconductor substrates made of silicon, silicon carbide, etc., polycrystalline semiconductor substrates, compound semiconductor substrates such as silicon germanium, and SOI substrates can be used. However, it is preferable to use a translucent glass substrate or the like for the substrate 83.

[0310] In particular, it is preferable to use a substrate 83 on which a semiconductor circuit including semiconductor elements such as transistors is formed on the semiconductor substrate or insulating substrate. It is preferable that the semiconductor circuit constitutes, for example, a pixel circuit, a gate line driving circuit (gate driver), a source line driving circuit (source driver), etc. In addition to the above, an arithmetic circuit, a memory circuit, etc. may also be configured.

[0311] Substrate 83 corresponds to substrate 101 in Embodiment 1.

[0312] The light-emitting element 61R has a pixel electrode 84R, a hole injection layer 85R, a hole transport layer 86R, an emissive layer 87R, an electron transport layer 88R, a common layer 89, and a common electrode 81. The light-emitting element 61G has a pixel electrode 84G, a hole injection layer 85G, a hole transport layer 86G, an emissive layer 87G, an electron transport layer 88G, a common layer 89, and a common electrode 81. The light-emitting element 61B has a pixel electrode 84B, a hole injection layer 85B, a hole transport layer 86B, an emissive layer 87B, an electron transport layer 88B, a common layer 89, and a common electrode 81.

[0313] Figure 20(B) shows an example of the cross-sectional configuration of the light-emitting element 61IR and the photodetector 62. The light-emitting element 61IR has a pixel electrode 84IR, a hole injection layer 85IR, a hole transport layer 86IR, a light-emitting layer 87IR, an electron transport layer 88IR, a common layer 89, and a common electrode 81. The photodetector 62 has a pixel electrode 84PD, a hole transport layer 86PD, a photodetector layer 90, an electron transport layer 88PD, a common layer 89, and a common electrode 81.

[0314] In the following, when explaining matters common to pixel electrodes 84R, 84G, 84B, 84IR, and 84PD, the symbols attached to the reference numerals may be omitted, and they may be referred to simply as pixel electrode 84. Similarly, when explaining matters common to hole injection layers 85R, 85G, 85B, and 85IR, the symbols attached to the reference numerals may be omitted, and they may be referred to simply as hole injection layer 85. Similarly, when explaining matters common to hole transport layers 86R, 86G, 86B, 86IR, and 86PD, the symbols attached to the reference numerals may be omitted, and they may be referred to simply as hole transport layer 86. Similarly, when explaining matters common to light-emitting layers 87R, 87G, 87B, and 87IR, the symbols attached to the reference numerals may be omitted, and they may be referred to simply as light-emitting layer 87. Similarly, when explaining matters common to electron transport layers 88R, 88G, 88B, 88IR, and 88PD, the symbols attached to the reference numerals may be omitted, and the explanation may simply refer to it as electron transport layer 88.

[0315] In the light-emitting element 61, the common layer 89 functions as an electron injection layer. On the other hand, in the photodetector 62, the common layer 89 functions as an electron transport layer. Therefore, the photodetector 62 may not need to have an electron transport layer 88PD.

[0316] The hole injection layer 85, hole transport layer 86, electron transport layer 88, and common layer 89 can also be called functional layers. In addition, in the light-emitting element 61 shown in Figure 20(A), the hole injection layer 85, hole transport layer 86, light-emitting layer 87, electron transport layer 88, and common layer 89 can be collectively called the light-emitting layer.

[0317] The pixel electrode 84, hole injection layer 85, hole transport layer 86, light-emitting layer 87, and electron transport layer 88 can be provided separately for each element. The common layer 89 and common electrode 81 are provided in common for the light-emitting element 61R, light-emitting element 61G, light-emitting element 61B, light-emitting element 61IR, and light-receiving element 62.

[0318] Furthermore, the light-emitting element 61 and the light-receiving element 62 may have a hole-blocking layer and an electron-blocking layer in addition to the layers shown in Figure 20(A), etc. Also, the light-emitting element 61 and the light-receiving element 62 may have layers containing bipolar material (a material with high electron transport and hole transport properties), etc.

[0319] An insulating layer 92 overlaps the edge of the pixel electrode 84, and a gap is provided between the insulating layer 92 and the common layer 89. This prevents the common layer 89 from coming into contact with the side surface of the light-emitting layer 87, the side surface of the light-receiving layer 90, the side surface of the hole transport layer 86, and the side surface of the hole injection layer 85. This prevents short circuits in the light-emitting element 61 and the light-receiving element 62.

[0320] The above-mentioned voids are more likely to form when, for example, the distance between the light-emitting layers 87 is shorter. For example, if the distance is 1 μm or less, preferably 500 nm or less, and more preferably 200 nm or less, 100 nm or less, 90 nm or less, 70 nm or less, 50 nm or less, 30 nm or less, 20 nm or less, 15 nm or less, or 10 nm or less, the above-mentioned voids can be suitably formed.

[0321] Figure 20(A), etc., shows a configuration in which the light-emitting element 61 is provided with, in order from the bottom layer, a pixel electrode 84, a hole injection layer 85, a hole transport layer 86, a light-emitting layer 87, an electron transport layer 88, a common layer 89 (electron injection layer), and a common electrode 81, and the light-receiving element 62 is provided with, in order from the bottom layer, a pixel electrode 84PD, a hole transport layer 86PD, a light-receiving layer 90, an electron transport layer 88PD, a common layer 89, and a common electrode 81. However, the present invention is not limited to this configuration. For example, the light-emitting element 61 may be provided with, in order from the bottom layer, a pixel electrode, an electron injection layer, an electron transport layer, a light-emitting layer, a hole transport layer, a hole injection layer, and a common electrode, and the light-receiving element 62 may be provided with, in order from the bottom layer, a pixel electrode, an electron transport layer, a light-receiving layer, a hole transport layer, and a common electrode. In this case, the hole injection layer of the light-emitting element 61 can be a common layer, and this common layer can be provided between the hole transport layer of the photodetector 62 and the common electrode. Furthermore, in the light-emitting element 61, the electron injection layer can be separated for each element.

[0322] Furthermore, by using an MML structure for the light-emitting element 61 and the light-receiving element 62, the configurations of the light-emitting element 61 and the light-receiving element 62 can be made different. For example, the light-emitting element 61 may be provided with, in order from the bottom layer, a pixel electrode 84, a hole injection layer 85, a hole transport layer 86, a light-emitting layer 87, an electron transport layer 88, a common layer 89 (electron injection layer), and a common electrode 81, while the light-receiving element 62 may be provided with, in order from the bottom layer, a pixel electrode 84PD, an electron transport layer 88PD, a light-receiving layer 90, a hole transport layer 86PD, a common layer 89, and a common electrode 81. With this configuration, the driving voltages of the light-emitting element 61 and the light-receiving element 62 can be made to be in the same direction. In this configuration, a hole injection layer may be provided between the hole transport layer 86PD and the common layer 89 in the light-receiving element 62.

[0323] In the following explanation, we will assume that the electron transport layer is located above the hole transport layer. However, by, for example, substituting "electron" with "hole" and "hole" with "electron," the following explanation can also be applied when the electron transport layer is located below the hole transport layer.

[0324] The hole injection layer is a layer that injects holes from the anode into the hole transport layer and is a layer containing a material with high hole injection properties. Examples of materials with high hole injection properties include aromatic amine compounds and composite materials containing a hole transport material and an acceptor material (electron-accepting material).

[0325] The hole transport layer is a layer that transports the holes injected from the anode by the hole injection layer to the light-emitting layer. The hole transport layer is a layer containing a hole transport material. As the hole transport material, a substance having a hole mobility of 1×10 -6 cm 2 / Vs or more is preferred. In addition, as long as the substance has higher hole transportability than electrons, other substances can also be used. As the hole transport material, hole transport materials with high hole transportability such as π-electron-excessive heteroaromatic compounds (such as carbazole derivatives, thiophene derivatives, furan derivatives, etc.) and aromatic amines (compounds having an aromatic amine skeleton) are preferred.

[0326] The electron transport layer is a layer that transports the electrons injected from the cathode by the electron injection layer to the light-emitting layer. The electron transport layer is a layer containing an electron transport material. As the electron transport material, a substance having an electron mobility of 1×10 -6 cm 2 / Vs or more is preferred. In addition, as long as the substance has higher electron transportability than holes, other substances can also be used. As the electron transport material, metal complexes having a quinoline skeleton, metal complexes having a benzoquinoline skeleton, metal complexes having an oxazole skeleton, metal complexes having a thiazole skeleton, etc. In addition, electron transport materials with high electron transportability such as oxadiazole derivatives, triazole derivatives, imidazole derivatives, oxazole derivatives, thiazole derivatives, phenanthroline derivatives, quinoline derivatives having a quinoline ligand, benzoquinoline derivatives, quinoxaline derivatives, dibenzoquinoxaline derivatives, pyridine derivatives, bipyridine derivatives, pyrimidine derivatives, and other nitrogen-containing heteroaromatic compounds can be used.

[0327] The electron injection layer is a layer that injects electrons from the cathode to the electron transport layer, and is a layer containing a material with high electron injection capabilities. Alkali metals, alkaline earth metals, or compounds thereof can be used as materials with high electron injection capabilities. Composite materials containing both electron transport materials and donor materials (electron-donating materials) can also be used as materials with high electron injection capabilities.

[0328] Examples of electron injection layers include lithium, cesium, lithium fluoride (LiF), cesium fluoride (CsF), calcium fluoride (CaF2), 8-(quinolinolato)lithium (abbreviated as Liq), 2-(2-pyridyl)phenolate (abbreviated as LiPP), 2-(2-pyridyl)-3-pyridinolatritium (abbreviated as LiPPy), 4-phenyl-2-(2-pyridyl)phenolate (abbreviated as LiPPP), and lithium oxide (LiO2). x ), alkali metals such as cesium carbonate, alkaline earth metals, or compounds thereof can be used.

[0329] Alternatively, an electron-transporting material may be used as the electron injection layer described above. For example, a compound having a lone pair of electrons and an electron-deficient heteroaromatic ring can be used as the electron-transporting material. Specifically, a compound having at least one of a pyridine ring, a diazine ring (pyrimidine ring, pyrazine ring, pyridazine ring), or a triazine ring can be used.

[0330] Furthermore, it is preferable that the lowest unoccupied molecular orbital (LUMO) of an organic compound containing a lone pair of electrons is between -3.6 eV and -2.3 eV. In general, the highest occupied molecular orbital (HOMO) level and LUMO level of an organic compound can be estimated by methods such as cyclic voltammetry (CV), photoelectron spectroscopy, optical absorption spectroscopy, and inverse photoelectron spectroscopy.

[0331] For example, 4,7-diphenyl-1,10-phenanthroline (abbreviated as BPhen), 2,9-di(naphthalene-2-yl)-4,7-diphenyl-1,10-phenanthroline (abbreviated as NBPhen), diquinoxalino[2,3-a:2',3'-c]phenazine (abbreviated as HATNA), and 2,4,6-tris[3'-(pyridine-3-yl)biphenyl-3-yl]-1,3,5-triazine (abbreviated as TmPPPyTz) can be used in organic compounds containing lone pairs of electrons. NBPhen has a higher glass transition temperature (Tg) and superior heat resistance compared to BPhen.

[0332] The luminescent layer is a layer containing a luminescent material. The luminescent layer may contain one or more types of luminescent materials. Suitable luminescent materials include those that emit light of various colors such as blue, purple, blue-violet, green, yellow-green, yellow, orange, and red. Furthermore, materials that emit near-infrared light may also be used as luminescent materials.

[0333] Examples of luminescent materials include fluorescent materials, phosphorescent materials, TADF materials, and quantum dot materials.

[0334] Examples of fluorescent materials include pyrene derivatives, anthracene derivatives, triphenylene derivatives, fluorene derivatives, carbazole derivatives, dibenzothiophene derivatives, dibenzofuran derivatives, dibenzoquinoxaline derivatives, quinoxaline derivatives, pyridine derivatives, pyrimidine derivatives, phenanthrene derivatives, and naphthalene derivatives.

[0335] Examples of phosphorescent materials include organometallic complexes (especially iridium complexes) having a 4H-triazole skeleton, 1H-triazole skeleton, imidazole skeleton, pyrimidine skeleton, pyrazine skeleton, or pyridine skeleton; organometallic complexes (especially iridium complexes) using phenylpyridine derivatives having electron-withdrawing groups as ligands; platinum complexes; and rare earth metal complexes.

[0336] The light-emitting layer may contain one or more types of organic compounds (host material, assist material, etc.) in addition to the light-emitting substance (guest material). One or more of these organic compounds may be hole-transporting materials and / or electron-transporting materials. Alternatively, one or more of these organic compounds may be bipolar materials or TADF materials.

[0337] The light-emitting layer preferably comprises, for example, a phosphorescent material and a combination of a hole-transporting material and an electron-transporting material that readily forms an excitation complex. This configuration allows for efficient emission using ExTET (Exciplex-Triplet Energy Transfer), which is energy transfer from the excitation complex to the light-emitting substance (phosphorescent material). By selecting a combination that forms an excitation complex that emits light overlapping with the wavelength of the lowest-energy absorption band of the light-emitting substance, energy transfer becomes smoother, and light emission can be obtained efficiently. This configuration simultaneously achieves high efficiency, low-voltage operation, and a long lifespan for the light-emitting element.

[0338] For a combination of materials to form an excited complex, it is preferable that the HOMO level (highest occupied orbital level) of the hole-transporting material is greater than or equal to the HOMO level of the electron-transporting material. It is also preferable that the LUMO level (lowest unoccupied orbital level) of the hole-transporting material is greater than or equal to the LUMO level of the electron-transporting material. The LUMO and HOMO levels of the materials can be derived from the electrochemical properties (reduction potential and oxidation potential) of the materials measured by cyclic voltammetry (CV).

[0339] The formation of excited complexes can be confirmed, for example, by comparing the emission spectra of a hole-transporting material, an electron-transporting material, and a mixed film made by mixing these materials, and observing that the emission spectrum of the mixed film shifts to a longer wavelength than the emission spectra of each individual material (or has a new peak on the longer wavelength side). Alternatively, it can be confirmed by comparing the transient photoluminescence (PL) of a hole-transporting material, an electron-transporting material, and a mixed film made by mixing these materials, and observing differences in the transient response, such as the transient PL lifetime of the mixed film having a longer lifetime component or a larger proportion of the delayed component than the transient PL lifetime of each individual material. Furthermore, the transient PL mentioned above can be read as transient electroluminescence (EL). That is, the formation of excited complexes can also be confirmed by comparing the transient EL of a hole-transporting material, an electron-transporting material, and a mixed film made by mixing these materials, and observing the differences in the transient response.

[0340] The light-emitting layer 87R of the light-emitting element 61R has a luminescent organic compound that emits light having a peak in at least the red wavelength range. The light-emitting layer 87G of the light-emitting element 61G has a luminescent organic compound that emits light having a peak in at least the green wavelength range. The light-emitting layer 87B of the light-emitting element 61B has a luminescent organic compound that emits light having a peak in at least the blue wavelength range. The light-emitting layer 87IR of the light-emitting element 61IR has a luminescent organic compound that emits light having a peak in at least the infrared wavelength range. The light-receiving layer 90 of the light-receiving element 62 has, for example, an organic compound that has detection sensitivity in the infrared wavelength range.

[0341] A conductive film that is transparent to visible light is used on either the pixel electrode 84 or the common electrode 81, and a conductive film that is reflective is used on the other. By making the pixel electrode 84 transparent and the common electrode 81 reflective, the display device 100 can be made into a bottom-emission type display device. On the other hand, by making the pixel electrode 84 reflective and the common electrode 81 transparent, the display device 100 can be made into a top-emission type display device. Furthermore, by making both the pixel electrode 84 and the common electrode 81 transparent, the display device 100 can also be made into a dual-emission type display device.

[0342] Furthermore, it is preferable that the light-emitting element 61 has a microcavity structure. This allows the light emitted from the light-emitting layer 87 to resonate between the pixel electrode 84 and the common electrode 81, thereby strengthening the light emitted from the light-emitting element 61. If the light-emitting device has an SBS structure, a microcavity structure can be applied to the light-emitting device.

[0343] When the light-emitting element 61 has a microcavity structure, it is preferable that one of the common electrode 81 or the pixel electrode 84 is an electrode that has both light transmission and reflectivity (a semi-transparent / semi-reflective electrode), and the other of the common electrode 81 or the pixel electrode 84 is a reflective electrode (a reflective electrode). Light emitted from the light-emitting layer is repeatedly reflected between the semi-transparent / semi-reflective electrode and the reflective electrode, and light of a desired wavelength can be extracted from the semi-transparent / semi-reflective electrode. Here, an electrode made by thinning a reflective electrode can be used as the semi-transparent / semi-reflective electrode. When a semi-transparent / semi-reflective electrode is applied to the pixel electrode 84 to form a microcavity structure, a transparent electrode can be placed on the semi-transparent / semi-reflective electrode, and the thickness of the transparent electrode can be varied. Such a transparent electrode is called an optical adjustment layer.

[0344] Alternatively, the light-emitting element 61 can have a microcavity structure by making the distance between the common electrode 81 and the pixel electrode 84IR of the light-emitting element 61IR that emits the longest wavelength light the longest, the distance between the common electrode 81 and the pixel electrode 84R of the light-emitting element 61R that emits the next longest wavelength light the next longest, the distance between the common electrode 81 and the pixel electrode 84G of the light-emitting element 61G that emits the next longest wavelength light the next longest, and the distance between the common electrode 81 and the pixel electrode 84G of the light-emitting element 61B that emits the shortest wavelength light the shortest. However, it is not limited to this, and the thickness of each layer can be adjusted considering the wavelength of light emitted by each light-emitting element, the optical properties of the layers constituting the light-emitting element, and the electrical properties of the light-emitting element.

[0345] The transmittance of visible light in the transparent electrode shall be 40% or more. For example, it is preferable to use an electrode as the transparent electrode that has a transmittance of 40% or more of visible light (light with a wavelength of 400 nm or more and less than 750 nm). Furthermore, the reflectance of visible light in the semi-transparent / semi-reflective electrode shall be 10% or more and 95% or less, preferably 30% or more and 80% or less. The reflectance of visible light in the reflective electrode shall be 40% or more and 100% or less, preferably 70% or more and 100% or less. Furthermore, the resistivity of these electrodes shall be 1 × 10⁻⁶ -2 It is preferable that the value be Ωcm or less. Furthermore, when a light-emitting element (IR) that emits near-infrared light is used in the display device, it is preferable that the transmittance and reflectance of these electrodes for near-infrared light (light with a wavelength of 750 nm to 1300 nm) are also within the above numerical range.

[0346] An insulating layer 92 is provided to cover the ends of the pixel electrode 84R, the ends of the pixel electrode 84G, the ends of the pixel electrode 84B, the ends of the pixel electrode 84IR, and the ends of the pixel electrode 84PD. The ends of the insulating layer 92 are preferably tapered. The insulating layer 92 may be omitted if it is not needed.

[0347] For example, the hole injection layer 85R, hole injection layer 85G, hole injection layer 85B, hole injection layer 85IR, and hole transport layer 86PD each have a region that overlaps with the pixel electrode 84 and a region that overlaps with the insulating layer 92. Furthermore, the ends of the hole injection layer 85R, the ends of the hole injection layer 85G, the ends of the hole injection layer 85B, the ends of the hole injection layer 85IR, and the ends of the hole transport layer 86PD are located in the region that overlaps with the insulating layer 92.

[0348] As shown in Figure 20(A), a gap is provided between light-emitting elements 61 that emit light of different colors, for example, between two light-emitting layers 87. In this way, it is preferable that light-emitting layers 87R, 87G, and 87B are provided so that they do not touch each other. This effectively prevents crosstalk, which occurs when current flows through two adjacent light-emitting layers 87, causing unintended light emission. As a result, the contrast of the display device 100 can be increased, and thus the display quality of the display device 100 can be improved.

[0349] A protective layer 91 is provided on the common electrode 81. The protective layer 91 has the function of preventing impurities such as water from diffusing to each light-emitting element from above.

[0350] The protective layer 91 can be, for example, a single-layer structure or a multilayer structure including at least an inorganic insulating film. Examples of inorganic insulating films include oxide films or nitride films such as silicon oxide film, silicon oxide nitride film, silicon nitride film, silicon nitride film, aluminum oxide film, aluminum oxide nitride film, and hafnium oxide film. Alternatively, semiconductor materials such as indium gallium oxide and indium gallium zinc oxide may be used as the protective layer 91.

[0351] In this specification, a silicon oxide-nitride film refers to a film in which the oxygen content is greater than the nitrogen content. Similarly, a silicon nitride-oxide film refers to a film in which the nitrogen content is greater than the oxygen content.

[0352] Furthermore, a laminated film of an inorganic insulating film and an organic insulating film can be used as the protective layer 91. For example, it is preferable to have a configuration in which an organic insulating film is sandwiched between a pair of inorganic insulating films. It is also preferable that the organic insulating film functions as a planarizing film. This makes the upper surface of the organic insulating film flat, thereby improving the coverage of the inorganic insulating film on top of it and enhancing its barrier properties. In addition, since the upper surface of the protective layer 91 is flat, it is preferable because it reduces the influence of uneven shapes caused by the structure below when a structure (for example, a color filter, touch sensor electrodes, or lens array, etc.) is provided above the protective layer 91.

[0353] Figure 20(C) shows an example of the cross-sectional configuration of the display device 100 in the Y direction, specifically showing an example of the cross-sectional configuration of the light-emitting element 61R and the light-receiving element 62. Note that the light-emitting elements 61G, 61B, and 61IR can also be arranged in the Y direction in the same way as the light-emitting element 61R.

[0354] Figure 20(D) shows a connection portion 93 where the connecting electrode 82 and the common electrode 81 are electrically connected. In the connection portion 93, the common electrode 81 is placed in contact with the connecting electrode 82, and a protective layer 91 is provided to cover the common electrode 81. In addition, an insulating layer 92 is provided to cover the end of the connecting electrode 82.

[0355] Figures 20(A) to 20(C) show a configuration in which an insulating layer 92 is provided covering the ends of the pixel electrode 84R, the ends of the pixel electrode 84G, the ends of the pixel electrode 84B, and the ends of the pixel electrode 84PD, but the present invention is not limited to this. As shown in Figure 20(E), the insulating layer 92 may not be provided.

[0356] Furthermore, insulating layers may be provided in the region between adjacent light-emitting elements 61, and in the region between adjacent light-emitting elements 61 and light-receiving elements 62. Figure 20(E) is a cross-sectional view corresponding to the dashed line A1-A2 in Figure 18(B). In Figure 20(E), insulating layer 94 and insulating layer 96 are provided in the region.

[0357] The sides of the pixel electrodes 84R, 84G, and 84B, the hole injection layer 85R, 85G, and 85B, the hole transport layer 86R, 86G, and 86B, the light-emitting layer 87R, 87G, and 87B, the electron transport layer 88R, 88G, and 88B are covered by insulating layers 94 and 96, respectively. A common layer 89 is provided on the electron transport layer 88R, 88G, 88B, insulating layer 94, and insulating layer 96, and a common electrode 81 is provided on the common layer 89.

[0358] By using the above configuration, it is possible to suppress the common layer 89 (or common electrode 81) from coming into contact with any of the side surfaces of the pixel electrode 84R, pixel electrode 84G, pixel electrode 84B, light-emitting layer 87R, light-emitting layer 87G, and light-emitting layer 87B, thereby suppressing a short circuit of the light-emitting element.

[0359] The insulating layer 94 preferably covers at least the sides of the pixel electrodes 84R, 84G, and 84B. Furthermore, the insulating layer 94 preferably covers the sides of the hole injection layer 85R, 85G, 85B, hole transport layer 86R, 86G, 86B, light-emitting layer 87R, 87G, 87B, electron transport layer 88R, 88G, and electron transport layer 88B. The insulating layer 94 can be configured to be in contact with the respective sides of the pixel electrode 84R, pixel electrode 84G, pixel electrode 84B, hole injection layer 85R, hole injection layer 85G, hole injection layer 85B, hole transport layer 86R, hole transport layer 86G, hole transport layer 86B, light-emitting layer 87R, light-emitting layer 87G, light-emitting layer 87B, electron transport layer 88R, electron transport layer 88G, and electron transport layer 88B.

[0360] The insulating layer 96 is provided on the insulating layer 94 so as to fill the recesses formed in the insulating layer 94. The insulating layer 96 can be configured to overlap with the sides of each of the pixel electrodes 84R, 84G, 84B, hole injection layer 85R, 85G, 85B, hole transport layer 86R, 86G, 86B, light-emitting layer 87R, 87G, 87B, electron transport layer 88R, 88G, and electron transport layer 88B via the insulating layer 94.

[0361] Furthermore, it is not necessary to provide either the insulating layer 94 or the insulating layer 96. If the insulating layer 94 is not provided, the insulating layer 96 can be configured to be in contact with the respective sides of the light-emitting layer 87R, the light-emitting layer 87G, and the light-emitting layer 87B. The display device may also have an insulating layer that covers the ends of the pixel electrodes. In this case, one or both of the insulating layer 94 and the insulating layer 96 may be provided on the insulating layer.

[0362] The common layer 89 and common electrode 81 are provided on electron transport layers 88R, 88G, 88B, insulating layer 94, and insulating layer 96. Before insulating layers 94 and 96 are provided, a step difference exists due to the region where the pixel electrode and light-emitting layer are provided and the region where the pixel electrode and light-emitting layer are not provided (the region between light-emitting elements). In one embodiment of the present invention, the presence of insulating layers 94 and 96 can flatten this step difference and improve the coverage of the common layer 89 and common electrode 81. Therefore, connection failures due to step breaks can be suppressed. Alternatively, the local thinning of the common electrode 81 due to the step difference and the resulting increase in electrical resistance can be suppressed.

[0363] To improve the flatness of the formation surfaces of the common layer 89 and the common electrode 81, it is preferable that the heights of the upper surfaces of the insulating layer 94 and the insulating layer 96 match or approximately match the height of at least one of the upper surfaces of the electron transport layers 88R, 88G, and 88B, respectively. Furthermore, it is preferable that the upper surface of the insulating layer 96 has a flat shape, but it may have convex or concave portions.

[0364] The insulating layer 94 has regions that are in contact with the sides of the light-emitting layers 87R, 87G, and 87B, and functions as a protective insulating layer for the light-emitting layers 87R, 87G, and 87B. By providing the insulating layer 94, it is possible to suppress the intrusion of impurities (oxygen, moisture, etc.) into the interior from the sides of the light-emitting layers 87R, 87G, and 87B, resulting in a highly reliable display device.

[0365] If the width (thickness) of the insulating layer 94 in the region in contact with the sides of the light-emitting layers 87R, 87G, and 87B in a cross-sectional view is large, the spacing between the light-emitting layers 87R, 87G, and 87B will increase, which may result in a lower aperture ratio. Conversely, if the width (thickness) of the insulating layer 94 in the region in contact with the sides of the light-emitting layers 87R, 87G, and 87B in a cross-sectional view is small, the effect of suppressing the intrusion of impurities into the interior from the sides of the light-emitting layers 87R, 87G, and 87B may be reduced. In a cross-sectional view, the width (thickness) of the insulating layer 94 in the region in contact with the sides of the light-emitting layer 87R, light-emitting layer 87G, and light-emitting layer 87B is preferably 3 nm to 200 nm, more preferably 3 nm to 150 nm, more preferably 5 nm to 150 nm, more preferably 5 nm to 100 nm, more preferably 10 nm to 100 nm, and more preferably 10 nm to 50 nm. By setting the width (thickness) of the insulating layer 94 within the above range, a display device with a high aperture ratio and high reliability can be obtained.

[0366] The insulating layer 94 can be an insulating layer having an inorganic material. For example, inorganic insulating films such as oxide insulating films, nitride insulating films, oxidative nitride insulating films, and nitride oxide insulating films can be used for the insulating layer 94. The insulating layer 94 may be a single layer or a laminated structure. Examples of oxide insulating films include silicon oxide film, aluminum oxide film, magnesium oxide film, indium gallium zinc oxide film, gallium oxide film, germanium oxide film, yttrium oxide film, zirconium oxide film, lanthanum oxide film, neodymium oxide film, hafnium oxide film, and tantalum oxide film. Examples of nitride insulating films include silicon nitride film and aluminum nitride film. Examples of oxidative nitride insulating films include silicon oxidative nitride film and aluminum oxidative nitride film. Examples of nitride oxide insulating films include silicon nitride oxide film and aluminum nitride oxide film. In particular, aluminum oxide is preferred because it has a high selectivity ratio with the light-emitting layer during etching and has the function of protecting the light-emitting layer during the formation of the insulating layer 96, which will be described later. In particular, by applying inorganic insulating films such as aluminum oxide films, hafnium oxide films, and silicon oxide films formed by the ALD method to the insulating layer 94, it is possible to form an insulating layer 94 with fewer pinholes and excellent function in protecting the light-emitting layer.

[0367] In this specification, the term "oxide-nitride" refers to a material in which the oxygen content is greater than the nitrogen content, and the term "nitride oxide" refers to a material in which the nitrogen content is greater than the oxygen content. For example, when "silicon oxynitride" is written, it refers to a material in which the oxygen content is greater than the nitrogen content, and when "silicon nitride oxide" is written, it refers to a material in which the nitrogen content is greater than the oxygen content.

[0368] The insulating layer 94 can be formed using sputtering, CVD, PLD, ALD, or the like. It is preferable to form the insulating layer 94 using the ALD method, which provides good coverage.

[0369] The insulating layer 96 provided on the insulating layer 94 has the function of flattening the recess in the insulating layer 94 formed between adjacent light-emitting devices. In other words, the presence of the insulating layer 96 improves the flatness of the surface on which the common electrode 81 is formed. Suitable insulating layers 96 include those made of organic materials. For example, acrylic resin, polyimide resin, epoxy resin, imide resin, polyamide resin, polyimidoamide resin, silicone resin, siloxane resin, benzocyclobutene resin, phenol resin, and precursors of these resins can be used as the insulating layer 96. Alternatively, organic materials such as polyvinyl alcohol (PVA), polyvinyl butyral, polyvinylpyrrolidone, polyethylene glycol, polyglycerin, pullulan, water-soluble cellulose, or alcohol-soluble polyamide resin can be used as the insulating layer 96. Furthermore, a photosensitive resin (also referred to as an organic resin) can be used as the insulating layer 96. A photoresist may be used as the photosensitive resin. The photosensitive resin can be a positive-type material or a negative-type material.

[0370] The difference between the height of the upper surface of the insulating layer 96 and the height of the upper surface of any of the light-emitting layers 87R, 87G, and 87B is preferably 0.5 times or less the thickness of the insulating layer 96, and more preferably 0.3 times or less. Alternatively, the insulating layer 96 may be provided such that the upper surface of any of the light-emitting layers 87R, 87G, and 87B is higher than the upper surface of the insulating layer 96. Alternatively, the insulating layer 96 may be provided such that the upper surface of the insulating layer 96 is higher than the upper surface of any of the light-emitting layers 87R, 87G, and 87B.

[0371] In the display device 100, the pixels 80 are composed of sub-pixels having an element-emitting element 61R, sub-pixels having an element-emitting element 61G, sub-pixels having an element-emitting element 61B, sub-pixels having an element-emitting element 61IR, and sub-pixels having a light-receiving element 62, but the present invention is not limited to this. Display devices different from the display device 100 shown in Figure 18(A), etc., are shown in Figures 21(A), 21(B), 22(A), 22(B), 23(A), and 23(B).

[0372] The display device 100 shown in Figure 21(A) differs from the display device 100 shown in Figure 18(A) in that the pixel 80 is composed of a light-emitting element 61R, a light-emitting element 61G, a light-emitting element 61B, and a light-receiving element 62. In this case, the light-emitting element 61IR may be provided between the display area 95 and the connecting electrode 82. Alternatively, as shown in Figure 21(B), the light-emitting element 61IR may be provided outside the display area 95 and the connecting electrode 82. When the light-emitting element 61IR is provided on the outside, the light-receiving element 62 can be formed over a larger area, thereby increasing the light detection sensitivity of the light-receiving element 62.

[0373] In the display device 100 shown in Figure 21(A), multiple light-emitting elements 61IR can be provided along the outer perimeter of the display area 95. For example, they may be provided along one side of the outer perimeter of the display area 95, or they may be provided across two or more sides of the outer perimeter of the display area 95. That is, if the top surface shape of the display area 95 is rectangular, the arrangement of the light-emitting elements 61IR in a top view can be in the shape of a strip, L-shape, U-shape (angle bracket shape), or square, etc.

[0374] In the display device 100 shown in Figure 21(B), multiple light-emitting elements 61IR can be provided along the outer circumference of the connecting electrode 82. For example, they may be provided along one side of the outer circumference of the connecting electrode 82, or they may be provided across two or more sides of the outer circumference of the connecting electrode 82. That is, if the top surface shape of the connecting electrode 82 is rectangular, the arrangement of the light-emitting elements 61IR in a top view can be in the shape of a strip, L-shape, U-shape (angle bracket shape), or square, etc.

[0375] Furthermore, Figures 21(A) and 21(B) show an example where the width of the light-emitting element 61IR in the Y direction is approximately the same as the width of the pixel 80 in the Y direction, but the present invention is not limited to this. The width of the light-emitting element 61IR in the Y direction may be greater than or less than the width of the pixel 80 in the Y direction. Also, Figures 21(A) and 21(B) show an example where the number of light-emitting elements 61IR in the Y direction is the same as the number of pixels 80, but the present invention is not limited to this. The number of light-emitting elements 61IR in the Y direction may be different from the number of pixels 80, and may be one or more. Also, Figures 21(A) and 21(B) show an example where there is one light-emitting element 61IR in the X direction, but the present invention is not limited to this. The number of light-emitting elements 61IR in the X direction may be multiple.

[0376] The display device 100 shown in Figure 22(A) differs from the display device 100 shown in Figure 18(A) in that the pixel 80 is composed of light-emitting elements 61R, 61G, 61B, and 61IR. In this case, the light-receiving element 62 may be provided between the display area 95 and the connecting electrode 82. Alternatively, as shown in Figure 22(B), the light-receiving element 62 may be provided outside the display area 95 and the connecting electrode 82. This allows the light-receiving element 62 to be formed over a larger area, thereby increasing the light detection sensitivity of the light-receiving element 62.

[0377] In the display device 100 shown in Figure 22(A), multiple light-receiving elements 62 can be provided along the outer perimeter of the display area 95. For example, they may be provided along one side of the outer perimeter of the display area 95, or they may be provided across two or more sides of the outer perimeter of the display area 95. That is, if the top surface shape of the display area 95 is rectangular, the arrangement of the light-receiving elements 62 in a top view can be in the shape of a strip, L-shape, U-shape (angle bracket shape), or square, etc.

[0378] In the display device 100 shown in Figure 22(B), multiple light-receiving elements 62 can be provided along the outer circumference of the connecting electrode 82. For example, they may be provided along one side of the outer circumference of the connecting electrode 82, or they may be provided across two or more sides of the outer circumference of the connecting electrode 82. That is, if the top surface shape of the connecting electrode 82 is rectangular, the arrangement of the light-receiving elements 62 in a top view can be in the shape of a strip, L-shape, U-shape (angle bracket shape), or square, etc.

[0379] Furthermore, Figures 22(A) and 22(B) show an example where the width of the light-receiving element 62 in the Y direction is approximately the same as the width of the pixel 80 in the Y direction, but the present invention is not limited to this. The width of the light-receiving element 62 in the Y direction may be greater than or less than the width of the pixel 80 in the Y direction. Also, Figures 22(A) and 22(B) show an example where the number of light-receiving elements 62 in the Y direction is the same as the number of pixels 80, but the present invention is not limited to this. The number of light-receiving elements 62 in the Y direction may be different from the number of pixels 80, and may be one or more. Also, Figures 22(A) and 22(B) show an example where there is one light-receiving element 62 in the X direction, but the present invention is not limited to this. The number of light-receiving elements 62 in the X direction may be multiple.

[0380] The display device 100 shown in Figure 23(A) differs from the display device 100 shown in Figure 18(A) in that the pixel 80 is composed of light-emitting elements 61R, 61G, and 61B. In this case, the light-emitting element 61IR and the light-receiving element 62 may be placed between the display area 95 and the connecting electrode 82. Alternatively, as shown in Figure 23(B), the light-emitting element 61IR and the light-receiving element 62 may be placed outside the display area 95 and the connecting electrode 82. Since the length of the light-emitting elements 61R, 61G, and 61B in the Y direction can be increased, the brightness of the light emitted by the pixel 80 can be increased.

[0381] In the display device 100 shown in Figure 23(A), multiple light-emitting elements 61IR and light-receiving elements 62 can be provided along the outer perimeter of the display area 95. For example, they may be provided along one side of the outer perimeter of the display area 95, or they may be provided across two or more sides of the outer perimeter of the display area 95. That is, if the top surface shape of the display area 95 is rectangular, the arrangement of the light-emitting elements 61IR and light-receiving elements 62 in a top view can be in the shape of a strip, L-shape, U-shape (angle bracket shape), or square, etc. Furthermore, the arrangement of the light-emitting elements 61IR and light-receiving elements 62 may be different. For example, the light-emitting elements 61IR in a top view may be arranged on two opposing sides of the display area 95, and the light-receiving elements 62 in a top view may be arranged on two sides other than the two sides mentioned above.

[0382] In the display device 100 shown in Figure 23(B), multiple light-emitting elements 61IR and light-receiving elements 62 can be provided along the outer circumference of the connecting electrode 82. For example, they may be provided along one side of the outer circumference of the connecting electrode 82, or they may be provided across two or more sides of the outer circumference of the connecting electrode 82. That is, if the top surface shape of the connecting electrode 82 is rectangular, the arrangement of the light-emitting elements 61IR and light-receiving elements 62 in a top view can be strip-shaped, L-shaped, U-shaped (angle bracket-shaped), or square, etc. Furthermore, the arrangement of the light-emitting elements 61IR and light-receiving elements 62 may be different. For example, the light-emitting elements 61IR in a top view may be arranged on two opposing sides of the connecting electrode 82, and the light-receiving elements 62 in a top view may be arranged on two sides other than the two sides mentioned above.

[0383] Furthermore, Figures 23(A) and 23(B) show an example where the sum of the Y-direction width of the light-emitting element 61IR and the Y-direction width of the light-receiving element 62 is approximately the same as the Y-direction width of the pixel 80, but the present invention is not limited to this. The Y-direction widths of the light-emitting element 61IR and the light-receiving element 62 may be greater than or less than the Y-direction width of the pixel 80. Also, Figures 23(A) and 23(B) show an example where the number of light-emitting elements 61IR, the number of light-receiving elements 62, and the number of pixels 80 are the same in the Y-direction, but the present invention is not limited to this. The number of light-emitting elements 61IR and the number of light-receiving elements 62 in the Y-direction may be different from the number of pixels 80, and may be one or more. Furthermore, the number of light-emitting elements 61IR and the number of light-receiving elements 62 in the Y-direction may be different. Furthermore, while Figures 23(A) and 23(B) show an example in which there is one light-emitting element 61IR and one photodetector 62 in the X direction, the present invention is not limited to this. The number of light-emitting elements 61IR and photodetectors 62 in the X direction may be multiple.

[0384] <Example of light-emitting element configuration> As shown in Figure 24(A), the light-emitting element has a functional layer 686 between a pair of electrodes (electrode 672, electrode 688). The functional layer 686 can be composed of multiple layers, such as layer 4420, light-emitting layer 4411, and layer 4430. Layer 4420 may include, for example, a layer containing a material with high electron injection properties (electron injection layer) and a layer containing a material with high electron transport properties (electron transport layer). Light-emitting layer 4411 may include, for example, a light-emitting compound. Layer 4430 may include, for example, a layer containing a material with high hole injection properties (hole injection layer) and a layer containing a material with high hole transport properties (hole transport layer).

[0385] A configuration having a layer 4420, an emissive layer 4411, and a layer 4430 provided between a pair of electrodes can function as a single emissive unit, and in this specification, the configuration shown in Figure 24(A) is referred to as a single structure.

[0386] Furthermore, Figure 24(B) shows a modified example of the functional layer 686 of the light-emitting element shown in Figure 24(A). Specifically, the light-emitting element shown in Figure 24(B) has a layer 4430-1 on electrode 672, a layer 4430-2 on layer 4430-1, an emissive layer 4411 on layer 4430-2, a layer 4420-1 on emissive layer 4411, a layer 4420-2 on layer 4420-1, and an electrode 688 on layer 4420-2. For example, when electrode 672 is the anode and electrode 688 is the cathode, layer 4430-1 functions as a hole injection layer, layer 4430-2 functions as a hole transport layer, layer 4420-1 functions as an electron transport layer, and layer 4420-2 functions as an electron injection layer. Alternatively, when electrode 672 is used as the cathode and electrode 688 as the anode, layer 4430-1 functions as an electron injection layer, layer 4430-2 functions as an electron transport layer, layer 4420-1 functions as a hole transport layer, and layer 4420-2 functions as a hole injection layer. This layer structure allows for efficient injection of carriers into the light-emitting layer 4411 and improves the efficiency of carrier recombination within the light-emitting layer 4411.

[0387] Furthermore, as shown in Figure 24(C), a configuration in which multiple light-emitting layers (light-emitting layer 4411, light-emitting layer 4412, light-emitting layer 4413) are provided between layer 4420 and layer 4430 is also a modified example of the single structure.

[0388] Furthermore, as shown in Figure 24(D), a configuration in which multiple light-emitting units (functional layers 686a and 686b) are connected in series via an intermediate layer (charge generation layer) 4440 is referred to as a tandem structure in this specification. The tandem structure may also be called a stacked structure. By using a tandem structure, a light-emitting element capable of high-brightness emission can be achieved.

[0389] Furthermore, in Figures 24(C) and 24(D), as shown in Figure 24(B), layer 4420 and layer 4430 may be a laminated structure consisting of two or more layers.

[0390] Furthermore, a structure that creates different light-emitting layers (for example, red (R), green (G), and blue (B)) for each light-emitting element is called an SBS structure.

[0391] Furthermore, when comparing the single structure, tandem structure, and SBS structure described above, power consumption can be reduced in the order of SBS structure, tandem structure, and single structure. If you want to keep power consumption low, the SBS structure is preferable. On the other hand, the single structure and tandem structure are preferable because their manufacturing process is simpler than that of the SBS structure, which can lead to lower manufacturing costs or higher manufacturing yields.

[0392] The light-emitting color of the light-emitting element can be red, green, blue, cyan, magenta, yellow, or white, depending on the material constituting the light-emitting layer 4411. Furthermore, the color purity can be further enhanced by adding a microcavity structure to the light-emitting element.

[0393] A light-emitting element that emits white light preferably has a configuration in which two or more light-emitting materials are included in the light-emitting layer. To obtain white light emission, light-emitting materials should be selected such that the light emitted by each of the two or more materials is complementary in color. For example, by making the light-emitting color of the first light-emitting layer and the light-emitting color of the second light-emitting layer complementary in color, a light-emitting element that emits white light as a whole can be obtained. The same applies to light-emitting elements that have three or more light-emitting layers.

[0394] The light-emitting layer preferably contains two or more light-emitting materials that emit light such as R (red), G (green), B (blue), Y (yellow), and O (orange). Alternatively, it is preferable to have two or more light-emitting materials, and for each light-emitting material to emit light that contains spectral components of two or more colors from R, G, and B.

[0395] <Example of light-emitting element and light-receiving element configuration> One embodiment of the present invention is a top-emission type display device that emits light in the direction opposite to the substrate on which the light-emitting element is formed. In this embodiment, a display device equipped with a top-emission type light-emitting element and a light-receiving element will be described as an example.

[0396] In this specification, unless otherwise specified, when describing a configuration having multiple elements (such as light-emitting elements and light-emitting layers), the letters will be omitted when describing matters common to each element. For example, when describing matters common to light-emitting layers 383R and 383G, etc., it may be written as light-emitting layer 383.

[0397] The display device 380A shown in Figure 25 includes a light-receiving element 370PD, a light-emitting element 370R that emits red (R) light, a light-emitting element 370G that emits green (G) light, a light-emitting element 370B that emits blue (B) light, and a light-emitting element 370IR that emits infrared (IR) light.

[0398] Each light-emitting element has a pixel electrode 371, a hole injection layer 381, a hole transport layer 382, ​​a light-emitting layer, an electron transport layer 384, an electron injection layer 385, and a common electrode 375 stacked in this order. Light-emitting element 370R has a light-emitting layer 383R, light-emitting element 370G has a light-emitting layer 383G, light-emitting element 370B has a light-emitting layer 383B, and light-emitting element 370IR has a light-emitting layer 383IR. Light-emitting layer 383R has a light-emitting material that emits red light, light-emitting layer 383G has a light-emitting material that emits green light, light-emitting layer 383B has a light-emitting material that emits blue light, and light-emitting layer 383IR has a light-emitting material that emits infrared light.

[0399] The light-emitting element is an electroluminescent element that emits light towards the common electrode 375 when a voltage is applied between the pixel electrode 371 and the common electrode 375.

[0400] The photodetector 370PD has a pixel electrode 371, a hole injection layer 381, a hole transport layer 382, ​​an active layer 373, an electron transport layer 384, an electron injection layer 385, and a common electrode 375 stacked in this order.

[0401] The light-receiving element 370PD is a photoelectric conversion element that receives light incident from outside the display device 380A and converts it into an electrical signal.

[0402] In this embodiment, both the light-emitting element and the light-receiving element are described as having a pixel electrode 371 functioning as the anode and a common electrode 375 functioning as the cathode. In other words, the light-receiving element can detect light incident on it, generate an electric charge, and extract it as an electric current by driving it with a reverse bias applied between the pixel electrode 371 and the common electrode 375.

[0403] In the display device of this embodiment, an organic compound is used for the active layer 373 of the light-receiving element 370PD. The layers of the light-receiving element 370PD other than the active layer 373 can have the same configuration as those of the light-emitting element. Therefore, by simply adding a step of forming the active layer 373 to the manufacturing process of the light-emitting element, the light-receiving element 370PD can be formed in parallel with the formation of the light-emitting element. Furthermore, the light-emitting element and the light-receiving element 370PD can be formed on the same substrate. Thus, the light-receiving element 370PD can be incorporated into the display device without significantly increasing the manufacturing process.

[0404] In the display device 380A, an example is shown where the light-receiving element 370PD and the light-emitting element have a common configuration, except that the active layer 373 of the light-receiving element 370PD and the light-emitting layer 383 of the light-emitting element are manufactured separately. However, the configuration of the light-receiving element 370PD and the light-emitting element is not limited to this. In addition to the active layer 373 and the light-emitting layer 383, the light-receiving element 370PD and the light-emitting element may have layers that are manufactured separately from each other. It is preferable that the light-receiving element 370PD and the light-emitting element have one or more layers that are used in common (common layers). This makes it possible to incorporate the light-receiving element 370PD into the display device without significantly increasing the manufacturing process.

[0405] Of the pixel electrode 371 and the common electrode 375, the electrode that extracts light preferably uses a conductive film that transmits visible light. Furthermore, it is preferable to use a conductive film that reflects visible light on the electrode that does not extract light.

[0406] In this embodiment, it is preferable that the light-emitting element of the display device has a microcavity structure. Therefore, it is preferable that one of the pair of electrodes of the light-emitting element has an electrode that is transparent to and reflective to visible light (a semi-transmitting / semi-reflective electrode), and the other has an electrode that is reflective to visible light (a reflective electrode). By having a microcavity structure in the light-emitting element, the light emitted from the light-emitting layer can be resonated between the two electrodes, thereby strengthening the light emitted from the light-emitting element.

[0407] The light-emitting element has at least an emissive layer 383. The light-emitting element may further have layers other than the emissive layer 383, which include a material with high hole injection properties, a material with high hole transport properties, a hole blocking material, a material with high electron transport properties, a material with high electron injection properties, an electron blocking material, or a bipolar material (a material with high electron transport and hole transport properties).

[0408] For example, a light-emitting element and a photodetector can have one or more layers from among the hole injection layer, hole transport layer, electron transport layer, and electron injection layer in common. Alternatively, the light-emitting element and the photodetector can have one or more layers from among the hole injection layer, hole transport layer, electron transport layer, and electron injection layer that are differently constructed for each other.

[0409] In a photodetector, the hole transport layer is a layer that transports holes generated in the active layer based on incident light to the anode. Similarly, in a photodetector, the electron transport layer is a layer that transports electrons generated in the active layer based on incident light to the cathode.

[0410] Furthermore, the above-mentioned considerations may be applied to the hole injection layer, hole transport layer, electron transport layer, electron injection layer, and light emission layer.

[0411] The active layer 373 contains a semiconductor. Examples of such semiconductors include inorganic semiconductors such as silicon, and organic semiconductors containing organic compounds. In this embodiment, an example is shown in which an organic semiconductor is used as the semiconductor of the active layer 373. Using an organic semiconductor is preferable because it allows the light-emitting layer 383 and the active layer 373 to be formed by the same method (for example, vacuum deposition), and the manufacturing equipment can be shared.

[0412] The n-type semiconductor material of the active layer 373 is fullerene (for example, C 60 , C 70 Examples include electron-accepting organic semiconductor materials such as fullerene derivatives. Fullerenes have a shape like a soccer ball, and this shape is energetically stable. Both the HOMO and LUMO levels of fullerenes are deep (low). Because the LUMO level of fullerenes is deep, they have extremely high electron-accepting properties. Normally, when π-electron conjugation (resonance) spreads out in a plane, as in benzene, electron-donating properties increase, but because fullerenes have a spherical shape, they have high electron-accepting properties despite the large spread of π-electron conjugation. High electron-accepting properties allow for fast and efficient charge separation, making them useful as photodetectors. 60 , C 70 Both have a broad absorption band in the visible light region, and especially C 70 is C 60 Compared to [another compound], it is preferable because it has a larger π-electron conjugation system and a broad absorption band in the long-wavelength region. Other examples of fullerene derivatives include [6,6]-Phenyl-C71-butyric acid methyl ester (abbreviated as PC70BM), [6,6]-Phenyl-C61-butyric acid methyl ester (abbreviated as PC60BM), and 1',1'',4',4''-Tetrahydro-di[1,4]methanonaphthaleno[1,2:2',3',56,60:2'',3''][5,6]fullerene-C60 (abbreviated as ICBA).

[0413] Furthermore, examples of n-type semiconductor materials include metal complexes having a quinoline skeleton, metal complexes having a benzoquinoline skeleton, metal complexes having an oxazole skeleton, metal complexes having a thiazole skeleton, oxadiazole derivatives, triazole derivatives, imidazole derivatives, oxazole derivatives, thiazole derivatives, phenanthroline derivatives, quinoline derivatives, benzoquinoline derivatives, quinoxaline derivatives, dibenzoquinoxaline derivatives, pyridine derivatives, bipyridine derivatives, pyrimidine derivatives, naphthalene derivatives, anthracene derivatives, coumarin derivatives, rhodamine derivatives, triazine derivatives, and quinone derivatives.

[0414] Examples of p-type semiconductor materials for the active layer 373 include electron-donating organic semiconductor materials such as copper(II) phthalocyanine (CuPc), tetraphenyldibenzoperiflanthene (DBP), zinc phthalocyanine (ZnPc), tin phthalocyanine (SnPc), and quinacridone.

[0415] Furthermore, examples of p-type semiconductor materials include carbazole derivatives, thiophene derivatives, furan derivatives, and compounds having an aromatic amine skeleton. In addition, examples of p-type semiconductor materials include naphthalene derivatives, anthracene derivatives, pyrene derivatives, triphenylene derivatives, fluorene derivatives, pyrrole derivatives, benzofuran derivatives, benzothiophene derivatives, indole derivatives, dibenzofuran derivatives, dibenzothiophene derivatives, indrocarbazole derivatives, porphyrin derivatives, phthalocyanine derivatives, naphthalocyanine derivatives, quinacridone derivatives, polyphenylenevinylene derivatives, polyparaphenylene derivatives, polyfluorene derivatives, polyvinylcarbazole derivatives, and polythiophene derivatives.

[0416] The HOMO level of electron-donating organic semiconductor materials is preferably shallower (higher) than the HOMO level of electron-accepting organic semiconductor materials. The LUMO level of electron-donating organic semiconductor materials is preferably shallower (higher) than the LUMO level of electron-accepting organic semiconductor materials.

[0417] It is preferable to use spherical fullerenes as electron-accepting organic semiconductor materials and organic semiconductor materials with a near-planar shape as electron-donating organic semiconductor materials. Molecules with similar shapes tend to aggregate, and when molecules of the same type aggregate, their molecular orbital energy levels are close, which can improve carrier transport.

[0418] For example, the active layer 373 is preferably formed by co-depositing an n-type semiconductor and a p-type semiconductor. Alternatively, the active layer 373 may be formed by stacking an n-type semiconductor and a p-type semiconductor.

[0419] The light-emitting element and the light-receiving element may use either low-molecular-weight compounds or high-molecular-weight compounds, and may also contain inorganic compounds. The layers constituting the light-emitting element and the light-receiving element can be formed by methods such as vapor deposition (including vacuum deposition), transfer, printing, inkjet, and coating.

[0420] For example, polymer compounds such as poly(3,4-ethylenedioxythiophene) / poly(styrenesulfonic acid) (PEDOT / PSS) and inorganic compounds such as molybdenum oxide and copper iodide (CuI) can be used as hole transport materials. In addition, inorganic compounds such as zinc oxide (ZnO) can be used as electron transport materials.

[0421] Furthermore, the active layer 373 can use polymer compounds such as Poly[[4,8-bis[5-(2-ethylhexyl)-2-thienyl]benzo[1,2-b:4,5-b']dithiophene-2,6-diyl]-2,5-thiophenediyl[5,7-bis(2-ethylhexyl)-4,8-dioxo-4H,8H-benzo[1,2-c:4,5-c']dithiophene-1,3-diyl]]polymer (abbreviated as PBDB-T) or PBDB-T derivatives, which function as donors. For example, a method of dispersing the acceptor material in PBDB-T or a PBDB-T derivative can be used.

[0422] Furthermore, the active layer 373 may contain a mixture of three or more materials. For example, to broaden the wavelength range, a third material may be mixed with an n-type semiconductor material and a p-type semiconductor material. In this case, the third material may be a low-molecular-weight compound or a high-molecular-weight compound.

[0423] <Example of display device configuration> Figure 26 is a cross-sectional view showing an example of the configuration of the display device 100. The display device 100 has a configuration in which a transistor 310 with a channel formed on a substrate 301 and a transistor 320 containing a metal oxide in the semiconductor layer where the channel is formed are stacked. Furthermore, the display device 100 has a light-emitting element 61 and a light-receiving element 62 on the transistor 320. The light-emitting element 61 can emit light in the direction indicated by the dashed line, and the light-receiving element 62 can receive light in the direction indicated by the dashed line. Although not shown, the display device 100 may also have a light-emitting element (IR) formed by the same process as the light-emitting element 61.

[0424] An insulating layer 261 is provided covering the transistor 310, and a conductive layer 251 is provided on the insulating layer 261. An insulating layer 262 is provided covering the conductive layer 251, and a conductive layer 252 is provided on the insulating layer 262. The conductive layers 251 and 252 each function as wiring. An insulating layer 263 and an insulating layer 332 are provided covering the conductive layer 252, and a transistor 320 is provided on the insulating layer 332. An insulating layer 265 is provided covering the transistor 320, and a capacitor 240 is provided on the insulating layer 265. The capacitor 240 and the transistor 320 are electrically connected by a plug 274. Two or more conductive layers may be used in the plug 274.

[0425] Transistor 320 can be used as a transistor constituting a pixel circuit or a transistor constituting a memory cell. Transistor 310 can be used as a transistor constituting a memory cell, a transistor constituting a drive circuit for driving the pixel circuit, or a transistor constituting an arithmetic circuit. Transistors 310 and 320 can also be used as transistors constituting various circuits such as arithmetic circuits or memory circuits.

[0426] The transistor 310 is a transistor having a channel-forming region in the substrate 301. The substrate 301 can be a semiconductor substrate such as a single-crystal silicon substrate. The transistor 310 comprises a portion of the substrate 301, a conductive layer 311, a low-resistance region 312, an insulating layer 313, and an insulating layer 314. The conductive layer 311 functions as a gate electrode. The insulating layer 313 is located between the substrate 301 and the conductive layer 311 and functions as a gate insulating layer. The low-resistance region 312 is a region of the substrate 301 doped with impurities and functions as either a source or a drain. The insulating layer 314 covers the side surface of the conductive layer 311 and functions as an insulating layer.

[0427] Furthermore, an element isolation layer 315 is provided between two adjacent transistors 310 so as to be embedded in the substrate 301.

[0428] A silicon transistor may also be used as transistor 310.

[0429] Transistor 320 is an OS transistor in which an oxide semiconductor is applied to the semiconductor layer where the channel is formed.

[0430] The transistor 320 has a semiconductor layer 321, an insulating layer 323, a conductive layer 324, a pair of conductive layers 325, an insulating layer 326, and a conductive layer 327.

[0431] The insulating layer 332 functions as a barrier layer that prevents impurities such as water or hydrogen from diffusing from the substrate 301 side to the transistor 320, and prevents oxygen from detaching from the semiconductor layer 321 to the insulating layer 332 side. As the insulating layer 332, for example, a film that is less susceptible to hydrogen or oxygen diffusion than a silicon oxide film can be used, such as an aluminum oxide film, a hafnium oxide film, or a silicon nitride film.

[0432] A conductive layer 327 is provided on an insulating layer 332, and an insulating layer 326 is provided covering the conductive layer 327. The conductive layer 327 functions as the first gate electrode of the transistor 320, and a portion of the insulating layer 326 functions as the first gate insulating layer. It is preferable to use an oxide insulating film, such as a silicon oxide film, for at least the portion of the insulating layer 326 that is in contact with the semiconductor layer 321. It is preferable that the upper surface of the insulating layer 326 is flattened.

[0433] The conductive layer 327 may be a single layer or a stack of two or more conductive layers. When the conductive layer 327 is configured as two stacked conductive layers, it is preferable that the conductive layer in contact with the insulating layer 326 is made of a conductive material that has the function of suppressing the diffusion of impurities such as water or hydrogen or oxygen. Examples of such conductive materials include titanium, titanium nitride, tantalum, tantalum nitride, ruthenium, and ruthenium oxide. This configuration suppresses the diffusion of impurities such as water or hydrogen into the semiconductor layer 321.

[0434] The insulating layer 326 may be an inorganic insulating film, either as a single layer or by stacking two or more layers. When two or more inorganic insulating films are stacked as the insulating layer 326, it is preferable that one of the inorganic insulating films of the insulating layer 326 functions as a barrier layer to prevent impurities such as water or hydrogen from diffusing from the substrate 301 to the transistor 320.

[0435] The semiconductor layer 321 is provided on the insulating layer 326. The semiconductor layer 321 preferably has an oxide semiconductor. The oxide semiconductor preferably uses a metal oxide containing at least one of indium, element M, and zinc. Element M can be one or more selected from, for example, aluminum, gallium, yttrium, copper, vanadium, beryllium, boron, titanium, iron, nickel, germanium, zirconium, molybdenum, lanthanum, cerium, neodymium, hafnium, tantalum, tungsten, or magnesium. An OS transistor using such a metal oxide in the channel formation region has the characteristic of having a very low off-current. Therefore, using an OS transistor as a transistor in a pixel circuit is preferable because it can retain analog data written to the pixel circuit for a long period of time. Similarly, using an OS transistor as a transistor in a memory cell is preferable because it can retain analog data written to the memory cell for a long period of time.

[0436] A pair of conductive layers 325 are provided in contact with the semiconductor layer 321 and function as source and drain electrodes.

[0437] Furthermore, an insulating layer 328 is provided covering the top and side surfaces of the pair of conductive layers 325, as well as the side surfaces of the semiconductor layer 321, and an insulating layer 264 is provided on the insulating layer 328. The insulating layer 328 functions as a barrier layer to prevent impurities such as water or hydrogen from diffusing into the semiconductor layer 321 from the insulating layer 264, etc., and to prevent oxygen from detaching from the semiconductor layer 321. As the insulating layer 328, an insulating film similar to that of the insulating layer 332 can be used.

[0438] An opening is provided in the insulating layer 328 and the insulating layer 264 that reaches the semiconductor layer 321. Inside this opening, the insulating layer 323 and the conductive layer 324 are embedded, in contact with the sides of the insulating layer 264, the insulating layer 328, and the conductive layer 325, as well as the upper surface of the semiconductor layer 321. The conductive layer 324 functions as a second gate electrode, and the insulating layer 323 functions as a second gate insulating layer.

[0439] As the insulating layer 323, for example, an inorganic insulating film such as a silicon oxide film or a silicon oxide nitride film can be used. Note that the insulating layer 323 is not limited to a single layer of inorganic insulating film, but may be used by laminating two or more inorganic insulating films. For example, an aluminum oxide film, a hafnium oxide film, a silicon nitride film, etc. may be provided as a single layer or in layers on the side in contact with the conductive layer 324. This can suppress oxidation of the conductive layer 324. Alternatively, for example, an aluminum oxide film or a hafnium oxide film may be provided on the side in contact with the insulating layer 264, insulating layer 328, and conductive layer 325. This can suppress oxygen desorption from the semiconductor layer 321, excessive oxygen supply to the semiconductor layer 321, oxidation of the conductive layer 325, etc.

[0440] The upper surfaces of the conductive layer 324, the insulating layer 323, and the insulating layer 264 are flattened so that their heights are roughly the same, and the insulating layers 329 and 265 are provided covering them.

[0441] Furthermore, it is preferable that the conductive layer 327 and the conductive layer 324 are superimposed on the outer side of the side surface in the channel width direction of the semiconductor layer 321, with an insulator in between. With this configuration, the channel formation region of the semiconductor layer 321 can be electrically surrounded by the electric field of the conductive layer 327, which functions as the first gate electrode, and the electric field of the conductive layer 324, which functions as the second gate electrode. In this specification, a transistor structure in which the channel formation region is electrically surrounded by the electric fields of the first gate electrode and the second gate electrode is called a surrounded channel (S-channel) structure.

[0442] In this specification, an S-channel transistor refers to a transistor structure in which the channel formation region is electrically surrounded by the electric fields of one and the other of a pair of gate electrodes. Furthermore, the S-channel structure disclosed in this specification is different from the Fin-type structure and the Planar-type structure. By adopting an S-channel structure, it is possible to create a transistor that has improved resistance to short-channel effects, or in other words, a transistor in which short-channel effects are less likely to occur.

[0443] By setting transistor 320 to normally off and adopting the above-described S-channel structure, the channel formation region can be electrically surrounded. Therefore, transistor 320 can also be considered as having a GAA (Gate All Around) structure or an LGAA (Lateral Gate All Around) structure. By making transistor 320 an S-channel structure, GAA structure, or LGAA structure, the channel formation region formed at or near the interface between the semiconductor layer 321 and the gate insulating film can be made to encompass the entire bulk of the semiconductor layer 321. Consequently, it becomes possible to improve the current density flowing through the transistor, which can be expected to improve the on-current of the transistor or increase the field-effect mobility of the transistor.

[0444] Insulating layers 264 and 265 function as interlayer insulating layers. Insulating layer 329 functions as a barrier layer to prevent impurities such as water or hydrogen from diffusing from insulating layer 265 to transistor 320. As insulating layer 329, an insulating film similar to that used for insulating layers 328 and 332 can be used.

[0445] A plug 274, which is electrically connected to one of the pair of conductive layers 325, is provided so as to be embedded in the insulating layer 265, insulating layer 329, and insulating layer 264. The plug 274 is configured such that the conductive layers are provided as a single layer or a laminated structure of two or more layers. When the plug 274 is configured as a laminated structure of two conductive layers, it is preferable to use a conductive material that does not easily allow hydrogen and oxygen to diffuse as the conductive layer that covers the sides of the openings of the insulating layer 265, insulating layer 329, insulating layer 264, and insulating layer 328, and a part of the upper surface of the conductive layer 325. This configuration makes it possible to suppress the mixing of impurities such as water or hydrogen from the insulating layer 264, etc., into the semiconductor layer 321 through the plug 274. It also makes it possible to suppress the absorption of oxygen contained in the insulating layer 264 into the plug 274.

[0446] Furthermore, an insulating layer 275 is provided in contact with the side surface of the plug 274. In other words, the insulating layer 275 may be provided in contact with the insulating layer 265, the insulating layer 329, and the inner wall of the opening of the insulating layer 264, and the plug 274 may be provided in contact with the side surface of the insulating layer 275 and a part of the upper surface of the conductive layer 325. Note that the insulating layer 275 may not be provided in some cases.

[0447] Capacitor 240 has a conductive layer 241, a conductive layer 245, and an insulating layer 243 located between them. The conductive layer 241 functions as one electrode of the capacitor 240, the conductive layer 245 functions as the other electrode of the capacitor 240, and the insulating layer 243 functions as the dielectric of the capacitor 240.

[0448] The conductive layer 245 is provided on the insulating layer 265 and embedded in the insulating layer 254. The conductive layer 245 is electrically connected to either the source or drain of the transistor 320 by a plug 274 embedded in the insulating layer 265. The insulating layer 243 is provided covering the conductive layer 245. The conductive layer 245 is provided in a region that overlaps with the conductive layer 241 via the insulating layer 243.

[0449] An insulating layer 255 is provided covering the capacitance 240, and a light-emitting element 61 and a light-receiving element 62 are provided on the insulating layer 255. A protective layer 91 is provided on the light-emitting element 61 and the light-receiving element 62, and a substrate 420 is bonded to the upper surface of the protective layer 91 by a resin layer 419. The resin layer 419 and the substrate 420 can be made of materials that are transparent to visible light and infrared light, respectively.

[0450] The pixel electrode 84 of the light-emitting element 61 and the pixel electrode 84PD of the light-receiving element 62 are electrically connected to either the source or drain of the transistor 320 by plugs 256 and 274. The transistor 320 and the light-emitting element 61 can be electrically connected via plug 256.

[0451] This configuration allows for the placement of OS transistors constituting pixel circuits and memory cells directly beneath the light-receiving and light-emitting elements, as well as the placement of drive circuits and arithmetic circuits, making it possible to miniaturize a high-performance display device.

[0452] Figure 26 shows a configuration in which the display device 100 has transistors 310 and 320 stacked on top of each other. However, the configuration of the display device 100 is not limited to this, and the display device 100 may have a configuration that includes transistor 310 but does not include transistor 320, or a configuration that does not include transistor 310 but includes transistor 320, or a configuration in which multiple transistors 320 are stacked on top of each other.

[0453] Furthermore, when a silicon substrate is used as the substrate 301, a photodiode with a photoelectric conversion layer may be formed on the silicon substrate, and this photodiode can be used as the light-receiving element 159 described in Embodiment 1. In other words, the light-receiving element 159 described in Embodiment 1 may be formed on a silicon substrate. In this case, a transistor 310 may or may not be formed.

[0454] The configuration examples illustrated in this embodiment, and the corresponding drawings, etc., can be appropriately combined with other configuration examples or drawings, etc., at least in part.

[0455] This embodiment can be implemented in appropriate combination with other embodiments described herein, at least in part. [Explanation of symbols]

[0456] 100 display device 100L display device 100R display device 120L drive circuit section 120R Drive Circuit Section 121L Image Processing Unit 121R Image Processing Unit 122L Frame Memory 122R Frame Memory 150 Electronic equipment 151L optical system 151R optical system 153 Motion detection unit 154 Audio 155 Camera 156 Control Unit 157 Communications Department 158 batteries 159L photodetector 159R photodetector 160 Infrared light source

Claims

1. An electronic device having eye-tracking capabilities, The electronic device comprises a first display device, a second display device, an infrared light source, an optical system, and a drive mechanism. The first display device comprises a first display element and a first light-receiving element, The second display device comprises a second display element and a second light-receiving element, The infrared light source has the function of emitting infrared light, The optical system includes a first optical element located in the optical path through which an image from the first display element is incident on one of the right and left eyeballs, and a second optical element located in the optical path through which an image from the second display element is incident on the other of the right and left eyeballs. The drive mechanism has the function of moving or rotating at least one of the first optical element and the second optical element so that the reflected infrared light from one of the right eyeball and the left eyeball reflects off the first optical element and the second optical element in that order and enters the first light-receiving element; and the function of moving or rotating at least one of the first optical element and the second optical element so that the reflected infrared light from the other of the right eyeball and the left eyeball reflects off the second optical element and the first optical element in that order and enters the second light-receiving element. electronic equipment.

2. An electronic device having an eye-tracking function, The electronic device comprises a first display device, a second display device, an infrared light source, an optical system, and a drive mechanism. The first display device comprises a first display element and a first light-receiving element, The second display device comprises a second display element and a second light-receiving element, The infrared light source has the function of emitting infrared light, The optical system includes a first optical element and a third optical element located in the optical path through which an image from the first display element is incident on one of the right and left eyeballs, and a second optical element and a fourth optical element located in the optical path through which an image from the second display element is incident on the other of the right and left eyeballs. The drive mechanism has the function of moving or rotating at least one of the first to fourth optical elements so that the infrared light reflected from one of the right and left eyeballs reflects off the first optical element, the second optical element, the fourth optical element, and the third optical element in that order and enters the first light-receiving element; and the drive mechanism has the function of moving or rotating at least one of the first to fourth optical elements so that the infrared light reflected from the other of the right and left eyeballs reflects off the second optical element, the first optical element, the third optical element, and the fourth optical element in that order and enters the second light-receiving element. electronic equipment.

3. In claim 1 or 2, The second optical element is an electronic device having an IR cut filter.

4. In claim 1 or 2, The infrared light source comprises a first infrared light source and a second infrared light source. The first infrared light source has the function of emitting infrared light towards one of the right and left eyeballs, The second infrared light source has the function of emitting infrared light to the other eye, the right eyeball and the left eyeball. electronic equipment.

Images