Display apparatus and near-eye display device

Abstract

The invention relates to a display apparatus and a near-to-eye display device. The device comprises a processing control chip used for sending a control instruction; the display sensing chip comprises a display area and a sensing area which are alternately arranged, and the display area is used for displaying virtual reality / augmented reality content; the invisible light emitter is installed on the peripheral side of the display sensing chip and used for emitting invisible light according to the control instruction; the sensing area is used for receiving reflected invisible reflected light and capturing an eye image of an operation body; the processing control chip is also used for analyzing the received invisible reflected light and the received eye image, and determining an adjustment behavior and a target position of the display area corresponding to the adjustment behavior; the display area is further used for displaying the virtual reality / augmented reality content at the target position according to the adjustment behavior. The size of the near-to-eye display device is reduced, and the portability of the near-to-eye display device is improved.

Description

Technical Field

[0001] The present invention relates to the field of near-eye display technology, and in particular to a display device and a near-eye display apparatus. Background Art

[0002] Eye tracking technology refers to the process of measuring and recording eye movements to determine a person's gaze point (i.e. where a person is looking), gaze direction, or the movement or position of the eye relative to the head. With the development of science and technology, eye tracking technology is closely integrated with current augmented reality (AR) / virtual reality (VR) applications, and usually uses non-contact optical methods to measure the position of the eye or the gaze angle.

[0003] Traditional eye tracking technology in AR / VR devices usually uses infrared light to illuminate the eyes and calculates the direction of the pupil by reflecting light. However, due to the presence of optical devices such as thermal mirrors and lenses, the device is relatively large. In addition, users need to interact with the display screen through a handheld controller, which greatly limits the user's freedom of movement, resulting in a low degree of natural interaction and a poor user experience. In addition, the infrared light information collected by traditional eye tracking technology is limited, resulting in low accuracy in the final eye tracking. Therefore, it is necessary to provide a display device and a near-eye display device. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a display device and a near-eye display apparatus, which improve the problem that the display device in the prior art is not very portable.

[0005] To achieve the above-mentioned purpose and other related purposes, the present invention provides a display device, comprising: a processing control chip, used to send control instructions; a display sensor chip, including a display area and a sensing area alternately arranged, the display area being used to display virtual reality / augmented reality content; a non-visible light emitter, installed on the outer peripheral side of the display sensor chip, for emitting non-visible light according to the control instructions; the sensing area being used to receive reflected non-visible reflected light and capture an eye image of an operating body; the processing control chip being further used to analyze the received non-visible reflected light and the eye image, and determine an adjustment behavior and a target position of the display area corresponding to the adjustment behavior; the display area being further used to display the virtual reality / augmented reality content at the target position according to the adjustment behavior.

[0006] In one embodiment of the present invention, the display area is an RGB pixel unit group, and a sensing area is disposed between adjacent RGB pixel unit groups.

[0007] In one embodiment of the present invention, the display area is an RGB pixel array, and a sensing area is disposed between adjacent RGB pixel arrays; wherein the RGB pixel array includes at least two RGB pixel unit groups arranged continuously.

[0008] In one embodiment of the present invention, there are a plurality of non-visible light emitters, and the plurality of non-visible light emitters are evenly distributed around the outer periphery of the display sensor chip.

[0009] In one embodiment of the present invention, the sensing area is a CIS image sensor.

[0010] In one embodiment of the present invention, the sensing area includes at least a first drive control layer and a CIS pixel array, wherein: the CIS pixel array is used to receive the invisible reflected light reflected by the eyes of the operating body, and to capture the visible light constituting the eye image, and to convert the optical signals of the invisible reflected light and the visible light into electrical signals; the first drive control layer is used to process the converted electrical signals to generate a form suitable for processing by the processing control chip.

[0011] In one embodiment of the present invention, the display area is Micro-OLED.

[0012] In one embodiment of the present invention, the display area includes at least a second driving control layer and a pixel display layer, wherein: the second driving control layer is used to receive the adjustment behavior and the target position, and convert the adjustment behavior and the target position to generate a driving voltage to drive the Micro-OLED pixel display layer; the Micro-OLED pixel display layer is arranged on the second driving control layer, and is used to update the display information of the virtual reality / augmented reality content at the target position under the action of the driving voltage.

[0013] In one embodiment of the present invention, the processing and control chip includes: a behavior analysis unit, used to extract and analyze the motion features of the eye image to determine the corresponding adjustment behavior; a pupil positioning unit, used to analyze the angle between the corneal reflected light and the pupil reflected light of the non-visible reflected light based on the pupil-corneal reflection vector method, determine the gaze position of the pupil on the display based on the angle, and use it as the target position of the display area.

[0014] In one embodiment of the present invention, the display device further includes a power supply unit, and the power supply unit is electrically connected to the display sensor chip, the processing control chip and the non-visible light emitter respectively.

[0015] In an embodiment of the present invention, the power supply unit is a power chip.

[0016] In one embodiment of the present invention, the non-visible light is infrared light or near-infrared light.

[0017] In one embodiment of the present invention, a near-eye display device is also provided, comprising: a bracket and a display device as described in any one of the above items, wherein the display device is arranged on the bracket and is used to track eye postures for near-eye display.

[0018] As described above, a display device and a near-eye display device of the present invention have the following beneficial effects: the processing control chip sends instructions to control the non-visible light emitter to emit non-visible light to the eye, and the sensing area on the display sensor chip captures the light and image reflected by the eye, and transmits the data to the processing control chip for analysis, thereby determining the target position of the operator in the display area and the corresponding adjustment behavior. The display area dynamically displays the content of virtual reality or augmented reality at the target position of the gaze according to the adjustment behavior generated by the processing control chip. The present invention not only greatly increases the number of sensing areas and optimizes the final imaging effect by alternately arranging the display area and the sensing area on the same substrate, but also makes the display area and the sensing area closely integrated, reduces the number of ports, and thus improves the data processing rate and accuracy. On the other hand, this method effectively reduces the number of connection ports between components and reduces the delay of signal transmission, so that the display area can respond to the eye movement data collected by the sensing area in real time, quickly adjust the display content, and ensure that the user's line of sight changes can be immediately fed back to the screen. In addition, this chip integration method greatly reduces the space occupancy rate of the device, realizes the lightweight design of the device, optimizes the user's wearing experience, and significantly improves the user's immersion and convenience of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the structure of a traditional display device;

[0020] Figure 2 Shown is a side view of a display device provided by an embodiment of the present invention;

[0021] Figure 3 Shown is a front view of a display device provided by an embodiment of the present invention;

[0022] Figure 4 Shown is a structural diagram of the corresponding functional area of ​​the integrated display sensor chip of the present invention;

[0023] Figure 5 Shown is a structural block diagram of the display device of the present invention;

[0024] Figure 6 Shown are diagrams showing different pupil positions in the eye.

[0025] Component number description:

[0026] 100. Processing control chip; 200. Display sensor chip; 210. Display area; 220. Sensing area; 300. Non-visible light emitter; 400. Power supply unit; 500. Eyes. DETAILED DESCRIPTION

[0027] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0028] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0029] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0030] In order to better understand and illustrate the method provided in the embodiments of the present application, some technical terms involved in the embodiments of the present application are first explained and illustrated below.

[0031] The design principle of eye tracking is to use an infrared transmitter to emit a beam of infrared light to illuminate the eyes. The infrared light reflected from the cornea and retina is collected by the camera. Due to the physiological structure and physical properties of the eyeball, under the premise that the relative position of the light source and the head remains unchanged, the light spot formed by the corneal reflection will not move, and the direction of the light reflected on the retina indicates the direction of the pupil (the light source enters from the pupil, and the retinal reflected light is emitted from the pupil). Finally, the direction of the eye movement can be calculated based on the angle between the cornea and the pupil reflected light. When the subject is staring at a fixed target, no matter how the head moves, the positional relationship between the pupil center and the corneal reflection will not change. Even if the subject is moving, the eye tracker can reflect that she is looking at the same position. Therefore, when the light source and the head position are relatively stable, the light reflected from the cornea and retina is collected, and their relative position relationship is analyzed, the data of the pupil direction can be obtained. This is the principle of the pupil-corneal reflection eye tracker.

[0032] In one known type of optical eye tracking technology, light (typically light in the infrared region) is reflected from the eye and sensed by a camera. The information sensed by the camera is then analyzed to extract the gaze direction or position of the pupil from the changes in the reflection. Video-based eye trackers sometimes use corneal reflections or pupil centers as features to track over time. A camera-based eye tracking system may include a rear-mounted camera attached to a bracket and pointed (directly or indirectly) at the user's eyes as a means of detecting the position of the user's eyes. The digital data generated by the camera is sent via wired or wireless means to an external device such as a computer (or alternatively, to a computer resource located on itself) for processing and analysis. Computer software in such a system runs an eye tracking algorithm known to those of ordinary skill in the art to detect the position of one or both of the user's eyes.

[0033] like Figure 1 As shown, eye tracking includes one or two small display areas with lenses and semi-transparent mirrors (i.e., "hot mirrors") embedded in various form factors, such as helmets, glasses (also called data glasses), or goggles. Hot mirrors provide one possible design approach for eye tracking and allow cameras or other eye tracking sensors (such as eye tracking cameras) to obtain a good view of the eye being tracked. Some hot mirrors reflect infrared ("IR") radiation and are transparent to visible light. The hot mirror in some eye tracking applications is tilted in front of the eye and allows the IR camera or other eye tracking sensor to obtain a reflected image of the eye while having a transparent view on the display. Figure 1 The display shows a traditional display device, where an infrared transmitter emits infrared light to the user's eyeball, and the light reflected from the eyeball is transmitted to the eye tracking camera through a lens and a thermal mirror, and the corresponding content is displayed on the display after being analyzed by the controller. This optical eye tracking method is widely used for gaze tracking. Such a tracker may require a relatively high-resolution camera that uses image processing and pattern recognition devices to capture at a high frame rate to track reflected light or known eye structures (e.g., iris or pupil).

[0034] Based on the above, traditional display devices monitor eye movements by reflecting and capturing light through optical components such as lenses and thermal mirrors. However, these components greatly increase the weight of the entire VR / AR glasses, making the entire device bulky and not conducive to long-term wear by users. In addition, traditional devices usually require users to hold controllers to assist in operation and target selection, but this interaction method is not natural enough. When using it, users need to switch back and forth between line of sight and hand operation, and cannot achieve natural line of sight control, resulting in poor immersion for users.

[0035] See also Figure 2Based on the above problems, the present invention provides a display device, in which a processing control chip sends instructions to control a non-visible light emitter to emit non-visible light to the eyes, and a sensing area on the display sensor chip captures the light and image reflected by the eyes, and transmits the data to the processing control chip for analysis, thereby determining the target position of the operator in the display area and the corresponding adjustment behavior. The display area dynamically displays the content of virtual reality or augmented reality at the target position of the gaze according to the adjustment behavior generated by the processing control chip. The present invention not only greatly increases the number of sensing areas and optimizes the final imaging effect by alternately arranging display areas and sensing areas on the same substrate, but also closely integrates the display area with the sensing area, reduces the number of ports, and thus improves the data processing rate and accuracy. On the other hand, this method effectively reduces the number of connection ports between components and reduces the delay in signal transmission, so that the display area can respond to the eye movement data collected by the sensing area in real time, quickly adjust the display content, and ensure that the user's line of sight changes can be immediately fed back to the screen. Furthermore, all key components of the present invention are implemented in the form of chips. The integrated design of this chip reduces the physical connection between modules, realizes the precise control of non-visible light and the real-time processing of eye movement data, thereby ensuring that the eye tracking has good response speed and accuracy. In addition, this chip integration method greatly reduces the space occupancy rate of the device, realizes the lightweight design of the device, optimizes the user's wearing experience, and significantly improves the user's immersion and ease of use. Furthermore, this method of the present invention can determine the target position and adjustment behavior through eye tracking, so as to realize the selection of the target at the target position on the display screen, and make it present according to the adjustment behavior, thereby eliminating the manual remote control operation action, and therefore has a milestone significance for VR glasses or AR glasses.

[0036] See also Figure 2 The display device includes: a processing control chip 100, a display sensor chip 200 and a non-visible light emitter 300. The processing control chip 100 is used to send control instructions; the display sensor chip 200 includes a display area 210 and a sensing area 220 that are alternately arranged, and the display area 210 is used to display the content of virtual reality / augmented reality; the non-visible light emitter 300 is installed on the outer peripheral side of the display sensor chip 200, and is used to emit non-visible light according to the control instructions; the sensing area 220 is used to receive the reflected non-visible reflected light and capture the eye image of the operating body; the processing control chip 100 is also used to analyze the received non-visible reflected light and the eye image, determine the adjustment behavior and the target position of the display area 210 corresponding to the adjustment behavior; the display area 210 is also used to display the content of virtual reality / augmented reality at the target position according to the adjustment behavior.

[0037] See also Figure 2 , the processing control chip 100 serves as the control center of the entire display device, and is responsible for sending control instructions to the non-visible light emitter 300, and receiving eye movement data such as non-visible reflected light and eye images reflected by the eyes 500 of the operator sent by the sensing area 220. After analyzing and processing these eye movement data, the gaze position of the operator and the specific adjustment behavior can be determined. The processing control chip 100 can also determine its target position in the display area 210 according to the gaze position based on the data fed back by the display area 210. Thereby, the display area 210 can dynamically adjust the display content according to the corresponding adjustment behavior according to the change of the operator's line of sight. Specifically, after the operator wears the near-eye display device equipped with the display device, the processing control chip 100 will send control instructions to the non-visible light emitter 300 at a preset time interval, so that the non-visible light emitter 300 emits non-visible light to the eyes 500 of the operator. When the sensing area 220 receives the non-visible reflected light and the eye image reflected from the eye, the eye movement data is sent to the processing control chip 100, which analyzes and processes the eye movement data, determines the gaze direction of the eye 500 based on the non-visible reflected light, and determines the specific eye movement based on the eye image, thereby determining the adjustment behavior. Exemplarily, when the operator gazes at a certain position of the display area 210, the processing control chip 100 can command the display area 210 to enlarge or focus the content at that position according to the gaze position. Among them, the processing control chip 100 is a CPU chip.

[0038] See also Figure 2 and Figure 3On the same silicon-based substrate, a display area 210 and a sensing area 220 are alternately arranged according to a preset area to form an integrated chip, namely, a display sensor chip 200. Among them, the display area 210 is used to display virtual reality or augmented reality content, and the sensing area 220 is responsible for receiving the non-visible light reflected by the eye and obtaining the eye image, and sending the acquired eye movement data to the processing control chip 100. After the processing control chip 100 analyzes and processes the data sent by the display area 210, it sends the generated target position and adjustment behavior to the display area 210, and the display area 210 dynamically adjusts the displayed content at the target position according to the adjustment behavior. In the present invention, the sensing area 220 and the display area 210 are alternately arranged on the same silicon-based substrate, and there is no need for complex internal connections between the display area 210 and the sensing area 220, which greatly reduces the wiring requirements, thereby reducing signal loss, not only allowing the processing control chip 100 to receive high-quality data, but also greatly improving the overall response rate, so that the line of sight changes of the operating body can be more accurately and quickly fed back to the display area 210 for adjustment, providing a good user experience. On the other hand, this method of arranging the display area 210 and the sensing area 220 at intervals on the same silicon-based substrate increases the number of sensing areas 220, thereby improving the collection coverage and accuracy of eye movement data. And because the sensing areas 220 are in different positions, the reflected light and image information of the eyes can be captured from multiple different angles, greatly reducing the monitoring blind spots. In addition, the present invention integrates the display area 210 and the sensing area 220 on the same chip, which greatly saves device space, reduces the overall weight and volume of the device, and makes the overall device more compact and portable. Furthermore, this integrated chip enables data transmission between the display area 210 and the sensing area 220 to be completed through internal integration, greatly reducing the demand for external interfaces and simplifying the data processing process to improve the processing rate.

[0039] See also Figure 2 and Figure 3The non-visible light emitter 300 is arranged on the outer peripheral side of the display sensor chip 200, and is used to emit non-visible light to the eyes 500 of the operator under the action of the control command. When the non-visible light irradiates the eyes 500 of the operator, a part of the light is reflected on the cornea to form a stable corneal reflection light spot, and the other part of the light passes through the pupil into the eye, and is emitted from the pupil after being reflected on the retina. These two kinds of non-visible reflected light will be captured by the sensing area 220 and transmitted to the processing control chip 100. Based on the relative position change of the corneal reflection light spot and the pupil reflection light, the processing control chip 100 uses the pupil-corneal reflection vector method to calculate the real-time gaze point position and gaze direction of the eye 500, so as to accurately track the line of sight of the operator. Furthermore, since the human eye cannot perceive the existence of infrared light and near-infrared light, it will not interfere with the user's visual experience. Therefore, in one embodiment of the present invention, the non-visible light is infrared light or near-infrared light.

[0040] See also Figure 2 and Figure 3 In one embodiment of the present invention, the display area 210 is an RGB pixel unit group, and a sensing area 220 is arranged between adjacent RGB pixel unit groups. The pixel unit group is composed of a plurality of pixel units arranged vertically, and each pixel unit includes three sub-pixels of red, green and blue. Specifically, the processing control chip 100 controls the brightness combination of each sub-pixel, and the RGB array can present a variety of colors, thereby achieving high-resolution color display. The sensing area 220 is arranged between two adjacent RGB pixel unit groups, and is used to receive non-visible light reflected by the eye and obtain eye image data, so as to achieve accurate tracking of the line of sight. By alternately arranging each RGB pixel unit group and the sensing area 220 on the same substrate, the signal transmission path is reduced and the data delay is reduced. On the other hand, this method can arrange the sensing area 220 and the display area 210 closely, simplify the circuit layout, and realize the lightweight design of the device. And because the number of sensing areas 220 is greatly increased, the overall imaging effect is clearer. Furthermore, in another embodiment of the present invention, the display area 210 is an RGB pixel array, and a sensing area 220 is arranged between adjacent RGB pixel arrays; wherein the RGB pixel array includes at least two RGB pixel unit groups arranged in series. Relative to setting the sensing area 220 between two adjacent RGB pixel unit groups, by setting the sensing area 220 between the RGB pixel arrays, the density of pixels in each array is effectively improved, so that the final presented image quality is clearer and more realistic. Although the number of sensing areas 220 is reduced, since multiple sensing areas 220 can be set on the same substrate, the coverage of data collection can still be guaranteed, and will not affect the subsequent content presentation. Those skilled in the art can choose any method based on actual needs, and there is no limitation here.

[0041] It is understandable that, whether the sensing area 220 is set between the RGB pixel array or between the RGB pixel unit groups, the display sensing chip has multiple sensing areas 220. In this way, the display and eye tracking functions can work together on the same chip, and the sensing area 220 can receive light reflected from the eyes of the operator from different angles. Through this more comprehensive data collection, the visual blind spot phenomenon that may be caused by a single perspective is effectively avoided. In addition, this method of alternately setting multiple sensing areas 220 and display areas 210 on the same substrate saves the actual required space and weight in space, and increases the actual AR / VR experience.

[0042] See also Figure 2 and Figure 5 In order to improve the comprehensiveness of data collection and take into account the actual needs of AR / VR glasses space and weight, the present invention removes the traditional lens and thermal mirror, thereby greatly reducing the space occupied by the entire device. In one embodiment of the present invention, there are multiple non-visible light emitters 300, and the multiple non-visible light emitters 300 are evenly distributed around the outer periphery of the display sensor chip 200. Multiple non-visible light emitters 300 are evenly arranged around the outer side of the display sensor chip 200, so that non-visible light can be emitted from different angles toward the eyes of the operator, which is convenient for all-round reflection of non-visible light from the cornea and pupil. Since the sensing area 220 and the display area 210 are alternately arranged, the non-visible light emitter 300 is arranged on the outer periphery of the display sensor chip 200, thereby realizing the alternating arrangement of the non-visible light emitter 300 and the sensing area 220. This method can ensure that the sensing area 220 can receive the non-visible reflected light reflected by the eyes 500 of the operator from different angles, so that the subtle movements and position changes of the eyes 500 can be captured more accurately, effectively improving the problem of blind spots that are prone to occur in single-angle monitoring. This method of alternately arranging multiple non-visible light emitters 300 and multiple sensing areas 220 can collect data from multiple perspectives, ensure the stability and reliability of the data, and provide more and more accurate data to analyze the rotation position of the eye 500. Figure 6 , which shows schematic diagrams of pupils at different positions in the eye 500, namely, the pupil is on the right side above the eye 500, in the middle above the eye 500, on the left side above the eye 500, and the pupil is on the right side below the eye 500, in the middle below the eye 500, and on the left side below the eye 500. It can be seen that the user's gaze point can be accurately determined by the change in the position of the pupil.

[0043] See also Figure 5In one embodiment of the present invention, the sensing area 220 is a CIS image sensor. The high-precision CIS image sensor can collect non-visible light information such as infrared light or near-infrared light and facial features in multiple viewing angles and directions, thereby providing more data support for the processing control chip 100. Unlike traditional image collectors, the present invention uses a CIS image sensor as a pixel image collector. Since the CIS image sensor has a strong photosensitivity, it can better capture eye images and non-visible reflected light, which not only greatly improves the clarity and accuracy of pixel collection, but also effectively reduces the installation space required on the main bracket, making the overall device more lightweight. Through the integration of image sensors and chip process optimization, the integrated chip can achieve efficient and accurate eye 500 tracking, and effectively improve the lightweight degree of AR / VR glasses to improve the comfort of users wearing them.

[0044] See also Figures 2 to 4 In one embodiment of the present invention, the sensing area 220 includes at least a first driving control layer and a CIS pixel array, wherein: the CIS pixel array is used to receive the non-visible reflected light reflected by the eye 500 of the operator, and to capture the visible light constituting the eye image, and to convert the optical signals of the non-visible reflected light and the visible light into electrical signals; the first driving control layer is used to process the converted electrical signals to generate a form suitable for processing by the processing control chip. Specifically, the CIS pixel array receives the visible light reflected by the eye and its surroundings of the operator to obtain the eye and its surrounding images of the operator, and extracts the eye area image from them through the built-in image processing algorithm. On the other hand, after the non-visible light emitted by the non-visible light emitter 300 irradiates the eye 500 of the operator, the non-visible light generates different reflections on the cornea and retina, forming corneal reflection spots and retinal reflection spots respectively. The CIS pixel array captures these two types of non-visible light reflections and converts the received optical signals into electrical signals. The first driving circuit layer is a CIS CMOS circuit, which converts the original data from the CIS pixel array into a format suitable for the processing and control chip 100, so that the processing and control chip 100 can effectively interpret and use the signal data.

[0045] See also Figure 2 and Figure 3In one embodiment of the present invention, the display area 210 is Micro-OLED. The display area 210 adopts a silicon-based CMOS technology driver chip, and a Micro-OLED pixel display layer is made on the upper surface of the silicon-based CMOS technology driver chip, so that the bottom CMOS drives the upper Micro-OLED pixels. This design not only improves the resolution of the pixels, but also greatly improves the brightness uniformity and brightness, making the visual effect clearer and brighter. Specifically, the silicon-based Micro-OLED chip realizes the combination of pixels and silicon-based CMOS chips, and has high resolution and clarity, so that a clear visual picture can be presented. The display area 210 receives the video signal (MIPI / RGB / LVDS and other formats) transmitted by the processing control chip 100, and is driven by the silicon-based Micro-OLED and displayed in the Micro-OLED pixel area.

[0046] See also Figures 2 to 4 In one embodiment of the present invention, the display area 210 at least includes a second drive control layer and a pixel display layer, wherein: the second drive control layer is used to receive the adjustment behavior and the target position, and convert the adjustment behavior and the target position to generate a driving voltage to drive the Micro-OLED pixel display layer; the Micro-OLED pixel display layer is arranged on the second drive control layer, and is used to update the display information of the virtual reality / augmented reality content at the target position under the action of the driving voltage. The display area 210 adopts a silicon-based Micro-OLED chip, including a second drive circuit layer and a Micro-OLED pixel display layer located thereon. Specifically, the second drive circuit layer is a CMOS micro-OLED drive circuit, which receives and processes the adjustment behavior and the corresponding target position sent by the control chip 100, and then performs data conversion on the received information, and generates a driving voltage signal suitable for driving the Micro-OLED pixel display layer through data format conversion. The Micro-OLED pixel display layer is composed of a large number of micro OLED pixels, and the OLED pixels include three sub-pixels of red, green and blue arranged in sequence, and these pixels can emit light independently under the drive of voltage. When the voltage signal provided by the second driving circuit layer is applied to the pixel, the OLED material emits visible light, so that the driving voltage signal controls the brightness and color of each pixel to form a complete image or video content.

[0047] See also Figure 2 and Figure 5In one embodiment of the present invention, the processing control chip 100 includes: a behavior analysis unit, which is used to extract and analyze the action features of the eye image to determine the corresponding adjustment behavior; a pupil positioning unit, which is used to analyze the angle between the corneal reflection light of the non-visible reflection light and the pupil reflection light according to the pupil-corneal reflection vector method, determine the gaze position of the pupil on the display based on the angle, and use it as the target position of the display area 210. The behavior analysis unit receives multiple eye images within a preset time length sent by each eye tracking pixel collector, and extracts eye features in the eye image through a pre-trained eye movement recognition model to determine the user's interactive action. The identified interactive action is analyzed. If the interactive action is a preset action category, the corresponding adjustment behavior can be determined according to the interactive action. Exemplarily, the interactive action is a preset number of blinks (such as 5 times) within a preset time length (such as 2 seconds), and the adjustment behavior is to amplify the information of the corresponding position. It can be understood that the eye movement recognition model that recognizes the interactive action through the eye image can be any applicable model (such as YOLO, MobileNet, LSTM, etc.), which is not limited here. After the pupil positioning unit receives the corneal reflected light and pupil emitted light sent by the eye tracking pixel collector, it determines the gaze angle at this time by calculating the angle between the corneal reflected light and the pupil reflected light. According to the preset mapping rules, after the gaze angle is mapped to the coordinate system of the display area 210, the specific gaze position (i.e., the target position) of the operating body in the display area 210 can be determined. The eye movement operation control content presentation can be realized according to the target position and the adjustment behavior. For example, the target position is A, and the adjustment behavior is to amplify the information, then the final effect is to amplify the content at A in the display area 210.

[0048] Please continue to see Figure 2 and Figure 5 In one embodiment of the present invention, the display device further includes a power supply unit 400, and the power supply unit 400 is electrically connected to the display sensor chip 200, the processing control chip 100 and the non-visible light emitter 300, respectively. The power supply unit 400 serves as the power supply center of the entire device and is electrically connected to other components to ensure the normal operation of the device by providing different power supply voltages to different components. In one embodiment of the present invention, the power supply unit 400 is a power chip. By setting the power supply unit 400 as a power chip, the space occupancy rate can be greatly reduced, thereby effectively reducing the overall volume of the device and improving the lightweight and portability of the device.

[0049] See also Figure 2 and Figure 3, the processing process of the display device in the present invention is as follows: the non-visible light emitter 300 is used to emit infrared light to the eyes 500 of the operator, and the infrared light emitted by the non-visible light emitter 300 is irradiated on the human cornea and reflected on the cornea to form a corneal reflection spot. According to the change in the relative position of the corneal reflection spot and the pupil, the pupil-corneal reflection method is used to calculate the position and gaze direction of the real-time gaze point of the human eye to track the eye sight of the monitored person. By adding multiple groups of non-visible light emitters 300 and fixing them in special positions, it is convenient to emit infrared light at multiple angles and reflect infrared light from the cornea and pupil without dead angles. Unlike the traditional design, non-visible light emitters 300 are installed in different positions around the display sensor chip 200 to ensure that infrared rays are emitted at different positions to the human eye, and infrared rays are reflected from different viewing angles. When a person wears AR / VR glasses, the display area 210 is located directly in front of the eye 500. The pupil of the human eye usually rotates continuously and searches for display target graphics at different positions on the display area 210. Under the action of the processing control chip 100, the non-visible light emitter 300 emits non-visible light to the eye, and the sensing area 220 obtains the image of the eye 500 and the non-visible reflected light, etc., and transmits these data to the processing control chip 100. The processing control chip 100100 receives and analyzes these data, and gives a display target graphic on the display area 210 where the eye 500 is locked. The display area 210 can receive the processing control chip signal and confirm the position where the human eye pupil is looking at, so the display area 210 can identify the content that the person wants to see and open the video signal content.

[0050] In one embodiment of the present invention, a near-eye display device is also provided, comprising a bracket and any of the display devices described above, the display device being mounted on the bracket and used to track eye postures for near-eye display. The near-eye display device includes but is not limited to VR glasses and AR glasses.

[0051] In summary, the present invention provides a display device and a near-eye display device, wherein a processing control chip sends instructions to control a non-visible light emitter to emit non-visible light to the eye, and a sensing area on the display sensor chip captures the light and image reflected by the eye, and transmits the data to the processing control chip for analysis, thereby determining the target position of the operator in the display area and the corresponding adjustment behavior. The display area dynamically displays the content of virtual reality or augmented reality at the target position of the gaze according to the adjustment behavior generated by the processing control chip. The present invention not only greatly increases the number of sensing areas and optimizes the final imaging effect by alternately arranging display areas and sensing areas on the same substrate, but also closely integrates the display area with the sensing area, reduces the number of ports, and thus improves the data processing rate and accuracy. On the other hand, this method effectively reduces the number of connection ports between components and reduces the delay in signal transmission, so that the display area can respond to the eye movement data collected by the sensing area in real time, quickly adjust the display content, and ensure that the user's line of sight changes can be immediately fed back to the screen. Furthermore, all key components of the present invention are implemented in the form of chips. The integrated design of this chip reduces the physical connection between modules, realizes the precise control of non-visible light and the real-time processing of eye movement data, thereby ensuring that the eye tracking has good response speed and accuracy. In addition, this chip integration method greatly reduces the space occupancy rate of the device, realizes the lightweight design of the device, optimizes the user's wearing experience, and significantly improves the user's immersion and convenience of use. Furthermore, this method of the present invention can determine the target position and adjustment behavior through eye tracking, so as to realize the selection of the target at the target position on the display screen, and make it present according to the adjustment behavior, thereby eliminating the manual remote control operation action, and therefore has a milestone significance for VR glasses or AR glasses. Therefore, the present invention effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0052] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A display device, characterized in that: The device comprises: Processing control chip, used to send control instructions; A display sensor chip, comprising a display area and a sensing area arranged alternately, wherein the display area is used to display virtual reality / augmented reality content; A non-visible light emitter, mounted on the outer peripheral side of the display sensor chip, for emitting non-visible light according to the control instruction; The sensing area is used to receive the reflected non-visible reflected light and capture the eye image of the operating body; The processing control chip is further used to analyze the received non-visible reflected light and the eye image to determine an adjustment behavior and a target position of the display area corresponding to the adjustment behavior; The display area is also used to display the virtual reality / augmented reality content at the target location according to the adjustment behavior.

2. The display device according to claim 1, characterized in that The display area is an RGB pixel unit group, and a sensing area is arranged between adjacent RGB pixel unit groups.

3. The display device according to claim 1, characterized in that The display area is an RGB pixel array, and a sensing area is arranged between adjacent RGB pixel arrays; wherein the RGB pixel array includes at least two RGB pixel unit groups arranged continuously.

4. The display device according to claim 1, characterized in that There are a plurality of non-visible light emitters, and the plurality of non-visible light emitters are evenly distributed around the outer periphery of the display sensor chip.

5. The display device according to claim 1, characterized in that The sensing area is a CIS image sensor.

6. The display device according to claim 5, characterized in that The sensing area at least includes a first driving control layer and a CIS pixel array, wherein: The CIS pixel array is used to receive the non-visible reflected light reflected by the eye of the operating body, and to capture the visible light constituting the eye image, and to convert the optical signals of the non-visible reflected light and the visible light into electrical signals; The first driving control layer is used to process the converted electrical signal and generate a form suitable for processing by the processing control chip.

7. The display device according to claim 1, characterized in that The display area is Micro-OLED.

8. The display device according to claim 7, characterized in that The display area at least includes a second drive control layer and a pixel display layer, wherein: The second driving control layer is used to receive the adjustment behavior and the target position, and convert the adjustment behavior and the target position to generate a driving voltage for driving the Micro-OLED pixel display layer; The Micro-OLED pixel display layer is arranged on the second driving control layer, and is used to update the display information of the virtual reality / augmented reality content at the target position under the action of the driving voltage.

9. The display device according to claim 1, characterized in that: The processing control chip comprises: A behavior analysis unit, used to extract and analyze the action features of the eye image to determine the corresponding adjustment behavior; The pupil positioning unit is used to analyze the angle between the corneal reflected light and the pupil reflected light of the non-visible reflected light according to the pupil-corneal reflection vector method, determine the gaze position of the pupil on the display based on the angle, and use it as the target position of the display area.

10. The display device according to claim 1, characterized in that The display device further includes a power supply unit, which is electrically connected to the display sensor chip, the processing control chip and the non-visible light emitter respectively.

11. The display device according to claim 10, characterized in that The power supply unit is a power chip.

12. The display device according to claim 1, characterized in that The non-visible light is infrared light or near infrared light.

13. A near-eye display device, characterized in that: include: Bracket; as well as The display device according to any one of claims 1 to 12, wherein the display device is arranged on the bracket and is used to track the eye posture of the operating body to perform near-eye display.

Images