Display apparatus and near-eye display device

Abstract

The invention relates to a display apparatus and a near-to-eye display device. The device comprises a processing control chip used for sending a control instruction; the display sensing chip comprises a display area and a sensing area arranged on the outer side of the display area, and the display area is used for displaying virtual reality / augmented reality content; the invisible light emitter is installed on the peripheral side of the display sensing chip and used for emitting invisible light according to the control instruction; the sensing area is used for receiving reflected invisible reflected light and acquiring an eye image of an operation body; the processing control chip is also used for analyzing the received invisible reflected light and the received eye image, and determining an adjustment behavior and a target position of the display area corresponding to the adjustment behavior; the display area is further used for displaying the virtual reality / augmented reality content at the target position according to the adjustment behavior. The size of the near-to-eye display device is reduced, and the portability of the near-to-eye display device is improved.

Description

Technical Field

[0001] The present invention relates to the field of near-eye display technology, and in particular to a display device and a near-eye display apparatus. Background Art

[0002] Eye tracking technology refers to the process of measuring and recording eye movements to determine a person's gaze point (i.e. where the person is looking), gaze direction, or the movement or position of the eye relative to the head. With the development of technology, eye tracking technology is closely integrated with current augmented reality (AR) / virtual reality (VR) applications, and usually uses non-contact optical methods to measure the position of the eye or the gaze angle.

[0003] Traditional eye tracking technology in AR / VR devices usually uses infrared light to illuminate the eyes and calculates the direction of the pupil by reflecting light. However, due to the presence of optical devices such as thermal mirrors and lenses, the device is relatively large. In addition, users need to interact with the display screen through a handheld controller, which greatly limits the user's freedom of movement, resulting in a low degree of natural interaction and a poor user experience. In addition, in traditional eye tracking technology, the image sensor and the display are usually separated, resulting in a high size and power consumption of the device. Therefore, it is necessary to provide a display device and a near-eye display device. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a display device and a near-eye display apparatus, which improve the problem that the display device in the prior art is not very portable.

[0005] To achieve the above-mentioned purpose and other related purposes, the present invention provides a display device, comprising: a processing control chip, used to send control instructions; a display sensor chip, including a display area and a sensing area arranged outside the display area, the display area is used to display virtual reality / augmented reality content; a non-visible light emitter, installed on the outer side of the display sensor chip, used to emit non-visible light according to the control instruction; the sensing area is used to receive reflected non-visible reflected light and obtain an eye image of an operating body; the processing control chip is also used to analyze the received non-visible reflected light and the eye image, determine an adjustment behavior and a target position of the display area corresponding to the adjustment behavior; the display area is also used to display the virtual reality / augmented reality content at the target position according to the adjustment behavior.

[0006] In an embodiment of the present invention, there are a plurality of sensing areas, and the plurality of sensing areas are distributed around an outer edge of the display area.

[0007] In one embodiment of the present invention, there are a plurality of non-visible light emitters, and the plurality of non-visible light emitters are evenly distributed around the outer periphery of the display sensor chip and are staggered with the plurality of sensing areas.

[0008] In one embodiment of the present invention, the sensing area is a CIS image sensor.

[0009] In one embodiment of the present invention, the sensing area includes at least a first driving circuit layer and a CIS pixel array, wherein: the CIS pixel array is used to receive the reflected non-visible reflected light, and to capture the visible light constituting the eye image, and to convert the optical signals of the non-visible reflected light and the visible light into electrical signals; The first driving circuit layer is used to process the converted electrical signal and generate a form suitable for processing by the processing control chip.

[0010] In one embodiment of the present invention, the display area is Micro-OLED.

[0011] In one embodiment of the present invention, the display area includes at least a second driving circuit layer and a Micro-OLED pixel display layer, wherein: the second driving circuit layer is used to receive the adjustment behavior and the target position, and convert the adjustment behavior and the target position to generate a driving voltage to drive the Micro-OLED pixel display layer; the Micro-OLED pixel display layer is arranged on the second driving circuit layer, and is used to update the display information of the virtual reality / augmented reality content at the target position under the action of the driving voltage.

[0012] In one embodiment of the present invention, the processing and control chip includes: a behavior analysis unit, used to extract and analyze the motion features of the eye image to determine the corresponding adjustment behavior; a pupil positioning unit, used to analyze the angle between the corneal reflected light and the pupil reflected light of the non-visible reflected light based on the pupil-corneal reflection vector method, and determine the gaze position of the pupil in the display area based on the angle, and use it as the target position of the display area.

[0013] In one embodiment of the present invention, the display device further includes a power supply unit, and the power supply unit is electrically connected to the display sensor chip, the processing control chip and the non-visible light emitter respectively.

[0014] In an embodiment of the present invention, the power supply unit is a power chip.

[0015] In one embodiment of the present invention, the non-visible light is infrared light or near-infrared light.

[0016] In one embodiment of the present invention, a near-eye display device is further provided, comprising: a bracket and a display device as described in any one of the above items, wherein the display device is arranged on the bracket and is used to track eye postures for near-eye display.

[0017] As described above, a display device and a near-eye display device of the present invention have the following beneficial effects: the processing control chip sends instructions to control the non-visible light emitter to emit non-visible light to the eye, the sensing area on the display sensor chip captures the light and image reflected by the eye, and transmits the data to the processing control chip for analysis, thereby determining the target position of the operator in the display area and the corresponding adjustment behavior. The display area dynamically displays the content of virtual reality or augmented reality at the target position of the gaze according to the adjustment behavior generated by the processing control chip. The present invention integrates the display area and the sensing area into a silicon-based substrate to form an integrated chip. This method effectively reduces the number of connection ports between components and reduces the delay of signal transmission, so that the display area can respond to the eye movement data collected by the sensing area in real time, quickly adjust the display content, and ensure that the user's line of sight changes can be fed back to the screen in real time. Furthermore, the key components of the present invention are all implemented in the form of chips. The integrated design of this chip reduces the physical connection between the modules, realizes the precise control of non-visible light and the real-time processing of eye movement data, thereby ensuring that the eye movement tracking has good response speed and accuracy. In addition, this chip integration method greatly reduces the space occupancy of the device, realizes the lightweight design of the device, optimizes the user's wearing experience, and significantly improves the user's immersion and ease of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a structural schematic diagram of a traditional display device; Figure 2 Shown is a side view of a display device provided by an embodiment of the present invention; Figure 3 Shown is a front view of a display device provided by an embodiment of the present invention; Figure 4 Shown is a structural diagram of the corresponding functional area of ​​the integrated display sensor chip of the present invention; Figure 5 Shown is a structural block diagram of the display device of the present invention; Figure 6 Shown are diagrams showing different pupil positions in the eye.

[0019] Component number description: 100. Processing control chip; 200. Display sensor chip; 210. Display area; 220. Sensing area; 300. Non-visible light emitter; 400. Power supply unit; 500. Eyeball. DETAILED DESCRIPTION

[0020] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0021] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0022] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0023] In order to better understand and illustrate the method provided in the embodiments of the present application, some technical terms involved in the embodiments of the present application are first explained and illustrated below.

[0024] The design principle of eye tracking is to use an infrared transmitter to emit a beam of infrared light to illuminate the eyes. The infrared light reflected from the cornea and retina is collected by the camera. Due to the physiological structure and physical properties of the eyeball, under the premise that the relative position of the light source and the head remains unchanged, the light spot formed by the corneal reflection will not move, and the direction of the light reflected on the retina indicates the direction of the pupil (the light source enters from the pupil, and the retinal reflected light is emitted from the pupil). Finally, the direction of the eye movement can be calculated based on the angle between the cornea and the pupil reflected light. When the subject is staring at a fixed target, no matter how the head moves, the positional relationship between the pupil center and the corneal reflection will not change. Even if the subject is moving, the eye tracker can reflect that she is looking at the same position. Therefore, when the light source and the head position are relatively stable, the light reflected from the cornea and retina is collected, and their relative position relationship is analyzed, the data of the pupil direction can be obtained. This is the principle of the pupil-corneal reflection eye tracker.

[0025] In one known type of optical eye tracking technology, light (typically light in the infrared region) is reflected from the eye and sensed by a camera. The information sensed by the camera is then analyzed to extract the gaze direction or position of the pupil from the changes in the reflection. Video-based eye trackers sometimes use corneal reflections or pupil centers as features to track over time. A camera-based eye tracking system may include a rear-mounted camera attached to a bracket and pointed (directly or indirectly) at the user's eyes as a means of detecting the position of the user's eyes. The digital data generated by the camera is sent via wired or wireless means to an external device such as a computer (or alternatively, to a computer resource located on itself) for processing and analysis. Computer software in such a system runs an eye tracking algorithm known to those of ordinary skill in the art to detect the position of one or both of the user's eyes.

[0026] like Figure 1 As shown, eye tracking includes one or two small display areas with lenses and semi-transparent mirrors (i.e., "hot mirrors") embedded in various form factors, such as helmets, glasses (also called data glasses), or goggles. Hot mirrors provide one possible design approach for eye tracking and allow cameras or other eye tracking sensors (such as eye tracking cameras) to get a good view of the eye being tracked. Some hot mirrors reflect infrared ("IR") radiation and are transparent to visible light. The hot mirror in some eye tracking applications is tilted in front of the eye and allows the IR camera or other eye tracking sensor to get a reflected image of the eye while having a transparent view on the display. Figure 1 The display shows a traditional display device, where an infrared transmitter emits infrared light to the user's eyeball, and the light reflected from the eyeball is transmitted to the eye tracking camera through a lens and a thermal mirror, and the corresponding content is displayed on the display after being analyzed by the controller. This optical eye tracking method is widely used for gaze tracking. Such a tracker may require a relatively high-resolution camera that uses image processing and pattern recognition devices to capture at a high frame rate to track reflected light or known eye structures (e.g., iris or pupil).

[0027] Based on the above, traditional display devices monitor eye movements by reflecting and capturing light through optical components such as lenses and thermal mirrors. However, these components greatly increase the weight of the entire VR / AR glasses, making the entire device bulky and not conducive to long-term wear by users. In addition, traditional devices usually require users to hold controllers to assist in operation and target selection, but this interaction method is not natural enough. When using it, users need to switch back and forth between line of sight and hand operation, and cannot achieve natural line of sight control, resulting in poor immersion for users.

[0028] See also Figure 2Based on the above problems, the present invention provides a display device, in which the processing control chip sends instructions to control the non-visible light emitter to emit non-visible light to the eyes, and the sensing area on the display sensor chip captures the light and image reflected by the eyes, and transmits the data to the processing control chip for analysis, thereby determining the target position of the operator in the display area and the corresponding adjustment behavior. The display area dynamically displays the content of virtual reality or augmented reality at the target position of the gaze according to the adjustment behavior generated by the processing control chip. The present invention integrates the display area and the sensing area on a silicon-based substrate to form an integrated chip. This method effectively reduces the number of connection ports between components and reduces the delay of signal transmission, so that the display area can respond to the eye movement data collected by the sensing area in real time, quickly adjust the display content, and ensure that the user's line of sight changes can be fed back to the screen in real time. Furthermore, all key components of the present invention are implemented in the form of chips. The integrated design of this chip reduces the physical connection between the modules, realizes the precise control of non-visible light and the real-time processing of eye movement data, thereby ensuring that the eye movement tracking has good response speed and accuracy. In addition, this chip integration method greatly reduces the space occupancy of the device, realizes the lightweight design of the device, optimizes the user's wearing experience, and significantly improves the user's immersion and convenience of use. Furthermore, this method of the present invention can determine the target position and adjustment behavior through eye tracking, so as to realize the selection of the target at the target position on the display screen and make it present according to the adjustment behavior, thereby eliminating the manual remote control operation action, and thus has a milestone significance for VR glasses or AR glasses.

[0029] See also Figure 2 The display device includes: a processing control chip 100, a display sensor chip 200 and a non-visible light emitter 300. The processing control chip 100 is used to send control instructions; the display sensor chip 200 includes a display area 210 and a sensing area 220 arranged outside the display area 210, and the display area 210 is used to display virtual reality / augmented reality content; the non-visible light emitter 300 is installed on the outer peripheral side of the display sensor chip 200, and is used to emit non-visible light to the eyes of the operating body according to the control instructions; the sensing area 220 is used to receive the non-visible reflected light reflected by the eyes of the operating body and obtain the eye image of the operating body; the processing control chip 100 is also used to analyze the received non-visible reflected light and the eye image, determine the adjustment behavior and the target position of the display area 210 corresponding to the adjustment behavior; the display area 210 is also used to update the display information of the virtual reality / augmented reality content at the target position according to the adjustment behavior.

[0030] See also Figure 2 and Figure 5, the processing control chip 100 serves as the control center of the entire display device, and is responsible for sending control instructions to the non-visible light emitter 300, and receiving eye movement data such as non-visible reflected light and eye images reflected by the eyes 500 of the operator sent by the sensing area 220. After analyzing and processing these eye movement data, the gaze position of the operator and the specific adjustment behavior can be determined. The processing control chip 100 can also determine its target position in the display area 210 according to the gaze position based on the data fed back by the display area 210. Thereby, the display area 210 can dynamically adjust the display content according to the corresponding adjustment behavior according to the change of the operator's line of sight. Specifically, after the operator wears the near-eye display device equipped with the display device, the processing control chip 100 will send control instructions to the non-visible light emitter 300 at a preset time interval, so that the non-visible light emitter 300 emits non-visible light to the eyes 500 of the operator. When the sensing area 220 receives the non-visible reflected light and the eye image reflected from the eye, the eye movement data is sent to the processing control chip 100, which analyzes and processes the eye movement data, determines the gaze direction of the eye based on the non-visible reflected light, and determines the specific eye movement based on the eye image, thereby determining the adjustment behavior. Exemplarily, when the operator gazes at a certain position of the display area 210, the processing control chip 100 can command the display area 210 to enlarge or focus the content of the position according to the gaze position. Among them, the processing control chip 100 is a CPU chip.

[0031] See also Figure 2 and Figure 3, the display area 210 and the sensing area 220 located outside the display area 210 constitute an integrated chip, namely the display sensor chip 200. In the display sensor chip 200, a silicon base is used as a substrate, and different functional areas are arranged on the same substrate. Among them, the display area 210 is used to display virtual reality or augmented reality content, and the sensing area 220 is responsible for receiving non-visible light reflected by the eye and obtaining eye images, and sending the acquired eye movement data to the processing control chip 100. After the processing control chip 100 analyzes and processes the data sent by the display area 210, it sends the generated target position and adjustment behavior to the display area 210, and the display area 210 dynamically adjusts the displayed content at the target position according to the adjustment behavior. In the present invention, by integrating the display and sensing functions on the same substrate, the CIS pixel part function and the silicon-based Micro-OLED display are integrated into a silicon-based CMOS chip to form an integrated chip. It not only greatly reduces the space occupancy rate and improves the system integration, but also effectively reduces the number of ports, thus achieving efficient and accurate data transmission, ensuring that the operator's line of sight changes can be fed back to the adjustment of the display content in real time. In addition, this integrated chip effectively improves the clarity and resolution of image display and acquisition, enhancing the user's immersive experience.

[0032] See also Figure 2 and Figure 3 The non-visible light emitter 300 is arranged on the outer peripheral side of the display sensor chip 200, and is used to emit non-visible light to the eyes 500 of the operator under the action of the control command. When the non-visible light irradiates the eyes 500 of the operator, a part of the light is reflected on the cornea to form a stable corneal reflection light spot, and the other part of the light passes through the pupil into the eye, and is emitted from the pupil after being reflected on the retina. These two kinds of non-visible reflected light will be captured by the sensing area 220 and transmitted to the processing control chip 100. Based on the relative position change of the corneal reflection light spot and the pupil reflection light, the processing control chip 100 uses the pupil-corneal reflection vector method to calculate the real-time gaze point position and gaze direction of the eye 500, so as to accurately track the line of sight of the operator. Furthermore, since the human eye cannot perceive the existence of infrared light and near-infrared light, it will not interfere with the user's visual experience. Therefore, in one embodiment of the present invention, the non-visible light is infrared light or near-infrared light.

[0033] See also Figures 2 to 4In order to realize that the non-visible reflected light reflected by the eye can be received at different angles, in one embodiment of the present invention, there are multiple sensing areas 220, and the multiple sensing areas 220 are distributed around the outer edge of the display area 210. Specifically, multiple sensing areas 220 are evenly distributed around the outer edge of the display area 210, and the silicon-based substrate is preset with a middle area and an edge area surrounding the middle area. The middle area integrates the display area 210 formed by the CMOS driving circuit of Micro-OLED and the RGB OLED display layer to achieve efficient display function. The edge area integrates the sensing area 220 formed by the CIS image acquisition pixel and the CIS CMOS circuit for capturing and processing eye movement data. In this way, the display and eye movement tracking functions can work together on the same chip, and the sensing area 220 can receive light reflected from the eyes of the operating body from different angles. Through this more comprehensive data collection, the visual blind spot phenomenon that may be caused by a single perspective is effectively avoided. In addition, the method of evenly distributing multiple sensing areas 220 around the outer edge of the display area 210 saves the actual required space and weight in space, and increases the actual AR / VR experience.

[0034] See also Figure 2 and Figure 5In order to improve the comprehensiveness of data collection and take into account the actual needs of AR / VR glasses space and weight, the present invention removes traditional lenses and thermal mirrors, thereby greatly reducing the space occupied by the entire device. In one embodiment of the present invention, there are multiple non-visible light emitters 300, and the multiple non-visible light emitters 300 are evenly distributed around the outer periphery of the display sensor chip 200, and are staggered with the multiple sensing areas 220. Multiple non-visible light emitters 300 are evenly arranged around the outside of the display sensor chip 200, so that non-visible light can be emitted toward the eyes of the operating body from different angles, which facilitates the all-round reflection of non-visible light from the cornea and pupil. Multiple sensing areas 220 are arranged around the outside of the display area 210, and form an integrated chip with the display area 210, and the sensing areas 220 and the non-visible light emitters 300 are alternately arranged on the outside of the display area 210. This method can ensure that the sensing area 220 can receive the non-visible reflected light reflected by the eyes of the operating body from different angles, so that the subtle movements and position changes of the eyes can be captured more accurately, and the problem of blind spots that are prone to occur in single-angle monitoring can be effectively improved. This method of alternating multiple non-visible light emitters 300 and multiple sensing areas 220 can collect data from multiple perspectives, ensure the stability and reliability of the data, and provide more and more accurate data to analyze the position of eye rotation. It can be understood that the arrangement order between the sensing area 220 and the non-visible light emitter 300 is not limited, and it can be "sensing area 220-non-visible light emitter 300-sensing area 220", or "sensing area 220-non-visible light emitter 300-non-visible light emitter 300-sensing area 220" and other methods. The specific combination method is not limited and can be adaptively selected based on actual application needs. Furthermore, if Figure 6 As shown in FIG. 1 , it shows schematic diagrams of different positions of the pupil in the eye, namely, the pupil is on the right side above the eye, in the middle above the eye, on the left side above the eye, and the pupil is on the right side below the eye, in the middle below the eye, and on the left side below the eye. It can be seen that the user's gaze point can be accurately determined by the change in the position of the pupil.

[0035] See also Figure 5In one embodiment of the present invention, the sensing area 220 is a CIS image sensor. The high-precision CIS image sensor can collect non-visible light information such as infrared light or near-infrared light and facial features in multiple viewing angles and directions, thereby providing more data support for the processing control chip 100. Unlike traditional image collectors, the present invention uses a CIS image sensor as a pixel image collector. Since the CIS image sensor has a strong photosensitivity, it can better capture eye images and non-visible reflected light, which not only greatly improves the clarity and accuracy of pixel collection, but also effectively reduces the installation space required on the main bracket, making the overall device more lightweight. Through the integration of image sensors and chip process optimization, the integrated chip can achieve efficient and accurate eye tracking, and effectively improve the lightweight degree of AR / VR glasses to improve the comfort of users wearing them.

[0036] See also Figures 2 to 4 In one embodiment of the present invention, the sensing area 220 includes at least a first driving circuit layer and a CIS pixel array, wherein: the CIS pixel array is used to receive the non-visible reflected light reflected by the eyes of the operating body, and to capture the visible light constituting the eye image, and to convert the optical signals of the non-visible reflected light and the visible light into electrical signals; the first driving circuit layer is used to process the converted electrical signals to generate a form suitable for processing by the processing control chip 100. Specifically, the CIS pixel array receives the visible light reflected by the eyes of the operating body and its surroundings to obtain the eyes of the operating body and its surroundings, and extracts the eye area image from them through the built-in image processing algorithm. On the other hand, after the non-visible light emitted by the non-visible light emitter 300 irradiates the eyes 500 of the operating body, the non-visible light generates different reflections on the cornea and retina, forming corneal reflection spots and retinal reflection spots respectively. The CIS pixel array captures these two types of non-visible light reflections and converts the received optical signals into electrical signals. The first driving circuit layer is a CIS CMOS circuit, which converts the original data from the CIS pixel array into a format suitable for the processing and control chip 100, so that the processing and control chip 100 can effectively interpret and use the signal data.

[0037] See also Figure 2 and Figure 3In one embodiment of the present invention, the display area 210 is Micro-OLED. The display area 210 adopts a silicon-based CMOS technology driver chip, and a Micro-OLED pixel display layer is made on the upper surface of the silicon-based CMOS technology driver chip, so that the bottom CMOS drives the upper Micro-OLED pixels. This design not only improves the resolution of the pixels, but also greatly improves the brightness uniformity and brightness, making the visual effect clearer and brighter. Specifically, the silicon-based Micro-OLED chip realizes the combination of pixels and silicon-based CMOS chips, and has high resolution and clarity, so that a clear visual picture can be presented. The display area 210 receives the video signal (MIPI / RGB / LVDS and other formats) transmitted by the processing control chip 100, and is driven by the silicon-based Micro-OLED and displayed in the Micro-OLED pixel area.

[0038] See also Figures 2 to 4 In one embodiment of the present invention, the display area 210 includes at least a second driving circuit layer and a Micro-OLED pixel display layer, wherein: the second driving circuit layer is used to receive the adjustment behavior and the target position, and convert the adjustment behavior and the target position to generate a driving voltage to drive the Micro-OLED pixel display layer; the Micro-OLED pixel display layer is arranged on the second driving circuit layer, and is used to update the display information of the virtual reality / augmented reality content at the target position under the action of the driving voltage. The display area 210 adopts a silicon-based Micro-OLED chip, including a second driving circuit layer and a Micro-OLED pixel display layer located thereon. Specifically, the second driving circuit layer is a CMOS micro-OLED driving circuit, which performs data conversion on the received information after receiving the adjustment behavior and the corresponding target position sent by the processing control chip 100, and generates a driving voltage signal suitable for driving the Micro-OLED pixel display layer through data format conversion. The Micro-OLED pixel display layer is composed of a large number of micro OLED pixels, and the OLED pixels include three sub-pixels of red, green and blue arranged in sequence, and these pixels can emit light independently under the drive of voltage. When the voltage signal provided by the second driving circuit layer is applied to the pixel, the OLED material emits visible light, so that the driving voltage signal controls the brightness and color of each pixel to form a complete image or video content.

[0039] See also Figure 2 and Figure 5In one embodiment of the present invention, the processing control chip 100 includes: a behavior analysis unit, which is used to extract and analyze the action features of the eye image to determine the corresponding adjustment behavior; a pupil positioning unit, which is used to analyze the angle between the corneal reflection light of the non-visible reflection light and the pupil reflection light according to the pupil-corneal reflection vector method, determine the gaze position of the pupil in the display area 210 based on the angle, and use it as the target position of the display area 210. The behavior analysis unit receives multiple eye images within a preset time length sent by each eye tracking pixel collector, and extracts eye features in the eye image through a pre-trained eye movement recognition model to determine the user's interactive action. The identified interactive action is analyzed. If the interactive action is a preset action category, the corresponding adjustment behavior can be determined according to the interactive action. Exemplarily, the interactive action is a preset number of blinks (such as 5 times) within a preset time length (such as 2 seconds), and the adjustment behavior is to amplify the information of the corresponding position. It can be understood that the eye movement recognition model that recognizes the interactive action through the eye image can be any applicable model (such as YOLO, MobileNet, LSTM, etc.), which is not limited here. After the pupil positioning unit receives the corneal reflected light and pupil emitted light sent by the eye tracking pixel collector, it determines the gaze angle at this time by calculating the angle between the corneal reflected light and the pupil reflected light. According to the preset mapping rules, after mapping the gaze angle to the coordinate system of the display area 210, the specific gaze position (i.e., the target position) of the operating body in the display area 210 can be determined. The eye movement operation control content presentation can be realized according to the target position and the adjustment behavior. For example, the target position is A, and the adjustment behavior is to magnify the information, then the final effect is to magnify the content at A in the display area 220.

[0040] Please continue to see Figure 2 and Figure 5 In one embodiment of the present invention, the display device further includes a power supply unit 400, and the power supply unit 400 is electrically connected to the display sensor chip 200, the processing control chip 100 and the non-visible light emitter 300, respectively. The power supply unit 400 serves as the power supply center of the entire device and is electrically connected to other components to ensure the normal operation of the device by providing different power supply voltages to different components. In one embodiment of the present invention, the power supply unit 400 is a power chip. By setting the power supply unit 400 as a power chip, the space occupancy rate can be greatly reduced, thereby effectively reducing the overall size of the device and improving the lightweight and portability of the device.

[0041] See also Figure 2 and Figure 3, the processing process of the display device in the present invention is as follows: the non-visible light emitter 300 is used to emit infrared light to the eyes 500 of the operator, and the infrared light emitted by the non-visible light emitter 300 is irradiated on the human cornea and reflected on the cornea to form a corneal reflection spot. According to the change of the relative position of the corneal reflection spot and the pupil, the pupil-corneal reflection method is used to calculate the position and gaze direction of the real-time gaze point of the human eye to track the eye sight of the monitored person. By adding multiple groups of non-visible light emitters 300 and fixing them in special positions, it is convenient to emit infrared light at multiple angles and reflect infrared light from the cornea and pupil without dead angles. Unlike the traditional design, non-visible light emitters 300 are installed in different positions around the display sensor chip 200 to ensure that infrared rays are emitted at different positions to the human eye, and infrared rays are reflected from different viewing angles. When a person wears AR / VR glasses, the display area 210 is located directly in front of the eye. The pupil of the human eye usually rotates continuously and searches for display target graphics at different positions on the display area 210. Under the action of the processing and control chip 100, the non-visible light emitter 300 emits non-visible light to the eye, and the sensing area 220 obtains the eye image and non-visible reflected light, etc., and transmits these data to the processing and control chip 100. The processing and control chip 100 receives and analyzes these data, and gives a display target graphic on the display area 210 where the eye 500 is locked. The display area 210 can receive the signal from the processing and control chip 100 and confirm the position where the human eye pupil is looking at, so the display area 210 can identify the content that the person wants to see and open the video signal content.

[0042] In one embodiment of the present invention, a near-eye display device is also provided, comprising a bracket and any of the display devices described above, the display device being mounted on the bracket and used to track eye postures for near-eye display. The near-eye display device includes but is not limited to VR glasses and AR glasses.

[0043] In summary, the present invention discloses a display device and a near-eye display device, in which the processing control chip sends instructions to control the non-visible light emitter to emit non-visible light to the eye, and the sensing area on the display sensor chip captures the light and image reflected by the eye, and transmits the data to the processing control chip for analysis, thereby determining the target position of the operator in the display area and the corresponding adjustment behavior. The display area dynamically displays the content of virtual reality or augmented reality at the target position of the gaze according to the adjustment behavior generated by the processing control chip. The present invention integrates the display area and the sensing area into a silicon-based substrate to form an integrated chip. This method effectively reduces the number of connection ports between components and reduces the delay of signal transmission, so that the display area can respond to the eye movement data collected by the sensing area in real time, quickly adjust the display content, and ensure that the user's line of sight changes can be fed back to the screen in real time. Furthermore, all key components of the present invention are implemented in the form of chips. The integrated design of this chip reduces the physical connection between the modules, realizes the precise control of non-visible light and the real-time processing of eye movement data, thereby ensuring that the eye movement tracking has good response speed and accuracy. In addition, this chip integration method greatly reduces the space occupancy of the device, realizes the lightweight design of the device, optimizes the user's wearing experience, and significantly improves the user's immersion and convenience of use. Furthermore, this method of the present invention can determine the target position and adjustment behavior through eye tracking, so as to realize the selection of the target at the target position on the display screen, and make it present according to the adjustment behavior, thereby eliminating the manual remote control operation action, and therefore has a milestone significance for VR glasses or AR glasses. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.

[0044] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A display device, characterized in that: The device comprises: Processing control chip, used to send control instructions; A display sensor chip, comprising a display area and a sensing area arranged outside the display area, wherein the display area is used to display virtual reality / augmented reality content; A non-visible light emitter, mounted on the outer peripheral side of the display sensor chip, for emitting non-visible light according to the control instruction; The sensing area is used to receive the reflected non-visible reflected light and obtain the eye image of the operating body; The processing control chip is further used to analyze the received non-visible reflected light and the eye image to determine an adjustment behavior and a target position of the display area corresponding to the adjustment behavior; The display area is also used to display the virtual reality / augmented reality content at the target location according to the adjustment behavior.

2. The display device according to claim 1, characterized in that There are multiple sensing areas, and the multiple sensing areas are distributed around the outer edge of the display area.

3. The display device according to claim 2, characterized in that There are a plurality of non-visible light emitters, and the plurality of non-visible light emitters are evenly distributed around the outer periphery of the display sensor chip and are staggered with the plurality of sensing areas.

4. The display device according to claim 1, characterized in that The sensing area is a CIS image sensor.

5. The display device according to claim 4, characterized in that The sensing area at least includes a first driving circuit layer and a CIS pixel array, wherein: The CIS pixel array is used to receive the reflected non-visible reflected light, capture the visible light constituting the eye image, and convert the optical signals of the non-visible reflected light and the visible light into electrical signals; The first driving circuit layer is used to process the converted electrical signal and generate a form suitable for processing by the processing control chip.

6. The display device according to claim 1, characterized in that The display area is Micro-OLED.

7. The display device according to claim 6, characterized in that The display area at least includes a second driving circuit layer and a Micro-OLED pixel display layer, wherein: The second driving circuit layer is used to receive the adjustment behavior and the target position, and convert the adjustment behavior and the target position to generate a driving voltage for driving the Micro-OLED pixel display layer; The Micro-OLED pixel display layer is arranged on the second driving circuit layer, and is used to update the display information of the virtual reality / augmented reality content at the target position under the action of the driving voltage.

8. The display device according to claim 1, characterized in that The processing control chip comprises: A behavior analysis unit, used to extract and analyze the action features of the eye image to determine the corresponding adjustment behavior; The pupil positioning unit is used to analyze the angle between the corneal reflected light and the pupil reflected light of the non-visible reflected light according to the pupil-corneal reflection vector method, determine the gaze position of the pupil in the display area based on the angle, and use it as the target position of the display area.

9. The display device according to claim 1, characterized in that: The display device further includes a power supply unit, which is electrically connected to the display sensor chip, the processing control chip and the non-visible light emitter respectively.

10. The display device according to claim 9, characterized in that The power supply unit is a power chip.

11. The display device according to claim 1, characterized in that The non-visible light is infrared light or near infrared light.

12. A near-eye display device, characterized in that: include: Bracket; as well as The display device according to any one of claims 1 to 11, wherein the display device is arranged on the bracket and is used to track eye postures for near-eye display.

Images