Tracking system and head-mounted display device

By using a combination of folding optical path and special-shaped prisms in head-mounted display devices, the problems of complex structure and low eye tracking accuracy caused by multiple cameras in the prior art are solved, and flexible equipment layout and high accuracy are achieved. eye tracking reduces costs.

CN222070963UActive Publication Date: 2024-11-26GRAVITYXR ELECTRONICS & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202420863134.5
Authority / Receiving Office
CN · China
Patent Type
Utility models(China)
Current Assignee / Owner
Filing Date
2024-04-24
Publication Date
2024-11-26
Estimated Expiration
2034-04-24

AI Technical Summary

Technical Problem

The eye tracking device of the existing head-mounted display device has a complex structure and difficult arrangement due to the installation of multiple cameras, which affects the accuracy of eye tracking. Especially when the display module is large, the optical axis direction of the camera and the display module. The angle of the optical axis of the group increases, making it easy to have blind spots in shooting.

Method used

A tracking system is adopted, including a display module, a fill light module and a first prism component. The optical path is folded through the first prism component, the position of the image acquisition device is changed, and the shooting direction of the binocular camera is realized using a monocular camera, saving equipment costs. , and through the combination of special-shaped prisms and regular prisms, the setting flexibility and accuracy of the acquisition device are improved.

Benefits of technology

The structure of the tracking system is simplified, the difficulty of equipment layout is reduced, the accuracy of eye tracking is improved, the cost of head-mounted display devices is reduced, and the user experience is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN222070963U_ABST
    Figure CN222070963U_ABST
Patent Text Reader

Abstract

The utility model provides a tracking system and a head-mounted display device, and relates to the technical field of tracking. The tracking system comprises a display module, a light supplementing module, a first prism assembly and a first image acquisition device, the display module faces the eyes of a user, and the light supplementing module is used for emitting infrared light to the face of the user. The first prism assembly is located between the display module and the eyes of a user, the first prism assembly comprises a first incident surface, a first emergent surface and at least one first reflecting surface, the first image acquisition device is located between the display module and the eyes of the user, and the first incident surface of the first image acquisition device is parallel to the first emergent surface. The first incident surface faces the eyes of a user, and infrared light entering from the first incident surface is reflected by the at least one first reflecting surface, then is emitted from the first emergent surface, and then enters the first image acquisition device to form an image of the eyes of the user. The tracking system is simple in structure, convenient to arrange and high in design flexibility.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The utility model relates to the field of tracking technology, in particular to a tracking system and a head-mounted display device. Background Art

[0002] With the development of technologies such as virtual reality (VR) and augmented reality (AR), as well as the emergence of the concept of the metaverse, head mounted display (HMD) has gradually entered the public eye. In order to enhance the user's immersive experience, it is usually necessary to track the user's head, gestures, expressions, and eyes. Among them, eye tracking is a technology that obtains information by tracking and recording human eye movements. Expressions are an intuitive way to express emotions. Expression tracking can be achieved by collecting images of eyebrows, foreheads, eyes, or the lower half of the face, combined with facial expression recognition algorithms. Eye tracking and expression tracking are key technologies in the field of human-computer interaction.

[0003] Usually, the eye tracking device on the head-mounted display device includes infrared light and a camera, which captures eye movement images and obtains the eye movement trajectory. By analyzing the changes in the eyes in time and space, the gaze point, gaze duration, scanning path and other information can be determined. In the related art, the eye tracking device of the head-mounted display device usually includes four cameras, two cameras for each eye, and the two cameras are respectively set on both sides of the eye, which can improve the accuracy of tracking.

[0004] The eye tracking device has a complex structure due to the multiple cameras, and the angle between the camera optical axis and the display module optical axis needs to be kept within a certain angle, which makes it difficult to arrange the eye tracking device. However, if one camera is used to capture eye movement images, the camera is often only on one side of the eye, and there is a blind spot when the eyeball moves, which affects the accuracy of the eye tracking system. Utility Model Content

[0005] The utility model provides a tracking system and a head-mounted display device. The tracking system has a simple structure, is convenient to arrange, and has high design flexibility.

[0006] A first aspect of an embodiment of the present utility model provides a tracking system, comprising:

[0007] A display module, wherein the display direction is toward the user's eyes;

[0008] A fill light module, the fill light module is used to emit infrared light to the face of the user;

[0009] a first prism assembly, disposed between the display module and the user's eyes, the first prism assembly comprising a first incident surface, a first exit surface and at least one first reflection surface, the first incident surface facing the user's eyes, the first incident surface being used to receive infrared light reflected by the user's eyes, and the infrared light entering from the first incident surface being reflected by the at least one first reflection surface and then emitted from the first exit surface;

[0010] The first image acquisition device is arranged between the display module and the user's eyes. The first image acquisition device includes a first light incident surface, which is parallel to the first light exit surface. The first light incident surface is used to receive infrared light emitted from the first light exit surface to form an image of the user's eyes.

[0011] The tracking system in the embodiment of the present application is provided with a display module, and the display direction of the display module is set toward the user's eyes, so that the picture displayed by the display module can be seen by the user. A fill light module is provided, and the fill light module is set to at least be a structure for emitting infrared light to the user's eyes, so that the fill light module can fill light to the user's eyes, so that the first image acquisition device can acquire an image of the user's eye area, thereby realizing eye tracking of the user.

[0012] By arranging the first prism assembly between the display module and the user's eyes and arranging at least one first reflection surface in the first prism assembly, the first prism assembly can change the direction of the infrared light reflected by the user's eyes, thereby improving the flexibility of the setting position of the first image acquisition device.

[0013] By setting the first prism assembly, a folded light path is achieved. The propagation path of the light is folded and bent, and the position of the first image acquisition device is changed. At the same time, a monocular camera can be used to achieve a shooting direction that could only be achieved with a binocular camera before, thereby saving equipment costs.

[0014] By setting at least one first reflection surface in the first prism assembly, the infrared light reflected from the user's eyes received by the first incident surface can be reflected at least once and then enter the first light incident surface of the first image acquisition device through the first exit surface, so that the first image acquisition device can capture the image of the user's eyes. In other words, the number and setting positions of the first reflection surfaces can change the angle of the infrared light finally emitted from the first exit surface of the first prism assembly.

[0015] By setting the first exit surface and the light incident surface of the first image acquisition device parallel to each other and at an angle to the plane where the display module is located, the structure and position of the first prism assembly can affect the setting position of the first image acquisition device. Therefore, the setting position of the first image acquisition device can be changed by changing the structure or position of the first prism assembly. Compared with the situation in the related art where the camera can only be directed toward the user's face for shooting, this can improve the flexibility of the setting position of the first image acquisition device. By setting the first light incident surface of the first prism assembly toward the user's eyes, the first image acquisition device can capture images of a specific area, thereby improving the accuracy of eye tracking.

[0016] In a possible implementation, the first prism assembly includes one or more prisms, wherein the prisms are regular prisms or special prisms, and each of the prisms includes at least one first reflection surface.

[0017] By configuring the first prism assembly to include one or more prisms, the first prism assembly may include at least one first reflective surface, so that the infrared light reflected by the user's eyes can enter the first image acquisition device after at least one reflection, that is, the optical path of the infrared light reflected by the user's eyes can be changed at least once, thereby improving the flexibility of the setting position of the first image acquisition device, so that the first image acquisition device does not need to be limited to a specific position or a specific angle, resulting in a blind spot when the eyeball rotates, thereby affecting the accuracy of the eye tracking system.

[0018] In a possible implementation manner, when the first prism assembly includes a plurality of prisms, two opposite surfaces of two adjacent prisms are parallel to each other.

[0019] By arranging the two opposite faces of two adjacent prisms to be parallel to each other when the first prism assembly has multiple prisms, the light loss between the two adjacent prisms can be reduced, so as to restore the image of the user's eyes to the greatest extent and improve the accuracy of the tracking system.

[0020] In a possible implementation, the regular-shaped prism includes a triangular prism, a quadrangular prism or a pentagonal prism. The special-shaped prism includes a first side surface and a second side surface that are opposite to each other, and a V-shaped groove extending perpendicularly from the first side surface to the second side surface is provided between the first side surface and the second side surface, and the inner surface of the V-shaped groove is the first reflecting surface.

[0021] By setting the regular prism to include a triangular prism, a quadrangular prism or a pentaprism, the structure of the prism can be simplified and the cost can be reduced. By setting the special-shaped prism to include a first side surface and a second side surface that are opposite to each other, so that the special-shaped prism can receive infrared light from the first side surface and the second side surface, the field of view of the first image acquisition device can be improved, the occurrence of shooting blind spots can be avoided, and the accuracy of the tracking system can be improved.

[0022] In a possible implementation manner, each of the first reflection surfaces is a total reflection surface.

[0023] By setting the first reflection surface as a total reflection surface, the refraction variable of light can be reduced, thereby reducing the field of view loss of the first prism assembly and improving the accuracy of the tracking system.

[0024] In a possible implementation manner, a total reflection film is provided on the first reflection surface.

[0025] By arranging a total reflection film on the first reflection surface, all reflections occurring on the first reflection surface are total reflections. This can reduce the refraction variable of light, thereby reducing the field of view loss of the first prism assembly and improving the accuracy of the tracking system.

[0026] In a possible implementation, a surface of the first prism assembly facing the display module is arranged parallel to the display module. The first emission surface of the first prism assembly is perpendicular to a plane where the display module is located.

[0027] By arranging the side of the first prism assembly facing the display module to be parallel to the display module, the first prism assembly can be easily installed, thereby improving the aesthetics of the tracking system.

[0028] In a possible implementation, the first prism assembly includes a plurality of regular prisms and a special-shaped prism. In the circumferential direction of the display module, the plurality of regular prisms are respectively arranged on both sides of the special-shaped prism. The first image acquisition device is located on a side of the special-shaped prism away from the central axis of the display module. The side of the special-shaped prism opposite to the first image acquisition device is the first exit surface of the first prism assembly.

[0029] By configuring the first prism assembly to include a plurality of regular prisms and a special-shaped prism and configuring the structure as described above, the range of the first prism assembly receiving light reflected from the user's eyes can be increased, that is, the tracking area can be expanded so that the first image acquisition device can accurately capture the image of the desired tracking area. In addition, the regular prisms located on both sides of the special-shaped prism can capture the image of the final area from two directions, thereby preventing blind spots in shooting and improving the accuracy of the tracking system.

[0030] In a possible implementation, in the circumferential direction of the display module, a plurality of regular prisms are symmetrically arranged on both sides of the special-shaped prism.

[0031] By symmetrically arranging multiple regular prisms on both sides of the special-shaped prism, the structure of the first prism assembly can be made symmetrical, which can enhance the industrial aesthetics of the tracking system, facilitate assembly, and reduce assembly difficulty.

[0032] In a possible implementation, a second image acquisition device and a second prism assembly are further included, wherein the second image acquisition device and the second prism assembly are both arranged between the display module and the area where the center of the eyebrows of the user is located.

[0033] The second prism assembly includes two second incident surfaces, two second reflection surfaces and one second exit surface. The two second incident surfaces are respectively facing the areas where the two eyebrows of the user are located. The second incident surfaces are used to receive infrared light reflected from the areas where the two eyebrows of the user are located. The infrared light entering from the two second incident surfaces is respectively reflected by the two second reflection surfaces and then emitted from the second exit surface.

[0034] The second image acquisition device includes a second light incident surface, which is parallel to the second light exit surface, and the second light incident surface is used to receive the infrared light emitted from the second light exit surface to form an image of the area where the two eyebrows of the user are located.

[0035] By setting up a second image acquisition device and a second prism assembly, and setting the second image acquisition device and the second prism assembly between the display module and the area where the user's eyebrows are located, the second image acquisition device is used to capture images of the area where the user's two eyebrows are located. In this way, the tracking system can track the user's expression through the shape of the eyebrows, so that the tracking system can also track expressions and enhance the user experience.

[0036] A second aspect of an embodiment of the utility model provides a head-mounted display device, comprising the tracking system provided in the first aspect.

[0037] The head-mounted display device in the embodiment of the present application is provided with the tracking system of the first aspect. Since the tracking system of the first aspect has a simple structure and the arrangement positions of the first prism assembly and the first image acquisition device in the tracking system are flexible, the difficulty of reconfiguration of the head-mounted display device can be reduced. Especially when the display module is large in size, the first prism assembly and the first image acquisition device can be more flexibly arranged in appropriate positions without affecting other components. Therefore, the cost of the head-mounted display device can be reduced and the user experience can be improved.

[0038] The structure of the present invention and its other purposes and beneficial effects will be more clearly understood through the description of the preferred embodiments in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the utility model or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the utility model. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 It is a structural schematic diagram of a tracking system provided by an embodiment of the utility model;

[0041] Figure 2 It is a structural schematic diagram of a special-shaped prism of a tracking system provided by an embodiment of the utility model;

[0042] Figure 3 It is a partial optical path schematic diagram of a tracking system provided by an embodiment of the utility model;

[0043] Figure 4 It is a schematic diagram of a partial optical path in another direction of a tracking system provided by an embodiment of the utility model;

[0044] Figure 5 It is a partial structural schematic diagram of a tracking system provided by an embodiment of the utility model;

[0045] Figure 6 It is a partial structural schematic diagram of a tracking system provided by an embodiment of the utility model;

[0046] Figure 7 It is a partial structural schematic diagram of a tracking system provided by an embodiment of the utility model;

[0047] Figure 8 It is a structural schematic diagram of a second prism assembly of a tracking system provided by an embodiment of the utility model;

[0048] Fig. 9 It is a partial optical path schematic diagram of a tracking system provided in an embodiment of the utility model.

[0049] Description of reference numerals:

[0050] 10- The user’s face;

[0051] 11- The user's eyes;

[0052] 12-Eyebrows;

[0053] 100-Tracking system;

[0054] 110-display module;

[0055] 120-fill light module;

[0056] 130-first prism assembly;

[0057] 131- first incident surface;

[0058] 132-first exit surface;

[0059] 133-first reflecting surface;

[0060] 134-First prism;

[0061] 135-Second prism;

[0062] 136- Shaped prism;

[0063] 1361-first side;

[0064] 1362-second side;

[0065] 1363-V-shaped groove;

[0066] 140-a first image acquisition device;

[0067] 141- first light incident surface;

[0068] 150: second prism assembly;

[0069] 151- second incident surface;

[0070] 152- second exit surface;

[0071] 153- second reflecting surface;

[0072] 154- triangular pyramidal groove;

[0073] 160- second image acquisition device;

[0074] 161 - second light incident surface. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solution and advantages of the embodiment of the utility model clearer, the technical solution in the embodiment of the utility model will be clearly and completely described below in conjunction with the drawings in the embodiment of the utility model. Obviously, the described embodiment is a part of the embodiment of the utility model, not all of the embodiments. Based on the embodiment of the utility model, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the utility model.

[0076] The tracking system and the head mounted display device in the embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0077] Figure 1 It is a structural diagram of a tracking system 100 provided in an embodiment of the utility model.

[0078] like Figure 1 As shown, the embodiment of the utility model provides a tracking system 100, including a display module 110, a fill light module 120, a first prism assembly 130 and a first image acquisition device 140. The display direction of the display module 110 is toward the user's eyes 11. The fill light module 120 includes a light emitting surface, the light emitting surface is toward the user's face 10, and the fill light module 120 is used to emit infrared light to the user's face 10.

[0079] The first prism assembly 130 is arranged between the display module 110 and the user's eye 11. The first prism assembly 130 includes a first incident surface 131, a first exit surface 132 and at least one first reflection surface 133. The first incident surface 131 faces the user's eye 11. The first incident surface 131 is used to receive the infrared light reflected by the user's eye 11. The infrared light entering from the first incident surface 131 is reflected by at least one first reflection surface 133 and then emitted from the first exit surface 132.

[0080] The first image acquisition device 140 is disposed between the display module 110 and the user's eye 11. The first image acquisition device 140 includes a first light incident surface 141. The first light incident surface 141 is parallel to the first exit surface 132. The first light incident surface 141 is used to receive the infrared light emitted from the first exit surface 132 to form an image of the user's eye 11.

[0081] Exemplarily, the first prism assembly 130 may be spaced apart from the display module 110, wherein in the embodiment of the present application, the distance between the first prism assembly 130 and the display module 110 is not limited. Of course, in some embodiments, the first prism assembly 130 may also be disposed on the outer frame of the display module 110, that is, there is no space between the first prism assembly 130 and the display module 110. In the embodiment of the present application, the relative position of the first prism assembly 130 and the display module 110 is not further limited.

[0082] It should be noted that the user's face may include expression areas such as eyebrows, eyelids, eyes, and forehead. This makes it convenient to apply the tracking system 100 to eye tracking or expression tracking. Of course, in some other embodiments, the face may also include the entire face of the user. In the embodiments of the present application, the specific position of the user's face is not further limited.

[0083] The tracking system 100 in the embodiment of the present application is provided with a display module 110, and the display direction of the display module 110 is set toward the user's eye 11, so that the picture displayed by the display module 110 can be seen by the user. A fill light module 120 is provided, and the fill light module 120 is set to at least be a structure for emitting infrared light to the user's eye 11, so that the fill light module 120 can fill light to the user's eye 11, so that the first image acquisition device 140 can acquire an image of the user's eye area, thereby realizing eye tracking of the user.

[0084] By disposing the first prism assembly 130 between the display module 110 and the user's eye 11 and disposing at least one first reflective surface 133 in the first prism assembly 130, the first prism assembly 130 can change the direction of the infrared light reflected by the user's eye, thereby improving the flexibility of the setting position of the first image acquisition device 140.

[0085] By setting the first prism assembly 130, a folded light path is achieved. The propagation path of the light is folded and bent, and the position of the first image acquisition device 140 is changed. At the same time, a monocular camera can be used to achieve a shooting direction that could only be achieved by a binocular camera before, thereby saving equipment costs.

[0086] By setting at least one first reflection surface 133 in the first prism assembly 130, the infrared light reflected by the user's eye 11 received by the first incident surface 131 can enter the first light incident surface 141 of the first image acquisition device 140 through the first exit surface 132 after at least one reflection, so that the first image acquisition device 140 can capture the image of the user's eye without facing the user's face. That is to say, the number and setting position of the first reflection surfaces 133 can change the angle of the infrared light finally emitted from the first exit surface 132 of the first prism assembly 130, thereby improving the setting position and direction of the first image acquisition device.

[0087] By arranging the first exit surface 132 and the light incident surface of the first image acquisition device 140 to be parallel to each other, the structure and position of the first prism assembly 130 can affect the setting position of the first image acquisition device 140. Therefore, the setting position of the first image acquisition device 140 can be changed by changing the structure or position of the first prism assembly 130, thereby improving the flexibility of the setting position of the first image acquisition device 140.

[0088] Compared with the related art, the tracking system is equipped with multiple cameras, and the angle between the camera optical axis and the display module optical axis needs to be maintained within a certain angle (for example, the camera is close to the user's face for shooting), otherwise the eyelids will cover part of the eye, affecting the accuracy of eye tracking, especially when the display module is large, which will cause the angle between the camera optical axis and the display module optical axis to increase (that is, the camera is diagonally facing the user's face, and the inclination angle is large), making the camera setting more difficult, and affecting the shooting angle, and easily resulting in shooting blind spots. The tracking system in the technical solution of this application is more accurate and can reduce the cost of the head-mounted display device that uses the tracking system.

[0089] By arranging the first light incident surface 141 of the first prism assembly 130 toward the user's eye 11, the first image acquisition device 140 can acquire an image of a specific area, thereby improving the accuracy of eye tracking.

[0090] Exemplarily, the angle between the first light incident surface 141 and the plane where the display module 110 is located can be 90°, 80°, 70°, 60°, 50°, etc., as long as it is greater than 0°. In the embodiment of the present application, the angle between the first light incident surface 141 and the plane where the display module 110 is located is not further limited.

[0091] It should be noted that the tracking system 100 implemented in the present application may also include an eye tracking module, wherein the eye tracking module can be connected to the first image acquisition device 140, and the eye tracking module is used to receive image information acquired by the first image acquisition device 140, establish an eye movement model after processing, and output the result.

[0092] Exemplarily, in the embodiment of the present application, each first reflection surface 133 in the first prism assembly 130 is a total reflection surface. By setting the first reflection surface 133 as a total reflection surface, the refraction variable of light can be reduced, thereby reducing the field angle loss of the first prism assembly 130 and improving the accuracy of the tracking system 100.

[0093] Exemplarily, a total reflection film may be provided on the first reflection surface 133. By providing a total reflection film on the first reflection surface 133, all reflections occurring on the first reflection surface 133 are total reflections, which can reduce the refraction variable of light, thereby reducing the field of view loss of the first prism assembly 130, and improving the accuracy of the tracking system 100.

[0094] Of course, in other embodiments, the shape of the first prism assembly 130 can be adjusted, or the refractive index of the first prism assembly 130 can be changed so that the first reflection surface 133 is a total reflection surface. In the embodiment of the present application, there is no further limitation on how to make the reflections occurring on the first reflection surface 133 total reflections.

[0095] Exemplarily, the side of the first prism assembly 130 facing the display module 110 can be arranged parallel to the display module 110. The first emission surface 132 of the first prism assembly 130 is perpendicular to the plane where the display module 110 is located. By arranging the side of the first prism assembly 130 facing the display module 110 to be parallel to the display module 110, the first prism assembly 130 can be easily installed, thereby improving the aesthetics of the tracking system 100.

[0096] It should be noted that, by setting the first emission surface 132 of the first prism assembly 130 perpendicular to the plane where the display module 110 is located, since the first emission surface 132 is parallel to the first light incident surface 141 of the first image acquisition device 140, the first light incident surface 141 of the first image acquisition device 140 can be perpendicular to the plane where the display module 110 is located, that is, the angle between the first light incident surface 141 and the plane where the display module 110 is located is 90°. In this way, the optical axis of the first light incident surface 141 of the first image acquisition device 140 can be parallel to the display module 110, or it can be said that the first image acquisition device 140 faces the central axis of the display module 110, so that the first image acquisition device 140 does not need to face the user's face 10 directly, and can collect image information of the user's eyes, thereby improving the flexibility of the first image acquisition device 140. In turn, the design flexibility of the head-mounted display device using the tracking system 100 is improved.

[0097] In addition, by adjusting the shape of the first prism assembly 130, the first image acquisition device 140 can have a larger field of view. Compared with the tracking system 100 using two cameras in the related art, the tracking system 100 in the embodiment of the present application has lower cost and higher accuracy.

[0098] It should be noted that one tracking system 100 of the embodiment of the present application can correspond to one eye of the user. If it is to correspond to two eyes of the user, two tracking systems 100 need to be set.

[0099] In the embodiment of the present application, the first prism assembly 130 may include one or more prisms, wherein the prisms may be regular prisms or special prisms, and each prism includes at least one first reflection surface 133 .

[0100] By configuring the first prism assembly 130 to include one or more prisms, the first prism assembly 130 may include at least one first reflection surface 133, so that the infrared light reflected by the user's eye 11 can enter the first image acquisition device 140 after at least one reflection. In other words, the optical path of the infrared light reflected by the user's eye 11 can be changed at least once, thereby improving the flexibility of the setting position of the first image acquisition device 140, so that the first image acquisition device 140 does not need to be limited to a specific position or a specific angle, resulting in a blind spot when the eyeball rotates, thereby affecting the accuracy of the eye tracking system 100.

[0101] It should be noted that a regular prism refers to a prism whose cross section perpendicular to the extension direction of the prism is a regular shape, for example, the cross section of a triangular prism is a triangle, the cross section of a quadrangular prism is a quadrilateral, etc. Of course, regular shapes include but are not limited to equilateral regular shapes, and may also be unequal-sided regular shapes.

[0102] In a possible implementation, the regular prism may include a triangular prism, a quadrangular prism, or a pentaprism. Of course, in some other embodiments, the regular prism may also include a hexagonal prism, etc. In the embodiment of the present application, the shape of the regular prism is not further limited. By configuring the regular prism to include a triangular prism, a quadrangular prism, or a pentaprism, the structure of the prism can be simplified and the cost can be reduced.

[0103] like Figure 2 As shown, the special-shaped prism 136 may include a first side surface 1361 and a second side surface 1362 which are opposite to each other, and a V-shaped groove 1363 extending perpendicularly from the first side surface 1361 to the second side surface 1362 is provided between the first side surface 1361 and the second side surface 1362, and the inner surface of the V-shaped groove 1363 is the first reflecting surface 133. A first emitting surface 132 is provided on a side of the special-shaped prism 136 which is opposite to the opening end of the V-shaped groove 1363.

[0104] By setting the special-shaped prism 136 to include a first side surface 1361 and a second side surface 1362 that are opposite to each other, the special-shaped prism 136 can receive infrared light from the first side surface 1361 and the second side surface 1362, thereby improving the field of view of the first image acquisition device 140, preventing the occurrence of blind spots in shooting, and improving the accuracy of the tracking system 100.

[0105] It should be noted that the irregular prism refers to an irregular shape in the cross section perpendicular to the extension direction of the prism, which can be set according to needs. In the embodiment of the present application, the structure of the irregular prism is not further limited.

[0106] It should be noted that when used, the side of each prism where light enters can be used as the incident surface, the side where light is reflected can be used as the reflection surface, and the side where light is emitted can be used as the exit surface. The opposite surfaces of two adjacent prisms can be the exit surface of one prism and the incident surface of another prism.

[0107] Continue to see Figure 1 As shown, the first prism assembly 130 includes a plurality of regular prisms and a special-shaped prism 136. In the circumferential direction of the display module, the plurality of regular prisms are respectively arranged on both sides of the special-shaped prism 136, and at least one surface of at least some of the regular prisms in the plurality of regular prisms is arranged opposite to the special-shaped prism. In the embodiment of the present application, it can be considered that the direction from the first side surface 1361 to the second side surface 1362 is the same as the circumferential direction of the display module.

[0108] Exemplarily, a plurality of regular prisms may be symmetrically arranged on both sides of the special-shaped prism 136. By symmetrically arranging a plurality of regular prisms on both sides of the special-shaped prism 136, the structure of the first prism assembly 130 can be made symmetrical, the industrial aesthetics of the tracking system 100 can be improved, the assembly is convenient, and the assembly difficulty can be reduced.

[0109] Of course, in some embodiments, multiple regular prisms can also be asymmetrically arranged on both sides of the special-shaped prism 136, which can be specifically set according to the setting positions of the first prism assembly and the first image acquisition device. The setting positions of the multiple regular prisms relative to the special-shaped prism are not further explained here.

[0110] In the radial direction of the display module 110, the first image acquisition device 140 can be located on a side of the special-shaped prism 136 away from the center of the display module 110, and the first image acquisition device 140 is arranged opposite to the special-shaped prism 136. The side of the special-shaped prism 136 opposite to the first image acquisition device 140 is the first emission surface 132 of the first prism assembly 130.

[0111] In some embodiments, in the radial direction of the display module 110 , the regular prism close to the central axis of the display module 110 includes a first incident surface 131 . That is, the first incident surface 131 of the first prism assembly 130 is located on the regular prism close to the central axis of the display module 110 .

[0112] Of course, in other embodiments, the first incident surface 131 of the first prism assembly 130 may also be arranged on other regular prisms. In the embodiment of the present application, the position of the first incident surface 131 of the first prism assembly 130 is not further limited. By arranging the first prism assembly 130 to include multiple regular prisms and one special-shaped prism 136, and arranging it in the above-mentioned structure, the range of the first prism assembly 130 receiving the reflected light from the user's eyes can be increased, that is, the tracking area can be expanded, so that the first image acquisition device 140 can accurately capture the image of the required tracking area. In addition, the regular prisms located on both sides of the special-shaped prism 136 can capture the image of the final area from two directions, preventing the occurrence of shooting blind spots, and improving the accuracy of the tracking system 100.

[0113] In a possible implementation, when the first prism assembly 130 includes multiple prisms, two opposite faces of two adjacent prisms are parallel to each other. When the first prism assembly 130 includes multiple prisms, two opposite faces of two adjacent prisms are arranged to be parallel to each other, so that the light loss between the two adjacent prisms can be reduced, so as to restore the image of the user's eyes to the greatest extent and improve the accuracy of the tracking system 100.

[0114] Exemplarily, the plurality of regular prisms in the first prism assembly 130 may include two quadrangular prisms of the same shape and two pentaprisms of the same shape, wherein the two quadrangular prisms and the two pentaprisms are symmetrically arranged on both sides of the special-shaped prism 136, and the quadrangular prism is closer to the central axis of the display module 110 relative to the pentaprism, one surface of the pentaprism is arranged opposite to the quadrangular prism, and the other surface is arranged opposite to the special-shaped prism 136. The opposite surfaces of the quadrangular prism and the pentaprism are parallel to each other, and the opposite surfaces of the pentaprism and the special-shaped prism 136 are also parallel to each other.

[0115] In the embodiment of the present application, a regular prism closer to the central axis of the display module 110 is used as the first prism 134, that is, a quadrangular prism in the present embodiment, and a regular prism relatively far from the central axis of the display module 110 is used as the second prism 135, that is, a pentaprism in the present embodiment. The first prism 134 and the second prism 135 are arranged at intervals, the second prism 135 is arranged opposite to the first side surface 1361 or the second side surface 1362 of the special-shaped prism 136, the first image acquisition device 140 is arranged opposite to the special-shaped prism 136, and the incident surface of the first prism 134 is the first incident surface 131 of the first prism assembly 130, and the exit surface of the special-shaped prism 136 is the first exit surface 132 of the first prism assembly 130.

[0116] It should be noted that the direction of the central axis of the display module 110 is perpendicular to the plane on which the display module 110 is used for display. When the display module 110 is a cylindrical structure, the central axis of the display module 110 is the central axis of the cylindrical structure.

[0117] Figure 3 is a schematic diagram of the light path in the first prism, as shown in Figure 3 As shown, the light path undergoes two total reflections at the two first reflection surfaces 133 in the first prism 134 of the first prism assembly 130, so that the light path is parallel to the plane where the display module 110 is located. The two first reflection surfaces 133 in the first prism 134 are coated with a total reflection film. Figure 3 As shown, the first incident surface 131 and the first reflective surface 133 may be located in the same plane, wherein a total reflection film is provided on a portion of the surface of the first reflective surface 133 .

[0118] Combination Figure 1 and Figure 4 As shown, the special-shaped prism 136 can be located at 6 o'clock on the clock or other positions suitable for capturing images of the user's eye 11 area from the cheek side, the first prism 134 and the second prism 135 located on the side of the first side 1361 of the special-shaped prism 136 can be arranged at 7 o'clock on the clock or other positions suitable for capturing images of the user's eye 11 area from the cheek side, and the first prism 134 and the second prism 135 located on the side of the second side 1362 of the special-shaped prism 136 can be deployed at 5 o'clock on the clock or other positions suitable for capturing images of the user's eye 11 area from the cheek side.

[0119] Of course, in other embodiments, the special-shaped prism 136, the first prism 134 and the second prism 135 may also be arranged at other positions, as long as the first prism 134 and the second prism 135 are respectively arranged on both sides of the special-shaped prism 136. The specific arrangement can be based on the specific situation and is not further limited in the embodiments of the present application.

[0120] like Figure 4 As shown, the light path is totally reflected once in the second prism 135 of the first prism assembly 130, and then enters perpendicularly to the first side surface 1361 or the second side surface 1362 of the special-shaped prism 136. Finally, the light paths on both sides of the special-shaped prism 136 are totally reflected once in the special-shaped prism 136, and then vertically enter the first exit surface 132, and then vertically enter the first image acquisition device 140.

[0121] That is, the first prism assembly 130 folds the light path through light reflection, and then folds the light path parallel to the plane where the display module 110 is located or other directions suitable for receiving by the special-shaped prism 136 to enter the special-shaped prism 136. The special-shaped prism 136 is located between the first prism 134 and the second prism 135, and can reflect the light on both sides of the special-shaped prism 136 to the first image acquisition device 140. Finally, the shooting effect of the binocular camera is achieved through the monocular camera, thereby saving costs and improving tracking accuracy.

[0122] By setting the first prism assembly 130, a folded light path is achieved. The propagation path of the light is folded and bent, and the position of the first image acquisition device 140 is changed. At the same time, a monocular camera can be used to achieve a shooting direction that could only be achieved by a binocular camera before, thereby saving equipment costs.

[0123] It should be noted that, in the embodiment of the present application, the spacing distance between adjacent prisms is not limited, and can be specifically set according to the size of the device of the tracking system 100, and is not further limited in the embodiment of the present application.

[0124] Of course, in other embodiments, the tracking system 100 may also have other structures. For example, the first prism assembly 130 in the tracking system 100 may include a prism.

[0125] like Figure 5 As shown, the first prism assembly 130 includes a prism whose cross-section is triangular, that is, a three-dimensional prism, and the first reflecting surface 133 of the three-dimensional prism is parallel to the display module 110, and there is a gap between the first prism assembly 130 and the display module 110 so that the incident light is totally reflected at the first reflecting surface 133; the first emitting surface 132 of the first prism assembly 130 is parallel to the first light incident surface 141 of the first image acquisition device 140, which can reduce the refraction variable of the light, reduce the field of view angle loss of the three-dimensional prism, and improve the accuracy of the tracking system 100.

[0126] Exemplarily, the first incident surface 131 may be perpendicular to the line between the geometric center of the prism and the center of the user's eye, so that the shooting area can be located at the center of the user's eyeball, reducing the field of view loss and improving the accuracy of the tracking system 100.

[0127] It should be noted that, in some embodiments, the structure of the first prism assembly 130 can be simplified by configuring the first prism assembly 130 as a triangular prism, thereby reducing the cost.

[0128] Of course, in other embodiments, the first prism assembly 130 may also be configured as other structures, such as Figure 6As shown, the first prism assembly 130 may also include two prisms arranged at intervals, wherein, in the embodiment of the present application, the prism closer to the central axis of the display module 110 is used as the first prism 134, and the prism relatively far from the central axis of the display module 110 is used as the second prism 135. The first prism 134 and the second prism 135 are both arranged parallel to the display module 110, the first image acquisition device 140 is arranged opposite to the second prism 135, and the incident surface of the first prism 134 is the first incident surface 131 of the first prism assembly 130, and the exit surface of the second prism 135 is the first exit surface 132 of the first prism assembly 130.

[0129] like Figure 6 As shown, the cross-section of the first prism 134 is a triangle, the incident surface of the first prism 134 is the first incident surface 131 of the first prism assembly 130, the first incident surface 131 can be perpendicular to the line connecting the center of the first prism 134 and the center of the tracked area, the angle of the reflecting surface in the first prism 134 can be determined according to the structural design, and the exit surface of the first prism 134 can be perpendicular to the exiting light.

[0130] The cross section of the second prism 135 is a quadrilateral, and the incident surface of the second prism 135 is perpendicular to the outgoing light of the first prism 134. The incident light is totally reflected on the reflection surface of the second prism 135 and then is emitted perpendicular to the outgoing surface of the second prism, and the light emitted from the second prism 135 is perpendicular to the first light incident surface 141 of the first image acquisition device 140. It should be noted that the angles of the reflection surface and the outgoing surface of the second prism 135 can be determined according to the structural design, and are not further limited in the embodiments of the present application.

[0131] In the embodiment of the present application, by forming a folded optical path between multiple prisms, light can be emitted at a specific angle, which can increase the flexibility of the arrangement of the first image acquisition device 140. Through this solution, the first image acquisition device 140 can capture an image of a specific area.

[0132] In other embodiments, Figure 7 As shown, the tracking system 100 may also include a second image acquisition device 160 and a second prism assembly 150. The second image acquisition device 160 and the second prism assembly 150 are both disposed between the display module 110 and the area where the center of the user's eyebrows is located. The second image acquisition device 160 can acquire images of the area where the user's two eyebrows 12 are located, thereby enabling the tracking system 100 to perform expression tracking.

[0133] Exemplarily, the second prism assembly 150 includes two second incident surfaces 151, two second reflection surfaces 153 and a second exit surface 152. The two second incident surfaces 151 are respectively facing the areas where the two eyebrows 12 of the user are located. The second incident surfaces 151 are used to receive infrared light reflected from the areas where the two eyebrows 12 of the user are located. The infrared light entering from the two second incident surfaces 151 is respectively reflected by the two second reflection surfaces 153 and then emitted from the second exit surface 152.

[0134] Among them, the second image acquisition device 160 includes a second light incident surface 161, the second light incident surface 161 is parallel to the second exit surface 152, and the second light incident surface 161 is used to receive infrared light emitted from the second exit surface 152 to form an image of the area where the two eyebrows 12 of the user are located.

[0135] By setting up a second image acquisition device 160 and a second prism assembly 150, and setting the second image acquisition device 160 and the second prism assembly 150 between the display module 110 and the area where the user's eyebrows are located, the second image acquisition device 160 is used to capture images of the area where the two eyebrows 12 of the user are located. In this way, the tracking system 100 can track the user's expression through the shape of the eyebrows 12, so that the tracking system 100 can also track expressions and enhance the user experience.

[0136] like Figure 8 As shown, the second prism assembly 150 is a special-shaped prism, wherein the two second incident surfaces 151 of the second prism assembly 150 are arranged at an angle and respectively face the areas where the two eyebrows 12 of the user are located, the second exit surface 152 is perpendicular to the two second incident surfaces 151, and a triangular pyramid-shaped groove 154 is provided on the side of the second prism assembly 150 away from the second exit surface 152, and the tip of the triangular pyramid-shaped groove 154 faces the second exit surface 152, and the inner sides of the two side surfaces of the triangular pyramid-shaped groove 154 form two second reflection surfaces 153 of the second prism assembly 150, and a total reflection film can be provided on the second reflection surface 153, so that the reflection occurring on the second reflection surface 153 is total reflection.

[0137] like Fig. 9 As shown, the light reflected by the user's eyebrow 12 is nearly perpendicular to the second incident surface 151, avoiding the light from being totally reflected on the second incident surface 151. The light is totally reflected on the second reflecting surface 153 of the second prism assembly 150, so that the light is vertically emitted to the second exit surface 152, and then vertically enters the second image acquisition device 160. Finally, an image acquisition device is used to capture the image of the expression tracking area such as the eyebrow 12 that originally could only be captured by two cameras, and the expression tracking of the two eyebrows 12 is synchronized at the same time. The accuracy of the tracking system 100 is improved.

[0138] It should be noted that, in other embodiments, the second prism assembly 150 may be configured as other structures, for example, a structure including multiple prisms. In the embodiment of the present application, the structure of the second prism assembly 150 is not further limited.

[0139] The tracking system 100 implemented in the present application may also include an expression tracking module, wherein the expression tracking module may be connected to the second image acquisition device 160, and the expression tracking module is used to receive image information acquired by the second image acquisition device 160, and after processing, establish an expression model to simulate the expression.

[0140] It should be noted that the drawings in the above embodiments are all schematic diagrams, and the proportions in the drawings are not to be used as a reference for the proportions of real devices.

[0141] An embodiment of the utility model further provides a head-mounted display device, comprising the tracking system in any of the above embodiments.

[0142] Exemplarily, the head-mounted display device may be MR glasses, VR glasses, or AR glasses, etc. In the embodiments of the present application, the specific type of the head-mounted display device is not further limited.

[0143] The head-mounted display device in the embodiment of the present application is provided with the above-mentioned tracking system. Since the tracking system has a simple structure and the arrangement positions of the first prism assembly and the first image acquisition device in the tracking system are flexible, the difficulty of reconfiguration of the head-mounted display device can be reduced. Especially when the display module is large in size, the first prism assembly and the first image acquisition device can be more flexibly arranged in appropriate positions without affecting other components. Therefore, the cost of the head-mounted display device can be reduced and the user experience can be improved.

[0144] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0145] In the description of the present invention, it should be understood that the terms "including" and "having" and any variations thereof used herein are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0146] Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral one; it can be directly connected or indirectly connected through an intermediate medium, and it can make the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this utility model can be understood according to specific circumstances. In addition, the terms "first", "second", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the utility model, rather than to limit it. Although the utility model has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiments of the utility model.

Claims

1. A tracking system, It is characterized in that include: A display module, wherein the display direction is toward the user's eyes; A fill light module, the fill light module is used to emit infrared light to the face of the user; a first prism assembly, disposed between the display module and the user's eyes, the first prism assembly comprising a first incident surface, a first exit surface and at least one first reflection surface, the first incident surface facing the user's eyes, the first incident surface being used to receive infrared light reflected by the user's eyes, and the infrared light entering from the first incident surface being reflected by the at least one first reflection surface and then emitted from the first exit surface; The first image acquisition device is arranged between the display module and the user's eyes. The first image acquisition device includes a first light incident surface, which is parallel to the first light exit surface. The first light incident surface is used to receive infrared light emitted from the first light exit surface to form an image of the user's eyes.

2. The tracking system according to claim 1, It is characterized in that The first prism assembly includes one or more prisms; wherein, The prisms are regular-shaped prisms or special-shaped prisms, and each of the prisms includes at least one first reflecting surface.

3. The tracking system according to claim 2, It is characterized in that When the first prism assembly includes a plurality of prisms, two opposite surfaces of two adjacent prisms are parallel to each other.

4. The tracking system according to claim 2 or 3, It is characterized in that The regular-shaped prism includes a triangular prism, a quadrangular prism or a pentaprism; The special-shaped prism includes a first side surface and a second side surface facing away from each other, a V-shaped groove extending perpendicularly from the first side surface to the second side surface is provided between the first side surface and the second side surface, and the inner surface of the V-shaped groove is the first reflecting surface.

5. A tracking system according to any one of claims 1 to 3, It is characterized in that Each of the first reflection surfaces is a total reflection surface; wherein, A total reflection film is disposed on the first reflection surface.

6. The tracking system according to claim 4, It is characterized in that A side of the first prism assembly facing the display module is arranged parallel to the display module; The first emission surface of the first prism assembly is perpendicular to the plane where the display module is located.

7. The tracking system according to claim 1, It is characterized in that The first prism assembly includes a plurality of regular prisms and one special-shaped prism; wherein, In the circumferential direction of the display module, a plurality of regular prisms are respectively arranged on both sides of the special-shaped prism; The first image acquisition device is located on a side of the special-shaped prism away from the central axis of the display module; The surface of the special-shaped prism opposite to the first image acquisition device is the first emission surface of the first prism assembly.

8. The tracking system according to claim 7, It is characterized in that In the circumferential direction of the display module, a plurality of regular prisms are symmetrically arranged on both sides of the special-shaped prism.

9. The tracking system according to any one of claims 1 to 3, It is characterized in that It also includes a second image acquisition device and a second prism assembly; wherein, The second image acquisition device and the second prism assembly are both arranged between the display module and the area where the center of the eyebrows of the user is located; The second prism assembly includes two second incident surfaces, two second reflection surfaces and one second exit surface, the two second incident surfaces are respectively facing the areas where the two eyebrows of the user are located, the second incident surfaces are used to receive infrared light reflected from the areas where the two eyebrows of the user are located, and the infrared light entering from the two second incident surfaces is respectively reflected by the two second reflection surfaces and then emitted from the second exit surface; The second image acquisition device includes a second light incident surface, which is parallel to the second light exit surface, and the second light incident surface is used to receive the infrared light emitted from the second light exit surface to form an image of the area where the two eyebrows of the user are located.

10. A head mounted display device, It is characterized in that A tracking system comprising any one of claims 1-9.