Optical Tracking Method and System for a Controller under Strong Outdoor Light

Abstract

The present invention provides an optical tracking method for a controller under strong outdoor light. First, for the optical tracking method and system of the controller under strong outdoor light, the surface of the tracking component under strong light is divided into a tracking area and a non-tracking area; wherein, a cursor combination or a tracking pattern is set in the tracking area; then, a tracking camera is used to capture the surface of the tracking component to form a display image of the tracking area; wherein, a filter is used to make the tracking area be normally exposed in the display image, and make the brightness of the non-tracking area darker than that of the tracking area, and then the position information of the tracking area is obtained according to the display image, and the position trajectory and attitude information of the tracking component are determined according to the position information and the inertial information obtained by a preset inertial sensor, thereby solving the problem of interference of ambient light on the light spot or fixed pattern used for tracking, and enabling the optical controller to be normally used outdoors or in an environment with strong light interference.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and more specifically, to a controller optical tracking method and system under outdoor strong light. Background Art

[0002] Due to the progress of technology and the diversified development of market demands, virtual reality systems are becoming increasingly common and are applied in many fields, such as computer games, health and safety, industry, and education and training. For example, hybrid virtual reality systems are being integrated into various aspects of life, such as mobile communication devices, game consoles, personal computers, cinemas, theme parks, university laboratories, student classrooms, and hospital fitness rooms.

[0003] Traditional optical tracking 6DOF controllers usually use visible light or infrared cameras to capture the spot combinations (cursor combinations) or fixed patterns (tracking patterns) that emit light actively or passively on the handle, and determine the position information of the controller in space and the attitude information of the controller through the relative position relationship of the spot combinations or fixed patterns on the controller and the inertial sensor (IMU). This tracking method depends on the shape and relative position of the spot combinations or fixed patterns captured by the camera. Therefore, this tracking method can only be realized indoors or in places with slightly darker ambient light. If the controller is transferred to outdoor sunlight or a place with strong ambient light, the light sources in the ambient light with similar wavelength bands will interfere with the spot combinations or fixed patterns that emit light actively or passively on the handle for tracking, resulting in overexposure on the surface of the controller, making it impossible for the camera to clearly obtain the boundary between the area where the spot combinations or fixed patterns are set and the area where no spot combinations or fixed patterns are set, thus unable to clearly capture the spot combinations or fixed patterns, and further causing the loss or abnormal situation of controller tracking.

[0004] Therefore, there is an urgent need for a controller optical tracking method under outdoor strong light that can still clearly capture the spot combinations or fixed patterns on the handle even in an outdoor strong light environment to accurately complete optical tracking. Summary of the Invention

[0005] In view of the above problems, the purpose of the present invention is to provide a controller optical tracking method under outdoor strong light to solve the problem that when the controller is transferred to outdoor sunlight or a place with strong ambient light, the light sources in the ambient light with similar wavelength bands will interfere with the spot combinations or fixed patterns that emit light actively or passively on the handle for tracking, resulting in overexposure on the surface of the controller, making it impossible for the camera to clearly obtain the boundary between the area where the spot combinations or fixed patterns are set and the area where no spot combinations or fixed patterns are set, thus unable to clearly capture the spot combinations or fixed patterns, and further causing the loss or abnormal situation of controller tracking.

[0006] An optical tracking method for a controller under strong outdoor light provided by the present invention includes:

[0007] Dividing the surface of the tracking component of the controller under strong light into a tracking area and a non-tracking area; wherein, a cursor combination or a tracking pattern is set in the tracking area;

[0008] Capturing the surface of the tracking component by a tracking camera to form a display image of the tracking area; wherein, the tracking area is normally exposed in the display image through a light filtering member, and the brightness of the non-tracking area is darker than that of the tracking area;

[0009] Obtaining the position information of the tracking area according to the display image, and determining the position trajectory and attitude information of the tracking component according to the position information and the inertial information obtained by a preset inertial sensor.

[0010] Preferably, the light filtering member is an absorbent material.

[0011] Preferably, the process of normally exposing the tracking area in the display image through a light filtering member includes:

[0012] Setting an absorbent material in the non-tracking area to filter the interference wavelength band under strong light in the non-tracking area; wherein,

[0013] The absorbent material is a sleeved absorbent material, a covering absorbent material or a spray-type light-shielding material.

[0014] Preferably, if the absorbent material is a sleeved absorbent material, the process of setting the sleeved absorbent material in the non-tracking area includes:

[0015] Sleeving the sleeved absorbent material on the tracking component;

[0016] Setting holes in the sleeved absorbent material so that the sleeved absorbent material covers the non-tracking area and the holes correspond to the tracking area.

[0017] Preferably, the sleeved absorbent material is a silica gel sleeve.

[0018] Preferably, the light filtering member is a filter.

[0019] Preferably, the process of normally exposing the tracking area in the display image through a light filtering member includes:

[0020] Setting the filter at the front end of the tracking camera to filter the ambient light when the tracking camera captures the tracking component; wherein,

[0021] If the tracking camera is a visible light camera, the filter is a heavy color filter;

[0022] If the tracking camera is an infrared light camera, the light transmission filtering range of the filter is 750 nanometers to 900 nanometers.

[0023] The present invention also provides an optical tracking system for a controller under strong outdoor light, which implements the optical tracking method for a controller under strong outdoor light as described above, and includes a tracking component of the controller under strong light, a tracking camera, a filtering component, a processor, and an inertial sensor for obtaining inertial information of the tracking component; wherein,

[0024] A tracking area and a non-tracking area are defined on the surface of the tracking component; and a cursor combination or a tracking pattern is set in the tracking area;

[0025] The tracking camera is used to capture the surface of the tracking component to form a display image of the tracking area;

[0026] The filtering component is used to make the tracking area normally exposed in the display image, and make the brightness of the non-tracking area darker than the brightness of the tracking area;

[0027] The processor is used to obtain the position information of the tracking area according to the display image, and determine the position trajectory and attitude information of the tracking component according to the position information and the inertial information obtained by the preset inertial sensor.

[0028] Preferably, the filtering component is a light-absorbing material;

[0029] The light-absorbing material is a sleeved light-absorbing material, a covering light-absorbing material, or a spray-type light-shielding material.

[0030] Preferably, the filtering component is a filter;

[0031] The filter is arranged at the front end of the tracking camera; and the type of the filter corresponds to the type of the tracking camera; wherein,

[0032] If the tracking camera is a visible light camera, the filter is a heavy color filter;

[0033] If the tracking camera is an infrared light camera, the light transmission filtering range of the filter is 750 nanometers to 900 nanometers.

[0034] As can be seen from the above technical solution, for the optical tracking method and system of the controller under strong outdoor light provided by the present invention, the surface of the tracking component under strong light is first divided into a tracking area and a non-tracking area; wherein, a cursor combination or a tracking pattern is set in the tracking area; then, the surface of the tracking component is captured by a tracking camera to form a display image of the tracking area; wherein, a filter is used to make the tracking area normally exposed in the display image, and make the brightness of the non-tracking area darker than that of the tracking area, and then the position information of the tracking area is obtained according to the display image, and the position trajectory and attitude information of the tracking component are determined according to the position information and the inertial information obtained by a preset inertial sensor, thereby solving the problem of interference of ambient light on the light spot or fixed pattern for tracking, and enabling the optical controller to be normally used outdoors or in an environment with strong light interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] By referring to the following description of the specification in conjunction with the drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more apparent and easier to understand. In the drawings:

[0036] Figure 1 is a flowchart of the optical tracking method of the controller under strong outdoor light according to an embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of the area of the tracking component in the optical tracking method of the controller under strong outdoor light according to an embodiment of the present invention;

[0038] Figure 3 is a schematic diagram of the optical tracking system of the controller under strong outdoor light according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] If the controller is transferred to outdoors in sunlight or a place with strong ambient light, the light sources in the ambient light with similar wavelength bands will interfere with the active or passive light spot combinations or fixed patterns for tracking on the handle, resulting in overexposure on the surface of the controller, making it impossible for the camera to clearly obtain the boundary between the area where the light spot combination or fixed pattern is set and the area where the light spot combination or fixed pattern is not set, thus unable to clearly capture the light spot combination or fixed pattern, and further causing the controller tracking to be lost or abnormal.

[0040] In view of the above problems, the present invention provides an optical tracking method and system of a controller under strong outdoor light, and the following will describe in detail the specific embodiments of the present invention with reference to the drawings.

[0041] In order to illustrate the optical tracking method and system of the controller under strong outdoor light provided by the present invention, Figure 1 an exemplary indication is made for the optical tracking method of the controller under strong outdoor light in the embodiment of the present invention;Figure 3 Exemplary labeling is performed on the controller optical tracking system under strong outdoor light in the embodiments of the present invention.

[0042] The following description of the exemplary embodiments is merely illustrative in nature and is in no way intended as a limitation on the present invention or its application or use. Technologies and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such technologies and devices should be regarded as part of the specification.

[0043] As Figure 1 shown, the controller optical tracking method under strong outdoor light in the embodiments of the present invention provided by the present invention includes:

[0044] S1: Divide the surface of the tracking component of the controller under strong light into a tracking area and a non-tracking area; wherein, a cursor combination or a tracking pattern is set in the tracking area;

[0045] S2: Capture the surface of the tracking component through a tracking camera to form a display image of the tracking area; wherein, the tracking area is normally exposed in the display image through a filter, and the brightness of the non-tracking area is darker than that of the tracking area;

[0046] S3: Obtain the position information of the tracking area according to the display image, and determine the position trajectory and attitude information of the tracking component according to the position information and the inertial information obtained by a preset inertial sensor.

[0047] In this embodiment, the tracking component can be any component with a tracking function in the controller of the VR system. In this embodiment, the tracking component is a handle, specifically the front end part of the handle.

[0048] Figure 2The tracking component in the controller is shown. Here, the preset light spot combination or fixed pattern area on the surface of the controller tracking component is defined as area A; the area on the surface of the controller tracking component other than the light spot combination or fixed pattern is defined as area B; the reflection area formed by the ambient light on the surface of the tracking component is defined as area C. In the traditional technology, if the tracking component in the controller is placed in a strong light environment, the boundary between area A and area B cannot be distinguished in area C, so the image of the light spot combination or fixed pattern cannot be clearly captured. Specifically, in an indoor environment or without ambient light interference, the light spots or fixed patterns in area A can be accurately captured by the tracking camera, and then the position and attitude information of the controller in space can be calculated based on the inertial sensor (IMU) and the relative position relationship. However, in an outdoor environment or under strong light interference, the ambient light reflection causes overexposure of the entire area C, resulting in a large-area exposure in the tracking camera covering the light spots or fixed patterns in area A, and further causing the camera to fail to obtain accurate information on the light spot combination and fixed pattern, so that the position information of the controller in space cannot be calculated.

[0049] As Figure 1 、 Figure 2 Collectively shown, step S1 is the process of dividing the surface of the tracking component of the controller under strong light into a tracking area and a non-tracking area; among them, a cursor combination or a tracking pattern is set in the tracking area.

[0050] Specifically, in this embodiment, the tracking component is a handle, that is, the front end part of the handle with a cursor combination or a tracking pattern is divided into areas. The cursor combination or the tracking pattern is used as the tracking area, and is marked as area A in Figure 2 ; the area other than the cursor combination or the tracking pattern is used as the non-tracking area, and is marked as area B in Figure 2 ; thus, the tracking area and the non-tracking area are processed differently, so as to reduce the exposure in the outdoor strong light environment.

[0051] As Figure 1 shown, step S2 is the process of capturing the surface of the tracking component through a tracking camera to form a display image of the tracking area. In this embodiment, it is to capture the cursor combination or the tracking pattern set on the handle through the tracking camera in the VR system to form a display image, so as to obtain the position information of the handle, and then track the handle according to the position information; among them, the tracking camera can be a visible light camera or an infrared camera, and no specific limitation is made here.

[0052] In step S2, the process of making the tracking area be normally exposed in the display image of the tracking camera through the light filter and making the brightness of the non-tracking area darker than that of the tracking area can enable the tracking camera to capture the cursor combination or the tracking pattern more clearly when the brightness of the non-tracking area is darker than that of the tracking area, thereby improving the accuracy of the tracking handle.

[0053] As Figure 1 , Figure 2 collectively shown, in a specific embodiment, the light filter is a light-absorbing material. In this embodiment, the process of making the tracking area be normally exposed in the display image through the light filter includes:

[0054] SA21: Set a light-absorbing material in the non-tracking area to make the light-absorbing material filter the interference bands under strong light in the non-tracking area; it should be noted that the light-absorbing material can be a sleeved light-absorbing material, that is, the sleeved light-absorbing material is sleeved in the non-tracking area; the light-absorbing material can also be a covering light-absorbing material, that is, the non-tracking area is directly made of a covering light-absorbing material that can absorb light, that is to say, the production material of the non-tracking area on the handle is a light-absorbing material; the light-absorbing material can also be a spray-type light-absorbing material, that is, the spray-type light-absorbing material is sprayed on the non-tracking area, and no specific limitation is made here.

[0055] To illustrate the process of setting the light-absorbing material in detail, in this embodiment, taking the light-absorbing material as a sleeved light-absorbing material as an example, if the light-absorbing material is a sleeved light-absorbing material, then the process of setting the sleeved light-absorbing material in the non-tracking area includes:

[0056] SA211: Sleeve the sleeved light-absorbing material on the tracking component;

[0057] SA212: Set holes in the sleeved light-absorbing material so that the sleeved light-absorbing material covers the non-tracking area and the holes correspond to the tracking area;

[0058] Specifically, as Figure 1 , Figure 2As shown together, in this specific embodiment, by adding a light-absorbing material in area B (non-tracking area), the specific type and material of the light-absorbing material are not specifically limited. In a more specific embodiment, the light-absorbing material is a sleeved light-absorbing material, and the sleeved light-absorbing material can be any material with elasticity, convenient for sleeving and light-absorbing. In this embodiment, the sleeved light-absorbing material is a silicone sleeve for easy sleeving, that is, a silicone sleeve is sleeved on the handle. The silicone sleeve can absorb and filter light. The silicone sleeve only covers the non-filtering area and has holes corresponding to the filtering area, so that the silicone sleeve does not cover the tracking area but only the non-tracking area, that is, only the tracking area is exposed; the silicone sleeve covering area B filters out the interference bands in the ambient light, so that area B does not reflect or only has a small amount of diffuse reflection; thus, only the spot combination (cursor combination) in area A or the surface of the tracking pattern reflects the full band or effective band of the ambient light, avoiding the situation where the reflection in area B affects the clear capture of the reflection in area A, and further preventing the generation of overexposure in area C, so that the cursor combination or tracking pattern in area A can be clearly captured by the tracking camera.

[0059] As Figure 1 shown, in another specific embodiment, the light filter is a filter film. In this specific embodiment, the process of enabling the tracking area to be normally exposed in the displayed image through the light filter includes:

[0060] SB21: Set a filter film at the front end of the tracking camera to enable the filter film to filter the ambient light when the tracking camera captures the tracking component; wherein,

[0061] if the tracking camera is a visible light camera, the filter film is a heavy color filter film;

[0062] if the tracking camera is an infrared light camera, the light transmission and filtering range of the filter film is 750 nanometers to 900 nanometers;

[0063] Specifically, as Figure 1 , Figure 2 shown together, in this specific embodiment, by adding a filter film at the front end of the tracking camera to weaken the strong light in the external environment, the filter film allows the band of the tracking light source to pass through, reduces or filters out the interference of other bands, and at the same time reduces the gain and exposure of the tracking camera, further reducing the interference of the ambient light in area B on area A until the difference between area A and area B can be clearly distinguished in the displayed image of the tracking camera, then the image of area A can be segmented, so as to accurately obtain the position information of the tracking area, and thus more accurately track the handle.

[0064] It should be noted that the above two specific embodiments can be applied separately, and of course, the above two embodiments can also be applied simultaneously, which is not limited here.

[0065] As Figure 1 shown, step S3 is a process of determining the position trajectory and attitude information of the tracking component according to the position information and the inertial information obtained by the preset inertial sensor. After step S2 can make the tracking area be normally exposed in the display image of the tracking camera through the filter, the position information of the tracking area can be accurately captured, and then the position trajectory and attitude information of the tracking component can be determined according to the accurately captured position information of the tracking area and the inertial information obtained by the preset inertial sensor, improving the accuracy of the overall position trajectory and attitude information.

[0066] As described above, the optical tracking method for the controller under outdoor strong light provided by the present invention first divides the surface of the tracking component under strong light into a tracking area and a non-tracking area; wherein, a cursor combination or a tracking pattern is set in the tracking area; then, the surface of the tracking component is captured by a tracking camera to form a display image of the tracking area; wherein, the tracking area is made to be normally exposed in the display image through a filter, and the brightness of the non-tracking area is darker than that of the tracking area, and then the position information of the tracking area is obtained according to the display image, and the position trajectory and attitude information of the tracking component are determined according to the position information and the inertial information obtained by the preset inertial sensor, thereby solving the problem of interference of ambient light on the light spot or fixed pattern for tracking, and enabling the optical controller to be normally used outdoors or in an environment with strong light interference.

[0067] As Figure 3 shown, the present invention also provides an optical tracking system 100 for a controller under outdoor strong light to implement the above optical tracking method for a controller under outdoor strong light, including a tracking component 101, a tracking camera 102, a filter 103, a processor 104 and an inertial sensor 105 for obtaining the inertial information of the tracking component of the controller under strong light; wherein,

[0068] a tracking area and a non-tracking area are divided on the surface of the tracking component 101; and, a cursor combination or a tracking pattern is set in the tracking area;

[0069] the tracking camera 102 is used to capture the surface of the tracking component to form a display image of the tracking area;

[0070] the filter 103 is used to make the tracking area be normally exposed in the display image, and make the brightness of the non-tracking area darker than that of the tracking area;

[0071] the processor 104 is used to obtain the position information of the tracking area according to the display image, and determine the position trajectory and attitude information of the tracking component according to the position information and the inertial information obtained by the preset inertial sensor.

[0072] InFigure 3 In one embodiment shown, the light filter 103 is a light-absorbing material;

[0073] The specific type of the light-absorbing material is not specifically limited herein. It can be a sleeved light-absorbing material, a covering light-absorbing material, or a spray-type light-shielding material. Among them, if a sleeved light-absorbing material is used, the sleeved light-absorbing material can be sleeved in the non-tracking area; if a covering light-absorbing material is used, the non-tracking area is directly made of a covering light-absorbing material capable of absorbing light, that is, the material for making the non-tracking area on the handle is a light-absorbing material; if a spray-type light-absorbing material is used, the spray-type light-absorbing material is sprayed on the non-tracking area.

[0074] In Figure 3 In another specific embodiment shown, the light filter is a filter film;

[0075] The filter film is arranged at the front end of the tracking camera (not shown in the figure); and the type of the filter film corresponds to the type of the tracking camera. Among them,

[0076] if the tracking camera is a visible light camera, the filter film is a heavy color filter film;

[0077] if the tracking camera is an infrared light camera, the light transmission filtering range of the filter film is 750 nanometers to 900 nanometers.

[0078] It should be noted that the above two specific embodiments can be applied alone or jointly, which is not limited herein.

[0079] As can be seen from the above embodiments, in the optical tracking system 100 of the controller under strong outdoor light provided by the present invention, a tracking area and a non-tracking area are defined on the surface of the tracking component 101 under strong light. Among them, a cursor combination or a tracking pattern is arranged in the tracking area; then, the surface of the tracking component is captured by the tracking camera 102 to form a display image of the tracking area; and the light filter 103 is used to make the tracking area be normally exposed in the display image, and make the brightness of the non-tracking area darker than that of the tracking area. Then, the processor 104 obtains the position information of the tracking area according to the display image, and determines the position trajectory and attitude information of the tracking component according to the position information and the inertial information obtained by the preset inertial sensor 105, thereby solving the problem of interference of ambient light on the spot or fixed pattern for tracking, and enabling the optical controller to be normally used outdoors or in an environment with strong light interference.

[0080] The method and system for optical tracking of a controller under strong outdoor light in augmented reality proposed according to the present invention have been described by way of example with reference to the accompanying drawings. However, those skilled in the art should understand that various improvements can be made to the above-mentioned method and system for optical tracking of a controller under strong outdoor light in the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the appended claims.

Claims

1. An optical tracking method for a controller under strong outdoor light, characterized in that, Including: Dividing the surface of the tracking component of the controller under strong light into a tracking area and a non-tracking area; wherein, the tracking area is a cursor combination or a tracking pattern, and the non-tracking area is the area other than the cursor combination or the tracking pattern; Capturing the surface of the tracking component through a tracking camera to form a display image of the tracking area; wherein, the tracking area is normally exposed in the display image through a light filter arranged in the non-tracking area, and the brightness of the non-tracking area is darker than that of the tracking area; Obtaining the position information of the tracking area according to the display image, and determining the position trajectory and attitude information of the tracking component according to the position information and the inertial information obtained by a preset inertial sensor.

2. The optical tracking method for a controller under strong outdoor light according to claim 1, characterized in that, The light filter is a light-absorbing material.

3. The optical tracking method for a controller under strong outdoor light according to claim 2, characterized in that, The process of making the tracking area be normally exposed in the display image through a light filter arranged in the non-tracking area includes: Arranging a light-absorbing material in the non-tracking area to filter the interference band under strong light in the non-tracking area; wherein, the light-absorbing material is a sleeved light-absorbing material, a covering light-absorbing material or a sprayed light-shielding material.

4. The optical tracking method for a controller under strong outdoor light according to claim 3, characterized in that, If the light-absorbing material is a sleeved light-absorbing material, the process of arranging the sleeved light-absorbing material in the non-tracking area includes: Sleeving the sleeved light-absorbing material on the tracking component; Arranging holes in the sleeved light-absorbing material so that the sleeved light-absorbing material covers the non-tracking area and the holes correspond to the tracking area.

5. The optical tracking method for a controller under strong outdoor light according to claim 4, characterized in that, The sleeved light-absorbing material is a silica gel sleeve.

6. The optical tracking method for a controller under strong outdoor light according to claim 1, characterized in that, The light filter is a filter.

7. The optical tracking method for a controller under strong outdoor light according to claim 6, characterized in that, The process of making the tracking area be normally exposed in the display image through a light filter includes: Arranging the filter at the front end of the tracking camera to filter the ambient light when the tracking camera captures the tracking component; wherein, If the tracking camera is a visible light camera, the filter is a heavy color filter; If the tracking camera is an infrared light camera, the light transmission filtering range of the filter is 750 nanometers to 900 nanometers.

8. An optical tracking system for a controller under strong outdoor light, which implements the optical tracking method for a controller under strong outdoor light according to any one of claims 1-7, including a tracking component of the controller under strong light, a tracking camera, a light filter disposed in the non-tracking area, a processor, and an inertial sensor for obtaining inertial information of the tracking component; wherein, A tracking area and a non-tracking area are divided on the surface of the tracking component; and a cursor combination or a tracking pattern is arranged in the tracking area; The tracking camera is used to capture the surface of the tracking component to form a display image of the tracking area; The light filter is used to make the tracking area be normally exposed in the display image and make the brightness of the non-tracking area darker than that of the tracking area; The processor is used to obtain the position information of the tracking area according to the display image, and determine the position trajectory and attitude information of the tracking component according to the position information and the inertial information obtained by a preset inertial sensor.

9. The optical tracking system for a controller under strong outdoor light according to claim 8, characterized in that, The light filter is a light-absorbing material; The light-absorbing material is a sleeved light-absorbing material, a covering light-absorbing material or a sprayed light-shielding material.

10. The optical tracking system for a controller under strong outdoor light according to claim 8, characterized in that, The light filter is a filter; The filter is arranged at the front end of the tracking camera; and the type of the filter corresponds to the type of the tracking camera; wherein, If the tracking camera is a visible light camera, the filter is a heavy color filter; If the tracking camera is an infrared light camera, the light transmission filtering range of the filter is 750 nanometers to 900 nanometers.

Images