Regulation and control method, tracking method, system, equipment and storage medium

Abstract

The invention discloses a control method and system, a tracking method and system, equipment and a storage medium. The control method comprises: acquiring an image of a mobile device (100) at a current frame time using a detector (200), the mobile device (100) being equipped with a plurality of light output units (110) for outputting signal light (S1); predicting a motion state of the mobile device (100) at the next frame time based on the image (S2); and obtaining configuration information of the light output state of each of the light output units (110) at the next frame time based on the motion state of the mobile device (100) at the next frame time to control the light output units (110) (S3). The invention facilitates adaptive adjustment of the light output states of all the light output units (110) based on the real-time state of the mobile device (100), which facilitates the light output units (110) to achieve an accurate and low-power light output state to ensure the use accuracy of the mobile device (100) and reduce the power consumption of the mobile device (100).

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of pose tracking, and more particularly to control methods and tracking methods, systems, devices, and storage media. Background Art

[0002] Virtual reality (VR) / augmented reality (AR) / mixed reality (MR) technologies can provide simulated three-dimensional digital experiences, allowing people to experience any virtual or real world in an immersive manner without being restricted by their actual location. A six-degree-of-freedom (6DoF) handheld controller is an important means of realizing human-computer interaction in VR devices. With the development of the VR technology field, head-mounted 6DoF all-in-one machines have gradually become mainstream, and 6DoF handheld controllers have gradually become a standard configuration for head-mounted 6DoF all-in-one machines.

[0003] Sensors used in 6DoF handheld controllers can be classified into optical, electromagnetic, and ultrasonic sensors according to their types. The main principle of tracking a 6DoF handheld controller is to use a camera to detect infrared light spots on the handheld controller and then use a vision algorithm to calculate the 6DoF pose of the handheld controller. Optical handheld controllers have become one of the mainstream implementation methods due to their high positioning accuracy, high tracking robustness, and good user experience.

[0004] However, there is often a trade-off between the positioning accuracy of current handheld controllers and the power consumption of the devices. Summary of the Invention

[0005] To solve the problems, embodiments of the present invention provide a control method and system, a tracking method and system, a device, and a storage medium, which are conducive to ensuring the usage accuracy of a movable device and reducing the power consumption of the movable device.

[0006] To solve the above problems, embodiments of the present invention provide a control method for a light output unit of a movable device, the control method including: obtaining an image of the movable device at a current frame time using a detector, the movable device being equipped with a plurality of light output units for outputting signal light; predicting a motion state of the movable device at a next frame time based on the image; and obtaining configuration information of a light output state of each light output unit in the light output units at the next frame time based on the motion state of the movable device at the next frame time to control the light output unit.

[0007] Accordingly, an embodiment of the present invention further provides a tracking method for a movable device. The movable device is equipped with a plurality of light output units for outputting signal light. The tracking method includes: obtaining configuration information of the light output state of each light output unit in the light output units by using any one of the control methods provided by the embodiments of the present invention; and adjusting the light output state of each light output unit in the light output units based on the configuration information of the light output state.

[0008] Accordingly, an embodiment of the present invention further provides a control system for a light output unit of a movable device. The control system includes: an image acquisition unit configured to acquire an image of the movable device at the current frame time by using a detector, wherein the movable device is equipped with a plurality of light output units for outputting signal light; a prediction unit configured to predict the motion state of the movable device at the next frame time based on the image; and a configuration information acquisition unit configured to obtain configuration information of the light output state of each light output unit in the light output units at the next frame time based on the motion state of the movable device at the next frame time to control the light output unit.

[0009] Accordingly, an embodiment of the present invention further provides a tracking system for a movable device. The movable device is equipped with a plurality of light output units for outputting signal light, and the tracking system includes: a calculation processing module configured to obtain configuration information of the light output state of each light output unit in the light output units by using any one of the control methods provided by the embodiments of the present invention; and a control module configured to adjust the light output state of each output unit in the output units based on the configuration information of the light output state.

[0010] Accordingly, an embodiment of the present invention further provides a device, which includes: at least one memory and at least one processor, wherein the memory stores one or more computer instructions, and the one or more computer instructions can be executed by the processor to implement the control method for the light output unit of the movable device or the tracking method for the movable device according to the embodiments of the present invention.

[0011] Accordingly, an embodiment of the present invention further provides a storage medium storing one or more computer instructions, wherein the one or more computer instructions are configured to implement the control method for the light output unit of the movable device or the tracking method for the movable device according to the embodiments of the present invention.

[0012] Compared with the prior art, the technical solution of the embodiment of the present invention has the following advantages:

[0013] In the control method of the light output unit for a mobile device according to an embodiment of the present invention, configuration information of the light output states of all light output units at the next frame time is obtained based on the motion state of the mobile device at the next frame time to control the light output unit. Obtaining the light output states of all light output units at the next frame time based on the motion state of the mobile device at the next frame time is conducive to obtaining the light output states corresponding to the light output units according to the prediction of the motion state at the next frame time. That is, it is conducive to adaptively adjusting the light output states of all light output units based on the real-time state of the mobile device, so that the light output units achieve accurate and low-power light output states. This is conducive to ensuring the usage accuracy of the mobile device and reducing the power consumption of the mobile device.

[0014] In the tracking method for a mobile device according to an embodiment of the present invention, configuration information of the light output states of all light output units at the next frame time is obtained based on the motion state of the mobile device at the next frame time to control the light output unit. Obtaining the light output states of all light output units at the next frame time based on the motion state of the mobile device at the next frame time is conducive to obtaining the light output states corresponding to the light output units according to the prediction of the motion state at the next frame time. That is, it is conducive to adaptively adjusting the light output states of all light output units based on the real-time state of the mobile device, so that the light output units achieve accurate and low-power light output states. This is conducive to ensuring the usage accuracy of the mobile device and reducing the power consumption of the mobile device. Description of the Drawings

[0015] Figure 1 is a flowchart of an embodiment of the control method of the light output unit for a mobile device according to the present invention;

[0016] Figure 2 is a schematic structural diagram of an embodiment of the mobile device according to the present invention;

[0017] Figure 3 is according to the present invention and Figure 2 a schematic structural diagram of the detector cooperating with the embodiment of the mobile device;

[0018] Figure 4 is a flowchart of an embodiment of the tracking method for a mobile device according to the present invention;

[0019] Figure 5 is a functional block diagram of an embodiment of the control system of the light output unit for a mobile device according to the present invention;

[0020] Figure 6 is a functional block diagram of an embodiment of the tracking system for a mobile device according to the present invention; and

[0021] Figure 7 It is a structural diagram of the hardware of the device according to an embodiment of the present invention. Detailed implementation manners

[0022] According to the background art section, it is currently difficult to balance the usage accuracy of a mobile device and the power consumption of the mobile device.

[0023] A handheld controller is one of the main means of human-computer interaction in a virtual reality device. According to the principle of the handheld controller, the handheld controller can be divided into a handheld controller based on electromagnetic tracking, a handheld controller based on ultrasonic tracking, and a handheld controller based on optical tracking. The handheld controller based on optical tracking has become one of the most mainstream handheld controller solutions in the VR scenario due to its high accuracy, high robustness, and easy construction characteristics.

[0024] According to the basic principle of the handheld controller based on optical tracking, the camera captures the spot information of the positioning lights on the handheld controller, and calculates the position and attitude information of the handheld controller in the three-dimensional space according to the computer vision algorithm.

[0025] The distribution of the positioning lights on the handheld controller plays a crucial role in the tracking effect. If there are more positioning lights, the camera will capture more spot information, thereby improving the tracking accuracy and robustness. Therefore, from the perspective of the tracking accuracy of the handheld controller, the more positioning lights, the better. However, as the number of positioning lights increases, the power consumption of the handheld controller will also increase accordingly, which will reduce the endurance of the handheld controller, and the increase in power consumption will also cause more heat to be generated, which is not conducive to the miniaturization design of the handheld controller. Therefore, from the perspective of power consumption, the fewer positioning lights, the better.

[0026] Currently, it is often necessary to balance the tracking accuracy and the device power consumption.

[0027] To solve the above technical problems, an embodiment of the present invention provides a control method for an optical output unit of a mobile device. Refer to Figure 1 , Figure 1 It is a flowchart of an embodiment of the control method for the optical output unit of a mobile device according to the present invention.

[0028] In an embodiment of the present invention, the control method for the optical output unit of a mobile device includes the following basic steps:

[0029] Step S1: Use a detector to obtain an image of the mobile device at the current frame time, and the mobile device is equipped with a plurality of optical output units for outputting signal light;

[0030] Step S2: Predict the motion state of the movable device at the next frame time based on the image; and

[0031] Step S3: Obtain the configuration information of the light output state of each light output unit in the light output unit at the next frame time based on the motion state of the movable device at the next frame time to control the light output unit.

[0032] In the control method according to an embodiment of the present invention, obtaining the light output state of all light output units at the next frame time based on the motion state of the movable device at the next frame time is beneficial to obtaining the light output state corresponding to the light output unit according to the prediction of the motion state at the next frame time. That is, it is beneficial to adaptively adjust the light output state of all light output units based on the real-time state of the movable device, so that the light output unit realizes an accurate and low-power light output state. This is beneficial to ensuring the usage accuracy of the movable device and reducing the power consumption of the movable device.

[0033] In order to make the above objects, features, and advantages of the embodiments of the present invention more obvious and understandable, the following will describe the specific embodiments of the present invention in detail with reference to the accompanying drawings.

[0034] Reference Figure 1 and in combination with Figure 2 and Figure 3 , Figure 2 is a schematic structural diagram of an embodiment of the movable device of the present invention, Figure 3 is the present invention related to Figure 2 A schematic structural diagram of the detector combined with the embodiment of the movable device in. Step S1 is executed, wherein the detector 200 is used to obtain the image of the movable device 100 at the current frame time, and the movable device 100 is equipped with a plurality of light output units 110 for outputting signal light.

[0035] The movable device 100 is a device that can move and be tracked. The movable device 100 is equipped with a plurality of light output units 110 to extract the light spots corresponding to the light output units 110 based on the image, so as to obtain the motion state of the movable device 100.

[0036] As an example, the movable device 100 is a handheld controller. For example, the movable device 100 is a handheld controller applied to VR, AR, or MR.

[0037] As Figure 2 (a) shows, in a specific embodiment, the handheld controller includes a lamp ring 105, and a plurality of light output units 110 are arranged on the lamp ring 105.

[0038] In this embodiment, each light output unit 110 is a positioning light. As an example, the positioning light can be an infrared LED light. The infrared LED light cannot be detected by human vision and is beneficial to reducing the interference of the operation process of the light output unit 110 on the experience of using the mobile device 100. In other embodiments, the positioning light can also be other types of LED lights.

[0039] In this embodiment, the light output unit 110 outputs light in a blinking mode, which means that the light output unit 110 outputs light in a blinking form. Accordingly, the capture of the detector 200 is at the same frequency as the blinking of the light output unit 110, and the light output time of the light output unit 110 is synchronized with the capture time of the detector 200. Therefore, the detector 200 can be used to obtain an image of the mobile device 100 at the current frame time.

[0040] In this embodiment, the light output unit 110 adopts a blinking mode, which is beneficial for the light output unit 110 to output light at intervals to reduce the power consumption of the light output.

[0041] In other embodiments, the light output unit can also output light in a constant-on mode, which means that the light output unit always outputs light. Accordingly, the detector does not need to capture images at the same frequency, and the time for the detector to obtain images is not restricted.

[0042] In this embodiment, the mobile device 100 further includes a control module 120, which is used to adjust the light output states of all the light output units 110 based on the configuration information of the light output states.

[0043] In this embodiment, the light output units 110 are grouped and connected in parallel, and each group of light output units 110 is connected in series with the control module 120. The light output units 110 in each group are connected in series, or the light output units 110 are connected in parallel. All the light output units are independently connected in series with the control module 120. Therefore, the light output states of all the light output units 110 can be flexibly adjusted accordingly based on the configuration information of the light output states of all the light output units 110.

[0044] As an example, referring to Figure 2 (b), the light output units 110 are all connected in parallel, and all the output units are independently connected in series with the control module 120.

[0045] The detector 200 is configured to capture an image of the mobile device 100 at the current frame time to prepare for extracting the light spot corresponding to the light output unit 110 and obtaining the motion state of the mobile device 100. Accordingly, the image will be processed subsequently.

[0046] In this embodiment, the mobile device 100 is used in combination with the detector 200. After obtaining the motion state of the mobile device 100, the display and calculation content of the detector 200 are updated based on the motion state of the mobile device 100. For example, the mobile device is a handheld controller applied to VR, AR, or MR, and the detector is a head-mounted display device used in combination with the handheld controller. Specifically, the head-mounted display device may be VR, AR, or MR smart glasses.

[0047] As an example, referring to Figure 3 , the image acquisition device 210 arranged on the detector 200 can be used to obtain an image of the mobile device 100 at the current frame time.

[0048] In this embodiment, multiple image acquisition devices 210 are used to capture an image of the mobile device 100 at the current frame time. As an example, the image acquisition device 210 may include a multi-lens camera, such as a multi-lens infrared camera, to obtain an image of the infrared LED lights. Accordingly, in this embodiment, the image at the current frame time is the image captured by multiple image acquisition devices 210, that is, multiple images at one frame time are obtained.

[0049] The mobile device 100 is equipped with multiple light output units 110 for outputting signal light. The mobile device 100 can be positioned by the signal light output by the light output unit 110. Accordingly, the image includes light spots corresponding to the light output units 110.

[0050] Specifically, after obtaining an image of the mobile device 100, light spots corresponding to the signal light output by the light output unit 110 can be obtained. Therefore, the image at the current frame time correspondingly includes light spots formed by the signal light. By extracting the light spot features corresponding to the light output unit 110, the current motion state of the mobile device 100 at the current frame time can be obtained based on the light spot features.

[0051] In this embodiment, after using the detector to obtain an image of the mobile device 100 at the current frame time and before predicting the motion state of the mobile device 100 at the next frame time based on the image, the control method further includes: performing step S4, where it is determined whether the tracking state of the mobile device 100 fails at the current frame time based on the image; if it fails, step S6 is performed, where the light output state of each light output unit in the light output unit 110 at the next frame time is set to output light to obtain the configuration information of the light output state of each light output unit in the light output unit 110 at the next frame time; otherwise, the step of predicting the motion state of the mobile device 100 at the next frame time based on the image is performed.

[0052] The image at the current frame time is classified by determining whether the tracking state of the movable device 100 fails at the current frame time. Only when the tracking state of the movable device 100 does not fail at the current frame time, the next calculation processing step is executed. When the tracking state of the movable device 100 fails at the current frame time, the next calculation processing step is not executed. Instead, the light output state of each light output unit in the light output unit 110 at the next frame time is directly set, which is beneficial to perform targeted calculation processing, thereby saving calculation power.

[0053] In this embodiment, when the tracking state of the movable device 100 fails at the current frame time, the light output state of each light output unit in the light output unit 110 at the next frame time is set to output light, which is beneficial to strengthen the positioning information provided by the light output unit 110 of the movable device 100 in the case of tracking failure, so that the tracking of the movable device 100 can succeed as quickly as possible.

[0054] As an example, the failure of the tracking state of the movable device 100 at the current frame time is usually that the movable device 100 moves out of the field of view. When the movable device 100 moves out of the field of view, the light output state of each light output unit in the light output unit 110 at the next frame time is set to output light, which is beneficial to track the movable device 100 for the first time when the movable device 100 moves back into the field of view.

[0055] In this embodiment, determining whether the tracking state of the movable device 100 fails at the current frame time based on the image includes: obtaining the number of light spots corresponding to the light output unit 110 on the image, where the number less than the preset number of light spots indicates that the tracking state of the movable device 100 fails at the current frame time.

[0056] After that, the motion state of the movable device 100 at the next frame time is predicted based on the image, which is achieved by processing the light spots. Therefore, it is necessary to obtain a sufficient number of light spots to perform more accurate calculation processing. Therefore, the number of light spots corresponding to the light output unit 110 on the image is obtained. When the number is less than the preset number of light spots, the number of light spots is difficult to support the subsequent calculation processing. This indicates that the tracking state of the movable device 100 fails at the current frame time.

[0057] As an example, the preset number of light spots is set to 4.

[0058] Continue to refer to Figure 1 , step S2 is executed, where the motion state of the movable device 100 at the next frame time is predicted based on the image.

[0059] Predict the motion state of the movable device 100 at the next frame time based on the image, and obtain the configuration information of the light output state of the light output unit 110 based on the motion state at the next frame time.

[0060] In this embodiment, predicting the motion state of the movable device 100 at the next frame time based on the image includes: obtaining the motion state of the movable device 100 at the current frame time based on the image.

[0061] Obtaining the motion state of the movable device 100 at the current frame time is the basis for predicting the motion state of the movable device 100 at the next frame time.

[0062] Specifically, in this embodiment, the motion state includes the pose, speed, and acceleration of the movable device 100.

[0063] The pose includes position and orientation; the position refers to the translational degrees of freedom (x, y, z) in the coordinate system of the detector 200; the orientation refers to the rotational degrees of freedom (rx, ry, rz) in the coordinate system of the detector 200; the speed includes linear velocity and angular velocity; the acceleration includes linear acceleration and angular acceleration.

[0064] In this embodiment, obtaining the motion state of the movable device 100 at the current frame time based on the image includes: extracting the light spot corresponding to the light output unit 110 on the image.

[0065] Extracting the light spot corresponding to the light output unit 110 on the image enables the subsequent obtaining of the pose of the movable device 100 based on the feature information of the light spot.

[0066] In one embodiment, during the process of obtaining the motion state of the movable device 100 at the current frame time based on the image, after extracting the light spot corresponding to the light output unit 110 on the image and before obtaining the motion state of the movable device 100 at the current frame time, the control method further includes: filtering the light spot to filter out the light spots with abnormal imaging.

[0067] Filtering the light spot facilitates removing abnormal environmental features, which prevents the abnormal environmental features from interfering with the light spot features, thereby improving the accuracy of the feature information of the extracted light spot and the accuracy of the subsequent obtained pose of the movable device 100.

[0068] Specifically, as an example, in actual use scenarios, there are often various environmental lighting interferences such as illumination and natural light. Therefore, the detected light spot features may not be the light spots of the positioning light of the movable device. Filter the light spot to remove abnormal features from the environmental lighting interference.

[0069] As an example, the light spot of the positioning light usually has high brightness, while the environmental lighting interference usually has light spots with low brightness. By setting a brightness threshold and filtering out the light spots with brightness less than the brightness threshold, the light spots of abnormal imaging can be filtered out.

[0070] As another example, the light spot characteristics of the positioning light are usually circular or elliptical with a certain size, while the environmental lighting interference is usually irregular in shape. By determining the shape and size of the characteristics, the light spots of abnormal imaging can be filtered out.

[0071] In one embodiment, the image based on the current frame time is used to detect the frame of the movable device, and the light spot corresponding to the light output unit 110 is extracted from the frame of the movable device.

[0072] More specifically, the movable device 100 is a handheld controller. Accordingly, the frame of the handheld controller is detected based on the image of the current frame time, and the light spot is extracted from the frame of the handheld controller.

[0073] In a specific implementation, a computer vision method or a deep learning method can be used to extract the light spot. For example, the computer vision method may include: performing binarization processing on the image, and then extracting the light spot with brightness greater than a preset brightness threshold from the binarized image as the light spot corresponding to the light output unit 110.

[0074] In this embodiment, the pose of the movable device 100 at the current frame time is obtained based on the light spot corresponding to the light output unit 110.

[0075] Obtaining the pose of the movable device 100 at the current frame time is the basis for predicting the pose of the movable device 100 at the next frame time.

[0076] Specifically, in this embodiment, the center position of the light spot is extracted based on the light spot corresponding to the light output unit 110, and the pose of the movable device 100 at the current frame time is calculated based on the center position of the light spot.

[0077] As an example, the method of ellipse fitting is used to extract the center position of each light spot.

[0078] As an example, the method for calculating the pose of the movable device 100 at the current frame time includes the Perspective-n-Point (PnP) algorithm.

[0079] In this embodiment, the speed of the movable device at the current frame time is obtained by combining the position of the movable device 100 at the previous frame time, the position of the movable device at the current frame time, and the time difference between the previous frame time and the current frame time.

[0080] In this embodiment, the acceleration of the mobile device 100 at the current frame time is obtained by combining the speed of the mobile device 100 at the previous frame time, the speed of the mobile device at the current frame time, and the time difference between the previous frame time and the current frame time.

[0081] The speed and acceleration of the mobile device 100 at the current frame time are obtained to predict the speed and acceleration of the mobile device 100 at the next frame time.

[0082] In a specific embodiment, the mobile device 100 is equipped with an inertial measurement unit (IMU) for obtaining inertial measurement data measured by the IMU, including speed and acceleration, specifically including linear acceleration and angular velocity. The inertial measurement data of the mobile device 100 at the current frame time obtained by the IMU is fused with the calculated motion state of the mobile device 100 at the current frame time to obtain more accurate data on the motion state of the mobile device 100 at the current frame time, and this data is used as the basic data for predicting the motion state of the mobile device 100 at the next frame time, which is beneficial for more accurately predicting the motion state of the mobile device 100 at the next frame time.

[0083] The high frame rate of inertial measurement is beneficial for increasing the output frequency of the current pose. In addition, by fusing the inertial measurement data of the mobile device 100 at the current frame time with the calculated motion state of the mobile device 100 at the current frame time, smooth filtering of the current motion state can be achieved, thereby reducing output jitter and outputting a smooth and low-latency motion state.

[0084] In one embodiment, the extended Kalman filter (EKF) is used for fusion to output the motion state of the mobile device 100 at the current frame image time. In some other embodiments, the square root Kalman filter can also be used to output the motion state of the mobile device 100 at the current frame image time.

[0085] In other embodiments, the mobile device can also directly obtain the motion state of the mobile device at the next frame time based on the calculated motion state of the mobile device at the current frame time without setting an inertial test unit.

[0086] In this embodiment, the motion state of the mobile device 100 at the next frame time is predicted based on the motion state of the mobile device 100 at the current frame time.

[0087] Predicting the motion state of the mobile device 100 at the next frame time facilitates obtaining the light output state of the light output unit 110 through the motion state of the mobile device 100 at the next frame time.

[0088] In this embodiment, during predicting the motion state of the movable device 100 at the next frame time based on the motion state of the movable device 100 at the current frame time, the motion state of the movable device 100 at the next frame time is predicted by combining the motion state of the movable device 100 at the previous frame time and the motion state of the movable device 100 at the current frame time.

[0089] Predicting the motion state of the movable device 100 at the next frame time by combining the motion state of the movable device 100 at the previous frame time and the motion state of the movable device 100 at the current frame time is conducive to making the prediction of the motion state of the movable device 100 at the next frame time accurate.

[0090] In this embodiment, by combining the motion states of the movable device 100 at both the previous frame time and the current frame time, the Taylor expansion formula is used to predict the motion state of the movable device 100 at the next frame time.

[0091] Specifically, in this embodiment, if the movable device 100 is set to move at a constant speed, the acceleration of the movable device 100 is always 0. The speed of the movable device 100 at the next frame time is equal to the speed at the current frame time. By combining the motion state of the movable device 100 at the previous frame time and the motion state of the movable device 100 at the current frame time, the Taylor first-order expansion formula is used to predict the position and attitude of the movable device 100, where X t+1 is the predicted variable at the next frame time; X t is the variable calculated at the current frame time; X t-1 is the variable calculated at the previous frame time; ΔT t-1 represents the time interval between the previous frame time and the current frame time; and ΔT t represents the time interval between the current frame time and the next frame time. The variable X here includes position and attitude.

[0092] In other embodiments, the movable device may also be set to move with a uniform acceleration, such that the acceleration of the movable device at the next frame time is equal to the acceleration at the current frame time. By combining the speeds of the movable device at both the previous frame time and the current frame time, the Taylor first-order expansion formula is used to predict the speed of the movable device. By combining the positions and attitudes of the movable device at the two previous frame times of the current frame and the position and attitude of the movable device at the current frame time, the Taylor second-order expansion formula is used to predict the position and attitude of the movable device.

[0093] In some other embodiments, the Taylor first-order expansion formula may also be used by combining the motion state of the movable device at the previous frame time and the motion state of the movable device at the current frame time to predict the motion state of the mobile device 100, where X t+1 is a prediction variable for the next frame time; X t is a variable calculated at the current frame time; X t-1 is a variable calculated at the previous frame time; ΔT t-1 represents the time interval between the previous frame time and the current frame time; and ΔT t represents the time interval between the current frame time and the next frame time. The variable X here includes position, attitude, velocity, and acceleration.

[0094] In still other embodiments, the motion state of the mobile device at the next frame time may also be predicted by combining the motion states of multiple consecutive frame times before the current frame time and the motion state of the mobile device at the current frame time.

[0095] Accordingly, in still other embodiments, the Taylor second-order expansion formula or a higher-order Taylor expansion formula may also be used to predict the motion state of the mobile device at the next frame time by combining the motion states of multiple consecutive frame times before the current frame time and the motion state of the mobile device at the current frame time.

[0096] In this embodiment, after predicting the motion state of the mobile device 100 at the next frame time based on the image and before obtaining the configuration information of the light output state of each light output unit in the light output unit 110 at the next frame time based on the motion state of the mobile device 100 at the next frame time, the control method further includes: performing step S5, where it is determined whether the tracking state of the mobile device 100 fails at the next frame time based on the motion state of the mobile device 100 at the next frame time; if it fails, then step S6 is performed, where the light output state of each light output unit in the light output unit 110 at the next frame time is set to output light to obtain the configuration information of the light output state of each light output unit in the light output unit 110 at the next frame time; otherwise, the configuration information of the light output state of each light output unit in the light output unit 110 at the next frame time is obtained based on the motion state of the mobile device 100 at the next frame time.

[0097] By determining whether the tracking state of the mobile device 100 fails at the next frame time, the motion state at the next frame time is classified. Only when the tracking state of the mobile device 100 does not fail at the next frame time is the next calculation processing step performed. When the tracking state of the mobile device 100 fails at the next frame time, the next calculation processing step is not performed. Instead, the light output state of each light output unit in the light output unit 110 at the next frame time is directly set, which is beneficial for performing targeted calculation processing, thereby saving computing power.

[0098] In this embodiment, when the tracking state of the movable device 100 fails in the next frame time, the light output state of each light output unit in the light output unit 110 in the next frame time is set to output light, which is beneficial to strengthening the positioning information provided by the light output unit 110 of the movable device 100 in the case of tracking failure, so that the tracking of the movable device 100 can succeed as quickly as possible.

[0099] As an example, the failure of the tracking state of the movable device 100 at the current frame time is usually that the movable device 100 moves out of the field of view. When the movable device 100 moves out of the field of view, the light output state of each light output unit in the light output unit 110 in the next frame time is set to output light, which is beneficial to tracking the movable device 100 for the first time when the movable device 100 moves back into the field of view.

[0100] In this embodiment, determining whether the tracking state of the movable device 100 fails in the next frame time based on the motion state of the movable device 100 in the next frame time includes: whether the speed or acceleration of the movable device 100 in the next frame time is abnormal, where an abnormal speed or acceleration indicates that the tracking state of the movable device 100 fails in the next frame time.

[0101] When the speed or acceleration is abnormal (for example, in the case of excessive speed or acceleration), it indicates that the motion of the movable device 100 is abnormal in the next frame time, thereby indicating that the tracking state of the movable device 100 fails in the next frame time.

[0102] Continue to refer to Figure 1 , perform step S3, where the configuration information of the light output state of each light output unit in the light output unit 110 in the next frame time is obtained based on the motion state of the movable device 100 in the next frame time to control the light output unit 110.

[0103] In this embodiment, obtaining the light output states of all the light output units 110 in the next frame time based on the motion state of the movable device 100 in the next frame time is beneficial to obtaining the light output states corresponding to the light output units 110 according to the prediction of the motion state in the next frame time. That is, it is beneficial to adaptively adjust the light output states of all the light output units 110 based on the real-time state of the movable device 100, so that the light output units 110 achieve an accurate and low-power light output state. This is beneficial to ensuring the usage accuracy of the movable device 100 and reducing the power consumption of the movable device.

[0104] In this embodiment, obtaining the configuration information of the light output state of each light output unit in the light output unit 110 based on the motion state of the mobile device 100 at the next frame time includes: configuring the light output state of each light output unit in the light output unit 110. This configuration includes determining whether the light output unit 110 is within the detection range of the detector 200 based on the motion state of the mobile device 100 at the next frame time.

[0105] Determine whether the light output unit 110 is within the detection range of the detector 200 based on the motion state of the mobile device 100 at the next frame time. Set the light output state of the light output unit 110 based on whether the light output unit is within the detection range of the detector 200. Classify the light output units 110 that are not within the detection range and the light output units 110 that are within the detection range, and adaptively adjust the light output state of the light output unit 110 based on the actual detection situation.

[0106] Specifically, in this embodiment, if the light output unit 110 is not within the detection range of the detector 200, then set the light output state of the light output unit 110 to not output light. Otherwise, set the light output state of the light output unit 110 to output light.

[0107] When the light output unit 110 is not within the detection range of the detector 200, the light output unit 110 is not used during the tracking of the mobile device 100 by the detector 200. Correspondingly, setting the light output state of the light output unit 110 to not output light is beneficial to saving the light output power consumption and does not affect the tracking accuracy of the detector 200 for the mobile device 100.

[0108] Otherwise, when the light output unit 110 is within the detection range of the detector 200, the light output unit 110 needs to be used during the tracking of the mobile device 100 by the detector 200. Correspondingly, setting the light output state of the light output unit 110 to output light is used to enable the detector 200 to track the mobile device 100.

[0109] In this embodiment, determining whether the light output unit 110 is within the detection range of the detector 200 based on the motion state of the mobile device at the next frame time includes: obtaining the pose of each light output unit in the light output unit 110 on the mobile device 100 based on the pose of the mobile device 100 at the next frame time; and determining whether the light output unit 110 is within the detection range of the detector 200 based on the pose of the light output unit 110.

[0110] The pose of the light output unit 110 represents the degrees of freedom of movement (x, y, z) of the light output unit 110 in the coordinate system of the detector 200, and the degrees of freedom of rotation (rx, ry, rz) of the light output unit 110 in the coordinate system of the detector 200. Therefore, it is possible to determine whether the light output unit 110 is within the detection range of the detector 200 based on the pose of the light output unit 110.

[0111] In this embodiment, during the process of obtaining the pose of each light output unit in the light output units 110 on the movable device 100 based on the pose of the movable device 100 at the next frame time, the pose of each light output unit in the light output units 110 on the movable device 100 is obtained based on the pose of the movable device 100 at the next frame time and the relative position relationship among all the light output units 110 on the movable device 100.

[0112] Specifically, in this embodiment, each light output unit 110 has a fixed relative position on the movable device 100, and the detector 200 has a model of the relative position of the light output unit 110 on the movable device 100. After obtaining the pose of the movable device 100 at the next frame time, the pose of each light output unit 110 on the movable device 100 can be obtained based on the model of the relative position of the light output unit 110 in the detector 200 on the movable device 100.

[0113] In this embodiment, determining whether the light output unit 110 is within the detection range of the detector 200 includes: obtaining the light output direction of the light output unit 110 as the first direction; obtaining the direction of the light output unit 110 towards the center of the coordinate system of the detector 200 as the second direction; and obtaining the included angle between the first direction and the second direction.

[0114] The light output direction of the light output unit 110 is the direction in which the light output unit 110 outputs light. The direction of the light output unit 110 towards the center of the coordinate system of the detector 200 is the direction in which the detector 200 receives the light output by the light output unit 110, so that the included angle between the first direction and the second direction can characterize the reception condition of the light output by the light output unit 110 by the detector 200.

[0115] Specifically, in this embodiment, the pose of the light output unit 110 represents the translational degrees of freedom (x, y, z) of the light output unit 110 in the coordinate system of the detector 200 and the rotational degrees of freedom (rx, ry, rz) of the light output unit 110 in the coordinate system of the detector 200. Therefore, the direction of the light output unit 110 towards the center of the coordinate system of the detector 200 is obtained based on the position of the light output unit 110, and the light output direction of the light output unit 110 is obtained based on the pose of the light output unit 110, where the center of the coordinate system of the detector 200 is the origin.

[0116] In this embodiment, it is determined whether the included angle is within the invalid viewing angle range, and the invalid viewing angle range is the viewing angle range that exceeds the detection viewing angle of the detector 200.

[0117] Determining whether the included angle is within the invalid viewing angle range means determining whether the detector 200 can receive the light output by the light output unit 110.

[0118] It should be noted that in this embodiment, the invalid viewing angle range should not be too large, that is, the smaller end value of the end values of the invalid viewing angle range should not be too small. During the actual detection process of the detector 200, the light spot obtained by the detector 200 and corresponding to the light output unit 110 is circular or elliptical, which means that the light spot corresponding to the light output unit 110 is an area rather than a point. During the calculation of the motion state, the center position of the light spot can be obtained, and the pose is calculated based on the center position of the light spot. Therefore, although the calculated included angle indicates that the center position of the light spot of the light output unit 110 cannot be received by the detector 200 (for example, the included angle between the calculated light output direction of the light output unit 110 and the direction of the light output unit 110 towards the center of the coordinate system of the detector 200 is 95°), in fact, some areas around the center position of the light spot corresponding to the light output unit 110 can still be received by the detector 200. At this time, it should also be determined that the light output unit 110 is not within the invalid viewing angle range. Therefore, in order to avoid determining that the light output unit 110 that can be detected by the detector 200 is within the invalid viewing angle range, in this embodiment, the invalid viewing angle range should not be too large. That is, the smaller end value of the end values of the invalid viewing angle range should not be too small. Therefore, in this embodiment, the invalid viewing angle range is 120° to 180°.

[0119] Specifically, in this embodiment, if the included angle is within the invalid viewing angle range, it is determined that the light output unit 110 is not within the detection range of the detector. Otherwise, it is determined that the light output unit 110 is within the detection range of the detector 200.

[0120] Specifically, in this embodiment, if the included angle is within the invalid viewing angle range, the light output unit 110 is set not to output light. If the included angle is not within the invalid viewing angle range, the light output unit 110 is set to output light.

[0121] Correspondingly, the present invention also provides a tracking method for a mobile device. Figure 4 It is a flowchart of an embodiment of the tracking method for a mobile device according to the present invention.

[0122] Refer to Figure 4 , the mobile device 100 (such as Figure 2 shown) is equipped with a plurality of light output units 110 (such as Figure 2 shown) for outputting signal light. Step S1 is executed, in which configuration information of the light output state of the light output unit 110 is obtained by using any control method provided by the present invention.

[0123] Obtaining the configuration information of the light output state of the light output unit 110 facilitates subsequent adjustment of the light output state of the light output unit 110 using this configuration information.

[0124] Specifically, in this embodiment, the configuration information of the light output state of the light output unit 110 is obtained in the calculation and processing module 220 (such as Figure 3 shown) of the detector 200 (such as Figure 3 shown).

[0125] Obtaining the light output states of all the light output units 110 at the next frame time based on the motion state of the mobile device 100 at the next frame time is beneficial for obtaining the light output states corresponding to the light output units 110 according to the prediction of the motion state at the next frame time. That is, it is beneficial to adaptively adjust the light output states of all the light output units 110 based on the real-time state of the mobile device 100, so that the light output units 110 achieve an accurate and low-power light output state. This is beneficial for ensuring the usage accuracy of the mobile device 100 and reducing the power consumption of the mobile device.

[0126] For the specific description of the mobile device 100 and obtaining the configuration information of the light output state of the light output unit 110, reference can be made to the relevant descriptions in the foregoing embodiments and will not be described in detail herein.

[0127] Continue to refer to Figure 4 , based on the configuration information of the light output state, adjust the light output states of all the light output units 110.

[0128] In this embodiment, each light output unit 110 adopts a flashing mode, which is beneficial for the light output unit 110 to output light at intervals to reduce the power consumption of the light output.

[0129] Accordingly, during the adjustment of the light output states of all the light output units 110, the light output units 110 output light in a blinking mode.

[0130] In other embodiments, the light output units may also output light in a normally-on mode. Accordingly, during the adjustment of the light output states of all the light output units, the light output units output light in a normally-on mode.

[0131] In this embodiment, during the adjustment of the light output states of all the light output units 110, the light output state of the light output units 110 is controlled to not output light by reducing the light output frequency to 0 or reducing the light output duration of each light output to 0.

[0132] In this embodiment, the mobile device 100 further includes a control module 120 (as Figure 2 shown) for adjusting the light output states of all the light output units 110 based on the configuration information of the light output states.

[0133] In this embodiment, the light output units 110 are grouped and connected in parallel, and each group of light output units 110 is connected in series with the control module 120. The light output units 110 in each group are connected in series, or the light output units 110 are connected in parallel. All the light output units are independently connected in series with the control module 120. Therefore, the light output states of all the light output units 110 can be flexibly adjusted accordingly based on the configuration information of the light output states of all the light output units 110.

[0134] As an example, referring to Figure 2 (b), the light output units 110 are all connected in parallel, and all the output units are independently connected in series with the control module 120.

[0135] In this embodiment, the circuit connection method for connecting the light output units 110 to the control module 120 includes a rigid circuit board, a flexible circuit board, or a flexible and rigid combined board.

[0136] In this embodiment, the mobile device 100 further includes an information receiving module 130 (as Figure 2 shown) for receiving the configuration information of the light output states of all the light output units 110.

[0137] Specifically, in this embodiment, the information receiving module 130 receives the configuration information of the light output states of all the light output units 110, and then transmits the configuration information to the control module 120. The control module 120 adjusts the light output states of all the light output units 110 based on the configuration information of the light output states of all the light output units 110.

[0138] In this embodiment, the information receiving module 130 receives information through Bluetooth, WiFi, or a wired connection.

[0139] In this embodiment, the control method described in the foregoing embodiment is used to obtain the configuration information of the light output state. For a specific description of the configuration information of the light output state, reference may be made to the relevant description in the foregoing embodiment and will not be described in detail herein.

[0140] Correspondingly, the present invention also provides a control system for a light output unit of a mobile device. Figure 5 is a functional block diagram of an embodiment of a control system for a light output unit of a mobile device according to the present invention.

[0141] Reference Figure 5 , the control system for the light output unit of the mobile device includes: an image acquisition unit 10, which is used to acquire an image of the mobile device at the current frame time using a detector, where the mobile device is equipped with a plurality of light output units for outputting signal light; a prediction unit 20, which is used to predict the motion state of the mobile device at the next frame time based on the image; a configuration information acquisition unit 30, which is used to obtain the configuration information of the light output state of each light output unit in the light output unit at the next frame time based on the motion state of the mobile device at the next frame time to control the light output unit.

[0142] The mobile device is a device that can move and be tracked. The mobile device is equipped with a plurality of light output units to extract light spots corresponding to the light output units based on the image, so as to obtain the motion state of the mobile device.

[0143] As an example, the mobile device is a handheld controller. For example, the mobile device is a handheld controller applied to VR, AR, or MR.

[0144] In a specific embodiment, the handheld controller includes a light ring, and a plurality of light output units are arranged on the light ring.

[0145] In this embodiment, each light output unit is a positioning light. As an example, the positioning light can be an infrared LED light. The infrared LED light cannot be detected by human vision and is beneficial to reducing the interference of the working process of the light output unit on the experience of using the mobile device. In other embodiments, the positioning light can also be other types of LED lights.

[0146] In this embodiment, the light output unit outputs light in a blinking mode, which means that the light output unit outputs light in a blinking form. Accordingly, the capture of the detector is at the same frequency as the blinking of the light output unit, and the light output time of the light output unit is synchronized with the capture time of the detector. Therefore, an image of the movable device at the current frame time can be obtained using the detector.

[0147] In this embodiment, the light output unit adopts a blinking mode, which is conducive to the light output unit outputting light at intervals to reduce the power consumption of light output.

[0148] In other embodiments, the light output unit can also output light in a constant-on mode, which means that the light output unit always outputs light. Accordingly, the detector does not need to capture images at the same frequency, and the time for the detector to obtain images is not restricted.

[0149] In this embodiment, the movable device further includes a control module, which is used to adjust the light output states of all light output units based on the configuration information of the light output states.

[0150] In this embodiment, the light output units are grouped and connected in parallel, and each group of light output units is connected in series with the control module. The light output units in each group are connected in series, or the light output units are connected in parallel. All light output units are independently connected in series with the control module. Therefore, the light output states of all light output units can be flexibly adjusted accordingly based on the configuration information of the light output states of all light output units.

[0151] As an example, the light output units are all connected in parallel, and all output units are independently connected in series with the control module.

[0152] The detector is configured to capture an image of the movable device at the current frame time to prepare for extracting the light spot corresponding to the light output unit and obtaining the motion state of the movable device. Accordingly, the image will be processed subsequently.

[0153] In this embodiment, the movable device is used in combination with the detector. After obtaining the motion state of the movable device, the display and calculation content of the detector are updated based on the motion state of the movable device. For example, the movable device is a handheld controller applied to VR, AR, or MR, and the detector is a head-mounted display device used in combination with the handheld controller. Specifically, the head-mounted display device can be a VR, AR, or MR smart glasses.

[0154] As an example, an image acquisition device arranged on the detector can be used to obtain an image of the movable device at the current frame time.

[0155] In this embodiment, multiple image acquisition devices are used to capture images of the movable device at the current frame time. As an example, the image acquisition device may include a multi-lens camera, such as a multi-lens infrared camera, so as to obtain images of infrared LED lights. Accordingly, in this embodiment, the image at the current frame time is the image captured by multiple image acquisition devices, that is, multiple images at one frame time are obtained.

[0156] The movable device is equipped with multiple light output units for outputting signal light. The movable device can be positioned by the signal light output by the light output unit. Accordingly, the image includes light spots corresponding to the light output units.

[0157] Specifically, after obtaining the image of the movable device, light spots corresponding to the signal light output by the light output unit can be obtained. Therefore, the image at the current frame time correspondingly includes light spots formed by the signal light. By extracting the light spot features corresponding to the light output units, the current motion state of the movable device at the current frame time can be obtained based on the light spot features.

[0158] In this embodiment, the control system further includes: a first determination unit 40, which is configured to determine whether the tracking state of the movable device fails at the current frame time based on the image after using a detector to obtain the image of the movable device at the current frame time and before predicting the motion state of the movable device at the next frame time based on the image; if it fails, set the light output state of each light output unit in the light output unit to output light at the next frame time to obtain the configuration information of the light output state of each light output unit in the light output unit at the next frame time; otherwise, perform predicting the motion state of the movable device at the next frame time based on the image.

[0159] The image at the current frame time is classified by determining whether the tracking state of the movable device fails at the current frame time. Only when the tracking state of the movable device does not fail at the current frame time, the next calculation processing step is performed. When the tracking state of the movable device fails at the current frame time, the next calculation processing step is not performed. Instead, the light output state of each light output unit in the light output unit at the next frame time is directly set, which is beneficial to performing targeted calculation processing, thereby saving computing power.

[0160] In this embodiment, when the tracking state of the movable device fails at the current frame time, setting the light output state of each light output unit in the light output unit to output light at the next frame time is beneficial to strengthening the positioning information provided by the light output unit of the movable device in the case of tracking failure, so that the tracking of the movable device can succeed as quickly as possible.

[0161] As an example, the failure of the tracking state of the movable device at the current frame time is usually that the movable device moves out of the field of view. When the movable device moves out of the field of view, the light output state of each light output unit in the light output unit at the next frame time is set to output light, which is beneficial to tracking the movable device for the first time when the movable device moves back into the field of view.

[0162] In this embodiment, determining whether the tracking state of the movable device fails at the current frame time based on the image includes: obtaining the number of light spots corresponding to the light output unit on the image, where the number less than the preset number of light spots indicates that the tracking state of the movable device fails at the current frame time.

[0163] After that, predicting the motion state of the movable device at the next frame time based on the image, which is achieved by processing the light spots. Therefore, it is necessary to obtain a sufficient number of light spots to perform more accurate calculation processing. Therefore, the number of light spots corresponding to the light output unit on the image is obtained. When the number is less than the preset number of light spots, the number of light spots is difficult to support subsequent calculation processing. This indicates that the tracking state of the movable device fails at the current frame time.

[0164] As an example, the preset number of light spots is set to 4.

[0165] The prediction unit 20 is configured to predict the motion state of the movable device at the next frame time based on the image.

[0166] Predicting the motion state of the movable device at the next frame time based on the image, and obtaining the configuration information of the light output state of the light output unit based on the motion state at the next frame time.

[0167] In this embodiment, predicting the motion state of the movable device at the next frame time based on the image includes: obtaining the motion state of the movable device at the current frame time based on the image.

[0168] Obtaining the motion state of the movable device at the current frame time is the basis for predicting the motion state of the movable device at the next frame time.

[0169] Specifically, in this embodiment, the motion state includes the pose, speed, and acceleration of the movable device.

[0170] The pose includes position and orientation; the position refers to the translational degrees of freedom (x, y, z) in the coordinate system of the detector; the orientation refers to the rotational degrees of freedom (rx, ry, rz) in the coordinate system of the detector; the speed includes linear velocity and angular velocity; the acceleration includes linear acceleration and angular acceleration.

[0171] In this embodiment, obtaining the motion state of the movable device at the current frame time based on the image includes: extracting the light spots corresponding to the light output unit on the image.

[0172] Extract the light spot corresponding to the light output unit on the image, so that the pose of the mobile device can be obtained based on the feature information of the light spot subsequently.

[0173] In one embodiment, in the process of obtaining the motion state of the mobile device at the current frame time based on the image, after extracting the light spot corresponding to the light output unit on the image and before obtaining the motion state of the mobile device at the current frame time, the control system is further configured to filter the light spot to filter out the abnormally imaged light spot.

[0174] Filtering the light spot facilitates removing abnormal environmental features, which prevents the abnormal environmental features from interfering with the features of the light spot, thereby improving the accuracy of the feature information of the extracted light spot and the accuracy of the pose of the mobile device obtained subsequently.

[0175] Specifically, as an example, in the actual use scenario, there are often various environmental lighting interferences such as illumination and natural light. Therefore, the detected light spot features may not be the light spots of the positioning light of the mobile device. Filter the light spot to remove abnormal features from the environmental lighting interference.

[0176] As an example, the light spot of the positioning light usually has high brightness, while the environmental lighting interference is usually a light spot with low brightness. By setting a brightness threshold, the light spot with a brightness less than the brightness threshold can be filtered out, and thus the abnormally imaged light spot can be filtered out.

[0177] As another example, the light spot features of the positioning light are usually circular or elliptical with a certain size, while the environmental lighting interference is usually irregular in shape. By determining the shape and size of the features, the abnormally imaged light spot can be filtered out.

[0178] In one embodiment, detect the mobile device frame based on the image at the current frame time, and extract the light spot corresponding to the light output unit from the mobile device frame.

[0179] More specifically, the mobile device is a handheld controller. Accordingly, detect the handheld controller frame based on the image at the current frame time, and extract the light spot from the handheld controller frame.

[0180] In a specific implementation, computer vision methods or deep learning methods can be used to extract the light spot. For example, the computer vision method may include: performing binarization processing on the image, and then extracting the light spot with a brightness greater than a preset brightness threshold from the binarized image as the light spot corresponding to the light output unit.

[0181] In this embodiment, the pose of the mobile device at the current frame time is obtained based on the light spot corresponding to the light output unit.

[0182] Obtaining the pose of the movable device at the current frame time is the basis for predicting the pose of the movable device at the next frame time.

[0183] Specifically, in this embodiment, the center position of the light spot is extracted based on the light spot corresponding to the light output unit, and the pose of the movable device at the current frame time is calculated based on the center position of the light spot.

[0184] As an example, the method of ellipse fitting is used to extract the center position of each light spot.

[0185] As an example, the method for calculating the pose of the movable device at the current frame time includes the PnP algorithm.

[0186] In this embodiment, the speed of the movable device at the current frame time is obtained by combining the position of the movable device at the previous frame time, the position of the movable device at the current frame time, and the time difference between the previous frame time and the current frame time.

[0187] In this embodiment, the acceleration of the movable device at the current frame time is obtained by combining the speed of the movable device at the previous frame time, the speed of the movable device at the current frame time, and the time difference between the previous frame time and the current frame time.

[0188] Obtaining the speed and acceleration of the movable device at the current frame time to predict the speed and acceleration of the movable device at the next frame time.

[0189] In a specific embodiment, the movable device is equipped with an IMU for obtaining inertial measurement data measured by the IMU, including speed and acceleration, specifically including linear acceleration and angular velocity. The inertial measurement data of the movable device at the current frame time obtained by the IMU is fused with the calculated motion state of the movable device at the current frame time to obtain more accurate data on the motion state of the movable device at the current frame time, and this data is used as the basic data for predicting the motion state of the movable device at the next frame time, which is beneficial to more accurately predicting the motion state of the movable device at the next frame time.

[0190] The high frame rate of inertial measurement is beneficial to improving the output frequency of the current pose. In addition, by fusing the inertial measurement data of the movable device at the current frame time with the calculated motion state of the movable device at the current frame time, smooth filtering of the current motion state can be achieved, thereby reducing output jitter and outputting a smooth and low-latency motion state.

[0191] In one embodiment, an Extended Kalman Filter (EKF) is used for fusion to output the motion state of the movable device at the current frame image time. In some other embodiments, a square root Kalman filter may also be used to output the motion state of the movable device at the current frame image time.

[0192] In other embodiments, the movable device may also directly obtain the motion state of the movable device at the next frame time based on the calculated motion state of the movable device at the current frame time, without setting an inertial measurement unit.

[0193] In this embodiment, the motion state of the movable device at the next frame time is predicted based on the motion state of the movable device at the current frame time.

[0194] Predicting the motion state of the movable device at the next frame time facilitates obtaining the light output state of the light output unit based on the motion state of the movable device at the next frame time.

[0195] In this embodiment, during the prediction of the motion state of the movable device at the next frame time based on the motion state of the movable device at the current frame time, the motion state of the movable device at the previous frame time and the motion state of the movable device at the current frame time are combined to predict the motion state of the movable device at the next frame time.

[0196] Combining the motion state of the movable device at the previous frame time and the motion state of the movable device at the current frame time to predict the motion state of the movable device at the next frame time is beneficial for making the prediction of the motion state of the movable device at the next frame time accurate.

[0197] In this embodiment, the Taylor expansion formula is used to predict the motion state of the movable device at the next frame time by combining the motion states of the movable device at both the previous frame time and the current frame time.

[0198] Specifically, in this embodiment, if the movable device is set to move at a constant speed, the acceleration of the movable device 100 is always 0. The speed of the movable device at the next frame time is equal to the speed at the current frame time. By combining the motion state of the movable device at the previous frame time and the motion state of the movable device at the current frame time, the Taylor first-order expansion formula is used to predict the position and orientation of the movable device, where X t+1 is the predicted variable at the next frame time; X t is the variable calculated at the current frame time; X t-1 is the variable calculated at the previous frame time; ΔT t-1 represents the time interval between the previous frame time and the current frame time; and ΔT tRepresents the time interval between the current frame time and the next frame time. The variable X here includes position and attitude.

[0199] In other embodiments, the movable device can also be set to move with a uniform acceleration such that the acceleration of the movable device at the next frame time is equal to the acceleration at the current frame time. Combining the velocities of the movable device at both the previous frame time and the current frame time, the Taylor first-order expansion formula is used to predict the velocity of the movable device. Combining the positions and attitudes of the movable device at the two frame times before the current frame and the position and attitude of the movable device at the current frame time, the Taylor second-order expansion formula is used to predict the position and attitude of the movable device.

[0200] In some other embodiments, the Taylor first-order expansion formula can also be used in combination with the motion state of the movable device at the previous frame time and the motion state of the movable device at the current frame time to predict the motion state of the movable device, where X t+1 is the predicted variable at the next frame time; X t is the variable calculated at the current frame time; X t-1 is the variable calculated at the previous frame time; ΔT t-1 represents the time interval between the previous frame time and the current frame time; and ΔT t represents the time interval between the current frame time and the next frame time. The variable X here includes position, attitude, velocity, and acceleration.

[0201] In still other embodiments, the motion state of the movable device at the next frame time can also be predicted by combining the motion states of a plurality of consecutive frame times before the current frame time and the motion state of the movable device at the current frame time.

[0202] Accordingly, in still other embodiments, the Taylor second-order expansion formula or a higher-order Taylor expansion formula can also be used in combination with the motion states of a plurality of consecutive frame times before the current frame time and the motion state of the movable device at the current frame time to predict the motion state of the movable device at the next frame time.

[0203] In this embodiment, the control system further includes: a second determination unit 50, configured to determine whether the tracking state of the movable device fails at the next frame time based on the motion state of the movable device at the next frame time after predicting the motion state of the movable device at the next frame time and before obtaining the configuration information of the light output state of each light output unit in the light output unit at the next frame time based on the motion state of the movable device at the next frame time; if it fails, set the light output state of each light output unit in the light output unit at the next frame time to output light to obtain the configuration information of the light output state of each light output unit in the light output unit at the next frame time; otherwise, obtain the configuration information of the light output state of each light output unit in the light output unit at the next frame time based on the motion state of the movable device at the next frame time.

[0204] By determining whether the tracking state of the movable device fails at the next frame time, the motion state at the next frame time is classified. Only when the tracking state of the movable device does not fail at the next frame time, the next calculation processing step is executed. When the tracking state of the movable device fails at the next frame time, the next calculation processing step is not executed. Instead, the light output state of each light output unit in the light output unit at the next frame time is directly set, which is beneficial for targeted calculation processing, thereby saving computing power.

[0205] In this embodiment, when the tracking state of the movable device fails at the next frame time, the light output state of each light output unit in the light output unit at the next frame time is set to output light, which is beneficial for strengthening the positioning information provided by the light output unit of the movable device in case of tracking failure, so that the tracking of the movable device can succeed as quickly as possible.

[0206] As an example, the failure of the tracking state of the movable device at the current frame time is usually that the movable device moves out of the field of view. When the movable device moves out of the field of view, the light output state of each light output unit in the light output unit at the next frame time is set to output light, which is beneficial for tracking the movable device for the first time when the movable device moves back into the field of view.

[0207] In this embodiment, determining whether the tracking state of the movable device fails at the next frame time based on the motion state of the movable device at the next frame time includes: determining whether the speed or acceleration of the movable device at the next frame time is abnormal, where an abnormal speed or acceleration indicates that the tracking state of the movable device fails at the next frame time.

[0208] When the speed or acceleration is abnormal (for example, when the speed or acceleration is too large), it indicates that the motion of the movable device is abnormal at the next frame time, thereby indicating that the tracking state of the movable device fails at the next frame time.

[0209] The configuration information unit 30 is configured to obtain configuration information of the light output state of each light output unit in the light output unit at the next frame time based on the motion state of the movable device at the next frame time to control the light output unit.

[0210] In this embodiment, obtaining the light output states of all the light output units at the next frame time based on the motion state of the movable device at the next frame time is beneficial to obtaining the light output states corresponding to the light output units according to the prediction of the motion state at the next frame time. That is, it is beneficial to adaptively adjust the light output states of all the light output units based on the real-time state of the movable device, so that the light output units can achieve accurate and low-power light output states. This is beneficial to ensuring the usage accuracy of the movable device and reducing the power consumption of the movable device.

[0211] In this embodiment, obtaining the configuration information of the light output state of each light output unit in the light output unit at the next frame time based on the motion state of the movable device at the next frame time includes: configuring the light output state of each light output unit in the light output unit. This configuration includes determining whether the light output unit is within the detection range of the detector based on the motion state of the movable device at the next frame time.

[0212] Determine whether the light output unit is within the detection range of the detector based on the motion state of the movable device at the next frame time. Set the light output state of the light output unit based on whether the light output unit is within the detection range of the detector. Classify the light output units not within the detection range and the light output units within the detection range, and adaptively adjust the light output state of the light output unit based on the actual detection situation.

[0213] Specifically, in this embodiment, if the light output unit is not within the detection range of the detector, the light output state of the light output unit is set to not output light. Otherwise, the light output state of the light output unit is set to output light.

[0214] When the light output unit is not within the detection range of the detector, the light output unit is not used during the tracking of the movable device by the detector. Accordingly, setting the light output state of the light output unit to not output light is beneficial to saving the light output power consumption and does not affect the tracking accuracy of the detector for the movable device.

[0215] Otherwise, when the light output unit is within the detection range of the detector, the light output unit needs to be used during the tracking of the movable device by the detector. Accordingly, setting the light output state of the light output unit to output light is used to enable the detector to track the movable device.

[0216] In this embodiment, determining whether the light output unit is within the detection range of the detector based on the motion state of the mobile device in the next frame time includes: obtaining the pose of each light output unit in the light output units on the mobile device based on the pose of the mobile device in the next frame time; and determining whether the light output unit is within the detection range of the detector based on the pose of the light output unit.

[0217] The pose of the light output unit characterizes the translational degrees of freedom (x, y, z) of the light output unit in the coordinate system of the detector, and the rotational degrees of freedom (rx, ry, rz) of the light output unit in the coordinate system of the detector. Therefore, it is possible to determine whether the light output unit is within the detection range of the detector according to the pose of the light output unit.

[0218] In this embodiment, during the process of obtaining the pose of each light output unit in the light output units on the mobile device based on the pose of the mobile device in the next frame time, the pose of each light output unit in the light output units on the mobile device is obtained based on the pose of the mobile device in the next frame time and the relative position relationship between all the light output units on the mobile device.

[0219] Specifically, in this embodiment, each light output unit has a fixed relative position on the mobile device, and the detector has a model of the relative position of the light output unit on the mobile device. After obtaining the pose of the mobile device in the next frame time, the pose of each light output unit on the mobile device can be obtained based on the model of the relative position of the light output unit in the detector on the mobile device.

[0220] In this embodiment, determining whether the light output unit is within the detection range of the detector includes: obtaining the light output direction of the light output unit as the first direction; obtaining the direction of the light output unit towards the center of the coordinate system of the detector as the second direction; and obtaining the angle between the first direction and the second direction.

[0221] The light output direction of the light output unit is the direction in which the light output unit outputs light. The direction of the light output unit towards the center of the coordinate system of the detector is the direction in which the detector receives the light output by the light output unit, so that the angle between the first direction and the second direction can characterize the reception condition of the light output by the light output unit by the detector.

[0222] Specifically, in this embodiment, the pose of the light output unit represents the translational degrees of freedom (x, y, z) of the light output unit in the coordinate system of the detector, and the rotational degrees of freedom (rx, ry, rz) of the light output unit in the coordinate system of the detector. Therefore, the direction of the light output unit towards the center of the coordinate system of the detector is obtained based on the position of the light output unit, and the light output direction of the light output unit is obtained based on the pose of the light output unit, where the center of the coordinate system of the detector is the origin.

[0223] In this embodiment, it is determined whether the included angle is within the invalid viewing angle range, and the invalid viewing angle range is the viewing angle range that exceeds the detection viewing angle of the detector.

[0224] Determining whether the included angle is within the invalid viewing angle range means determining whether the detector can receive the light output by the light output unit.

[0225] It should be noted that in this embodiment, the invalid viewing angle range should not be too large, that is, the smaller end value of the end values of the invalid viewing angle range should not be too small. During the actual detection process of the detector, the light spot obtained by the detector and corresponding to the light output unit is circular or elliptical, which means that the light spot corresponding to the light output unit is a region rather than a point. During the calculation of the motion state, the center position of the light spot can be obtained, and the pose is calculated based on the center position of the light spot. Therefore, although the calculated included angle indicates that the center position of the light spot of the light output unit cannot be received by the detector (for example, the included angle between the calculated light output direction of the light output unit and the direction of the light output unit towards the center of the coordinate system of the detector is 95°), in fact, some regions around the center position of the light spot corresponding to the light output unit can still be received by the detector. At this time, it should also be determined that the light output unit is not within the invalid viewing angle range. Therefore, in order to avoid determining that the light output unit that can be detected by the detector is within the invalid viewing angle range, in this embodiment, the invalid viewing angle range should not be too large. That is, the smaller end value of the end values of the invalid viewing angle range should not be too small. Therefore, in this embodiment, the invalid viewing angle range is 120° to 180°.

[0226] Specifically, in this embodiment, if the included angle is within the invalid viewing angle range, it is determined that the light output unit is not within the detection range of the detector. Otherwise, it is determined that the light output unit is within the detection range of the detector.

[0227] Specifically, in this embodiment, if the included angle is within the invalid viewing angle range, the light output unit is set to not output light. If the included angle is not within the invalid viewing angle range, the light output unit is set to output light.

[0228] Correspondingly, the present invention also provides a tracking system for a movable device. Figure 6 It is a functional block diagram of an embodiment of the tracking system for a movable device according to the present invention.

[0229] The mobile device is equipped with a plurality of light output units for outputting signal light, and the tracking system for the mobile device includes: a calculation and processing module 60, which is used to obtain configuration information of the light output state of each light output unit in the light output units by using any control method provided by the present invention; and a control module 70, which is used to adjust the light output state of each output unit in the output units based on the configuration information of the light output state.

[0230] Obtaining the configuration information of the light output state of the light output units by the calculation and processing module 60 facilitates subsequent adjustment of the light output state of the light output units using this configuration information.

[0231] Obtaining the light output state of all light output units at the next frame time based on the motion state of the mobile device at the next frame time is beneficial for obtaining the light output state corresponding to the light output units according to the prediction of the motion state at the next frame time. That is, it is beneficial to adaptively adjust the light output state of all light output units based on the real-time state of the mobile device, so that the light output units achieve an accurate and low-power light output state. This is beneficial to ensuring the usage accuracy of the mobile device and reducing the power consumption of the mobile device.

[0232] For the specific description of the mobile device and obtaining the configuration information of the light output state of the light output units, reference can be made to the relevant descriptions in the foregoing embodiments and will not be described in detail here.

[0233] The control module 70 is configured to adjust the light output state of each light output unit in the light output units based on the configuration information of the light output state.

[0234] In this embodiment, the light output unit adopts a blinking mode, which is beneficial for the light output unit to output light at intervals to reduce the power consumption of the light output.

[0235] Correspondingly, during the adjustment of the light output state of all light output units, the light output unit outputs light in a blinking mode.

[0236] In other embodiments, the light output unit can also output light in a normally-on mode. Correspondingly, during the adjustment of the light output state of all light output units, the light output unit outputs light in a normally-on mode.

[0237] In this embodiment, during the adjustment of the light output state of all light output units, the light output state of the light output unit is controlled to not output light by reducing the light output frequency to 0 or reducing the light output duration of each light output to 0.

[0238] In this embodiment, the optical output units are grouped and connected in parallel, and each group of optical output units is connected in series with the control module 70. The optical output units in each group are connected in series, or the optical output units are connected in parallel. All the optical output units are independently connected in series with the control module 70. Therefore, the optical output states of all the optical output units can be flexibly adjusted accordingly based on the configuration information of the optical output states of all the optical output units.

[0239] As an example, the optical output units are all connected in parallel, and all the output units are independently connected in series with the control module 70.

[0240] In this embodiment, the circuit connection method for connecting the optical output units to the control module 70 includes a rigid circuit board, a flexible circuit board, or a flexible and rigid combined board.

[0241] In this embodiment, the mobile device further includes an information receiving module 80, which is used to receive the configuration information of the optical output states of all the optical output units.

[0242] Specifically, in this embodiment, the information receiving module 80 receives the configuration information of the optical output states of all the optical output units, and then transmits the configuration information to the control module 70. The control module 70 adjusts the optical output states of all the optical output units based on the configuration information of the optical output states of all the optical output units.

[0243] In this embodiment, the information receiving module 80 receives information via Bluetooth, WiFi, or wired transmission.

[0244] In this embodiment, the control method described in the foregoing embodiment is used to obtain the configuration information of the optical output states. For the specific description of the configuration information of the optical output states, reference can be made to the relevant description in the foregoing embodiment and will not be described in detail herein.

[0245] For the specific description of the tracking system in this embodiment, reference can be made to the corresponding description of the tracking method in the foregoing embodiment. This embodiment will not be described in detail herein.

[0246] The embodiment of the present invention further provides a device, which can implement the above-mentioned control method by loading the control method provided by the embodiment of the present invention in the form of a program, or can implement the above-mentioned tracking method by loading the tracking method provided by the embodiment of the present invention in the form of a program.

[0247] In this embodiment, the device includes a detector and a movable device. The movable device is a device that can move and be tracked. In a specific implementation, the movable device is used in combination with the detector. After obtaining the motion state of the movable device, the display and calculation content of the detector can be updated based on the motion state of the movable device.

[0248] For example, the movable device is a handheld controller applied to VR, AR, or MR, and the detector is a head-mounted display device used in combination with the handheld controller. Specifically, as an example, the head-mounted display device can be VR, AR, or MR smart glasses.

[0249] Correspondingly, as an example, the device provided by this embodiment is an all-in-one head-mounted device with a handheld controller. For example, the device is an all-in-one head-mounted 6DoF device, etc.

[0250] Figure 7 The structural diagram of the hardware of the device according to the embodiment of the present invention is shown. The device of this embodiment includes: at least one processor 01, at least one network interface 02, at least one memory 03, and at least one communication bus 04.

[0251] In this embodiment, there is at least one processor 01, a communication interface 02, a memory 03, and a communication bus 04, and the processor 01, the communication interface 02, and the memory 03 complete mutual communication through the communication bus 04.

[0252] Optionally, the communication interface 02 can be an interface of a communication module for network communication, such as an interface of a Global System for Mobile Communications (GSM) module.

[0253] Optionally, the processor 01 can be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the control method according to the embodiment of the present invention.

[0254] Optionally, the memory 03 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0255] The memory 03 stores one or more computer instructions. The one or more computer instructions are executed by the processor 01 to implement the control method or tracking method provided by the foregoing embodiment.

[0256] It should be noted that the specific implementation terminal device mentioned above may also include other devices (not shown) that are not necessary for the disclosure of the embodiment of the present invention. In view of the fact that these other devices may not be necessary for understanding the disclosure of the embodiment of the present invention, the embodiment of the present invention will introduce them one by one.

[0257] Embodiments of the present invention also provide a storage medium. The storage medium stores one or more computer instructions. The one or more computer instructions are configured to implement the control method or the tracking method provided by the foregoing embodiments.

[0258] The above specific implementations of the present invention are combinations of elements and features of the present invention. Unless otherwise specified, these elements or features can be considered optional. Each element or feature can be practiced without being combined with other elements or features. In addition, specific implementations of the present invention can be constructed by combining some elements and / or features. The order of operations described in the embodiments of the present invention can be rearranged. Some configurations of any specific implementation can be included in another specific implementation and can be replaced by corresponding configurations of another specific implementation. For those skilled in the art, it is obvious that claims that are not explicitly cross-referenced in the appended claims can be combined into specific implementations of the present invention or can be used as new claims in amendments after the filing of this application.

[0259] Specific implementations of the present invention can be realized by various components such as hardware, firmware, software, or combinations thereof. Among the hardware configuration components, the method according to the exemplary specific implementation of the present invention can be realized by one or more ASICs, digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0260] Among the firmware or software configuration components, specific implementations of the present invention can be realized in the form of modules, processes, functions, etc. The software code can be stored in a memory unit and executed by a processor. The memory unit is located inside or outside the processor, and data can be transmitted to and received from the processor through various known measures.

[0261] The foregoing explanations of the disclosed embodiments enable those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to these embodiments shown herein, but rather conforms to the broadest scope consistent with the principles and novel features disclosed herein.

[0262] Although the embodiments of the present invention are disclosed above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and the protection scope of the present invention is defined by the appended claims.

Claims

1. A control method for a light output unit of a mobile device, characterized in that Including: Using a detector to obtain an image of a movable device at the current frame time, the movable device being equipped with a plurality of light output units for outputting signal light; Predicting a motion state of the movable device at the next frame time based on the image; And Obtaining configuration information of a light output state of each of the light output units at the next frame time based on the motion state of the movable device at the next frame time to control the light output units.

2. The control method according to claim 1, wherein Obtaining the configuration information of the light output states of the plurality of light output units at the next frame time based on the motion state of the movable device at the next frame time includes: Configuring the light output state of each of the light output units; wherein the configuration includes: Determining whether the light output unit is within a detection range of the detector based on the motion state of the movable device at the next frame time; If the light output unit is not within the detection range of the detector, setting the light output state of the light output unit to no light output; and Otherwise, setting the light output state of the output unit to output light.

3. The control method according to claim 2, wherein In predicting the motion state of the movable device at the next frame time, the motion state includes a pose of the movable device; Determining whether the light output unit is within a detection range of the detector based on the motion state of the movable device at the next frame time includes: Obtaining a pose of each light output unit on the movable device based on the pose of the movable device at the next frame time; And Determining whether the light output unit is within the detection range of the detector based on the pose of the light output unit.

4. The control method according to claim 3, wherein In obtaining the pose of each light output unit on the movable device based on the pose of the movable device at the next frame time, the pose of each light output unit on the movable device is obtained based on the pose of the movable device at the next frame time and a relative position relationship of each light output unit on the movable device.

5. The control method according to claim 2 or 3, characterized in that Determining whether the light output unit is within the detection range of the detector includes: Obtaining a light output direction of the light output unit as a first direction; Obtaining a direction of the light output unit toward the center of the coordinate system of the detector as a second direction; Obtaining an included angle between the first direction and the second direction; Determining whether the included angle is within a range of an invalid viewing angle, the invalid viewing angle being a viewing angle range exceeding a detection viewing angle of the detector; If the included angle is within the invalid viewing angle range, determining that the light output unit is not within the detection range of the detector; and Otherwise, determining that the light output unit is within the detection range of the detector.

6. The control method according to claim 5, characterized in that The invalid viewing angle range is from 120° to 180°.

7. The control method according to claim 1, characterized in that Predicting the motion state of the movable device in the next frame time based on the image includes: Obtaining the motion state of the movable device at the current frame time based on the image; and Predicting the motion state of the movable device at the next frame time based on the motion state of the movable device at the current frame time.

8. The control method according to claim 7, wherein Obtaining the motion state of the movable device at the current frame time based on the image includes: Extracting the corresponding light spot of the light output unit on the image; Obtaining the pose of the movable device at the current frame time based on the light spot corresponding to the light output unit; Obtaining the speed of the movable device at the current frame time by combining the position of the movable device at the previous frame time, the position of the movable device at the current frame time, and the time difference between the previous frame time and the current frame time; and Obtaining the acceleration of the movable device at the current frame time by combining the speed of the movable device at the previous frame time, the speed of the movable device at the current frame time, and the time difference between the previous frame time and the current frame time, and Predicting the motion state of the movable device at the next frame time based on the motion state of the movable device at the current frame time includes: Predicting the pose, speed, and acceleration of the movable device at the next frame time based on the pose, speed, and acceleration of the movable device at the current frame time.

9. The control method according to claim 7, wherein In the step of predicting the motion state of the movable device at the next frame time based on the motion state of the movable device at the current frame time, the motion state of the movable device at the next frame time is predicted by combining the motion state of the movable device at the current frame time and the motion state of the movable device at the previous frame time or the motion states of the movable device at multiple consecutive frame times before the current frame time.

10. The control method according to claim 9, characterized in that, The motion state of the movable device at the next frame time is predicted using the Taylor expansion formula by combining the motion state of the movable device at the current frame time and the motion state of the movable device at the previous frame time or the motion states of the movable device at multiple consecutive frame times before the current frame time.

11. The control method according to claim 1, wherein After obtaining the image of the movable device at the current frame time using the detector and before predicting the motion state of the movable device at the next frame time based on the image, the method further includes: Determining whether the tracking state of the movable device at the current frame time fails based on the image; If it fails, setting the light output state of each light output unit in the light output unit to output light at the next frame time to obtain the configuration information of the light output state of each light output unit at the next frame time; Otherwise, perform predicting the motion state of the movable device at the next frame time based on the image.

12. The control method according to claim 11, characterized in that, Determining whether the tracking state of the movable device fails at the current frame time based on the image includes: Obtaining the number of light spots corresponding to the light output unit on the image, where the number less than a preset number of light spots indicates that the tracking state of the movable device fails at the current frame time.

13. The control method according to claim 1, characterized in that, After predicting the motion state of the movable device at the next frame time based on the image and before obtaining the configuration information of the light output state of each light output unit in the light output unit at the next frame time based on the motion state of the movable device at the next frame time to control the light output unit, the method further includes: Determining whether the tracking state of the movable device fails at the next frame time based on the motion state of the movable device at the next frame time; If it fails, setting the light output state of each light output unit in the light output unit at the next frame time to output light to obtain the configuration information of the light output state of each light output unit in the light output unit at the next frame time; Otherwise, obtaining the configuration information of the light output state of each light output unit in the light output unit at the next frame time based on the motion state of the movable device at the next frame time.

14. The control method according to claim 13, wherein The motion state includes speed and acceleration; and determining whether the tracking state of the movable device fails at the next frame time based on the motion state of the movable device at the next frame time includes: Determining whether the speed or the acceleration of the movable device at the next frame time is abnormal, where the speed or the acceleration being abnormal indicates that the tracking state of the movable device fails at the next frame time.

15. A tracking method for a mobile device, characterized in that, The movable device is equipped with a plurality of light output units for outputting signal light, and the tracking method includes: Using the control method according to any one of claims 1 to 14 to obtain the configuration information of the light output state of the light output unit; and Adjusting the light output state of each light output unit in the light output unit based on the configuration information of the light output state.

16. A control system for a light output unit of a mobile device, characterized in that Including: An image acquisition unit for acquiring an image of the movable device at the current frame time using a detector, where the movable device is equipped with a plurality of light output units for outputting signal light; And A prediction unit for predicting the motion state of the movable device at the next frame time based on the image; A configuration information acquisition unit for obtaining the configuration information of the light output state of each light output unit in the light output unit at the next frame time based on the motion state of the movable device at the next frame time to control the light output unit.

17. A tracking system for a mobile device, characterized in that, The movable device is equipped with a plurality of light output units for outputting signal light, and the tracking system includes: A calculation and processing module, which is configured to obtain configuration information of the optical output state of the optical output unit by using the control method according to any one of claims 1 to 14; and A control module, which is configured to adjust the optical output state of each output unit in the output unit based on the configuration information of the output state.

18. The tracking system according to claim 17, wherein The control module and the optical output unit are connected in the following manner: The optical output units are grouped and the groups are connected in parallel, each group of optical output units is connected in series with the control module, and the optical output units in each group are connected in series, or All optical output units are connected in parallel, and each output unit is individually connected in series with the control module.

19. A device, characterized in that Comprising: At least one memory and at least one processor, the memory stores one or more computer instructions, wherein the one or more computer instructions can be executed by the processor to implement the control method according to any one of claims 1 to 14 or to implement the tracking method according to claim 15.

20. A storage medium, characterized in that, The storage medium stores one or more computer instructions for implementing the control method according to any one of claims 1 to 14 or for implementing the tracking method according to claim 15.

Images