Control method and device, head-mounted display equipment and medium

Abstract

The invention provides a control method and device, head-mounted display equipment and a medium, and the method comprises the steps: obtaining the exposure time of a tracking camera disposed on the head-mounted display equipment according to the current system load of the head-mounted display equipment; and under the condition that the tracking camera is controlled to start exposure according to the exposure duration, transmitting a control instruction to a handle to control a luminous body on the handle to be lightened.

Description

Technical Field

[0001] This disclosure relates to the field of head-mounted display technology, and more specifically, to a control method, a control device, a head-mounted display device, and a computer-readable storage medium. Background Technology

[0002] Typically, head-mounted display devices, such as virtual reality (VR) head-mounted display devices, are equipped with controllers, which can be tracked and positioned using the VR head-mounted display device.

[0003] In related technologies, infrared lights are installed on the handle, and the handle is positioned by capturing the infrared lights through an infrared camera on the head-mounted display device. Furthermore, to avoid problems such as motion blur in the images captured by the infrared camera, the duration of each infrared light illumination and the exposure time of the infrared camera cannot be too long, for example, around a few milliseconds. This requires a strict synchronization relationship between the infrared camera and the infrared lights.

[0004] Generally, the head-mounted display device sends a control command to turn on the infrared lights only after the infrared camera has started exposure. The exposure time of the infrared camera is usually fixed. Therefore, when the system load of the head-mounted display device increases, the transmission time of the control command will become longer, even though the exposure time is fixed. This can lead to the problem that the infrared lights only turn on after the infrared camera has finished exposure, causing the infrared camera to fail to capture the infrared lights and affecting the tracking accuracy and stability of the controller. Summary of the Invention

[0005] One objective of this disclosure is to provide a new technical solution for control.

[0006] According to a first aspect of the present disclosure, a control method is provided, the method comprising:

[0007] Based on the current system load of the head-mounted display device, obtain the exposure time of the tracking camera set on the head-mounted display device;

[0008] When the tracking camera is controlled to start exposure for the specified exposure duration, a control command is transmitted to the handle to control the light source on the handle to light up.

[0009] Optionally, obtaining the exposure duration of the tracking camera set on the head-mounted display device based on the current system load of the head-mounted display device includes:

[0010] The transmission time of the control command is obtained based on the current system load of the head-mounted display device;

[0011] The exposure time of the tracking camera set on the head-mounted display device is determined based on the transmission time.

[0012] Optionally, obtaining the transmission time of the control command based on the current system load of the head-mounted display device includes:

[0013] Obtain the mapping relationship between the system load and transmission time of the head-mounted display device;

[0014] The transmission time of the control command is obtained based on the current system load and the mapping relationship.

[0015] Optionally, the mapping relationship is a mapping function, and obtaining the mapping relationship between the system load and transmission time of the head-mounted display device includes:

[0016] When the head-mounted display device is operating under different system loads, the first actual transmission time of the head-mounted display device transmitting the control command to the handle is collected under the different system loads.

[0017] The mapping function is obtained based on the different system loads and the first actual transmission time corresponding to the different system loads.

[0018] Optionally, the method further includes:

[0019] When the tracking camera is controlled to start exposure at the exposure duration, after the control command is transmitted to the handle, the second actual transmission time of the control command is obtained.

[0020] The mapping function is corrected based on the current system load and the second actual transmission time.

[0021] Optionally, determining the exposure duration of the tracking camera set on the head-mounted display device based on the transmission time includes:

[0022] Based on the transmission time and the set delay, the exposure time of the tracking camera set on the head-mounted display device is obtained;

[0023] The set delay is obtained by sampling the head-mounted display device under extreme system load.

[0024] Optionally, the method further includes the step of obtaining the set delay.

[0025] Obtaining the set delay includes:

[0026] When the head-mounted display device is operating under the extreme system load, the third actual transmission time of the head-mounted display device transmitting the control command to the handle under the extreme system load is collected;

[0027] Based on the third actual transmission time, the delay is gradually increased, and for each delay, the sum of the third actual transmission time and the delay is taken as the actual exposure time corresponding to the delay.

[0028] If an abnormality occurs in the image acquired by the tracking camera at the corresponding actual exposure time, the delay corresponding to the actual exposure time shall be taken as the limit delay.

[0029] The set delay is determined based on the limit delay.

[0030] According to a second aspect of the present disclosure, a control device is provided, the device comprising:

[0031] The acquisition module is used to obtain the exposure duration of the tracking camera set on the head-mounted display device based on the current system load of the head-mounted display device;

[0032] The control module is used to transmit the control command to the handle when controlling the tracking camera to start exposure at the exposure duration; wherein the control command is used to control the light source on the handle to light up.

[0033] According to a third aspect of the present disclosure, a head-mounted display device is provided, the head-mounted display device comprising:

[0034] Memory is used to store executable computer instructions;

[0035] A processor, configured to execute the control method described in the first aspect above, under the control of the executable computer instructions.

[0036] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, perform the method described in the first aspect above.

[0037] One beneficial effect of this disclosure is that the head-mounted display device can obtain the exposure time of the tracking camera set on the head-mounted display device according to its current system load, and when the tracking camera starts to expose at the exposure time, it transmits control commands to the handle to control the light source on the handle to light up, so that the exposure time of the tracking camera is controlled in real time according to the current system load, ensuring that the light source can be captured in every frame of the image captured by the tracking camera, thereby improving the accuracy and stability of the handle tracking.

[0038] Other features and advantages of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of this specification.

[0040] Figure 1 This is a schematic diagram of the hardware configuration of a head-mounted display device according to an embodiment of the present disclosure;

[0041] Figure 2 This is a flowchart illustrating a control method according to an embodiment of the present disclosure;

[0042] Figure 3 This is a flowchart illustrating a control method according to an example of this disclosure;

[0043] Figure 4 This is a schematic block diagram of a control device according to an embodiment of the present disclosure;

[0044] Figure 5 This is a schematic block diagram of a head-mounted display device according to an embodiment of the present disclosure. Detailed Implementation

[0045] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the embodiments of the present disclosure.

[0046] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0047] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0048] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0049] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0050] <Hardware Configuration>

[0051] like Figure 1 The diagram shows a block diagram of the hardware configuration of the head-mounted display system 100 provided in this embodiment of the invention.

[0052] See Figure 1As shown, the head-mounted display system 100 includes a head-mounted display device 1000 and a handle 2000.

[0053] The head-mounted display device 1000 can be, for example, a VR device, an MR (Mixed Reality) device, etc.

[0054] In one embodiment, the head-mounted display device 1000 can be as follows: Figure 1 As shown, it includes a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, an audio device 1700, a tracking camera 1800, etc.

[0055] The processor 1100 can be, for example, a central processing unit (CPU) or a microprocessor (MCU). The memory 1200 includes, for example, ROM (Read-Only Memory), RAM (Random Access Memory), or non-volatile memory such as a hard disk. The interface device 1300 includes, for example, a serial bus interface (including a USB interface), a parallel bus interface, or an HDMI interface. The communication device 1400 is capable of wired or wireless communication. The display device 1500 is, for example, a liquid crystal display (LCD), an LED display, or a touch screen. The input device 1600 includes, for example, a touchscreen or motion-sensing input. The audio device 1700 can be used for inputting / outputting voice information. The tracking camera 1800 can be used to acquire image information. At least two tracking cameras 1800 can be set.

[0056] Despite Figure 1 The present disclosure illustrates multiple devices for a head-mounted display device 1000; however, this disclosure may relate only to a portion of these devices. For example, the head-mounted display device 1000 may relate only to a memory 1200 and a processor 1100. The memory 1200 of the head-mounted display device 1000 is used to store instructions for controlling the processor 1100 to execute corresponding steps, thereby providing relevant support for the control method of this embodiment.

[0057] In one embodiment, the handle 2000 can be as follows: Figure 1 As shown, it includes at least a light-emitting element 2100. Multiple light-emitting elements 2100 may be provided for identifying the handle 2000. The light-emitting element 2100 may be, for example, a visible light source or an infrared light source, such as an LED (Light Emitting Diode) lamp.

[0058] It should be understood that, despite Figure 1 Only one head-mounted display device 1000 and one handle 2000 are shown, but this does not mean that the number of each is limited. The head-mounted display system 100 may contain multiple head-mounted display devices 1000 and multiple handles 2000.

[0059] In the above description, those skilled in the art can design instructions based on the solutions provided in this disclosure. How the instructions control the processor to operate is well known in the art, and therefore will not be described in detail here.

[0060] Figure 1 The head-mounted display system shown is illustrative only and is by no means intended to limit this disclosure, its application, or its use.

[0061] <Method Implementation>

[0062] Figure 2 This illustration shows a control method according to an embodiment of the present disclosure, applied to a head-mounted display device. The control method of this embodiment can be, for example, by... Figure 1 The illustrated head-mounted display device 1000 can be a VR device.

[0063] like Figure 2 As shown, the control method provided in this embodiment may include the following steps S2100 to S2200.

[0064] Step S2100: Based on the current system load of the head-mounted display device, obtain the exposure time of the tracking camera set on the head-mounted display device.

[0065] Typically, head-mounted display devices include at least two tracking cameras for acquiring images. Exemplarily, a head-mounted display device may include four tracking cameras, positioned within the device according to a predetermined first position constraint rule to ensure that each tracking camera meets the field-of-view requirements of the head-mounted display device. According to embodiments of this disclosure, positioning and tracking the handle based on images acquired by the four tracking cameras can improve robustness and accuracy.

[0066] System load typically includes CPU load, which refers to the number of processes running and waiting to run in the system. CPU load is usually expressed as a percentage, reflecting the degree to which the CPU is occupied by tasks at a certain moment or time period.

[0067] In a specific embodiment, step S2100, which obtains the exposure time of the tracking camera set on the head-mounted display device based on the current system load of the head-mounted display device, may further include the following steps S2110 to S2120:

[0068] Step S2110: Obtain the transmission time of the control command based on the current system load of the head-mounted display device.

[0069] The control commands are used to illuminate the light-emitting elements of the handle for positioning. Typically, the handle has multiple light-emitting elements positioned according to a predetermined second position constraint rule. This prevents overlapping or merging of local pattern information in images captured by the tracking camera when it is close to the handle, and ensures that at least four light spots corresponding to the light-emitting elements appear in images captured by the tracking camera at any angle. This allows for handle positioning based on the captured images. Furthermore, the light-emitting elements can be, for example, visible light or infrared light sources, such as LEDs.

[0070] The transmission time of the control command refers to the time it takes for the head-mounted display device to transmit the control command to the controller. For example, if the time when the head-mounted display device starts transmitting the control command to the controller is t1, and the time when the controller receives the control command is t2, then t2-t1 is the transmission time of the control command.

[0071] Optionally, obtaining the transmission time of the control command based on the current system load of the head-mounted display device in step S2110 may further include the following steps S2111 to S2112:

[0072] Step S2111: Obtain the mapping relationship between the system load and transmission time of the head-mounted display device.

[0073] Among them, system load loading_cur and transmission time T cur The mapping relationship between them can be represented by a mapping function f(loading_cur), where the independent variable of the mapping function f(loading_cur) is the system load loading_cur, and the dependent variable f(loading_cur) is the transmission time T determined by the system load loading_cur. cur Understandably, the mapping function f(loading_cur) can be pre-fitted based on the historical system load and the transmission time of the corresponding control commands.

[0074] Specifically, the mapping relationship is a mapping function. Step S2111, which obtains the mapping relationship between the system load and transmission time of the head-mounted display device, may further include: when the head-mounted display device is running under different system loads, collecting the first actual transmission time of the head-mounted display device transmitting the control command to the handle under the different system loads; and obtaining the mapping function based on the different system loads and the first actual transmission time corresponding to the different system loads.

[0075] Generally, the more data collected on the actual transmission times of different system loads and their corresponding control commands, the more accurate the fitted mapping function will be.

[0076] For example, the control system operates in different application scenarios, and by reading CPU load parameters, the CPU load of the system under different application scenarios is obtained. The transmission time of control commands under different CPU loads is then collected; this transmission time is the actual transmission time of the control commands under the corresponding CPU load. For instance, if the system is operating in scenario 1 with a CPU load of 5%, the transmission time of the control commands is collected as T1; if the system is operating in scenario 2 with a CPU load of 10%, the transmission time of the control commands is collected as T2; and if the system is operating in scenario 3 with a CPU load of 15%, the transmission time of the control commands is collected as T3. After obtaining multiple sets of CPU loads and their corresponding control command transmission times, the system load (loading_cur) and transmission time (T) can be fitted to obtain the system load (loading_cur) and transmission time (T). cur The mapping function between them is f(loading_cur).

[0077] In other words, it can statistically analyze different system loads and the transmission time of control commands sent by head-mounted displays to the controllers under different system loads, and fit a mapping function between system load and the transmission time of control commands.

[0078] Step S2112: Obtain the transmission time of the control command based on the current system load and the mapping relationship.

[0079] In step S2112, after obtaining the current system load of the head-mounted display device, the current system load can be substituted into the mapping function f(loading_cur) to obtain the transmission time of the control commands. The transmission time of the control commands obtained from the mapping function f(loading_cur) and the current system load can be understood as the theoretical transmission time, that is, the predicted transmission time.

[0080] Based on steps S2111 to S2112 above, the transmission time of control commands can be obtained based on the current system load of the head-mounted display device and the mapping function that reflects the mapping relationship between system load and transmission time, thereby effectively improving the accuracy and effectiveness of obtaining the transmission time of control commands.

[0081] Step S2120: Determine the exposure duration of the tracking camera set on the head-mounted display device based on the transmission time.

[0082] The longer the exposure time of the tracking camera, the more information about the ambient light is acquired when capturing images.

[0083] In a specific embodiment, step S2120, which determines the exposure duration of the tracking camera set on the head-mounted display device based on the transmission time, may further include: obtaining the exposure duration of the tracking camera set on the head-mounted display device based on the transmission time and a set delay.

[0084] The set delay is obtained by sampling the head-mounted display device under extreme system load. Typically, extreme system load can be understood as the maximum load the CPU can withstand during operation. This load is usually expressed as a percentage, reflecting the degree to which the CPU is occupied by tasks at a given moment or time period. When the CPU load approaches or reaches its limit, the system may experience performance degradation, slower response times, or even crashes.

[0085] Specifically, the exposure time D_cur of the tracking camera satisfies the following formula:

[0086] D cur =T cur +x (1)

[0087] Among them, T cur This represents the transmission time of the control command, and x represents the set delay. In other words, after obtaining the transmission time of the control device based on the current system load, the transmission time T of the control command can be set. cur Substitute into formula (1) to obtain the exposure time D of the tracking camera. cur .

[0088] Optionally, embodiments of this disclosure further include the step of obtaining the set delay. Obtaining the set delay may further include step S3100: when the head-mounted display device is operating under the extreme system load, acquiring the third actual transmission time of the head-mounted display device transmitting the control command to the handle under the extreme system load; gradually increasing the delay based on the third actual transmission time, and for each delay, using the sum of the third actual transmission time and the delay as the actual exposure time corresponding to the delay; if an abnormality occurs in the image acquired by the tracking camera with the corresponding actual exposure time, using the delay corresponding to the actual exposure time as the extreme delay; and determining the set delay based on the extreme delay.

[0089] The set delay is usually less than the limit delay.

[0090] For example, if the maximum system load is 100%, the actual transmission time of the sampled control command is T. n In this T n Add a delay, for example, T n +1ms, and with T n+1ms is used as the exposure duration for the tracking camera. Specifically, the tracking camera is controlled to operate with an exposure duration of T. n At the +1ms start of exposure, immediately transmit a control command to the handle to illuminate the light source on the handle. If the image captured by the tracking camera is normal, continue increasing the delay. Continue at T... n Add a delay, for example, T n +2ms, and with T n +2ms is used as the exposure time for the tracking camera. Specifically, the tracking camera is controlled to operate with an exposure time of T. n At the +2ms exposure start, immediately transmit a control command to the handle to illuminate the light source on the handle. If the image captured by the tracking camera is normal, continue increasing the delay. Continue at T... n Add a delay, for example, T n +3ms, and with T n +3ms is used as the exposure time for the tracking camera. Specifically, the tracking camera is controlled to operate with an exposure time of T. n When the exposure begins at +3ms, a control command is immediately transmitted to the handle to illuminate the light source on the handle. If the image captured by the tracking camera is abnormal, such as having motion blur, 3ms can be used as the limit delay. The delay can be set to be less than 3ms.

[0091] After obtaining the exposure time of the tracking camera set on the head-mounted display device based on the current system load of the head-mounted display device, proceed to:

[0092] In step S2200, while controlling the tracking camera to start exposure for the specified exposure duration, a control command is transmitted to the handle to control the light source on the handle to light up.

[0093] In this embodiment, after obtaining the exposure time of the tracking camera, a control command can be immediately transmitted to the handle to control the light source on the handle to light up when the tracking camera starts to expose for the exposure time, so as to ensure that the image captured by the tracking camera can capture the light source and improve the accuracy and stability of the handle tracking.

[0094] According to embodiments of this disclosure, the head-mounted display device can obtain the exposure time of the tracking camera set on the head-mounted display device based on its current system load. When the tracking camera is controlled to start exposure at the exposure time, a control command is transmitted to the handle to control the light source on the handle to light up. This allows the exposure time of the tracking camera to be controlled in real time according to the current system load, ensuring that the light source can be captured in every frame of the image taken by the tracking camera, thereby improving the accuracy and stability of the handle tracking.

[0095] In one embodiment, the control method of this disclosure further includes: when controlling the tracking camera to start exposure at the exposure duration, after transmitting the control command to the handle, obtaining a second actual transmission time of the control command; and correcting the mapping function based on the current system load and the second actual transmission time.

[0096] In this embodiment, when the tracking camera is controlled to start exposure for the specified exposure duration according to step S2200 above, a control command is transmitted to the handle. At this time, the actual transmission time of the control command is also collected. Thus, after transmitting the control command to the handle when the tracking camera starts exposure for the specified exposure duration, the pre-fitted mapping function can be corrected based on the current system load obtained in step S2100 and the collected actual transmission time, so that the prediction of the transmission time is more accurate.

[0097] <Example 1>

[0098] Typically, to ensure synchronization between the tracking camera and the light source, the tracking camera must be operating in slave mode. This can be achieved by sending signals from the main program of the head-mounted display device to control the tracking camera's active exposure. Taking 30Hz as an example, the main program triggers the tracking camera to expose and illuminate the light source every 33.3 milliseconds.

[0099] Each time the main program needs to trigger the tracking camera exposure and the illumination of the light source, it first obtains the current CPU load loading_cur, and then determines the load based on T. cur =f(loading_cur), calculate the transmission time T corresponding to the current CPU load loading_cur. cur and the transmission time T cur Substitute the values ​​into formula (1) above to obtain the exposure time of the tracking camera. After the main program controls the tracking camera to start the exposure at the exposure time, it immediately sends a control command to the handle to trigger the illumination of the light source. Since the transmission time of the control command is less than the exposure time, the tracking camera is still in the exposure state when the light source is illuminated, thus maintaining synchronization.

[0100] <Example 2>

[0101] The following shows an example of a control method, in which reference... Figure 3 The control method may further include:

[0102] Step 401: When the main program triggers the exposure of the tracking camera and the lighting of the light source, it will detect the current system load. If the current system load changes, it will proceed to step 402 to calculate the synchronization delay.

[0103] Step 402: The main program performs synchronization delay calculation.

[0104] Specifically, the main program obtains the current CPU load loading_cur, and then... cur =f(loading_cur), calculate the transmission time T corresponding to the current CPU load loading_cur. cur and the transmission time T cur Substitute into the above formula (1) to obtain the exposure time of the tracking camera.

[0105] Step 403: After the main program controls the tracking camera to start the exposure for the specified exposure time, it immediately sends a control command to the handle to trigger the illumination of the light source. Synchronous control is completed, and the above step 401 is executed again.

[0106] Based on this example, it can control the exposure time of the tracking camera in real time according to the system load, ensuring that the tracking camera can capture the light source in every frame, thereby improving the accuracy and stability of the handle tracking.

[0107] <Device Embodiment>

[0108] Figure 4 This is a schematic diagram of the structure of a control device according to one embodiment. For example... Figure 4 As shown, the control device 400 includes an acquisition module 410 and a control module 420.

[0109] The acquisition module 410 is used to obtain the exposure duration of the tracking camera set on the head-mounted display device based on the current system load of the head-mounted display device;

[0110] The control module 420 is used to transmit control commands to the handle to control the light source on the handle to light up when the tracking camera starts to expose for the exposure duration.

[0111] In one embodiment, the acquisition module 410 is specifically used to obtain the transmission time of the control command based on the current system load of the head-mounted display device; and to determine the exposure duration of the tracking camera set on the head-mounted display device based on the transmission time.

[0112] In one embodiment, the acquisition module 410 is specifically used to acquire the mapping relationship between the system load and transmission time of the head-mounted display device; and to obtain the transmission time of the control command based on the current system load and the mapping relationship.

[0113] In one embodiment, the mapping relationship is a mapping function. The acquisition module 410 is specifically used to collect the first actual transmission time of the head-mounted display device transmitting the control command to the handle under different system loads; and to obtain the mapping function based on the different system loads and the first actual transmission time corresponding to the different system loads.

[0114] In one embodiment, the acquisition module 410 is further configured to acquire the second actual transmission time of the control command after the control module 420 transmits the control command to the handle when the tracking camera is controlled to start exposure at the exposure duration.

[0115] The control module 420 is also used to correct the mapping function based on the current system load and the second actual transmission time.

[0116] In one embodiment, the acquisition module 410 is specifically used to obtain the exposure duration of the tracking camera set on the head-mounted display device based on the transmission time and the set delay;

[0117] The set delay is obtained by sampling the head-mounted display device under extreme system load.

[0118] In one embodiment, the acquisition module 410 is further configured to: acquire a third actual transmission time of the head-mounted display device transmitting the control command to the handle under the extreme system load; gradually increase the delay based on the third actual transmission time; and for each delay, use the sum of the third actual transmission time and the delay as the actual exposure time corresponding to the delay; if an abnormality occurs in the image acquired by the tracking camera with the corresponding actual exposure time, use the delay corresponding to the actual exposure time as the limit delay; and determine the set delay based on the limit delay.

[0119] According to embodiments of this disclosure, the head-mounted display device can obtain the exposure time of the tracking camera set on the head-mounted display device based on its current system load. When the tracking camera is controlled to start exposure at the exposure time, a control command is transmitted to the handle to control the light source on the handle to light up. This allows the exposure time of the tracking camera to be controlled in real time according to the current system load, ensuring that the light source can be captured in every frame of the image taken by the tracking camera, thereby improving the accuracy and stability of the handle tracking.

[0120] <Equipment Example>

[0121] Figure 5 This is a schematic diagram of the hardware structure of a head-mounted display device according to one embodiment. Figure 5 As shown, the head-mounted display device 500 includes a processor 510 and a memory 520.

[0122] The memory 520 can be used to store executable computer instructions.

[0123] The processor 510 can be used to execute the control method according to the method embodiments of this disclosure, under the control of the executable computer instructions.

[0124] The head-mounted display device 500 can be as follows: Figure 1 The head-mounted display device 1000 shown may also be a device with other hardware structures, and is not limited herein. The head-mounted display device 500 may be, for example, a VR device, a MR device, etc., and is not limited in this embodiment.

[0125] In another embodiment, the head-mounted display device 500 may include the control device 400 described above.

[0126] In one embodiment, each module of the control device 400 can be implemented by the processor 510 running computer instructions stored in the memory 520.

[0127] Computer-readable storage media

[0128] This disclosure also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, perform the control method provided in this disclosure.

[0129] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0130] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0131] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0132] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0133] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0134] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0135] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0137] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A control method characterized by, The method comprises: According to the current system load of the head-mounted display device, the exposure time of the tracking camera arranged on the head-mounted display device is obtained; In the case of controlling the tracking camera to start exposure for the exposure time, a control instruction is transmitted to the handle to control the light emitter on the handle to light up.

2. The method of claim 1, wherein, According to the current system load of the head-mounted display device, the exposure time of the tracking camera arranged on the head-mounted display device is obtained, comprising: According to the current system load of the head-mounted display device, the transmission time of the control instruction is obtained; According to the transmission time, the exposure time of the tracking camera arranged on the head-mounted display device is determined.

3. The method of claim 2, wherein, According to the current system load of the head-mounted display device, the transmission time of the control instruction is obtained, comprising: Obtain the mapping relationship between the system load and the transmission time of the head-mounted display device; According to the current system load and the mapping relationship, the transmission time of the control instruction is obtained.

4. The method of claim 3, wherein, The mapping relationship is a mapping function, and the mapping relationship between the system load and the transmission time of the head-mounted display device is obtained, comprising: Under the condition that the head-mounted display device respectively runs under different system loads, the first actual transmission time of the head-mounted display device respectively transmitting the control instruction to the handle under the different system loads is collected; According to the different system loads and the first actual transmission time corresponding to the different system loads, the mapping function is obtained.

5. The method of claim 4, wherein, The method further comprises: After transmitting the control instruction to the handle under the condition of controlling the tracking camera to start exposure for the exposure time, the second actual transmission time of the control instruction is obtained; According to the current system load and the second actual transmission time, the mapping function is corrected.

6. The method of claim 2, wherein, According to the transmission time, the exposure time of the tracking camera arranged on the head-mounted display device is determined, comprising: According to the transmission time and the set delay, the exposure time of the tracking camera arranged on the head-mounted display device is obtained; Wherein, the set delay is obtained by sampling when the head-mounted display device is under the limit system load.

7. The method of claim 6, wherein, The method further comprises the step of obtaining the set delay, The method further comprises the step of obtaining the set delay, The method further comprises the step of obtaining the set delay, Under the condition that the head-mounted display device runs under the limit system load, the third actual transmission time of the head-mounted display device transmitting the control instruction to the handle under the limit system load is collected; On the basis of the third actual transmission time, the delay is gradually increased, and for each delay, the sum of the third actual transmission time and the delay is taken as the actual exposure time corresponding to the delay; In the case that the image collected by the tracking camera under the corresponding actual exposure time is abnormal, the delay corresponding to the actual exposure time is taken as the limit delay; 8. A control device characterized by comprising: According to the limit delay, the set delay is determined. The device comprises: An acquisition module is configured to obtain the exposure time of the tracking camera arranged on the head-mounted display device according to the current system load of the head-mounted display device; The control module is configured to transmit the control instruction to the handle when the tracking camera is controlled to start exposure for the exposure time length, and the control instruction is configured to control the light emitter on the handle to light up.

9. A head-mounted display device, comprising: The head-mounted display device comprises: a memory configured to store executable computer instructions; a processor configured to execute the control method according to any one of claims 1-7 under control of the executable computer instructions. 10.A computer readable storage medium having computer instructions stored thereon, wherein the computer instructions are executed by a processor to perform the control method according to any one of claims 1-7.

Images