VR display system

Abstract

According to the VR display system provided by the embodiment of the invention, the VR display system comprises a head-mounted host device and a slave device, the head-mounted host device comprises a camera, a micro-control unit and a first radio frequency chip, the slave device comprises a second radio frequency chip, and after the micro-control unit obtains a camera signal sent by the camera, the camera signal is sent to the first radio frequency chip; whether the camera is in a standby state or not is judged based on the frame rate of the vertical synchronizing signal in the camera signal, and under the condition that the frame rate of the vertical synchronizing signal in the camera signal is smaller than a preset threshold value, the micro-control unit controls to close a radio frequency link between the first radio frequency chip and the second radio frequency chip. And after the first radio frequency chip closes the radio frequency link with the second radio frequency chip, data interaction between the first radio frequency chip and the second radio frequency chip is stopped, so that when the camera is in a standby state, data interaction between the first radio frequency chip and the second radio frequency chip is stopped, and the camera is in a standby state. The resource occupation of the master and slave machines is avoided, and the power consumption of the master and slave machines is reduced.

Description

Technical Field

[0001] The present application relates to the field of display technology, and in particular to a VR display system. Background Art

[0002] With the development of science and technology, virtual reality (VR) technology is becoming increasingly popular because it can provide a better visual experience, making people feel as if they are actually there. VR technology is a computer simulation system that can create and experience a virtual world. It uses computers to generate a simulated environment. It is a multi-source information fusion, interactive three-dimensional dynamic visuals and physical behavior simulation system that immerses users in the environment.

[0003] The VR display system includes a head-mounted host device and a slave device. The head-mounted host device is a device that receives user responses and adds images and files to the images viewed by the user through a device worn on the user's head, thereby achieving an enhanced virtual reality effect.

[0004] In the prior art, when the user removes the head-mounted host device, the slave device will continue to transmit data from the inertial measurement unit of the slave device to the head-mounted host for calculation, causing the VR display system to consume high power and waste resources. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a VR display system. The specific technical solutions are as follows:

[0006] In a first aspect, an embodiment of the present application provides a VR display system, including a head-mounted host device and a slave device;

[0007] The head-mounted host device includes a camera, a micro control unit, and a first radio frequency chip; the slave device includes a second radio frequency chip;

[0008] The micro control unit is used to: obtain a camera signal sent by the camera, and when a frame rate of a vertical synchronization signal in the camera signal is less than a preset threshold, the micro control unit controls to close the radio frequency link between the first radio frequency chip and the second radio frequency chip.

[0009] In a possible implementation, the microcontroller unit is specifically configured to:

[0010] Generate a first control signal using pulse width modulation, and send the first control signal to the first radio frequency chip;

[0011] The first radio frequency chip is specifically configured to stop sending data packets to the second radio frequency chip in response to the first control signal.

[0012] In a possible implementation manner, the micro control unit is further configured to:

[0013] When the frame rate of the vertical synchronization signal in the camera signal is not less than a preset threshold, the micro control unit controls the first RF chip and the second RF chip to communicate based on a first transceiver rate; wherein, in a working state, the transceiver rates of the first RF chip and the second RF chip include the first transceiver rate and the second transceiver rate, and the first transceiver rate is less than the second transceiver rate.

[0014] In a possible implementation, the microcontroller unit is specifically configured to:

[0015] generating a second control signal by using pulse width modulation, and sending the second control signal to the first radio frequency chip;

[0016] The first radio frequency chip is specifically configured to send a data packet to the second radio frequency chip based on the first transceiver rate in response to the second control signal.

[0017] In a possible implementation manner, the micro control unit is further configured to:

[0018] When the vertical synchronization signal in the camera signal is a null signal, the first radio frequency chip and the second radio frequency chip are controlled to communicate at the second transceiver rate for a preset time period.

[0019] In a possible implementation, the slave device further includes an inertial measurement unit, and the microcontroller unit is further configured to:

[0020] Acquire a slave shooting status signal of the camera; the slave shooting status signal is used to indicate whether the camera has captured the position of the slave device;

[0021] When the camera captures the position of the slave device, a first control instruction is generated and sent to the first RF chip; the first RF chip is used to send the first control instruction to the second RF chip; the second RF chip is used to send the first control instruction to the inertial measurement unit; the inertial measurement unit is used to sample based on a first sampling frequency in response to the first control instruction; wherein, in a working state, the sampling frequency of the inertial measurement unit includes the first sampling frequency and the second sampling frequency, and the first sampling frequency is less than the second sampling frequency.

[0022] In a possible implementation manner, the micro control unit is further configured to:

[0023] When the camera fails to capture the position of the slave device, a second control instruction is generated and sent to the first RF chip; the first RF chip is used to send the second control instruction to the second RF chip; the second RF chip is used to send the second control instruction to the inertial measurement unit; the inertial measurement unit is used to sample based on a second sampling frequency in response to the second control instruction.

[0024] In a possible implementation, the slave device further includes an inertial measurement unit.

[0025] The microcontroller unit is also used for:

[0026] When the vertical synchronization signal in the camera signal is a null signal, a third control instruction is generated and the third control instruction is sent to the first RF chip; the first RF chip is used to send the third control instruction to the second RF chip; the second RF chip is used to send the third control instruction to the inertial measurement unit; the inertial measurement unit is used to sample based on a second sampling frequency in response to the third control instruction; wherein, in a working state, the sampling frequency of the inertial measurement unit includes a first sampling frequency and a second sampling frequency, and the first sampling frequency is less than the second sampling frequency.

[0027] In a possible implementation, the microcontroller unit is further configured to obtain an image captured by the camera; determine whether the camera has captured the position of the slave device based on the image and information of the slave device, and generate a slave shooting status signal based on the determination result.

[0028] In a possible implementation, the VR display system further includes a core processing unit;

[0029] The core processing unit is configured to obtain an image captured by the camera; determine whether the camera has captured the position of the slave device based on the image and information about the slave device, and generate a slave shooting status signal based on the determination result;

[0030] The slave shooting status signal is sent to the micro control unit.

[0031] Beneficial effects of the embodiments of the present application:

[0032] In the VR display system provided in the embodiment of the present application, after the microcontroller obtains the camera signal sent by the camera, it determines whether the camera is in a standby state based on the frame rate of the XVS signal. When the frame rate of the XVS signal is less than a preset threshold, the microcontroller controls the closure of the RF link between the first RF chip and the second RF chip. After the first RF chip closes the RF link with the second RF chip, data interaction between the first RF chip and the second RF chip is stopped. In this way, when the camera is in the standby state, data interaction between the first RF chip and the second RF chip is stopped, thereby avoiding resource occupation of the master and slave machines and reducing the power consumption of the master and slave machines.

[0033] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0035] Figure 1 A schematic diagram of the first structure of the VR display system provided in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of the interaction between a head-mounted host device and a slave device of a VR display system provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of the MCU decision process in the VR display system provided in an embodiment of the present application;

[0038] Figure 4 A second structural diagram of the VR display system provided in an embodiment of the present application;

[0039] Figure 5 A flowchart of a VR display system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0041] like Figure 1 As shown, an embodiment of the present application provides a VR display system, including:

[0042] Head-mounted host device 10, slave device 20;

[0043] The head-mounted host device 10 includes a camera 101, a micro control unit 102 and a first radio frequency chip 103; the slave device 20 includes a second radio frequency chip 201;

[0044] The micro control unit 102 is configured to obtain a camera signal sent by the camera 101 . When a frame rate of a vertical synchronization signal in the camera signal is less than a preset threshold, the micro control unit 102 controls to shut down a radio frequency link between the first radio frequency chip 103 and the second radio frequency chip 201 .

[0045] The camera 101 can be a common high-definition camera or a wide-angle camera, which is used to capture the scene within the viewing angle in a short time. When the camera 101 captures an image, it generates an XVS (vertical synchronization signal) signal to indicate the start of a frame of image data.

[0046] Radio frequency technologies can include Wi-Fi (mobile hotspot) routers, Bluetooth, GPS (Global Positioning System), ESB, and other technologies. The ESB (Enhanced Shock Burst) technology protocol is a basic protocol that supports bidirectional packet communication. An Enhanced Shock Burst packet transaction is a packet exchange between two transceivers, one acting as a primary receiver (PRX) and the other as a primary transmitter (PTX). The ESB protocol technology is always initiated by a packet transmission from the PTX. When the PTX receives an acknowledgment packet (ACK packet) from the PRX, the link transmission is completed.

[0047] The first RF chip 103 and the second RF chip 201 are integrated circuits specifically designed to process radio signals. The RF chip is responsible for sending and receiving radio waves, thereby realizing wireless communication functions. In one example, the RF chip in this application can be a RF chip with ESB technology.

[0048] The communication between the head-mounted host device 10 and the slave device 20 is carried out through the first RF chip 103 and the second RF chip 201. The slave device includes an IMU (inertial measurement unit) transmission unit 202. The IMU transmission unit is an inertial measurement device that can measure the three-axis acceleration and angular velocity of an object, and is usually composed of an accelerometer, a gyroscope, and a magnetometer. The communication data transmission between the head-mounted host device 10 and the slave device 20 involves data acquisition, format conversion, packaging and sending. For example, the IMU transmission unit 202 transmits the generated inertial data to the second RF chip 201 through the serial port, and then transmits it to the first RF chip 103 through the second RF chip 201. The first RF chip 103 sends the inertial data of the slave device to the processing unit of the head-mounted host device 10 for processing, thereby completing the data interaction between the head-mounted host device 10 and the slave device 20. In an example, the processing unit of the head-mounted host device 10 is a core processor.

[0049] Exemplarily, the first RF chip 103 and the second RF chip 201 can adopt the ESB protocol transmission mode, because the ESB protocol transmission mode is divided into master-slave modules, that is, including a PTX master end and a PRX slave end, the second RF chip 201 is a PRX slave end, and the first RF chip 103 is a PTX master end. The transmission process of the head-mounted host device 10 and the slave device 20 is: the first RF chip 103 actively sends a data packet to the second RF chip 201, and then the first RF chip 103 sends an event judgment (the first RF chip 103 determines whether the data packet sent by itself to the second RF chip 201 is sent successfully. If it is not sent successfully, it is resent. If it is sent successfully, a message indicating successful sending is generated), and the second RF chip 201 channel monitors to receive the first RF chip 103. The second RF chip 201 receives the data packet sent, and determines the reception event (the second RF chip 201 determines whether it has successfully received the data packet sent by the first RF chip 103. If not, it receives it again. If successful, it generates a message indicating successful reception). The second RF chip 201 calls the data collected by the inertial measurement unit and puts it into a return packet, and sends the return packet to the first RF chip 103. The first RF chip 103 receives the return packet sent by the second RF chip 201. After the first RF chip 103 receives the return packet, the data interaction between the first RF chip 103 and the second RF chip 201 is completed. After the first RF chip 103 receives the return packet, the data collected by the inertial measurement unit in the return packet can be sent to the core processor of the head-mounted host device 10 for processing.

[0050] The core processor is an integrated chip used in the head-mounted host device 10, which is a SoC (System on Chip), and may include a CPU, memory, various interface control modules, and various interconnection buses.

[0051] The core processor is in communication with the first RF chip 103. In one example, the first RF chip 103 and the core processor communicate via SPI (Serial Peripheral Interface). The SPI transmission rate can reach tens of MHz, and a full packet of data can be quickly retrieved within a few microseconds, meeting the requirements of millisecond-level wireless data reporting rates and continuous transmission and reception.

[0052] For example, the core SoC is the control end of the SPI chip select signal, that is, the core SoC is the SPIM (SPI Master, SPI master device), the first RF chip 103 is the SPIS (SPI Slave, SPI slave device), the core SoC is responsible for controlling the initiation and timing of communication, generating a clock signal (SCLK), and selecting the first RF chip 103. The first RF chip 103 responds to the instructions of the master device and passively receives or sends data. Figure 2 As shown, the core SoC of the core processor is SPIM, the core processor and the first RF chip 103 communicate via SPI, the first RF chip 103 and the second RF chip 201 communicate via ESB protocol transmission mode, the second RF chip 201 is a PRX slave end, and the first RF chip 103 is a PTX master end.

[0053] The second RF chip 201 communicates with the IMU transmission unit 202 via UART (Universal Asynchronous Receiver / Transmitter).

[0054] In one example, the head-mounted host device 10 may further include sensors, which may include position sensing, specific data detection information collection, and other sensors. These sensors are devices that can detect and respond to changes in physical quantities and convert them into electrical signals or other forms of signal output. The core processor of the head-mounted host device 10 can process the information detected by the sensors and the data collected by the inertial measurement unit of the slave device to generate corresponding image information for the user.

[0055] The XVS signal is usually used to indicate the end of a frame of image and the beginning of a new frame, ensuring that the frame rate of the image acquisition or display device is synchronized with the camera. After the camera 101 captures an image, it generates an XVS signal.

[0056] After the head-mounted host device 10 and the slave device 20 are started, the camera 101 will send a camera signal to the microcontroller unit 102 according to a preset time period. The camera signal includes at least an XVS signal. After the microcontroller unit 102 obtains the camera signal sent by the camera 101, it can determine the camera's operating status based on the XVS signal. The camera's operating status includes a standby state, a normal operating state, and an undriven state. The standby state indicates that the camera image is still for a period exceeding a threshold time, such as 10 seconds. The normal operating state is when the camera 101 is normally capturing images. The normal operating state is further divided into a first normal operating state and a second normal operating state. The first normal operating state indicates that the camera 101 is normally capturing images and can capture the image of the slave device, that is, the camera captures the position of the slave device. The second normal operating state indicates that the camera 101 is normally capturing images but fails to capture the image of the slave device, that is, the camera fails to capture the position of the slave device. The undriven state indicates that the camera 101 is in an unoperating state after failing to capture images in a dark environment.

[0057] When the camera is in standby mode, the XVS signal frame rate is less than the preset threshold. When the camera is in normal operation, the XVS signal frame rate is not less than the preset threshold. When the camera is not in operation, the XVS signal is a null signal. The preset threshold can be determined based on actual conditions and is not limited here.

[0058] Exemplarily, the preset threshold is 30 fps.

[0059] After the microcontroller unit 102 obtains the camera signal sent by the camera, it determines whether the camera 101 is in a standby state based on the frame rate of the XVS signal. When the frame rate of the XVS signal is less than a preset threshold, the microcontroller unit 102 controls the closure of the RF link between the first RF chip 103 and the second RF chip 201. After the first RF chip 103 closes the RF link with the second RF chip 201, data interaction between the first RF chip 103 and the second RF chip 201 stops. In this way, when the camera is in the standby state, data interaction between the first RF chip 103 and the second RF chip 201 stops, avoiding resource occupation of the master and slave devices and reducing the power consumption of the master and slave devices. In other words, by effectively monitoring the status of the head-mounted host device and intelligently managing transmission, the interaction between the head-mounted host device and the slave device is efficiently operated, and the overall power consumption of the VR display system is reduced.

[0060] In one possible implementation, when the frame rate of the XVS signal is less than a preset threshold, that is, when it is determined that the camera 101 is in a standby state, the micro control unit 102 controls the RF link between the first RF chip 103 and the second RF chip 201 to be turned off, and at the same time controls the inertial measurement unit to stop collecting data, thereby further reducing the power consumption of the slave device 20 and avoiding waste of resources.

[0061] In a possible implementation, the micro control unit 102 is specifically configured to:

[0062] Generate a first control signal by using pulse width modulation, and send the first control signal to the first RF chip 103;

[0063] The first RF chip 103 is specifically configured to stop sending data packets to the second RF chip 201 in response to the first control signal.

[0064] The microcontroller unit 102 includes a pulse width modulation circuit. When the frame rate of the XVS signal is less than a preset threshold, the microcontroller unit 102 generates a first control signal using internal pulse width modulation, and controls the shutdown of the RF link between the first RF chip and the second RF chip based on the first control signal.

[0065] The first control signal may be a low level signal.

[0066] The microcontroller unit MCU not only detects the normal operation of the host peripherals and calls wireless transmission, but also serves as the control end of the SPI chip select signal. After the first RF chip receives the IMU data packet, it needs to inform the processing to restore the initial SPI chip select signal. Therefore, the camera-side XVS signal sends a signal to the microcontroller unit. First, the SPIM connected GPIO is pulled low, and the RF chip GPIO level is pulled high / low to complete the data transmission. Then, the SPIM GPIO is pulled high to restore the potential, so that the next time the camera XVS signal comes, the GPIO level change can be repeatedly controlled.

[0067] In one possible implementation, the first RF chip 103 communicates with the core processor using SPI. After the microcontroller unit 102 obtains the camera signal sent by the camera 101, the microcontroller unit first connects to the enable signal of the core processor through an independent GPIO (general-purpose input / output), indirectly controlling the GPIO of the first RF chip to pull down the level. Then, after controlling the GPIO level of the first RF chip 103 to be pulled up / low to complete data transmission, the microcontroller unit 102 first connects to the enable signal of the core processor through an independent GPIO, indirectly controlling the GPIO of the first RF chip 103 to pull up the recovery potential, so that the next time the camera signal sent by the camera 101 comes, the GPIO level change can be repeatedly controlled.

[0068] like Figure 3 As shown, the microcontroller unit receives the XVS signal from the camera and makes an XVS signal judgment;

[0069] The electrical signal comparator receives the actual signal (the actual XVS signal sent by the camera) and the reference XVS signal. The electrical signal comparator determines whether the actual signal is normal or abnormal. If the XVS signal is normal, that is, the frame rate of the XVS signal is not less than a preset threshold, the microcontroller generates a fixed-frequency level signal and outputs it to the first RF chip 103, enabling normal communication between the first RF chip 103 and the second RF chip 201. If the XVS signal is abnormal, that is, the frame rate of the XVS signal is less than the preset threshold, the microcontroller controls the shutdown of the RF link between the first RF chip and the second RF chip, enabling interruption.

[0070] Figure 3 It is shown in the figure that the microcontroller unit MCU not only detects the normal operation of the host peripherals and calls wireless transmission, but also serves as the control end of the SPI chip select signal, and the core processor provides the SPI selection control signal to the core processor.

[0071] The microcontroller unit may also be equipped with other peripherals to intelligently determine whether to perform master-slave data transmission.

[0072] In a possible implementation, the micro control unit 102 is further configured to:

[0073] When the frame rate of the vertical synchronization signal in the camera signal is not less than a preset threshold, the micro control unit 102 controls the first RF chip 103 and the second RF chip 201 to communicate based on a first transceiver rate; wherein, in a working state, the transceiver rates of the first RF chip 103 and the second RF chip 201 include the first transceiver rate and the second transceiver rate, and the first transceiver rate is less than the second transceiver rate.

[0074] The 2.4G private protocol ESB supports a maximum transmission of 2 Mbps. The wireless rate between the first RF chip 103 and the second RF chip 201 can be achieved by setting two rates, the first transceiver rate is 1 Mbps, the second transceiver rate is 2 Mbps, and the default transceiver rate is 1 Mbps.

[0075] In the working state, that is, when the first RF chip 103 and the second RF chip 201 are communicating data, data communication can be performed based on two transceiver rates: a first transceiver rate and a second transceiver rate, wherein the first transceiver rate is lower than the second transceiver rate.

[0076] For example, the first transceiver rate is 500 Hz and the second transceiver rate is 800 Hz. When the camera is operating normally, that is, when the frame rate of the vertical synchronization signal is not less than a preset threshold, the first RF chip communicates with the second RF chip based on the first transceiver rate.

[0077] In a possible implementation, the micro control unit 102 is specifically configured to:

[0078] Generate a second control signal by using pulse width modulation, and send the second control signal to the first RF chip 103;

[0079] The first RF chip 103 is specifically configured to send a data packet to the second RF chip 201 based on the first transceiver rate in response to the second control signal.

[0080] The microcontroller unit 102 includes a pulse width modulation circuit. When the frame rate of the XVS signal is not less than a preset threshold, the microcontroller unit 102 generates a second control signal using internal pulse width modulation. The second control signal can be a square wave signal. The duty cycle of the square wave signal can be determined based on actual conditions. For example, the duty cycle of the square wave signal can be 30%, 40%, 50%, 60%, etc., which is not limited here.

[0081] After receiving the second control signal, the first RF chip 103 sends a data packet to the second RF chip 201 based on the first transceiver rate in response to the second control signal, so that the first transceiver rate and the second RF chip 201 communicate based on the first transceiver rate.

[0082] In a possible implementation, the micro control unit 102 is further configured to:

[0083] When the vertical synchronization signal in the camera signal is a null signal, the first RF chip 103 and the second RF chip 201 are controlled to communicate at the second transceiver rate for a preset time period.

[0084] When the head-mounted host device and the slave device are in a dark environment, the camera 101 fails to capture light without light. At this time, the XVS signal of the camera 101 is in an undriven state, and the camera XVS signal has no output, that is, the XVS signal is a null signal. At this time, the XVS signal is in a high-impedance state. For example, the VIL of the XVS signal is 0.5V and the VIH is 3V. If the signal voltage is continuously in the middle range (such as 1.5V±0.5V), the XVS signal is determined to be a null signal. The first RF chip 103 and the second RF chip 201 are controlled to communicate based on a preset time period based on the high transceiver rate. The preset time period is determined based on the actual situation, for example, the preset time period is 10s, so that the second RF chip 201 sends the data collected by the IMU to the head-mounted host device, so that the head-mounted host device generates corresponding image data based on the data of the slave device. In addition, the preset communication time period is used to inform the user that the scene camera is not turned on, which can prevent the entire VR display system from operating in high-speed mode for a long time and avoid increasing power consumption. While meeting the large-scale data transmission of IMU, the core processor load of the head-mounted host device and the load drive of the slave device are reduced. The overall power consumption of the VR display system is greatly reduced, making the VR display system efficient and energy-saving, and extending the working life of the master and slave devices.

[0085] The first RF chip 103 and the second RF chip 201 can be set up with a wireless retransmission mechanism. When the master end (the first RF chip 103) does not receive ACK (Acknowledgement) data, the slave end (the second RF chip 201) will retransmit the same data packet until the data content in the ACK is received. When the first RF chip 103 and the second RF chip 201 communicate at the maximum transmission rate, the number of data frames per unit time of the slave device can be more effectively increased. When the camera is not operating normally, the head-mounted host device can call the data for supplementary adjustments, thereby improving the overall data validity and real-time performance, and also ensuring the efficient operation of the entire transmission system.

[0086] In a possible implementation, the microcontroller unit is specifically configured to:

[0087] When the vertical synchronization signal in the camera signal is a null signal, generating a third control signal by pulse width modulation, and sending the third control signal to the first RF chip 103;

[0088] The first RF chip 103 is specifically configured to send a data packet to the second RF chip 201 based on the second transceiver rate in response to the third control signal.

[0089] The micro control unit 102 includes a pulse width modulation circuit. When the XVS signal is a null signal, the micro control unit 102 generates a third control signal using internal pulse width modulation. The third control signal may be a high level signal.

[0090] In a possible implementation, the slave device further includes an inertial measurement unit, and the micro control unit 102 is further configured to:

[0091] Acquire a slave shooting status signal of the camera; the slave shooting status signal is used to indicate whether the camera has captured the position of the slave device;

[0092] When the camera captures the position of the slave device, a first control instruction is generated and sent to the first RF chip; the first RF chip is used to send the first control instruction to the second RF chip; the second RF chip is used to send the first control instruction to the inertial measurement unit; the inertial measurement unit is used to sample based on a first sampling frequency in response to the first control instruction; wherein, in a working state, the sampling frequency of the inertial measurement unit includes the first sampling frequency and the second sampling frequency, and the first sampling frequency is less than the second sampling frequency.

[0093] The camera's slave shooting status signal can be sent by the camera 101 to the microcontroller unit 102, or generated by the microcontroller unit 102 based on the camera's shooting information, or sent by the core processor of the head-mounted host device 10 to the microcontroller unit 102. The specific method can be determined based on actual conditions and is described in the following embodiments for clarity.

[0094] After the head-mounted host device 10 and the slave device 20 are started, the camera of the head-mounted host device 10 is used to capture images. The camera may or may not capture the slave device. The slave capture state signal indicates that the camera has captured the position of the slave device.

[0095] If the camera captures the slave device, it means that the head-mounted host device 10 can obtain image information of the slave device 20 from the camera, and the head-mounted host device 10 and the slave device 20 are both operating normally. At this time, the inertial measurement unit is required to operate based on the normal operating frequency. The microcontroller unit 102 generates a first control instruction and sends the first control instruction to the first RF chip. The first RF chip sends the first control instruction to the second RF chip. The second RF chip sends the first control instruction to the inertial measurement unit. The inertial measurement unit samples based on the first sampling frequency in response to the first control instruction.

[0096] The first sampling frequency may be 500 Hz, and the second sampling frequency may be 800 Hz.

[0097] Exemplarily, when the frame rate of the XVS signal is not less than 20 fps and the camera captures the position of the slave device, the first RF chip communicates with the second RF chip based on a transceiver rate of 1 Mbps, and the inertial measurement unit samples based on a sampling frequency of 500 Hz.

[0098] In a possible implementation, when the camera 101 fails to capture the position of the slave device 20, a second control instruction is generated and the second control instruction is sent to the first RF chip 103; the first RF chip 103 is used to send the second control instruction to the second RF chip 201; the second RF chip 201 is used to send the second control instruction to the inertial measurement unit; and the inertial measurement unit is used to sample based on a second sampling frequency in response to the second control instruction.

[0099] If the camera does not capture the slave device, it means that the head-mounted host device 10 has not obtained the image information of the slave device 20 from the camera, that is, the head-mounted host device 10 cannot obtain the image information of the slave device 20 based on its own device. At this time, the inertial measurement unit needs to operate based on a relatively high sampling frequency to collect more data for the head-mounted host device 10, so that the head-mounted host device 10 can obtain the compensated motion trajectory data sent by the slave device, and the head-mounted host device compensates the motion trajectory of the slave device based on the compensated motion trajectory data. The head-mounted host device 10 can obtain more motion trajectory data of the slave device to make up for the problem of missing data caused by the camera's inability to capture, ensuring the normal operation of the system.

[0100] Exemplarily, when the frame rate of the XVS signal is not less than 20fps and the camera does not capture the position of the slave device, the first RF chip and the second RF chip communicate based on a transceiver rate of 1 Mbps, and the inertial measurement unit samples based on a sampling frequency of 800 Hz.

[0101] In a possible implementation, the slave device further includes an inertial measurement unit.

[0102] The microcontroller unit is also used for:

[0103] When the vertical synchronization signal in the camera signal is a null signal, a third control instruction is generated and the third control instruction is sent to the first RF chip; the first RF chip is used to send the third control instruction to the second RF chip; the second RF chip is used to send the third control instruction to the inertial measurement unit; the inertial measurement unit is used to sample based on a second sampling frequency in response to the third control instruction; wherein, in a working state, the sampling frequency of the inertial measurement unit includes a first sampling frequency and a second sampling frequency, and the first sampling frequency is less than the second sampling frequency.

[0104] When the vertical synchronization signal in the camera signal is an empty signal, the camera 101 of the head-mounted host device fails to capture light. At this time, the XVS signal of the camera 101 is in an undriven state, the camera XVS signal has no output, and the inertial measurement unit samples based on the second sampling frequency.

[0105] In a possible implementation, the microcontroller unit is further configured to obtain an image captured by the camera; determine whether the camera has captured the position of the slave device based on the image and information of the slave device, and generate a slave shooting status signal based on the determination result.

[0106] After capturing an image, the camera sends it to the microcontroller. The microcontroller may pre-store information about the slave device and compare the information in the image with the slave device information to determine whether the camera has captured the position of the slave device. If the camera has captured the position of the slave device, a slave capture status signal is generated indicating that the position of the slave device has been captured. If the camera has not captured the position of the slave device, a slave capture status signal is generated indicating that the position of the slave device has not been captured.

[0107] The method for the microcontroller unit to determine whether the camera has captured the position of the slave device based on the image and the information of the slave device can refer to the determination method in the relevant technology and will not be repeated here.

[0108] In a possible implementation, the VR display system further includes a core processing unit;

[0109] The core processing unit is configured to obtain an image captured by the camera; determine whether the camera has captured the position of the slave device based on the image and information about the slave device, and generate a slave shooting status signal based on the determination result;

[0110] The slave shooting status signal is sent to the micro control unit.

[0111] After capturing an image, the camera sends it to the core processing unit. The core processing unit may pre-store information about the slave device and compare the information in the image with the slave device information to determine whether the camera has captured the slave device's position. If the camera has captured the slave device's position, it generates a slave capture status signal indicating that the slave device's position has been captured. If the camera has not captured the slave device's position, it generates a slave capture status signal indicating that the slave device's position has not been captured. After generating the slave capture status signal, the core processing unit transmits it to the microcontroller unit.

[0112] Among them, the method for the core processing unit to determine whether the camera captures the position of the slave device based on the image and the information of the slave device can refer to the determination method in the relevant technology, which is not repeated here.

[0113] In a possible implementation, the camera determines whether it has captured the position of the slave device based on its own captured image information, and generates a slave shooting status signal based on the determination result; and sends the slave shooting status signal to the micro control unit.

[0114] After capturing an image, the camera itself determines whether it has captured the slave device's position. The camera may pre-store information about the slave device and compare the information in the image with the slave device's information to determine whether the camera has captured the slave device's position. If the slave device's position has been captured, a slave capture status signal is generated to indicate that the position has been captured. If the slave device's position has not been captured, a slave capture status signal is generated to indicate that the position has not been captured. After generating the slave capture status signal, the camera transmits it to the microcontroller unit.

[0115] Among them, the method of judging whether the position of the slave device is captured by the self-shot image information can refer to the judgment method in the relevant technology, which will not be repeated here.

[0116] like Figure 4 As shown, in a possible implementation, the head-mounted host device includes: a display screen, a sensor, a core processor, a camera, a first radio frequency chip, and a micro control unit.

[0117] After the head-mounted host device is turned on, the camera is started by default. The camera immediately captures the current surrounding environment and sends the captured image to the core processor. The core processor determines whether there is continuous frame visible light image input. If so, the core processor detects whether the slave device is within the range of the head-mounted host device. If no slave device is detected, the core processor generates feedback information and feeds the feedback information back to the display screen. Based on the feedback information, the user is prompted whether the slave device is turned on or whether it is worn.

[0118] The display screen can be a small-size LCD or OLED module, which is used to display and output image signals and show the specific scenes generated by the virtual system in real time.

[0119] After the camera of the head-mounted host device captures the slave device, it uses the captured image information of the slave device as input. Generally, the position data of the current slave device is inferred through image feature points or pixels. At the same time, the IMU transmission unit of the slave device reports the data to the second RF chip, and the second RF chip transmits it to the first RF chip. The first RF chip sends the IMU data to the core processor, and the IMU data is transmitted back to the head-mounted host device to complete the master-slave communication. The core processor of the head-mounted host device performs algorithm fusion calculations based on the IMU data and the camera trajectory data detected by the head-mounted host device's own sensors, converts it into three-dimensional coordinate system information data for VR display, and renders a simulated image of the slave device in the virtual display, so that the user's device interaction delay is reduced in specific sports scenarios such as 3D games.

[0120] The camera sends an XVS signal to the microcontroller, which controls the first RF chip based on the PWM signal, enabling the first RF chip to communicate at different transmit and receive rates. The microcontroller MCU is the control end of the SPI chip select signal, and the core processor provides the SPI selection control signal to the core processor.

[0121] In a possible implementation, the micro control unit determines whether the vertical synchronization signal has a voltage. If not, the GPIO state is pulled low, and the radio frequency link between the first radio frequency chip and the second radio frequency chip is controlled to be closed.

[0122] If there is voltage, it is determined whether the voltage is normal. If it is normal, the GPIO state is in a periodic high and low state, and the first radio frequency chip communicates with the second radio frequency chip based on the first transceiver rate, so that the slave device operates normally.

[0123] If the voltage is abnormal, the GPIO state is pulled high, so that the first radio frequency chip and the second radio frequency chip communicate based on the second transceiver rate.

[0124] Based on the above embodiments, Figure 5 As shown, the VR display system flow chart:

[0125] The head-mounted host device and the slave device are powered on;

[0126] After the head-mounted host device is turned on, the camera is started by default. The camera immediately captures the current surrounding environment and generates an XVS signal based on the captured image. If the image captured by the camera is obtained by a non-infrared camera and the information of the slave device is captured, it is determined that the camera's working state is in the first normal working state, and the frame rate of the generated XVS signal is not less than the preset threshold; for example, the frame rate of the generated XVS signal is 30fps.

[0127] If the image captured by the camera is obtained by a non-infrared camera and no information of the slave device is captured, the camera is determined to be in the second normal working state, the frame rate of the generated XVS signal is not less than the preset threshold, and the microcontroller unit controls the sampling frequency of the inertial measurement unit to sample based on the second sampling frequency. Exemplarily, the microcontroller unit calls a program to send a "not captured" status code to the first RF chip, the first RF chip sends the "not captured" status code to the second RF chip; the second RF chip sends the "not captured" status code to the inertial measurement unit; the inertial measurement unit is used to sample based on the second sampling frequency in response to the "not captured" status code.

[0128] If the image captured by the camera is obtained by an infrared camera, the generated XVS signal is in a high-impedance state, indicating that the camera is in an undriven state;

[0129] If the camera image remains stationary for longer than a threshold time, it is determined that the camera is in standby mode, and the frame rate of the generated XVS signal is 20 fps.

[0130] The camera sends the XVS signal to the micro control unit and determines whether the XVS signal is an empty signal. If so, it means that the head-mounted host device and the slave device are in a dark environment, and the camera 101 has no light to capture. At this time, the XVS signal of the camera 101 is in an undriven state, and the camera XVS signal has no output, that is, the XVS signal is a high-impedance signal. The micro control unit controls the first RF chip and the second RF chip to operate in high-speed mode, that is, the micro control unit 102 uses PWM to generate a high-level waveform to control the first RF chip 103, so that the first RF chip 103 and the second RF chip 201 are in high-speed mode. In high-speed mode, the first RF chip 103 and the second RF chip 201 communicate based on the preset time length based on the second transceiver rate, and the inertial measurement unit samples based on the second sampling frequency.

[0131] If the XVS signal is not a null signal, the microcontroller unit determines whether the frame rate of the XVS signal is less than 30 fps. If so, the microcontroller unit determines that the camera 101 is in a standby state and controls the RF link between the first RF chip and the second RF chip to be shut down. If it is not less than 30 fps, the microcontroller unit determines whether the camera has captured the position of the slave device 20. If so, the microcontroller unit controls the first RF chip and the second RF chip to communicate normally. In response to the control instruction of the microcontroller unit, the first RF chip communicates with the second RF chip, and the second RF chip of the slave device sends the IMU data to the SPI buffer of the first RF chip. The microcontroller unit determines whether data transmission and reception are normal. If so, the first RF chip sends the received data to the core processor of the head-mounted host device. If transmission and reception are not normal, the microcontroller unit returns the control instruction of the microcontroller unit, and the first RF chip communicates with the second RF chip to continue execution. The first RF chip 103 and the second RF chip 201 communicate at a first transmission and reception rate for a preset duration, and the inertial measurement unit samples at a first sampling frequency.

[0132] If the camera fails to capture the position of the slave device 20, that is, the XVS signal frame rate of the camera is normally 30fps, but the image of the slave device is lost, that is, it is determined that the camera has not captured the position of the slave device 20, the micro control unit controls the first RF chip and the second RF chip to operate in a data compensation mode, that is, the micro control unit 102 sends a non-capture status signal to the first RF chip 103, the first RF chip 103 sends the non-capture status signal to the second RF chip 201, and the second RF chip 201 sends the non-capture status signal to the inertial measurement unit. In response to the non-capture status signal, the first RF chip 103 and the second RF chip 201 communicate based on the first transceiver rate for a preset time length, and the inertial measurement unit samples based on the second sampling frequency.

[0133] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0134] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0135] Each embodiment in this specification is described in a related manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0136] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A VR display system, characterized in that: include: Head-mounted host device, slave device; The head-mounted host device includes a camera, a micro control unit, and a first radio frequency chip; the slave device includes a second radio frequency chip; The micro control unit is used to: obtain a camera signal sent by the camera, and when a frame rate of a vertical synchronization signal in the camera signal is less than a preset threshold, the micro control unit controls to close the radio frequency link between the first radio frequency chip and the second radio frequency chip.

2. The system according to claim 1, wherein: The micro control unit is specifically used for: Generate a first control signal using pulse width modulation, and send the first control signal to the first radio frequency chip; The first radio frequency chip is specifically configured to stop sending data packets to the second radio frequency chip in response to the first control signal.

3. The system according to claim 1, wherein: The microcontroller unit is also used for: When the frame rate of the vertical synchronization signal in the camera signal is not less than a preset threshold, the micro control unit controls the first RF chip and the second RF chip to communicate based on a first transceiver rate; wherein, in a working state, the transceiver rates of the first RF chip and the second RF chip include the first transceiver rate and the second transceiver rate, and the first transceiver rate is less than the second transceiver rate.

4. The system according to claim 3, characterized in that The micro control unit is specifically used for: generating a second control signal by using pulse width modulation, and sending the second control signal to the first radio frequency chip; The first radio frequency chip is specifically configured to send a data packet to the second radio frequency chip based on the first transceiver rate in response to the second control signal.

5. The system according to claim 3, wherein: The microcontroller unit is also used for: When the vertical synchronization signal in the camera signal is a null signal, the first radio frequency chip and the second radio frequency chip are controlled to communicate for a preset time period based on the second transceiver rate.

6. The system according to claim 3, wherein: The slave device further includes an inertial measurement unit, and the micro control unit is further configured to: Acquire a slave shooting status signal of the camera; the slave shooting status signal is used to indicate whether the camera has captured the position of the slave device; When the camera captures the position of the slave device, generating a first control instruction and sending the first control instruction to the first radio frequency chip; The first RF chip is used to send the first control instruction to the second RF chip; the second RF chip is used to send the first control instruction to the inertial measurement unit; the inertial measurement unit is used to sample based on a first sampling frequency in response to the first control instruction; wherein, in a working state, the sampling frequency of the inertial measurement unit includes the first sampling frequency and the second sampling frequency, and the first sampling frequency is less than the second sampling frequency.

7. The system according to claim 6, characterized in that The microcontroller unit is also used for: If the camera fails to capture the position of the slave device, generating a second control instruction and sending the second control instruction to the first radio frequency chip; The first RF chip is used to send the second control instruction to the second RF chip; the second RF chip is used to send the second control instruction to the inertial measurement unit; and the inertial measurement unit is used to sample based on a second sampling frequency in response to the second control instruction.

8. The system according to claim 5, wherein: The slave device further includes an inertial measurement unit, The microcontroller unit is also used for: When the vertical synchronization signal in the camera signal is a null signal, generating a third control instruction, and sending the third control instruction to the first radio frequency chip; The first RF chip is used to send the third control instruction to the second RF chip; the second RF chip is used to send the third control instruction to the inertial measurement unit; the inertial measurement unit is used to sample based on a second sampling frequency in response to the third control instruction; wherein, in a working state, the sampling frequency of the inertial measurement unit includes a first sampling frequency and a second sampling frequency, and the first sampling frequency is less than the second sampling frequency.

9. The system according to claim 6, wherein: The micro control unit is further configured to obtain an image captured by the camera; determine whether the camera has captured the position of the slave device based on the image and information about the slave device, and generate a slave shooting status signal based on the determination result.

10. The system according to claim 6, wherein: The VR display system also includes a core processing unit; The core processing unit is configured to obtain an image captured by the camera; determine whether the camera has captured the position of the slave device based on the image and information about the slave device, and generate a slave shooting status signal based on the determination result; The slave shooting status signal is sent to the micro control unit.

Images