Spread-spectrum video transport integration with virtual reality headset

Abstract

The video stream is encoded using spread spectrum video transmission and sent as an analog signal to the VR headset's display. A decoder integrated with the source driver decodes the analog signal and drives the display. The analog signal is sent wirelessly to the display, where it is received, converted to a wired format, decoded, and displayed. Wireless SSVT analog signals are received at the headset processor and forwarded to the VR headset for reception, conversion, decoding, and display. The wireless SSVT analog signal is received at the processor, converted to a wired format, and sent wirelessly to the display. The receiver receives the signal, converts it to a wired format, decodes, and displays it. The video stream is stored in persistent memory in the headset processor using SSVT encoding. The decoder integrated with the display's source driver is implemented directly on the display panel's glass.

Description

Technical Field

[0001] [Cross-reference to related applications] []

[0002] This application claims the rights to U.S. Provisional Patent Application No. 63 / 245,650 (HYFYP010P), filed September 17, 2021, entitled “Spread Spectrum Video Transport and AR / VR Headset Processor”; U.S. Provisional Patent Application No. 63 / 299,137 (Case No.: HYFYP011P), filed January 13, 2022; U.S. Provisional Patent Application No. 63 / 280,017 (Case No.: HYFYP009P2), filed November 16, 2021; U.S. Provisional Patent Application No. 63,317,336 (Case No.: HYFYP014P), filed March 7, 2022; and U.S. Provisional Patent Application No. 63,317,336 (Case No.: HYFYP014P), filed May 26, 2022. The priority of 63 / 346,064 (Case No.: HYFYP014P2) is incorporated herein by reference in its entirety.

[0003] This application also incorporates by reference U.S. Application No. 15 / 925,123 (file number: HYFYP001) filed March 19, 2018; U.S. Application No. 16 / 494,901 (file number: HYFYP002) filed September 17, 2019; U.S. Application No. 17 / 879,499 (file number: HYFYP003) filed August 2, 2022; U.S. Application No. 17 / 686,790 (file number: HYFYP004AX1) filed March 4, 2022; U.S. Application No. 17 / 887,849 (file number: HYFYP006) filed August 15, 2022; and U.S. Application No. 28, 2022. U.S. Application No. 17 / 851,821 (Case No.: HYFYP007), U.S. Application No. 63 / 398,460 (Case No.: HYFYP008P) filed on August 16, 2022, U.S. Application No. 17 / 900,570 (Case No.: HYFYP009) filed on August 31, 2022, and U.S. Application No. 63 / 391,226 (Case No.: HYFYP013P3) filed on July 21, 2022.

[0004] This invention generally relates to virtual reality head-mounted devices. More specifically, this invention relates to the transmission of video signals within, to, and from a head-mounted device. Prior Technology

[0005] Image sensors, display panels, and video processors are constantly racing to achieve larger formats, greater color depth, higher playback rates, and higher resolutions. Local web video transmission includes performance scaling bottlenecks that limit throughput and reduce performance, while consuming more cost and power. Eliminating these bottlenecks can provide advantages.

[0006] To date, data has been transmitted within VR headsets using digital video signals, both to and from the headset. This digital information is then converted in real-time to analog pixel information using a digital-to-analog converter at the display's source driver. Transmitting digital video signals requires compression, meaning higher power consumption (generating additional heat), more EMI transmission, greater latency, and difficulty in providing the required color depth, high frame rates, and high resolution. Latency—the time required for all the computations needed to perform digital transmission—is a particularly critical issue in VR systems, as any user-perceptible delay can cause nausea and render the system unusable. Furthermore, the digital-to-analog conversion at the source driver requires more space and cost.

[0007] What is needed is a VR headset that uses improved video transmission technology to solve the above problems. Summary of the Invention

[0008] To achieve the above objectives, and in accordance with the present invention, a virtual reality (VR) headset using an improved video transmission technology is disclosed.

[0009] In one embodiment, the video stream at the VR headset processor is encoded using Spread Spectrum Video Transmission (SSVT) technology and sent as an SSVT analog signal to one or more displays of the VR headset, wherein a decoder integrated with the source driver decodes the SSVT analog signal and drives the displays using the raw video stream. Multiple displays can be driven in the same manner.

[0010] In the second embodiment, after the video stream is encoded at the head-mounted device processor, the SSVT analog signal is wirelessly transmitted to one or more displays on the mask. The SSVT analog signal is received by a wireless receiver at the display, converted back to wired format, and then decoded and displayed.

[0011] In the third embodiment, the head-mounted device processor receives a wireless SSVT analog signal and then forwards it to the VR mask for reception, conversion, decoding, and display.

[0012] In the fourth embodiment, the wireless SSVT analog signal is received at the head-mounted device processor, converted back to wired format, and wirelessly transmitted to one or more displays on the mask, where it is received at a wireless receiver, converted back to wired format, and then decoded and displayed.

[0013] In the fifth embodiment, the video stream is stored in persistent storage on the head-mounted device processor using SSVT encoding. The stored analog data can then be read from the persistent storage, decoded, and then sent from the storage as the raw video stream.

[0014] In the sixth embodiment, the decoder integrated with the source driver of the VR mask's display is directly implemented on the glass of the display panel. Part or all of the decoder integrated with the source driver can be implemented on the TFT display substrate.

[0015] This invention applies encoding techniques to uncompressed video samples; the resulting uncompressed video transmission enables advanced virtual reality displays. Advantages include: negligible latency (partly due to the elimination of video signal compression); low display chipset power consumption (less heat, longer battery life, lighter, cheaper, and more robust cables); a wider field of view; greater color depth; high frame rates and resolution; improved noise immunity; readily available EMI emission compliance; longer signal range; greater video throughput; and the advantages of SWaP-C (size, weight, power, and cost). This invention is particularly applicable to displays such as LCD and OLED panels used in VR headsets. The low power consumption advantage is especially important for "tetherless" VR systems that rely on the headset's own battery for power, rather than on cables connected to the headset. Regarding the video throughput advantage, a wider field of view (typically expressed in degrees) provides a more immersive experience, but this wider field of view requires more video information; therefore, higher throughput also enables a larger field of view. Simple Explanation of the Diagram

[0016] The present invention and its further advantages can be best understood by referring to the following description taken in conjunction with the accompanying drawings, wherein: Figure 1 illustrates a virtual reality (VR) headset that embodies various aspects of the present invention. Figure 2 illustrates an embodiment of the GPU direct display platform. Figure 3 illustrates a system-on-module (SoM) embodiment of GPU direct display. Figure 4 illustrates a fully integrated GPU direct display embodiment. Figure 5 is a block diagram of the SSVT transmitter. Figure 6 shows the display panel that transmits the SSVT signal to the VR mask. Figure 7 shows the display source driver. Figure 8 shows a more detailed view of the SSVT receiver with the source driver. Figure 9 illustrates an alternative embodiment of an array for implementing a source driver. Figure 10 is a block diagram of one of the decoders in Figure 9. Figure 11 is a block diagram of the collector in Figure 9 and shows more details of the hierarchical library in Figure 8. Figure 12 is a block diagram illustrating the wireless implementation of the transmission medium. Figure 13A is a schematic diagram showing the transmitter. Figure 13B is a schematic diagram showing the receiver. Figure 14 shows a simple example illustrating how a signal sample (in this case, an analog value) is encoded within the encoder and then transmitted via an electromagnetic path. Figure 15 illustrates this novel encoding technique applicable to signal samples that are digital values. Figure 16 illustrates the decoding of the analog input potential level encoded using the encoder of Figure 14. Figure 17A illustrates the use of an analog encoder and a corresponding analog decoder. Figure 17B illustrates the use of a digital encoder and a corresponding analog decoder. Figure 17C illustrates the use of a digital decoder that decodes encoded analog signals that arrive via an electromagnetic path on a transmission medium. Figure 18 shows the analog (similar to an ideal oscilloscope trace) SSVT waveform transmitted via an electromagnetic path after being output from the analog encoder (or after digital encoding and then after DAC conversion). Implementation

[0017] It can be recognized that the digitization of the video signal used for a virtual reality (VR) mask can be generated at the system's signal source (e.g., at the GPU in the headset's processor); the digital signal is then transmitted to the display in the mask, where it is returned to analog and loaded onto the display. Alternatively, the system's video content may initially be digital. Therefore, the sole purpose of this digital signal is data transmission to the mask's display. Thus, it is advantageous to avoid digitization or the digital signal altogether and transmit analog data directly from the video source to the display. This can be accomplished using SSVT encoding, which decodes the precise analog voltage again in the display driver. Analog data has high precision, therefore high bit depth is not required. This means the sampling rate is at least ten times lower than in the case of digital data transmission, leaving greater bandwidth for expansion.

[0018] Therefore, instead of transmitting the digital signal from the video source all the way to the location where the analog signal needs to be generated, we transmit the analog signal to the display at a much lower sampling rate than is typically used with digitization. This means that instead of sending billions of bits per second in several lines, we only need a few megasamples per second in the case of analog signals, thus reducing the bandwidth of the channels that must be used. Furthermore, with prior art digital transmission, each bit would occupy approximately 1.25 cm, while transmitting analog data would result in a 10-fold increase in available space, which translates to additional available bandwidth. More precisely, the SSVT technology described herein achieves more than 10 times the video transmission capacity for any given set of conductors (e.g., wires or cables) and transmission clock rate. Therefore, all other things being equal, converting the digital video signal to SSVT at the source (or other location) can achieve more than 10 times the video information transmitted to the VR headset or mask itself. This tenfold increase in video information can be allocated to any desired improvement in headset performance: resolution, color bit depth, refresh rate, field of view, etc.

[0019] Furthermore, the bits in the digital data must be well defined. This means that digital transmission is quite sensitive to errors and noise, and needs to be able to detect highs and lows very accurately. However, using the proposed analog transmission, its sensitivity is much lower. This means that the cable quality does not need to be high. Overview of Virtual Reality Headsets

[0020] Figure 1 illustrates a virtual reality (VR) head-mounted device 10 embodying various aspects of the present invention. As discussed in more detail below, the head-mounted device 10 can be used as a VR head-mounted device (in which a user views computer-generated images). Therefore, the head-mounted device can be referred to as a VR head-mounted device or simply as a head-mounted device.

[0021] As is known in the art, VR headsets are typically worn on a user's head and include a mask 20 covering the user's eyes and a processor 60 typically integrated with or mounted on the back of the user's head. The mask 20 has a left display 32 and a right display 34 for displaying virtual reality or augmented reality images to the user. Once the left and right displays receive the images the user wants to see, these images can be displayed to the user using various techniques. In a simple technique, the left display 32 and right display 34 are placed in front of the user's eyes. In another technique, often referred to as a head-up display (HUD), displays 32 and 34 are not directly viewed by the user; instead, their images are projected and reflected onto glass or other surfaces within the mask in front of the user's eyes.

[0022] The processor 60 includes a core AI / ML module 62 (including a processor for executing artificial intelligence or machine learning applications, and other suitable processors, programs, memory, etc.), a GPU 64, SSVT memory 66, and any suitable interface for connecting to the outside world, such as an RF access point 72 for wireless communication (using digital or SSVT signals) with a network, the Internet, other computers, etc. A USB port 74 may also be provided for communication with another computer. The processor 60 can be mounted on the user's head and communicate with the mask 20 via wires, cables, or wirelessly. Alternatively, the processor 60 can be mounted anywhere else on the user's body (such as in a backpack or belt) or located away from the user (such as in a nearby computer, vehicle, building, etc.) and communicate wirelessly with the mask 20.

[0023] During operation, the head-mounted device 10 provides numerous advantages (such as less heat dissipation, less power consumption, greater immunity to interference, less EMI emission, negligible latency, and better image quality) by using a novel spread spectrum video transmission (SSVT) technology to transmit video signals from the processor to the mask and wirelessly transmit video signals between the processor and another computer.

[0024] As shown in the figure, the SSVT transmitter 82 within the processor transmits the SSVT signal 92 to each display 32 and 34 using a wired or wireless connection. A technique for inputting digital video signals, transmitting SSVT signals to displays, and integrating the source driver of the display with the decoding of the SSVT signal is described by reference in U.S. Patent Application No. 17 / 900,570 (Document No.: HYFYP009), which is incorporated herein by reference above.

[0025] A VR headset may consist of only a single display, in which case the SSVT transmitter 92 sends a single SSVT signal 92 to that single display. In the case of multiple displays (typically two displays), there may be two (or more) SSVT transmitters 82, each receiving a video stream from the GPU 64 (or from a VR bridge, video board, combined SoC / TCON / GPU, video distributor, etc., depending on the specific VR headset implementation), and each sending an SSVT signal 92 to each of the displays 32 and 34. Typically, the video stream sent to each SSVT transmitter will be the same video stream so that the same image is displayed in front of each eye, although the video stream sent to each SSVT transmitter may differ depending on the implementation.

[0026] In an alternative embodiment using only a single SSVT transmitter 82, the input will be a single video stream, and the output from each encoder (i.e., each EM signal) will be split or copied and transmitted to each of the two display panels. Those skilled in the art will find that splitting or copying the signal to send the same synchronization signal to both display panels is straightforward. In this embodiment, each panel will display the same image according to the input video stream.

[0027] As described above, any SSVT signal shown in Figure 1 can be transmitted wirelessly. For example, SSVT signal 92, SSVT signal 94, or even any SSVT signal transmitted via interface 72 or 74 can be a wireless SSVT signal. A technique for implementing a wireless SSVT signal is described in, by reference, U.S. Patent Application No. 63 / 299,137 (Document No.: HYFYP011P), which is incorporated herein by reference, and is described below. For example, if an SSVT wireless signal is received at RF access point 72, it can be directly and wirelessly relayed 99 to each display 32 and 34 to display its encoded video signal, as described herein. Alternatively, the wireless SSVT signal received at point 72 can be received by wireless receiver 206, converted into (P) wired SSVT EM signals as described above, and then these (P) SSVT EM signals are input to wireless transmitter 202 for wireless transmission 99 to displays 32 and 34. Alternatively, (P) wired SSVT EM signals can be transmitted to both displays via wired or fiber optic cables.

[0028] Furthermore, processor 60 may also include SSVT memory 66, which stores video or other data in a technique represented by SSVT. Techniques for implementing SSVT memory 66 are described by reference to U.S. Patent Application No. 17 / 887,849 (Case No.: HYFYP006), which is incorporated herein by reference.

[0029] Furthermore, the integrated decoder with source driver described below can be fully or partially implemented directly on the glass of display 32 or 34 or 32 and 34, as described by reference to the aforementioned U.S. Patent Application No. 63 / 346064 (Case No.: HYFYP014P2). SSVT Transmitter Integration Implementation

[0030] The following are various embodiments describing different levels of integration between the SSVT transmitter 82 and the GPU 64. These embodiments offer the advantages discussed above. In each of these embodiments, the SSVT signal is generated within the processor 60 near the GPU 64 and then transmitted to the source drivers of the displays 32, 34 to display video data. Compared to conventional digital video transmission technologies, these embodiments offer greater coverage, greater interference immunity, and lower power consumption (depending on the integration level).

[0031] Figure 2 illustrates a GPU-to-display platform embodiment 800, where the platform is a custom printed circuit board 802 including a current-generation (or "conventional") GPU IC 804 and an SSVT transmitter IC 82. In this embodiment, the SSVT transmitter is not integrated within the GPU. Video data is received from storage, a camera, the internet, core 62, etc., 808, processed by the GPU 804, and then converted and encoded into an SSVT signal 92 by the SSVT transmitter 82. This signal is then transmitted to displays 32, 34 via any suitable EM path (physical wire, RF, or fiber optic cable), where the signal is passed to each source driver 812-818, then decoded and displayed on the display panel, as described herein. The display can include any display suitable for a head-mounted device. This PCB-level integration embodiment saves 20% of power compared to conventional technologies.

[0032] Figure 3 illustrates a GPU-to-Display Module-on-Module (SoM) embodiment 820, where SoM 822 includes a current-generation GPU IC 824 and an SSVT transmitter IC 82; these ICs can be known-qualified chips (KGD). In this embodiment, the SSVT transmitter is not integrated within the GPU. Video data is received from storage, a camera, the internet, core 62, etc., 828, processed by the GPU 824, and then converted and encoded into an SSVT signal 92 by the SSVT transmitter 82. This signal is then transmitted to displays 32, 34 via any suitable EM path (physical wire, RF, or fiber optic cable), where the signal is passed to each source driver 832-838, then decoded and displayed on the display panel, as described herein. The display can include any display suitable for a head-mounted device. This advanced integration embodiment saves 50% of power compared to conventional technologies.

[0033] Figure 4 illustrates a fully integrated GPU-to-display embodiment 840, wherein the enhanced GPU 844 includes a fully integrated SSVT transmitter 82 within the GPU chip. Video data is received from storage, camera, internet, core 62, etc., 848, processed by the GPU 844, and then converted and encoded into an SSVT signal 92 by the SSVT transmitter 82. This signal is then transmitted to displays 32, 34 via any suitable EM path (physical wire, RF, or fiber optic cable), where the signal is passed to each source driver 852-858, then decoded and displayed on the display panel, as described herein. The display can include any display suitable for a head-mounted device. This highest level of integration embodiment saves 60% of power compared to conventional technologies. Furthermore, the GPU has an SSVT transmitter integrated on the same silicon chip. Therefore, all digital transmissions occur over very short distances, and thus, high data rates on a single chip are not critical. SSVT transmitter

[0034] Figure 5 is a block diagram of the SSVT transmitter 82. As discussed herein, the output of encoder 142 (EM signals 0 to P-1) is transmitted to a transmission medium such as cable, fiber optic, or wireless. Distributor 440 includes assembly library 450, hierarchical library 452, representation library 454, and controller 456. Encoder block 460 includes a set of digital-to-analog converters (DACs) 462 and any number of P encoders 142, one for each EM path, for transmitting EM signals over the transmission medium.

[0035] During operation, a timing video sample stream containing color values ​​and pixel-related information is received from the video source at GPU 64 and transmitted to SSVT transmitter 82 via SoC and TCON (both not shown). The number and content of the input video samples received from the video source depend on the color space being used at the source (and the samples can be black and white). Regardless of the color space used, each video sample represents light in the specified color space.

[0036] When an input video sample stream is received within the assembly library 450, the input video samples are repeatedly (1) assigned to encoder input vectors according to a predetermined arrangement (one vector per encoder), and (2) encoded by applying SSDS-based modulation to each input vector using orthogonal codes to generate multiple composite EM signals, one analog signal per encoder. The EM signals are then transmitted (3) via a transmission medium, one signal per path.

[0037] For illustrative purposes, one possible permutation of the four vectors V0, V1, V2, and V3 (assuming four encoders, P=4) implemented by the allocator is an arrangement where each vector contains N color information samples. In this example, the exposed color information of the sample set is "RGB". The exposed RGB samples in the sample set are assigned to vectors V0, V1, V2, and V3 from left to right. In other words, the "R", "G", and "B" values ​​of the first sample and the "R" signal of the next set of samples are assigned to vector V0, while the next (from left to right) "G", "B", "R", and "G" values ​​are assigned to vector V1, the next (from left to right) "B", "R", "G", and "B" values ​​are assigned to vector V2, and the next (from left to right) "R", "G", "R", and "R" values ​​are assigned to vector V3. Once the fourth vector V3 has been assigned its samples, the above process is repeated until each of the four vectors V0, V1, V2, and V3 has N samples. In various embodiments, the number of N samples can vary considerably.

[0038] As an example, consider an implementation with N=60. In this case, the total number of N samples included in the four vectors V0, V1, V2, and V3 is 240 (60 x 4 = 240). The four encoder input vectors V0, V1, V2, and V3, when fully established, include 80 samples from 80 different sample sets (240 / 3 = 80) (where S = 3). In other words: ● Vector V0 includes samples P0,N0 to P0,NN-1; ● Vector V1 includes samples P1,N0 to P1,NN-1; ● Vector V2 includes samples P2, N0 to P2,N-1; and ● Vector V3 includes samples P3,N0 to P3,NN-1.

[0039] It should be understood that the above examples are merely illustrative and should not be construed as restrictive. The sample size N can be greater than or less than 60. Furthermore, it should be understood that the exposure color information for each sample group can be any color information (e.g., Y, C, Cr, Cb, etc.), and is not limited to RGB. The number of EM paths on the transmission medium can also vary considerably. Correspondingly, the number of vectors V and the number of encoders can also vary widely from 1 to any number greater than 1. It should also be understood that the permutation scheme used to construct the vectors, regardless of its number, is arbitrary. Any permutation scheme can be used.

[0040] Distributor 440 is configured to sequentially receive exposure color information (e.g., RGB) of streams of sample groups. In response, assembly library 450 constructs vectors V0, V1, V2, and V3 from the exposure color information of the incoming streams of sample groups. When a sample set is received, it is stored in assembly library 450 according to a predetermined arrangement.

[0041] The hierarchical library 452 helps to cross the N samples of each of the four vectors V0, V1, V2, and V3 from a first clock frequency (or first timing domain) used by the re-timer to a second clock frequency (or second domain) used for encoding and outputting the resulting analog output level. For example, with N=60 and S=3, the four encoder input vectors V0, V1, V2, and V3 contain samples representing exactly 80 sets of RGB samples.

[0042] In various embodiments, the first clock frequency can be faster, slower, or the same as the second clock frequency. The first clock frequency f_pix is ​​determined by the video format selected by the video source. The second clock frequency f_ssvt is a function of f_pix, the number of EM paths P in the transmission medium, the number of samples S in each input / output sample group, and the SSVT transform parameters N (the number of input / output vector positions) and L (the length of each SSDS code), where f_ssvt = (f_pix * S * L) / (P * N). With this arrangement, the input clock (pix_clk) oscillates at one rate, and the SSVT clock (ssvt_clk) oscillates at another rate. These rates can be the same or different. The encoder performs encoding while preparing the next input vector. The representation library 454 presents N samples of each of the four encoder input vectors V0, V1, V2, and V3 to the encoder block 460 (e.g., vector V0 includes sample 0, 0 to sample 0, N-1).

[0043] Controller 456 controls the operation and timing of the assembly library 450, the hierarchical library 452, and the representation library 454. Specifically, when constructing the four encoder input vectors V0, V1, V2, and V3, the controller is responsible for defining the permutations and the number of samples N used. Controller 456 is also responsible for coordinating the cross-clock domain operations from the first clock frequency to the second clock frequency performed by the hierarchical library 452. Controller 456 is further responsible for coordinating the timing of when the representation library 454 presents N samples of each encoder input vector V0, V1, V2, and V3 to the encoder block 460.

[0044] Within encoder block 460, multiple digital-to-analog converters (DACs) 462 are provided, each DAC arranged to receive one of P*N samples (sample 0, 0 to sample P-1, N-1) collectively assigned to four encoder input vectors V0, V1, V2, and V3. Each DAC 462 converts its received sample from the digital domain into a differential voltage signal pair with an amplitude proportional to its input digital value. The output of DAC 462 can range from a maximum voltage to a minimum voltage.

[0045] Four encoders 142 are provided with four encoder input vectors V0, V1, V2, and V3, respectively. Each encoder 142 receives a differential pair signal for each of N samples of its encoder input vector, modulates each of the N differential pair voltage signals using the chip modulated code corresponding to each sample, accumulates the modulated values, and then produces L differential output levels, as shown in Figures 14 and 15. These L levels produce one of the EM signals. Since there are four encoders 142 in this example, EM signals 0 through EM signal 3 are output simultaneously. Preferably, L>=N>=2. As previously mentioned, the encoding can be analog (with the DAC placed before the encoder) or digital (where the L levels are converted to analog by the DAC before transmission). The L analog output levels are then transmitted via their EM paths as part of the SSVT signal to the SSVT receiver, in this case, the SSVT receiver being part of the source driver 586. Advantageously, the SSVT signal is an analog signal, eliminating the need for a DAC at the source driver.

[0046] The sequencer circuit 465 coordinates the timing of the operations of the DAC 462 and the encoder 142. The sequencer circuit 465 is responsible for controlling the clocks of the DAC 462 and the encoder 142. The sequencer circuit 465 is also responsible for generating two clock phase signals, "clk1" and "clk2", which control the operation of the encoder 142. Transmit SSVT signal to mask display

[0047] Figure 6 illustrates the display panel 34 that transmits the SSVT signal to the VR headset 20. The SoC and TCON of the head-mounted device processor 60 are not shown. Integration may have three or more embodiments: a discrete implementation, where the SSVT transmitter is embedded in a mixed-signal integrated circuit, and the TCON and SoC are discrete components (the SSVT transmitter is inserted between a conventional TCON and the novel source driver described herein, converting the TCON output signal into an SSVT signal); a hybrid implementation, where the SSVT transmitter and TCON are integrated in a single IC, and the SoC is discrete; and a fully integrated implementation, where as many functions as possible are integrated into a custom mixed-signal integrated circuit (the SSVT transmitter is integrated with the TCON and SoC).

[0048] In this example of Figure 6, the display panel 34 can be of any size within a VR headset and can be a head-up display (HUD) where the display is projected onto the mask's screen, etc. The left display 32, not shown, receives its own SSVT signal for display. Typically, in a multi-display headset, each panel will receive its own signal.

[0049] Figure 6 also shows an SSVT transmitter 82, which generates an SSVT signal 592 for the source drivers 586 and power and control signals 590 for the gate drivers 560. A rigid PCB 582 and a separate flexible PCB 584, each holding a source driver 586, generate source voltages for the display panel. As will be described in more detail below, signal 608 optionally provides information about the display panel to the transmitter 82 to assist in encoding the SSVT signal. The generation of the gate driver control signal 590 can be performed by a timing controller (or other specific hardware) based on synchronization information from the source drivers. Typically, due to pin count limitations, most panels with more than 1024 columns are implemented using an array of source driver chips, one source driver per chip. For panels with fewer columns, a single source driver is sufficient.

[0050] Typically, the SSVT transmitter and SSVT receiver are connected by a transmission medium. In various embodiments, the transmission medium can be a cable (such as HDMI, ribbon cable, fiber optic cable, metallic cable, non-metallic carbon rail flexible cable) or it can be wireless. The transmission medium can have multiple EM paths, one path per encoder. The SSVT receiver will include multiple decoders, the same number as the encoders. Display panel source driver

[0051] Figure 7 illustrates a display source driver 586. As shown and as known in this art, multiple source drivers are cascaded; these multiple source drivers then drive the display panel. As shown, source driver 586 does not require the DAC (used in the signal path to convert digital samples into analog samples for display) required in prior art source drivers. The input to the SSVT receiver 610 of each source driver is an analog SSVT signal 592 that has been encoded upstream as described herein. As shown, the SSVT signal 592 is daisy-chained between the source drivers. In an alternative embodiment, each source driver will have its own SSVT signal, and the TCON provides timing information to each source driver chip.

[0052] The SSVT receiver 610 can have any number (P) of decoders, or only a single decoder. Each receiver 610 decodes the SSVT signal (described in more detail below) and outputs a large number of reconstructed analog sample streams 612, i.e., analog voltages (the number of sample streams corresponds to the number of outputs from the source drivers). Since these analog outputs 612 may not be within the voltage range required by the display panel, they may need to be scaled and may be input to a quasi-shifter 620, which uses analog conversion to shift the voltage to the voltage range used to drive the display panel. Any suitable quasi-shifter known in the art can be used, such as latch-type or inverter-type. A quasi-shifter can also be referred to as an amplifier.

[0053] For example, the voltage range from the SSVT receiver can be 0 to 1V, while the voltage range from the quasi-shifter can be -8 to +8V (using the inverting signal 622 to instruct the quasi-shifter to toggle the voltage every frame; that is, the voltage range of one frame will be -8 to 0V, and then the voltage range of the next frame will be 0V to +8V). In this way, the SSVT signal does not need to toggle its voltage every frame; the SSVT receiver provides a positive voltage range (e.g.), and the quasi-shifter toggles the voltage every frame, as expected by the display panel. The SSVT receiver can also implement line inversion and dot inversion. The inverting signal tells the quasi-shifter which voltage to switch. Some display panels, such as OLEDs, do not require voltage toggle every frame; in this case, the inverting signal is unnecessary, and the quasi-shifter does not toggle the voltage every frame. Display panels such as LCDs do require this voltage toggle. The inverting signal 622 is recovered from the SSVT receiver, as described below.

[0054] The input quasi-shifter 620 can also have gain and gamma values; gain determines how much amplification is applied, and the gamma curve correlates luminous flux with perceived brightness, thereby linearizing human optical perception of luminous flux. Typically, in prior art source drivers, gain and gamma are setpoints determined by the manufacturing characteristics of the display panel. In the analog quasi-shifter 620, gain and gamma can be implemented as follows. In one embodiment, gamma is implemented in the digital portion of the system, and quasi-shifting and gain are implemented in the driver by setting the output stage amplification. In the case of gamma, implementation in the output driver is also possible by implementing nonlinear amplification characteristics. Once shifted, a sample is output 634, which is used to drive the source electrodes in the corresponding column of the display panel, as known in the art.

[0055] In order to correctly encode SSVT signals for eventual display on a specific display panel (whether encoded inside the head-mounted display or further upstream outside the head-mounted display), the GPU (or other display controller) or any entity performing SSVT encoding requires various physical characteristics or properties of that display panel. These physical characteristics are labeled 608 and include resolution, checkerboard layout, backlight layout, color profile, aspect ratio, and gamma curve. Resolution is a constant for a specific display panel; checkerboard layout refers to the way the plane of the panel is divided into multiple regions in a regular, predetermined manner, in pixels; backlight layout refers to the resolution and diffusion characteristics of the backlight panel; color profile is the precise luminance response of all primary colors, providing accurate colors for the image; the aspect ratio of the display panel will have discrete known values.

[0056] These physical characteristics of a specific display panel can be delivered, hard-connected to, or provided to a specific display controller in a variety of ways. In one example shown in Figure 6, signal 608 transmits the values ​​of these physical characteristics directly from the display panel (or from another location within the display unit) to the SSVT transmitter 82. Alternatively, the SSVT transmitter 82, embedded within a specific display unit, carries these values ​​hard-coded within the transmitter. Or, a specific display controller means that it is only used with a specific type of display panel, and its characteristic values ​​are hard-coded into that display controller.

[0057] The input to the display panel can also be a backlight signal 604, which indicates when the backlight LEDs are turned on and at what voltage level. In other words, it is typically a low-resolution representation of the image, meaning that the backlight LEDs need to be lit in bright areas and dimmed in dim areas. The backlight signal is a monochrome signal and can also be embedded within an SSVT signal; for example, it can be another parallel and independent video signal traveling with other parallel video signals R, G, and B (e.g.), and can be low-resolution or high-resolution.

[0058] The output from the SSVT receiver 610 is a gate driver control signal 606, which shares timing control information with the gate driver 560 on the left edge of the display panel to synchronize the gate driver with the source driver. Typically, each SSVT receiver includes timing acquisition circuitry to acquire the same timing control information for the gate driver, and one or more of the source driver flex foils (typically the leftmost and / or rightmost source drivers) pass this timing control information to the gate driver. The timing control information for the gate driver is embedded within the SSVT signal and recovered from the signal using established spread spectrum techniques.

[0059] Traditional display drivers typically use "COF" (Flexible On-Foil) IC packages directly attached to the glass; conventional COG (Copper On-Glass) packages are also feasible. These drivers can be replaced with the novel source drivers described in this paper, thereby converting existing display panels to support SSVT. The inputs of these ICs are typically connected together by a PCBA, providing input signals from the video source and timing controller. These can be located close to or far from the display panel, transmitting video and control signals via inexpensive wiring. Details of SSVT decoding and integration with source drivers

[0060] On the receiving side, the decoder of each source driver is responsible for decoding the stream of differential EM potential quasi-signals received via the transmission medium back into a format suitable for display. Once in the appropriate format, the video content contained in the sample can be displayed frame by frame on the video display. Therefore, video captured from any video source can be recreated by the video receiver.

[0061] Figure 8 shows a more detailed view of the SSVT receiver 610 of the source driver. P represents the number of input electromagnetic pairs, each carrying an SSVT signal independent of the others, except that they are isochronous signals known to be generated synchronously by the encoders on the transmitting side. The source driver contains P decoders 780 and collectors (blocks 782 and 786). The decoder 780 performs the inverse transform of its paired encoder on the transmitting side and reconstructs its input differential EM potential quasi-signals into an output vector of N reconstructed samples (although single-ended inputs can be used instead of differential inputs). The collectors assign the decoder output vector samples (or “reconstructed samples”) to predetermined positions in the source driver input 612. The source driver input 612 includes reconstructed samples corresponding to the driven column groups in the display panel. A retimer function is included in the collector.

[0062] P decoders 780 (labeled 0 to P-1) are arranged to receive differential EM signals 0 to P-1, 702-704 respectively. In response, each decoder 780 generates N differential pairs of reconstructed samples (sample 0 to sample N-1). With four decoders 780 (P=4), four vectors V0, V1, V2, and V3 are constructed respectively. The number of samples N is equal to the number of orthogonal codes used for early coding; that is, N orthogonal codes are used, which means N codes from the codebook.

[0063] Reconstruction library 782 samples each of the four decoder output vectors V0, V1, V2, and V3 at the end of each decoding interval and stores N reconstructed samples for each differential pair (sample 0 to sample N-1). These received differential voltage signal pairs are then output as samples (sample N-1 to sample 0) of each of the four vectors V0, V1, V2, and V3. Essentially, each reconstruction library reconstructs a single voltage from the differential pair. Hierarchical library 786 receives all reconstructed samples (Nn-1 to N0) of each of the four decoder output vectors V0, V1, V2, and V3 and acts as an analog output buffer, as described in more detail below. Once the samples are moved into hierarchical library 786, they are triggered by latching signals 632 derived from the decoded SSVT signals. The latching signals can be daisy-chained between source drivers. Once the samples are released from the hierarchical library, they are sent to potential quasi-shifters 620.

[0064] The SSVT receiver 610 also includes a channel aligner 787 and a hierarchy controller 789, which receive framing and aperture information from each decoder 780. In response, the hierarchy controller 789 coordinates the timing of the hierarchy library 786 to ensure that all samples originate from the common time interval of the potential quasi-signals transmitted by the SSVT transmitter. Therefore, individual channels of the transmission medium do not necessarily have the same length, as the channel aligner 787 and the hierarchy controller 789 compensate for any timing differences. A gate driver control signal 606 provides timing information to the gate driver (or to intermediate circuitry), which in turn provides the gate driver with correct timing and control signals, and may originate from the channel aligner 787.

[0065] Please note that Figure 8 reveals a decoder that buffers samples in the hierarchical library 786 and then shifts the potential level (amplifies); alternatively, it can shift the potential level and then buffer the samples for output. Display panel source driver array

[0066] Figure 9 illustrates an alternative embodiment of the array used to implement the source drivers. In this embodiment, each source driver includes a single decoder (i.e., an SSVT receiver for one decoder), followed by a collector and an amplifier, while Figures 7 and 8 show that each source driver may have multiple decoders within the SSVT receiver of the source driver. Either approach can be used. This array is suitable for high-resolution (8K) display panels. Since the resolution of VR display panels ranges from 960x1080 to 3840x2160, those skilled in the art will be able to adjust the number of source drivers used (and the number of outputs per source driver) to suit typical VR display panels.

[0067] The diagram shows 24 720MHz SSVT signals 652-654, each signal being a twisted-pair pair from SSVT transmitter 540, meaning each twisted-pair pair originates from the encoder of the transmitter. Each pair is input to one of decoders 656-658, each decoder outputting 64 analog samples at a frequency of 11.25MHz. These samples are then input to one of 24 collectors 662-664, each collector collecting 15 sets of these samples, and the output is updated every 15 decoding intervals, as detailed below. As mentioned above, each collector consists of a reconstruction library and a hierarchical library (not explicitly shown in this diagram). These 960 analog samples from each collector are then amplified at a frequency of 750kHz by one of amplifiers 666-668, and then output at a frequency of 750kHz (11.25MHz x 64 / 960) as the amplified analog level 670 to the display column of the display panel. For clarity, signals 604, 606, 608, 622, and 632 shown in Figures 7 and 8 are not shown.

[0068] If the encoded SSVT signal is a higher voltage, and the decoded signal results in the sampling voltage required by the display, then an amplifier or potential shifter can be omitted. However, since SSVT signals are typically low voltage (and require higher voltage outputs for large-format displays), amplification is usually necessary. However, in the case of displays within VR headsets, an amplifier may not be necessary and can be optional.

[0069] Note that Figure 9 discloses a decoder that buffers samples in collector 664 and then amplifies them; alternatively, samples can be amplified and then collected (buffered) for output. Either embodiment can be used.

[0070] Figure 10 is a block diagram of one of the decoders 656 in Figure 9. One of the SSVT signals 652 input to the decoder is shown. The decoder includes a chip counter 680, a codebook 682 containing orthogonal codes for encoding and decoding, typically stored in RAM, and a block diagram 684 for each of the 64 output analog samples 688. One output is "active" per L cycles at 11.25 MHz for each group of 64 analog samples. Decoding will be explained in more detail below.

[0071] Figure 11 is a block diagram of the collector in Figure 9 and shows more details of the hierarchical library 786 in Figure 8. Essentially, a single collector performs a serial-to-parallel conversion to the partition row buffer. The input to each collector 662-664 shown is a set of 64 analog samples 690-692 (not shown, the reconstruction library 782) from each decoder at a frequency of 11.25 MHz. As shown, during each decoding interval, a new set of 64 incoming reconstructed samples is stored within the collector, with each collector being filled every 15 decoding intervals. After every 15 decoding intervals, 960 stored samples 698 from each collector are output to their respective amplifiers 666-668 and then passed to the corresponding column on the display panel, as shown.

[0072] In one particular embodiment, each source driver of FIG9 (e.g., decoder 658, collector 664, and amplifier 668) is implemented within an integrated circuit, and each such integrated circuit can be mounted on a flexible PCB 584, as shown in FIG6. Wireless transmission of SSVT encoded signals

[0073] As previously mentioned, one of the possible options for the transmission medium of the P EM signals from the SSVT transmitter 82 is wireless. A wireless embodiment for transmitting and receiving SSVT-coded electromagnetic signals is provided in detail below.

[0074] Referring to Figure 12, a block diagram 200 of a wireless implementation of the transmission medium is shown (not shown because the medium is wireless in this embodiment). In this embodiment, a wireless transmitter 202 and an antenna 204 are provided on the transmitting side, and a receiver 206 and an antenna 208 are provided on the receiving side. The transmitter 202 modulates the SSVT electromagnetic signal generated by the SSVT transmitter 82 onto one or more carrier frequency signals. Once modulated, the carrier frequency signal is then broadcast by the antenna 204. On the receiving side, the antenna 208 receives the broadcast and provides the modulated carrier signal to the wireless receiver 206. In response, the receiver 206 demodulates and generates the SSVT electromagnetic signal, which is then provided to the SSVT receiver 610. The SSVT electromagnetic signal is not provided to the SSVT receiver 610 (which is integrated with the source driver of the display), but can also be provided to any SSVT receiver described in application numbers 17 / 686,790 (HYFYP004AX1) or 17 / 887,849 (HYFYP006). Once decoded, the video signal can be used to drive a video display, as described above.

[0075] Referring to Figure 13A, a schematic diagram of transmitter 202 is shown. Transmitter 202 includes antenna 204, one or more (P) modulators 210, carrier frequency generator 212, one or more (P) bandpass filters 214, summing node 216, and power amplifier 218. The number of encoders 142 in SSVT transmitter 82, and therefore the number of EM signal outputs, is represented by (P). In the case where encoder 142 is used only once in the SSVT transmitter (i.e., P=1), summing node 216 is not required.

[0076] During operation, one or more electromagnetic (EM) signals (P signals) generated by the SSVT transmitter 82 are provided to one or more modulators 210. In response, each modulator 210 modulates one of the electromagnetic signals onto (P) different carrier frequency signals. Preferably, the (P) carrier signals are different frequencies, but all derived from the same fundamental sine frequency. By performing modulation, the (P) electromagnetic signals are substantially superimposed onto the (P) carrier frequency signals respectively. Then, a bandpass filter 214 filters each modulated carrier frequency signal. Next, the outputs of the bandpass filters are summed at a summing node 216, effectively summing all P voltage waveforms to generate a composite signal. An amplifier 218 amplifies the composite signal from the antenna 204. In response, the antenna 204 wirelessly broadcasts the composite signal (i.e., the amplified, summed, filtered, and modulated carrier frequency signal). Preferably, both the amplifier and the antenna are selected to handle the additional bandwidth generated by the composite signal.

[0077] As long as the SSVT transmitter 82 generates (P) electromagnetic signals from the video sample stream, it continues to perform the above modulation and broadcasting operations. Therefore, the coded wireless signal representing the video sample stream is continuously broadcast.

[0078] Referring to Figure 13B, a schematic diagram of receiver 206 is shown. Receiver 206 includes an antenna 208, a gain controller 220, one or more (P) demodulators 222, one or more (P) low-pass filters 224, a discriminator circuit 226, and a voltage-controlled frequency source (VCFS) 228.

[0079] During operation, the composite signal broadcast by transmitter 202 is received by antenna 208. Gain controller 220 adjusts the gain of the received composite signal; the gain controller can be implemented using an automatic gain controller (AGC) or a programmable gain amplifier (PGA). Regardless of the method used, the gain-adjusted composite signal is provided to each demodulator 222.

[0080] In response, each demodulator 222 demodulates the composite signal and generates one of (P) electromagnetic signals. In one embodiment, each demodulator 222 is a superheterodyne receiver that uses frequency mixing to convert the received signal to an intermediate frequency (IF) that is easier to process than the original incoming composite signal. Alternatively, each demodulator 222 is a direct conversion receiver (DCR), which is a radio receiver designed to demodulate the input composite signal using synchronization detection driven by a local oscillator with a frequency that is the same as or very close to the carrier frequency of the input composite signal. Regardless of the type of demodulator used, each of the (P) demodulated signals is provided to one of the low-pass filters 224. Each low-pass filter filters its received demodulated electromagnetic signal and provides its output to the SSVT receiver 610, as described above.

[0081] The discriminator circuit 226 provides a feedback loop between the output 230 of each demodulator 222 and the VCFS 228. When the demodulator 222 is used to demodulate one or more frequencies that are drifting, the discriminator circuit 226 functions to adjust the demodulation frequency (or frequencies) to lock onto and match the received carrier frequency.

[0082] The above describes a wireless embodiment in which any number of electromagnetic signals are modulated, filtered, and then summed for amplification by an amplifier and output by an antenna. This reduces cost because only a single amplifier and a single antenna are required. In an alternative embodiment of the wireless transmitter 202, there is no summing node 216, and each of the (P) electromagnetic signals from the SSVT transmitter 82 is modulated and filtered accordingly, and then each signal is amplified and output using a power amplifier and an antenna. In other words, there will be (P) amplifiers and antennas instead of a single power amplifier and antenna. Similarly, the wireless receiver 206 can be implemented using (P) antennas, (P) gain controllers, and demodulators and filters for each signal as described. SSVT signal, encoding and decoding

[0083] As previously described, various embodiments of the present invention disclose the use of analog SSVT signals to transmit video information within a VR headset, or to or from a VR headset. The techniques for encoding and decoding this signal are described below.

[0084] For the purposes of this disclosure, an electromagnetic signal (EM signal) is a variable representing electromagnetic energy whose amplitude varies with time. EM signals propagate from the transmitter to the receiver via EM paths, such as wire pairs (or cables), free space (or wireless), and light or waveguides (optical fibers). EM signals can be independently characterized as continuous or discrete in each of two dimensions (time and amplitude). A "pure analog" signal is a continuous-time, continuous-amplitude EM signal; a "digital" signal is a discrete-time, discrete-amplitude EM signal; and a "sampled analog" signal is a discrete-time, continuous-amplitude EM signal. This invention discloses a novel discrete-time, continuous-amplitude EM signal called a "spread spectrum video transmission" (SSVT) signal, which is an improvement on existing SSDS-CDMA signals. SSVT refers to the transmission of electromagnetic signals over one or more EM paths using improved spread spectrum direct sequence (SSDS) modulation.

[0085] Code Division Multiple Access (CDMA) is a well-known channel access protocol commonly used in radio communication technologies, including cellular phones. CDMA is an example of multiple access where several different transmitters can simultaneously transmit information on a single communication channel. In telecommunications applications, CDMA allows multiple users to share a given frequency band without interference from other users. CDMA uses Spread Spectrum Direct Sequence (SSDS) coding, which relies on a unique code to encode each user's data. By using a unique code, transmissions from multiple users can be combined and sent without interference between users. On the receiving side, each user uses the same unique code to demodulate the transmission, recovering each user's data separately.

[0086] SSVT signals differ from CDMA signals. When an input video (e.g., a sample stream) is received at the encoder, it is encoded by applying SSDS-based modulation to each of the multiple encoder input vectors to generate an SSVT signal. The SSVT signal is then transmitted over a transmission medium. At the receiving end, the incoming SSVT signal is decoded by applying appropriate SSDS-based demodulation to reconstruct the encoded sample. Therefore, the original timing video sample stream, containing color and pixel-related information, is transmitted from a single video source to a single video receiver, unlike CDMA, which transmits data from multiple users to multiple receivers.

[0087] Figure 14 illustrates a simple example of how signal samples (analog values ​​in this example) are encoded in an encoder and then transmitted via an electromagnetic path. An input vector of N analog values ​​902-908 is shown, representing the voltage of each pixel within the video frame. These voltages can represent the brightness of a black-and-white image or the brightness of a specific color value in a pixel (e.g., the R, G, or B color value of a pixel), meaning each value represents the amount of light sensed or measured in a specified color space. While pixel voltages are used in this example, this encoding technique can be used with voltages representing any of a variety of signals from a sensor, such as LiDAR values, sound values, tactile values, aerosol values, etc., and analog values ​​can represent other samples, such as current. Signal samples with digital values ​​can also be encoded, as will be explained below. Furthermore, even though only one encoder and one EM path are shown, embodiments of the invention work well with multiple encoders, each transmitting via the EM path.

[0088] Preferably, the startup signal sample voltage is typically higher than the encoded voltage in the SSVT signal. After encoding, for efficiency, the voltage range is typically from 0 to 1V, although different ranges are possible. Lower voltage means less power consumption and potential for future use.

[0089] These voltages are typically acquired from pixels in a row of the frame in a specific order, but a different convention can be used to select and sort these pixels. Regardless of which convention is used to select and sort these pixels for encoding, the decoder will use the same convention at the receiving end to decode these voltages in the same order and then place them in their respective result frames. Similarly, if the frame is color and uses RGB, the convention in this encoder could be that all R pixel voltages are encoded first, followed by G and B voltages, or the convention could be that voltages 902-906 are the RGB values ​​of the pixels in that row, the next three voltages 908-912 represent the RGB values ​​of the next pixel, and so on. Again, the same convention used by this encoder to sort and encode the voltages will be used by the decoder at the receiving end. As long as the decoder uses the same convention, any specific convention can be used to sort the analog values ​​902-908 (whether by color value, by row, etc.). As shown, any number of N analog values ​​902-908 can be represented at once for encoding using codebook 920, limited only by the number of entries in the codebook.

[0090] As previously stated, codebook 920 has any number of N codes 932-938; in this simple example, the codebook has four codes, meaning that four analog values ​​902-908 are encoded at once. More codes, such as 127 codes, 255 codes, etc., can be used, but fewer codes are preferred for practical considerations such as circuit complexity. As is known in the art, codebook 920 comprises N mutually orthogonal codes, each of length L; in this example, L=4. Typically, each code is an SSDS code, but it does not necessarily have to be the spread spectrum code discussed herein. As shown, each code is divided into L time intervals (also called “chips”), and each time interval includes the binary value of that code. As shown in code representation 942, code 934 can be represented in the conventional binary form “1100”, although the same code can also be represented as “1 1 -1 -1” as shown in code representation 944 for use when modulating values, as explained below. Codes 932 and 936-938 can also be represented as in 942 or 944. Note that each code of length L is not associated with a different computing device (such as a telephone), a different person, or a different transmitter, as is the case in CDMA.

[0091] Therefore, in order to transmit the four analog values ​​902-908 to the receiver (with a corresponding decoder) via transmission medium 34, the following technique is used. Each analog value is modulated by each chip in the representation 944 of its corresponding code; for example, the value 902, i.e., .3, is modulated 948 sequentially by each chip in the representation 944 of code 932. Modulation 948 can be a multiplication operator. Thus, modulating .3 with code 932 yields the series ".3, .3, .3, .3". Modulating .7 with code 934 becomes ".7, .7, -.7, -.7"; the value "0" becomes "0, 0, 0, 0"; and the value "1" becomes "1, -1, 1, -1". Typically, the first chip of each code modulates its corresponding analog value, and then the next chip of each code modulates its analog value, although one implementation may also modulate a specific analog value by all chips of its code before moving to the next analog value.

[0092] At each time interval, the modulated analog values ​​are summed at 951 (vertically sensed in this figure) to obtain analog output potential levels 952-958; for example, the sum of the modulated values ​​for these time intervals results in output potential levels of 2, 0, .6, -1.4. These analog output potential levels 952-958 can be further normalized or amplified to conform to the voltage limits of the transmission line, and can then be transmitted in the order in which they are generated via the electromagnetic path (e.g., differential twisted pair) of the transmission medium 34. The receiver then receives these output potential levels 952-958 in this order and decodes them using the same codebook 920, in a manner opposite to the encoding scheme shown here. The resulting pixel voltages 902-908 can then be displayed in the frame of the display at the receiving end according to the convention used. Thus, the analog values ​​902-908 are effectively synchronously encoded and transmitted via a single electromagnetic path in a sequential sequence of L analog output potential levels 952-958. As shown and described in this paper, multiple encoders and electromagnetic paths can also be used. Furthermore, the number of N samples that can be encoded in this way depends on the number of orthogonal codes used in the codebook.

[0093] Advantageously, even though the use of robust SSDS techniques (such as spread spectrum codes) leads to a significant bandwidth reduction, the use of mutually orthogonal codes, modulating and summing each sample by chips with corresponding codes, and transmitting N samples in parallel using L output potential levels results in a significant bandwidth gain. Compared to conventional CDMA technology where binary digits are serially encoded and then summed, this invention first modulates the entire sample (i.e., the entire analog or digital value, not a single bit) by each chip in the corresponding code, and then sums these modulations at each time interval of the code to obtain the generated analog voltage level for each specific time interval, thereby utilizing the amplitude of the resulting waveform. It is these analog output potential levels that are transmitted via the transmission medium, rather than a binary digital representation. Furthermore, this invention facilitates the transmission of analog voltages from one video source to another video receiver, i.e., from end to end, unlike CDMA technology which allows multiple accesses and transmissions to multiple receivers by different people, different devices, or different sources. Moreover, the transmission of sample values ​​does not require compression.

[0094] Figure 15 illustrates the application of this novel encoding technique to signal samples of digital values. Here, digital values ​​902'-908' are digital representations of voltage. Using different voltage examples, value 902' is "1101", value 904' is "0011", value 906' is "0001", and value 908' is "1000". Each digital value is modulated (digital multiplication) by the representation 944 of each code, i.e., "1" or "-1", depending on the chip corresponding to the code of the digital value to be modulated. Considering only the first time interval 940 for each code and adding the most significant bit (MSB), i.e., the sign bit, modulating "1101" yields "01101" (MSB "0" indicates a positive value), modulating "0011" yields "00011", modulating "0001" yields "00001", and modulating "1000" yields "01000". These modulation values ​​are shown over the first time interval. (Although not shown, modulation using a -1 chip produces negative values, which can be represented in binary using a suitable binary representation for negative values.)

[0095] The bitwise summation of these modulation values ​​within the first time interval yields the bitwise value 952' "011001" (again, the MSB is the sign bit); the other bitwise values ​​954'-958' are not shown in this example but are calculated in the same way. Considering this base-10 sum, it can be verified that the sum of the modulation values ​​13, 3, 1, and 8 is indeed 25. Although not shown in this example, additional MSBs would typically be available for the generated level 952'-958', as the sum may require more than 5 bits. For example, if the values ​​902'-908' are represented using 4 bits, then with 64 codes, the level 952'-958' could be represented using up to 10 bits (plus 64 bits log2). Alternatively, if 32 modulation values ​​are summed, another 5 bits would be added. The number of bits required for the output level depends on the number of codes.

[0096] The output level 950' can be normalized to meet the input requirements of the DAC, and then sequentially fed to the DAC 959 to convert each digital value to its corresponding analog value for transmission via the EM path. The DAC 959 can be a MAX5857 RF DAC (including a clock multiplied by a PLL / VCO and a 14-bit RF DAC core, with complex paths that can be bypassed to directly access the RF DAC core), then a bandpass filter, and then a variable gain amplifier (VGA), not shown. In some cases, the number of bits used in level 950' is greater than the number of bits allowed by the DAC 959; for example, level 952' is represented with 10 bits, but the DAC 959 is an 8-bit DAC. In these cases, an appropriate number of LSBs are discarded, and the remaining MSBs are processed by the DAC, resulting in no loss of visual quality of the image on the display.

[0097] Advantageously, the entire digital value is modulated, and then these modulated digital values ​​are digitally summed to produce the digital output levels used for conversion and transmission. This technique differs from CDMA, which modulates each binary digit of the digital value and then sums these modulated bits to produce the output. For example, assuming each digital value has B bits, using CDMA, a total of B*L output levels need to be transmitted, while using this new digital (or analog) coding technique, only L output levels need to be transmitted, thus offering an advantage.

[0098] Figure 16 illustrates the decoding of the analog input potential levels encoded using the encoder of Figure 14. As shown, L input potential levels 950 have been received by a single electromagnetic path of transmission medium 34. As mentioned herein and previously, codebook 920 includes N orthogonal codes 932-938, which will be used to decode the input potential levels 950 to produce an output vector of N analog values ​​902-908, identical to the analog values ​​902-908 encoded above. To perform the decoding, as indicated by the vertical arrows, each input potential level 952-958 is modulated 961 by each chip of each code corresponding to a specific index in the output vectors 902-908. Consider modulating potential levels 952-958 with the first code 932, which produces a series of modulation values ​​"2, 0, .6, -1.4". Modulating potential levels 952-958 with the second code 934 produces a series of modulation values ​​"2, 0, -.6, 1.4". The third code 936 modulates to produce “2, 0, -.6, -1.4”, and the fourth code 938 modulates to produce “2, 0, .6, 1.4”.

[0099] Next, as indicated by the horizontal arrow, each series of modulation values ​​is summed to produce one of the analog values ​​902-908. For example, summing the first series yields the analog value "1.2" (which becomes ".3" after being normalized with a scaling factor of "4"). Similarly, the modulation values ​​of the other three series are summed to obtain the analog values ​​"2.8", "0", and "4", and normalized to obtain the output vector of analog values ​​902-908. Each code can modulate the input level and then sum the series, or all codes can modulate the input level before summing each series. Therefore, the output vector of N analog values ​​902-908 has been transmitted in parallel using L output levels.

[0100] These examples do not show any instance of decoding the digital input voltage level, but those skilled in the art will find that performing such decoding is straightforward after reading the encoding of the digital values ​​described above.

[0101] Figures 17A, 17B, and 17C illustrate that encoders and decoders can operate on analog or digital samples; various analog and digital encoders and decoders have been described previously. As mentioned above, there can be more than one EM path, and therefore more than one encoder / decoder pair and a corresponding number of DACs or ADCs, depending on the situation.

[0102] Figure 17A illustrates the use of an analog encoder and a corresponding analog decoder. The input to the analog encoder 900 is either an analog sample 970 or a digital sample 971 converted to analog by the DAC 972 located in the analog encoder. In this way, the analog or digital sample arriving at the analog encoder can be encoded for transmission via an electromagnetic path over the transmission medium 34. The analog decoder 900' decodes the encoded analog sample to produce the analog sample 970 for output. The analog sample 970 can be used as is, or it can be converted to a digital sample using an ADC (not shown).

[0103] Figure 17B illustrates the use of a digital encoder and a corresponding analog decoder. The input to the digital encoder 901 is either a digital sample 971 or an analog sample 970 that has been converted to digital by the ADC 973 located at the digital encoder. Since the encoder is digital, the DAC 959 located at the encoder converts the encoded sample to analog before transmission via the electromagnetic path. In this way, analog or digital samples arriving at the digital encoder can be encoded for transmission via the electromagnetic path on the transmission medium 34. The analog decoder 900' decodes the encoded analog sample to produce an analog sample 970 for output. The analog sample 970 can be used as is, or it can be converted to digital using an ADC (not shown).

[0104] Figure 17C illustrates the decoding of an encoded analog signal arriving via an electromagnetic path on transmission medium 34 using a digital decoder. The encoded analog signal can be transmitted using either the analog encoder or the digital encoder described above. An ADC 974 located at the digital decoder 976 receives the encoded analog samples transmitted via the electromagnetic path and converts the samples into digital samples. These encoded digital samples are then decoded by the digital decoder 976 into digital samples 978 (corresponding to the values ​​of the input vector of the samples initially encoded before transmission via the electromagnetic path). Digital samples 978 can be used as is, or they can be converted into analog samples using a DAC.

[0105] Figure 18 illustrates the analog (similar to an idealized oscilloscope trace) of the SSVT waveform 602 transmitted via an electromagnetic path after being output from an analog encoder (or digitally encoded and then converted by a DAC). The vertical scale represents voltage, and the horizontal scale represents 100 ps oscilloscope measurement time intervals. Note that the SSVT signal 602 is an analog waveform, not a digital signal (i.e., the signal does not represent a binary digit), and in this embodiment, a voltage range from approximately -15V to approximately +15V can be transmitted. The voltage values ​​of the analog waveform are (or at least can be) fully analog. Furthermore, the voltage is not limited to a certain maximum value, although high values ​​are impractical.

[0106] As previously described, analog voltage levels are transmitted sequentially via an electromagnetic path, each level being the sum of modulated samples for each time interval, such as the analog output levels 952-958 described above or the digital output levels 952'-958' described above (after passing through the DAC). When transmitted, these output levels are then displayed as waveforms, such as waveform 602. In particular, voltage level 980 represents the sum of modulated samples within a specific time interval (i.e., the output level). To give a simple example, sequential voltage levels 980-986 represent the transmission of four output levels. In this example, 32 codes are used, meaning 32 samples can be transmitted in parallel; therefore, voltage levels 980-986 (followed by many subsequent voltage levels, depending on the number of chips in the code, L) form the parallel transmission of 32 coded samples (such as pixel voltages from a video source). After this transmission, the next set of L voltage levels in waveform 602 represents the transmission of the next 32 samples. Typically, waveform 602 represents encoding analog or digital values ​​into analog output levels and transmitting these levels at discrete time intervals to form a composite analog waveform.

[0107] Due to phenomena such as attenuation, reflections caused by impedance mismatch, and intrusive signal impact, each electromagnetic path degrades the electromagnetic signal propagating through it. Therefore, measurements of the input potential level at the receiver are always inaccurate relative to the corresponding output potential level available at the transmitter. Thus, as is known in the art, scaling of the input potential level at the receiver (or normalization or amplification of the output potential level at the transmitter) can be performed to compensate for this. Furthermore, due to process gain (i.e., increased electroelasticity due to the increase in L), the decoded input potential level at the decoder is normalized by a scaling factor using the code length to recover the transmitted output potential level, as is known in the art. SSVT-coded chemical and tactile samples

[0108] While the foregoing description describes video samples encoded and transmitted via SSVT for displaying images on a VR headset panel, this document also references chemical or tactile samples that can also be encoded and transmitted via SSVT. In other words, any sample value arriving at SSVT transmitter 82 or the SSVT transmitter that generates the SSVT signal received at head-mounted device processor 60 (e.g., arriving via access points 72 or 74, via paths 808, 828, 848, etc.) can represent a chemical (such as an odor) or a tactile (such as touch). For example, the analog sample value 902 in Figure 14 is described as a video sample representing light, but this particular sample value could represent a chemical or tactile sensation. By convention, certain sample locations within the frame of the video (e.g., the last dozen or so positions in the first row of the frame) can be predetermined to store chemical or tactile sample values ​​instead of video sample values ​​between the SSVT transmitter and SSVT receiver (or between the video source and the target processor).

[0109] For example, sample values ​​representing chemical substances associated with the scent of a particular tree (e.g., a value ".1" for eucalyptus, a value ".2" for gerberas, etc.) can be embedded in a frame or multiple frames, where an image of the tree appears on the VR headset's display in the video stream presented to the user. When the user wearing the VR headset turns to look at the tree, approaches the tree, attempts to touch the tree, etc., the headset processor or VR headset can use those chemical sample values ​​sent via the SSVT to synthesize the scent of that tree at that moment. An olfactory sensor associated with the VR headset then presents the synthesized scent to the user via a mask with a controllable flow valve. An example of such a mask for delivering synthesized scents is described in "The Smell Engine: a System for Artificial Odor Synthesis in Virtual Environments," presented at the IEEE Virtual Reality and 3D User Interface (VR) Conference 2022. Other scents can include: fresh water, polluted water, smoke, multiple odors, etc. Advantageously, the identification of specific odors is embedded together with the SSVT-coded images of the objects that produce these odors, so as to combine the objects viewed by the user on the VR mask to synthesize specific odors.

[0110] Furthermore, sample values ​​representing tactile sensations (e.g., touch) associated with a specific tree (e.g., a value ".6" for smooth eucalyptus bark, a value ".7" for rough eucalyptus bark, etc.) can be embedded in a frame or multiple frames, where an image of the tree appears on the VR headset's display in the video stream presented to the user. When a user wearing the VR headset attempts to touch a tree, the headset processor or VR headset can use these tactile sample values ​​to reproduce the sensation on the surface the user actually touches (e.g., a haptic pad, joystick, handheld controller, etc.) or on a surface near the user's hand. Other examples of tactile sensations that can be transmitted via SSVT-encoded sample values ​​include: heat, cold, wind, wetness, dryness, etc. Advantageously, the recognition of a specific tactile sensation is embedded along with SSVT-encoded images of the objects that generate these sensations, facilitating the easy synthesis of specific tactile sensations by combining them with objects viewed or touched by the user on the VR headset.

[0111] 10: Head-mounted device 20: Face mask 32: Left monitor 34: Right monitor, display panel, transmission medium 60: Processor 62: Core AI / ML Module 64: GPU 66:SSVT storage 72: RF Access Point 74: USB port 82, 540: SSVT transmitter 92, 592, 652-654: SSVT signal 142: Encoder 200, 684: Flowchart 202: Wireless Transmitter 204, 208: Antenna 206: Receiver 210: Modulator 212: Carrier frequency generator 214: Bandpass filter 216: Summation node 218: Power Amplifier 220: Gain Controller 222: Demodulator 224: Low-pass filter 226: Discriminator Circuit 228: Voltage-Controlled Frequency Source (VCFS) 230, 634: Output 440: Distributor 450: Assembly Library 452, 786: Hierarchical Library 454: indicates a library 456: Controller 460: Encoder Block 462, 959, 972: Digital-to-Analog Converters (DACs) 465: Sequencer Circuit 560: Gate Driver 582: Rigid PCB 584: Flexible PCB 586, 812-818, 832-838, 852-858: Source drivers 590, 608: Signals 602: SSVT waveform, SSVT signal 604: Backlight signal 606: Gate driver control signal 610: SSVT Receiver 612: Analogous sample stream, analogous output 620: Potential Quasi-Shifter 622: Inverted signal 632: Latch signal 656-658, 780: Decoder 662-664: Collectors 666-668: Amplifier 670: Analog Potential Standard 680: Chip Counter 682, 920: Codebook 688: Output analog sample 690-692, 970: Analogous Samples 698: Stored Samples 702-704: EM signal 782: Rebuild the library 787: Channel Aligner 789: Hierarchical Controller 800: GPU Direct Display Platform Example 802: Custom Printed Circuit Board 804, 824: Current generation (or "traditional") GPU ICs 808, 828, 848: Path 820: GPU Direct Display Module on System (SoM) Example 822:SoM 840: Fully Integrated GPU Direct Display Example 844: Enhanced GPU 900: Analog Encoder 900': Analog Decoder 901: Digital Encoder 902-908: Analog Values 902'-908': Place value 932, 934, 936-938: Codes 940: Time interval 942, 944: Code representation 948, 961: Modulation 950: Input potential accuracy 950': Output potential accuracy 952-958: Analog Output Potential Standard 952'-958': Digital output potential standard 971, 978: Digital Samples 973, 974: ADC 976: Digital Decoder EM: Electromagnetic SSVT: Spread Spectrum Video Transmission VR: Virtual Reality

Claims

1. A virtual reality (VR) head-mounted device, comprising: A head-mounted device processor includes a transmitter arranged to receive a video sample stream, encode each of N consecutive groups of the video samples into a set of L output levels, and transmit the set of L output levels as an analog waveform via an electromagnetic path, where L >= N >= 2; and a VR mask including at least one display having at least one source driver, the source driver including: a receiver arranged to receive the set of L output levels of the analog waveform from the transmitter; a decoder arranged to decode each set of L output levels into an output vector of N analog samples; and a buffer arranged to collect the output vector of each of the N analog samples from the decoder and output the output vectors of the N analog samples in parallel, each analog sample being output to a column of the display, wherein the video sample stream is substantially displayed on the display.

2. The VR headset as claimed in claim 1, wherein encoding each group of N video samples into a set of L output levels is performed with reference to a predetermined code set of N mutually orthogonal codes, each code having a length of L, each of the codes being used to encode one of the N video samples, and wherein the decoding is performed with reference to the predetermined code set of N mutually orthogonal codes.

3. The VR headset as claimed in claim 1, wherein the source driver further comprises: Multiple amplifiers are arranged to amplify the analog sample of the output vector output from the buffer before it is output to the display.

4. The VR headset as claimed in claim 1, wherein the source driver further comprises: Multiple amplifiers are arranged to amplify the analog sample of the output vector from the decoder before it is input to the buffer.

5. The VR headset as claimed in claim 1, wherein the source driver does not include a digital-to-analog converter (DAC) for converting digital pixel data into analog pixel data.

6. The VR headset as claimed in claim 1, wherein the display panel comprises C columns, wherein the group of L output levels is received in series at a frequency freq(SSVT), wherein each of the output vectors of N analog samples is output from the decoder at a frequency freq(sample) = freq(SSVT) / N, and wherein the buffer outputs the output vectors of N analog samples in parallel at a frequency freq(line) = freq(sample) x N / C.

7. The VR headset as claimed in claim 1, wherein the set of L output potential levels is received in series at the receiver, and wherein each output vector of the N analog samples is output in parallel from the decoder.

8. The VR headset as claimed in claim 1, wherein the transmitter further transmits the set of L output potential levels as a second analog waveform via a second electromagnetic path, and wherein the mask further includes a second display having a second source driver that receives the set of L output potential levels from the second analog waveform.

9. The VR headset as claimed in claim 1, wherein the headset processor further includes a wireless transmitter arranged to transmit the analog waveform in an RF electromagnetic path, and wherein the VR mask further includes a wireless receiver arranged to receive the analog waveform.

10. The VR headset of claim 1, wherein one of the video samples has a value representing a chemical odor or a tactile sensation, the VR headset further comprising means for reproducing the chemical odor or the tactile sensation for a user based on a value of one of the N analog samples corresponding to the one in the video sample.

11. A virtual reality (VR) head-mounted device, comprising: A head-mounted device processor includes a wireless receiver arranged to receive an RF analog waveform comprising a group of L output levels representing a video stream, each group of L output levels being an encoded form of N video samples, where L >= N >= 2; an electromagnetic path through which the group of L output levels is transmitted as an analog waveform from the head-mounted device processor; and a VR headset including at least one display having at least one source driver, the source driver comprising: a receiver arranged to receive the group of L output levels of the analog waveform from the head-mounted device processor; a decoder arranged to decode each group of L output levels into an output vector of N analog samples; and a buffer arranged to collect the output vector of each of the N analog samples from the decoder and output the output vectors of the N analog samples in parallel, each of the analog samples being output to a column of the display, wherein the video sample stream is substantially displayed on the display.

12. The VR headset as claimed in claim 11, wherein the electromagnetic path is a wired path.

13. The VR headset as claimed in claim 11, wherein the electromagnetic path is a wireless path, the headset processor further includes a wireless transmitter arranged to transmit the set of L output potential levels to the receiver, and wherein the VR headset further includes a wireless receiver arranged to receive the set of L output potential levels and input the set of L output potential levels to the receiver.

14. The VR headset as claimed in claim 11, wherein the decoding of each of the L output levels is performed with reference to a predetermined set of N mutually orthogonal codes, each code having a length of L, and each code being used to decode one of the N video samples.

15. The VR headset as claimed in claim 11, wherein the source driver further comprises: Multiple amplifiers are arranged to amplify the analog sample of the output vector output from the buffer before it is output to the display.

16. The VR headset as claimed in claim 11, wherein the source driver further includes a plurality of amplifiers arranged to amplify the analog sample of the output vector output from the decoder before being input to the buffer.

17. The VR headset as claimed in claim 11, wherein the source driver does not include a digital-to-analog converter (DAC) for converting digital pixel data into analog pixel data.

18. The VR headset of claim 11, wherein the display panel comprises C columns, wherein the group of L output levels is received in series at a frequency (SSVT), wherein each of the output vectors of N analog samples is output from the decoder at a frequency freq(sample) = freq(SSVT) / N, and wherein the buffer outputs the output vectors of N analog samples in parallel at a frequency freq(line) = freq(sample) x N / C.

19. The VR headset as claimed in claim 11, wherein the set of L output potential levels is received in series at the receiver, and each output vector of N analog samples is output in parallel from the decoder.

20. The VR headset as claimed in claim 11, further comprising: A second electromagnetic path is provided, through which the set of L output potential levels is sent from the head-mounted device processor as a second analog waveform, wherein the VR mask further includes a second display having a second source driver that receives the set of L output potential levels from the second analog waveform.

21. The VR headset as claimed in claim 11, wherein one of the N video samples has a value representing a chemical odor or a tactile sensation, the VR headset further comprising means for reproducing the chemical odor or the tactile sensation for a user based on a value of one of the N analog samples corresponding to the one of the video samples.

Images