A precise extended reality system based on circularly polarized signals

Abstract

The application is suitable for the cross field of circularly polarized light and extended reality technology, and provides a precise extended reality system based on a circularly polarized signal, which comprises a circularly polarized emission module, a circularly polarized receiving module, a wearable display module, an extended reality processing module and a reality interaction module; the circularly polarized receiving module comprises a circularly polarized lens group, a light splitting prism, a CMOS / CCD camera and a differential processing system; the wearable display module is virtual reality glasses with a position tracking and environment sensing function; the extended reality processing module comprises a hand tracking unit, a scene mapping unit and a virtual-real calibration unit; and the reality interaction module is a mechanical arm and a mechanical hand with a multi-degree-of-freedom steering engine. The application effectively improves the anti-interference performance and precision of hand signal extraction, and effectively solves the problems of serious signal interference and low signal extraction accuracy of the extended reality system in a complex interference environment.

Description

Technical Field

[0001] This invention relates to the intersection of circularly polarized light emission and extended reality technology, and more particularly to a precise extended reality system based on circularly polarized signals. Background Technology

[0002] Extended reality (XR) technology combines virtual information with reality to provide users with an immersive experience, and it has great potential for development in fields such as gaming, education, healthcare, and industrial design. Extended reality technology refers to virtual reality (VR), augmented reality (AR), and mixed reality (MR). However, existing extended reality systems still have limitations in the accuracy of interactive control, especially in complex environments where the accuracy rate is difficult to reach 100%.

[0003] Traditional extended reality systems typically use intensity or wavelength as the information carrier for optical signals. These methods are inevitably affected by the external environment during information transmission, leading to control deviations. Furthermore, the calibration methods for traditional systems using interactive devices are quite cumbersome.

[0004] In recent years, circularly polarized light technology has shown great potential in the fields of optical communication and display. Because it has two orthogonal intensity distributions after phase delay, it can provide information in another dimension, thereby improving the system's information transmission capability and anti-interference ability. However, circularly polarized signals have not yet been used as the main signal propagation carrier in extended reality systems, nor have high-precision, anti-interference extended reality systems been achieved by utilizing the characteristics of circularly polarized signals.

[0005] Therefore, developing a precise extended reality system based on high-dimensional circularly polarized signals can utilize the high-dimensional characteristics of circularly polarized signals, which have advantages such as highly robust hand recognition, high anti-interference interactive perception, and highly immersive real-time virtual-real fusion. It is particularly suitable for application scenarios such as virtual reality (VR), augmented reality (AR), and mixed reality (MR) where signal accuracy and interactive effectiveness are extremely important, and has significant theoretical and practical implications. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the purpose of this invention is to propose a precise extended reality system based on circularly polarized signals, which aims to solve the technical problems mentioned in the background art.

[0007] A precise extended reality system based on circularly polarized signals, the extended reality system comprising a circularly polarized transmitting module, a circularly polarized receiving module, a wearable display module, an extended reality processing module, and a reality interaction module;

[0008] The operator wears a circularly polarized emission module (gloves) and a wearable display module (XR glasses). Hand movements drive the circularly polarized emission module to emit circularly polarized signals carrying hand posture information. The circularly polarized receiving module receives this signal and performs differential processing to eliminate noise, obtaining pure hand posture data. The extended reality processing module analyzes the hand posture data in real time to obtain an accurate hand skeletal model, and then fuses and calibrates it with a digital twin model of a virtual or real-world scene. The fused virtual-real scene is displayed to the operator by the wearable display module. Simultaneously, the obtained operation command information is output to the reality interaction module (robotic arm), which drives physical reality interaction. The operator obtains the immediate results of the physical interaction through the XR glasses and receives visual feedback, thus forming a low-latency closed-loop interactive operation.

[0009] The circular polarization emission module is a left / right circular polarization emission glove worn on the operator's hand, used to output circular polarization signals from the operator's hand in real time;

[0010] The circular polarization receiving module includes: a circular polarization lens group, a beam splitter prism, a CMOS / CCD camera, and a differential processing system, which identifies and separates left / right circularly polarized signals and significantly improves the signal noise ratio through a differential algorithm;

[0011] The wearable display module is a virtual reality glasses with position tracking and environmental awareness functions, supporting spatial tracking and immersive display;

[0012] The extended reality processing module includes a hand tracking unit, a scene mapping unit, and a virtual-real calibration unit to achieve high-precision extended reality effects;

[0013] The real-world interaction module consists of a robotic arm and a robotic hand equipped with multi-degree-of-freedom servo motors, responsible for completing the interaction in the physical world.

[0014] Furthermore, the orthogonal decomposition and noise reduction process of circularly polarized signals is based on the following principles:

[0015] Circularly polarized signal This can be represented as the superposition of two orthogonally linearly polarized signals:

[0016] ;

[0017] Among them, E x (t) and E y (t) represent the electric field components in the x and y directions, respectively, and satisfy:

[0018] ;

[0019] Where T is the signal period.

[0020] During transmission, the circularly polarized signal becomes [distorted / inverted] after being interfered with by noise. :

[0021] ;

[0022] Where, n x (t) and n y (t) represents the noise component. In nature, the noise component does not exhibit circular polarization characteristics, therefore n x (t) and n y (t) The noise components are equal;

[0023] By processing with a circular polarization difference system and after phase delay, the noise components n in the x and y directions can be separated. x (t) and n y (t) Subtraction and removal effectively separates the target signal, restoring it to the original circularly polarized signal, achieving the purpose of physical noise reduction and effectively improving the signal-to-noise ratio.

[0024] Furthermore, the circular polarization emission module is a left-handed or right-handed circular polarization emission wearable glove with a wavelength range of 350-1500nm and stable high-dimensional circular polarization emission function, which can output circular polarization signals with different characteristic parameters according to the operator's hand gestures.

[0025] Preferably, the left-handed or right-handed circularly polarized emission glove needs to cover all key nodes in the finger joints and palm area, such as the fingertips, finger joints and the center of the palm, to accurately correspond to the key points of the hand bones.

[0026] Preferably, the left-handed or right-handed circularly polarized emission glove has a circular polarization luminescence asymmetry factor glum ≥ 0.3 to ensure a high-purity circularly polarized state and suppress background incident interference.

[0027] Preferably, the circularly polarized light can be generated based on chiral materials. Chiral quantum dots, chiral perovskite nanocrystals, or chiral conjugated polymers can be used as luminescent materials to directly generate circularly polarized light with a high asymmetry factor.

[0028] Preferably, the circularly polarized light is generated based on micro / nano optical structures. Integrating a linearly polarized light-emitting unit with a miniaturized quarter-wave plate or chiral metasurface converts linearly polarized light into circularly polarized light.

[0029] Furthermore, the circularly polarized lens component in the circularly polarized receiving module is composed of a left-handed circularly polarized filter and a right-handed circularly polarized filter;

[0030] Preferably, the left-hand circularly polarizing filter is used to transmit left-hand circularly polarized light and block right-hand circularly polarized light; the right-hand circularly polarizing filter is used to transmit right-hand circularly polarized light and block left-hand circularly polarized light.

[0031] Preferably, the operating wavelength of the circular polarizing filter is selected from 350-1500 nm.

[0032] The beam splitter is used to split the signal into two beams of equal intensity.

[0033] Preferably, the beam splitter is a non-polarizing beam splitter, which decomposes the received mixed light signal into two identical beams of equal intensity, which are then directed toward the two subsequent imaging channels.

[0034] The CMOS / CCD camera is an imaging camera with visible light wavelengths, used to capture light intensity signals in the x and y directions, respectively.

[0035] Preferably, the CMOS / CCD camera is selected from a global shutter CMOS camera or a CCD camera.

[0036] Preferably, the CMOS / CCD camera has a frame rate of ≥30 fps.

[0037] Preferably, the CMOS / CCD camera has a resolution greater than 1920×1080.

[0038] Furthermore, the differential processing system performs differential processing on the light intensity matrix captured by the CMOS / CCD camera. Utilizing the orthogonality of circularly polarized signals, noise is canceled out during the differential calculation process, achieving separation of the target signal and interference signal, and reducing the circularly polarized emission asymmetry factor. The calculation formula is as follows:

[0039] ;

[0040] This represents the image light intensity value of a pixel in a left-handed image. This represents the image light intensity value of a pixel in a right-handed image.

[0041] Preferably, the maximum and minimum values ​​of the circularly polarized emission asymmetry factor are controlled within [-2, 2] to prevent data overflow.

[0042] Furthermore, the wearable display module is a virtual reality glasses with position tracking and environmental awareness functions, which realizes high-precision acquisition and modeling of the user's head movement and the surrounding environment, and is used to present a virtual-real fusion scene. It requires a display latency of ≤20ms, a field of view of ≥100°, and integrates a head tracking sensor.

[0043] Preferably, the wearable display module needs to integrate TOF / LiDAR environmental perception and binocular cameras to assist in environmental modeling and virtual-real fusion.

[0044] Preferably, the wearable display module needs to support adjustable focus optics and refractive compensation to improve comfort during long-term wear.

[0045] Preferably, the wearable display module can be a high-resolution screen such as an OLED to enhance the immersive experience.

[0046] Furthermore, the extended reality processing module includes: a hand tracking unit, a scene mapping unit, and a virtual-real calibration unit, which are used to connect the circular polarization receiving module and the wearable display module, and are used to analyze hand spatial pose data in real time, construct a three-dimensional model of the environment, and generate synchronized extended reality content;

[0047] The hand tracking unit is based on the circularly polarized light emission asymmetry factor in the differential processing system, and extracts a multi-degree-of-freedom hand skeleton model from it through multi-point solution and inverse kinematics algorithm.

[0048] Preferably, the hand tracking unit is a Mediapipe gesture recognition module, which can recognize multiple key gesture points based on circularly polarized signal images.

[0049] The scene mapping unit is used to create a virtual environment through Unreal Engine or to construct a real-time environment using a stereo camera.

[0050] Preferably, the virtual scene in the scene mapping unit is generated using a rendering engine such as Unreal Engine or Unity to achieve high-fidelity virtual scene generation.

[0051] Preferably, the mixed reality scene in the scene mapping unit is constructed by using an RGB camera and depth sensor (such as ToF or structured light) on the wearable display module, and a real-time visual synchronous positioning and mapping algorithm to build a high-precision three-dimensional mesh model of the physical environment.

[0052] The virtual-real calibration unit uses the ICP algorithm to align the virtual model with the physical space coordinates.

[0053] Preferably, the virtual-real calibration unit uses multi-level correction algorithms such as ICP, RGB-D registration, and automatic calibration board matching to improve alignment accuracy.

[0054] Preferably, during initial calibration, a rapid coarse alignment can be performed using a method based on known markers, followed by fine-tuning using the ICP algorithm.

[0055] Furthermore, the reality interaction module is a robotic arm and manipulator with multi-degree-of-freedom servo motors, and its input end is connected to the extended reality processing module to execute physical interaction actions according to the parsed operation instructions.

[0056] Preferably, the robotic arm and robotic hand are 6-axis or 7-axis collaborative robots, and the robotic hand is a multi-finger bionic dexterous hand, with each finger having multiple joints, capable of performing fine movements such as grasping, holding, and pinching.

[0057] Preferably, the ends or fingertips of the robotic arm and hand are integrated with tactile or torque sensors. These sensors can detect information such as contact force and pressure distribution with an object.

[0058] Furthermore, the extended reality system includes virtual reality, augmented reality, or mixed reality domains to provide an accurate visual experience.

[0059] Furthermore, the extended reality system establishes an operation closed loop, wherein the operation closed loop is: hand movement - circular polarization signal acquisition - virtual and real scene generation - real interaction - visual feedback to virtual reality glasses, and the closed loop delay is ≤50ms.

[0060] Preferably, the extended reality system can be applied to security and rescue, telemedicine equipment, precision manufacturing, immersive education, and highly immersive gaming entertainment, etc.

[0061] Preferably, the extended reality system is suitable for working conditions where traditional optical hand recognition is prone to failure, such as high light pollution and high disturbance environments.

[0062] The beneficial effects achieved by this invention are as follows:

[0063] 1. High precision: The system uses circularly polarized light as the carrier and differential measurement to eliminate interference from non-polarized and linearly polarized noise such as ambient light and stray light. It can still obtain stable and high-precision tracking results under external interference conditions such as complex changes in lighting, and is not affected by background characteristics such as color and texture.

[0064] 2. Ultra-low computational load: Compared to complex image recognition algorithms, the differential signal processing flow of this invention has a very small computational load and can be completed at high speed on low-power hardware, which greatly reduces the latency and computational power consumption of the hand tracking process. This allows more computing resources to be allocated to scene rendering and physical simulation, thus improving the overall XR experience.

[0065] 3. Achieves highly immersive closed-loop interaction: This invention not only meets the needs of accurate input and high-quality display, but also, with the help of the real-world interaction module and optional force feedback, closely links virtual operations with real-world results to form a closed loop, achieving multi-sensory, low-latency immersive interaction involving vision, hearing, and touch.

[0066] 4. Wide range of applications and strong scalability: The modular design can be configured according to different uses, allowing for the improvement, modification and replacement of various system components. For example, the transmitter can be integrated into the work tool, and the interactive component can be a drone or vehicle, enabling it to be flexibly applied to many fields such as industry, medical care, military, and entertainment. Attached Figure Description

[0067] Figure 1 A flowchart of a precise extended reality system based on circularly polarized signals provided by the present invention;

[0068] Figure 2 This is a virtual reality (VR) gesture demonstration of the precise extended reality system based on circularly polarized signals obtained in Experiment Example 1 of this invention;

[0069] Figure 3 The image shown is from VR glasses in the precise extended reality system based on circularly polarized signals obtained in Experiment Example 1 of this invention.

[0070] Figure 4 This is an augmented reality (AR) demonstration of the precise extended reality system based on circularly polarized signals obtained in Experiment Example 2 of this invention;

[0071] Figure 5 This is a mixed reality (MR) demonstration of the precise extended reality system based on circularly polarized signals obtained in Experiment Example 3 of this invention;

[0072] Figure 6 The mixed reality (MR) glasses image of the precise extended reality system based on circularly polarized signals obtained in Experiment Example 3 of this invention. Detailed Implementation

[0073] This invention provides a precise extended reality system based on circularly polarized signals. The operator wears circularly polarized emission gloves to make gestures; the signals, after differential noise reduction, are input into the ROS platform and parsed into control commands to drive a robotic arm to perform actions. Simultaneously, the system monitors the glasses to provide real-time feedback on the execution status. To make the objectives and technical solutions of this invention clearer, the invention is further described below with reference to specific embodiments. However, the following descriptions are merely illustrative and should not be construed as limiting the scope of protection of this invention.

[0074] Please see Figure 1 This invention provides a precise extended reality system based on circularly polarized signals. The extended reality system includes a circularly polarized transmission module, a circularly polarized reception module, a wearable display module, an extended reality processing module, and a reality interaction module.

[0075] The circular polarization emission module is a left / right circular polarization emission glove worn on the operator's hand, used to output circular polarization signals from the operator's hand in real time;

[0076] The circular polarization receiving module includes: a circular polarization lens group, a beam splitter prism, a CMOS / CCD camera, and a differential processing system, which identifies and separates left / right circularly polarized signals and significantly improves the signal noise ratio through a differential algorithm;

[0077] The wearable display module is a virtual reality glasses with position tracking and environmental awareness functions, supporting spatial tracking and immersive display;

[0078] The extended reality processing module includes a hand tracking unit, a scene mapping unit, and a virtual-real calibration unit to achieve high-precision extended reality effects;

[0079] The real-world interaction module consists of a robotic arm and a robotic hand equipped with multi-degree-of-freedom servo motors, responsible for completing the interaction in the physical world.

[0080] In this embodiment, the orthogonal decomposition and noise reduction process of the circularly polarized signal is based on the following principle:

[0081] Circularly polarized signal This can be represented as the superposition of two orthogonally linearly polarized signals:

[0082] ;

[0083] Among them, E x (t) and E y (t) represent the electric field components in the x and y directions, respectively, and satisfy:

[0084] ;

[0085] Where T is the signal period.

[0086] During transmission, the circularly polarized signal becomes [distorted / inverted] after being interfered with by noise. :

[0087] ;

[0088] Where, n x (t) and n y (t) represents the noise component. In nature, the noise component does not exhibit circular polarization characteristics, therefore n x (t) and n y (t) The noise components are equal;

[0089] By processing with a circular polarization difference system and after phase delay, the noise components n in the x and y directions can be separated. x (t) and n y (t) Subtraction and removal effectively separates the target signal, restoring it to the original circularly polarized signal, achieving the purpose of physical noise reduction and effectively improving the signal-to-noise ratio.

[0090] In this embodiment, the circular polarization emission module is a left-handed or right-handed circular polarization emission wearable glove with a wavelength range of 350-1500nm and stable high-dimensional circular polarization emission function. It can output circular polarization signals with different characteristic parameters according to the operator's hand gestures.

[0091] In this embodiment, the left-handed or right-handed circularly polarized emission glove needs to cover all key nodes in the finger joints and palm area, such as the fingertips, finger joints and the center of the palm, in order to accurately correspond to the key points of the hand bones.

[0092] In this embodiment, the circular polarization emission glove or the right-hand circular polarization emission glove has a circular polarization emission asymmetry factor glum ≥ 0.3, ensuring a high-purity circular polarization state and suppressing background incident interference.

[0093] In this embodiment, the circularly polarized light can be generated based on chiral materials. Chiral quantum dots, chiral perovskite nanocrystals, or chiral conjugated polymers can be used as luminescent materials to directly generate circularly polarized light with a high asymmetry factor.

[0094] In this embodiment, the circularly polarized light can be generated based on micro / nano optical structures. Integrating a linearly polarized light-emitting unit with a miniaturized quarter-wave plate or chiral metasurface converts linearly polarized light into circularly polarized light.

[0095] In this embodiment, the circular polarization lens component in the circular polarization receiving module is composed of a left-handed circular polarization filter and a right-handed circular polarization filter;

[0096] In this embodiment, the left-hand circularly polarized filter is used to transmit left-hand circularly polarized light and block right-hand circularly polarized light; the right-hand circularly polarized filter is used to transmit right-hand circularly polarized light and block left-hand circularly polarized light.

[0097] In this embodiment, the working wavelength of the circular polarizing filter is selected from 350-1500 nm.

[0098] The beam splitter is used to split the signal into two beams of equal intensity.

[0099] In this embodiment, the beam splitter is a non-polarizing beam splitter, which decomposes the received mixed light signal into two identical beams of equal intensity, and directs them to the two subsequent imaging channels respectively.

[0100] The CMOS / CCD camera is an imaging camera with visible light wavelengths, used to capture light intensity signals in the x and y directions, respectively.

[0101] In this embodiment, the CMOS / CCD camera is selected from a global shutter CMOS camera or a CCD camera.

[0102] In this embodiment, the CMOS / CCD camera has a frame rate of ≥30 fps.

[0103] In this embodiment, the CMOS / CCD camera has a resolution greater than 1920×1080.

[0104] In this embodiment, the differential processing system performs differential processing on the light intensity matrix captured by the CMOS / CCD camera. Utilizing the orthogonality of circularly polarized signals, noise is canceled out during the differential calculation process, achieving separation of the target signal and interference signal, and reducing the circularly polarized emission asymmetry factor. The calculation formula is as follows:

[0105] ;

[0106] This represents the image light intensity value of a pixel in a left-handed image. This represents the image light intensity value of a pixel in a right-handed image.

[0107] In this embodiment, the maximum and minimum values ​​of the circularly polarized emission asymmetry factor are controlled within [-2,2] to prevent data overflow.

[0108] In this embodiment, the wearable display module is a virtual reality glasses with position tracking and environmental perception functions, which realizes high-precision acquisition and modeling of the user's head movement and the surrounding environment, and is used to present a virtual-real fusion scene. It requires a display latency of ≤20ms, a field of view of ≥100°, and integrates a head tracking sensor.

[0109] In this embodiment, the wearable display module needs to integrate TOF / LiDAR environmental perception and binocular cameras to assist in environmental modeling and virtual-real fusion.

[0110] In this embodiment, the wearable display module needs to support adjustable focus optics and refractive compensation to improve comfort during long-term wear.

[0111] In this embodiment, the wearable display module may use a high-resolution screen such as an OLED to enhance the immersive experience.

[0112] In this embodiment, the extended reality processing module includes: a hand tracking unit, a scene mapping unit, and a virtual-real calibration unit, which are used to connect the circular polarization receiving module and the wearable display module, and are used to analyze hand spatial pose data in real time, construct a three-dimensional environmental model, and generate synchronized extended reality content;

[0113] The hand tracking unit is based on the circularly polarized light emission asymmetry factor in the differential processing system, and extracts a multi-degree-of-freedom hand skeleton model from it through multi-point solution and inverse kinematics algorithm.

[0114] Preferably, the hand tracking unit is a Mediapipe gesture recognition module, which can recognize multiple key gesture points based on circularly polarized signal images.

[0115] The scene mapping unit is used to create a virtual environment through Unreal Engine or to construct a real-time environment using a stereo camera.

[0116] In this embodiment, the virtual scene in the scene mapping unit is generated using rendering engines such as Unreal Engine or Unity to achieve high-fidelity virtual scene generation.

[0117] In this embodiment, the mixed reality scene in the scene mapping unit is constructed by using an RGB camera and depth sensor (such as ToF or structured light) on the wearable display module, and a real-time visual synchronous positioning and mapping algorithm to build a high-precision three-dimensional mesh model of the physical environment.

[0118] The virtual-real calibration unit uses the ICP algorithm to align the virtual model with the physical space coordinates.

[0119] In this embodiment, the virtual-real calibration unit preferably uses multi-level correction algorithms such as ICP, RGB-D registration, and automatic calibration board matching to improve alignment accuracy.

[0120] In this embodiment, during the initial calibration, a method based on known markers can be used for rapid coarse alignment, followed by fine-tuning using the ICP algorithm.

[0121] In this embodiment, the reality interaction module is a robotic arm and robotic hand with multi-degree-of-freedom servo motors. Its input end is connected to the extended reality processing module, which executes physical interaction actions according to the parsed operation instructions.

[0122] In this embodiment, the robotic arm and robotic hand are 6-axis or 7-axis collaborative robots. The robotic hand is a multi-finger bionic dexterous hand, with each finger having multiple joints, capable of performing fine movements such as grasping, holding, and pinching.

[0123] In this embodiment, the ends or fingertips of the robotic arm and robotic hand are integrated with tactile or torque sensors. These sensors can detect information such as contact force and pressure distribution with objects.

[0124] In this embodiment, the extended reality system includes virtual reality, augmented reality, or mixed reality domains, used to provide an accurate visual experience.

[0125] In this embodiment, the extended reality system establishes an operation closed loop, which consists of: hand movement - circular polarization signal acquisition - virtual and real scene generation - real interaction - visual feedback to virtual reality glasses, and the closed loop delay is ≤50ms.

[0126] In this embodiment, the extended reality system can be applied to security and rescue, remote medical equipment, precision manufacturing, immersive education, and highly immersive gaming entertainment, etc.

[0127] In this embodiment, the extended reality system is suitable for working conditions where traditional optical hand recognition is prone to failure, such as high light pollution and high disturbance environments.

[0128] To facilitate a better understanding of the technical solution of this invention by those skilled in the art, specific experimental examples of this invention are provided below:

[0129] Experiment Example 1 verifies the virtual reality system. The specific steps are as follows:

[0130] 1. System initialization and environment loading:

[0131] The operator, wearing circularly polarized emission gloves and a pair of GOOVIS VR glasses, starts the system, and Unity loads a pre-built virtual hand scene.

[0132] 2. Hand operation and signal transmission:

[0133] The operator extends his right hand, revealing five fingers. He is wearing a perovskite left-handed circularly polarized emission glove that emits a 530nm CPL signal carrying fine attitude information.

[0134] 3. Signal Processing and Attitude Analysis:

[0135] The circular polarization receiver module was run on a computer to capture the signal. A Thorlabs CP1L532 left-handed circular polarizer and a Thorlabs CP1R532 right-handed circular polarizer were used as phase decomposition devices. The signal was split using a commercial beam splitter prism and then captured by visible light cameras with left-handed and right-handed circular polarization channels. Differential analysis was performed according to the calculation steps for the emission asymmetry factor to obtain the circularly polarized image of the glove. The hand tracking unit then analyzed the differential image to extract a complete hand skeletal model and identified the gesture's posture information.

[0136] 4. Virtual Reality Hybrid Interaction:

[0137] In VR scenarios, operators see their virtual hands precisely replicate hand gestures (such as...). Figure 2 , Figure 3 ).

[0138] Experiment 2 verifies the augmented reality system, using the first three steps from Experiment 1, but with the following additional steps:

[0139] Environmental perception and digital information overlay:

[0140] AR glasses use ToF sensors and cameras to scan the real environment and build a 3D mesh model of it in real time. The extended reality processing platform then precisely registers this real-time model.

[0141] Augmented Reality Interaction:

[0142] In the AR scenario, the operator sees their virtual hand accurately recreating hand gestures and can perform simple interactions with the environment (such as...). Figure 4 ).

[0143] Experiment 3 verifies the mixed reality system, using the first three steps of Experiment 1, but with the following additional steps:

[0144] Building mixed reality scenarios:

[0145] Image data is transmitted to the operator via a real-time scanning device. The operator wears an MR head-mounted display that integrates the system of this invention.

[0146] Virtual reality hybrid interaction:

[0147] In the MR scene, the operator sees their virtual hand accurately reproduce the gesture.

[0148] Simultaneously, the operation command is transmitted via a low-latency network to a remote reality interaction module—a 6-axis robotic arm. The robotic arm performs a grasping motion in a synchronized manner (e.g., ...). Figure 5 ).

[0149] Closed-loop feedback:

[0150] Operators can clearly perceive the completion of tasks such as grasping through stereoscopic vision, achieving highly immersive and secure remote operation (e.g. Figure 6 ).

[0151] In summary, this invention, through innovative differential processing technology based on high-dimensional circularly polarized signals, fundamentally solves the problem of signal accuracy and gesture recognition accuracy in traditional optical gesture recognition systems affected by complex environments. It also cleverly achieves an extended reality closed loop experience of high-precision gesture input, highly immersive display, and seamless interaction with the real world. The above embodiments are merely preferred implementations of this invention; its modular design can flexibly meet the application needs of a wider range of scenarios such as security and rescue, precision manufacturing, and immersive education and entertainment, demonstrating significant application value. Any obvious modifications or substitutions made by those skilled in the art based on this invention should also be included within the scope of protection of this invention.

Claims

1. A precise extended reality system based on circularly polarized signals, characterized in that, The extended reality system includes a circularly polarized transmission module, a circularly polarized reception module, a wearable display module, an extended reality processing module, and a reality interaction module; The circular polarization emission module is a left-handed or right-handed circular polarization emission wearable glove worn on the operator's hand; The circular polarization receiving module includes: a circular polarization lens group, a beam splitter, a CMOS / CCD camera, and a differential processing system; The wearable display module is a virtual reality glasses with position tracking and environmental perception functions; The extended reality processing module includes: a hand tracking unit, a scene mapping unit, and a virtual-real calibration unit; The real-world interaction module consists of a robotic arm and a robotic hand equipped with multi-degree-of-freedom servo motors.

2. The precise extended reality system based on circularly polarized signals according to claim 1, characterized in that, The circular polarization emission module is a left-handed or right-handed circular polarization emission wearable glove with a wavelength range of 350-1500nm, which outputs circular polarization signals with different characteristic parameters according to the operator's hand gestures.

3. The precise extended reality system based on circularly polarized signals according to claim 1, characterized in that, The circular polarization lens component in the circular polarization receiving module consists of a left-handed circular polarization filter and a right-handed circular polarization filter; The beam splitter is used to split the signal into two beams of equal intensity. The CMOS / CCD camera is an imaging camera used to capture light intensity signals in the x and y directions, respectively.

4. A precise extended reality system based on circularly polarized signals according to claim 1, characterized in that, The differential processing system performs differential processing on the light intensity matrix captured by the CMOS / CCD camera. Utilizing the orthogonality of circularly polarized signals, noise is canceled out during the differential calculation process, achieving separation of the target signal and interference signal, and reducing the circularly polarized emission asymmetry factor. The calculation formula is as follows: ； This represents the image light intensity value of a pixel in a left-handed image. This represents the image light intensity value of a pixel in a right-handed image.

5. A precise extended reality system based on circularly polarized signals according to claim 1, characterized in that, The wearable display module is a virtual reality glasses with position tracking and environmental perception functions, which realizes high-precision acquisition and modeling of the user's head movement and the surrounding environment, and is used to present a virtual-real fusion scene. It requires a display latency of ≤20ms, a field of view of ≥100°, and integrates a head tracking sensor.

6. A precise extended reality system based on circularly polarized signals according to claim 1, characterized in that, The extended reality processing module includes a hand tracking unit, a scene mapping unit, and a virtual-real calibration unit, which are used to connect the circular polarization receiving module and the wearable display module, and are used to analyze hand spatial pose data in real time, construct a three-dimensional model of the environment, and generate synchronized extended reality content. The hand tracking unit extracts a multi-degree-of-freedom hand skeleton model based on the circularly polarized light emission asymmetry factor in the differential processing system. The scene mapping unit is used to create a virtual environment through Unreal Engine or to construct a real-time environment using a stereo camera. The virtual-real calibration unit uses the ICP algorithm to align the virtual model with the physical space coordinates.

7. A precise extended reality system based on circularly polarized signals according to claim 1, characterized in that, The reality interaction module is a robotic arm and a robotic hand with multi-degree-of-freedom servo motors. Its input end is connected to the extended reality processing module, which executes physical interaction actions according to the parsed operation instructions.

8. A precise extended reality system based on circularly polarized signals according to claim 1, characterized in that, The extended reality system includes the fields of virtual reality, augmented reality, or mixed reality.

9. A precise extended reality system based on circularly polarized signals according to claim 1, characterized in that, The extended reality system establishes an operation closed loop, which consists of: hand movement - circular polarization signal acquisition - virtual and real scene generation - real interaction - visual feedback to virtual reality glasses, and the closed loop delay is ≤50ms.

Images