Single-person first visual angle interaction system and method based on light guide film

Abstract

The invention discloses a single-person first visual angle interaction system and method based on a light guide film. The system modulates high-frequency light through the micro-nano grating array to form a reflected light pattern which can be collected after being reflected by the cornea; the image sensor collects and solves the head track at the frequency of more than or equal to 120Hz, and the precision is less than or equal to 0.05 pixel; synchronously encoding the first visual angle picture and the track into a space-time capsule; the playing end adjusts the visual angle of the display picture in real time based on the head motion trail. The grating array is aligned with pixels through photoetching, is necessary hardware for realizing high-precision and low-delay interaction under the constraint of a mobile terminal, and can be used for scenes such as medical teaching, family memory, Vlog, remote interaction and the like.

Description

[0001] Citation of relevant applications This application references the following prior applications filed by the applicant on or before the same date, the entire technical content of their descriptions, claims and abstracts of which is incorporated herein by reference: Patent Application No.: 2026103269949, Application Date: 2026-03-17, Invention Title: Unauthorized Eye-Tracking Touch Terminal Control Method and System Based on Optical Guide Interaction Layer; Patent Application No.: 2026102814971, Application Date: 2026-03-10, Invention Title: An Unauthorized AI Interaction Implementation Method Based on Optical Guide Tempered Glass Film. Technical Field

[0002] This invention relates to the fields of human-computer interaction, virtual reality, and artificial intelligence, and specifically to a single-person first-person perspective interaction system and method based on a photoconductive film, applicable to smart devices with touchscreens such as smartphones, tablets, and computers. Background Technology

[0003] Existing short video platforms only allow users to passively watch third-party video content, lacking the ability to change perspectives based on user head movements, engage in real-time dialogue with characters in the video, or perceive the emotional state of the person filming. While existing VR / AR technologies can achieve immersive experiences, the content is mostly artificially modeled, lacking first-person perspective recordings of real people, and relies on specialized hardware. In current technologies, photoconductive films are only used for screen display enhancement or physical touch control, and have not yet been applied to head movement trajectory acquisition and perspective transformation. Summary of the Invention

[0004] I. Technical Problems to be Solved This invention aims to solve the following technical problems: achieving high-precision head motion trajectory acquisition without adding dedicated sensors; realizing real-time perspective transformation of the playback screen based on the head motion trajectory; making the light guide film a necessary hardware structure for realizing the above functions; achieving compatible operation under different hardware configurations; and ensuring low latency and high precision of perspective transformation.

[0005] II. Technical Solution 2.1 Hardware Structure of the Photoconductive Film The photoconductor film of this invention is a multi-layer composite optical structure, comprising: a micro / nano grating array with a period of 200-500nm. Experiments have verified that this period can achieve the optimal signal-to-noise ratio under the power consumption constraints of mobile devices; below 200nm, it is susceptible to ambient light interference, while above 500nm, the phase calculation accuracy is insufficient; through photolithography combined with mask alignment, hardware alignment with screen pixels is achieved, with an alignment error of < 0.01 pixels, used to modulate incident light and convert minute head movements into phase changes of reflected light; an edge micro-LED light source: integrated around the perimeter of the film, emitting high-frequency modulated light with a frequency ≥ 1MHz; an optical waveguide layer: uniformly conducts the high-frequency modulated light emitted by the edge micro-LED light source to the entire film surface; a temperature compensation layer: the coefficient of thermal expansion matches the screen glass, ensuring the stability of the grating optical characteristics in an environment of -10℃ to 60℃; and an upper MgF2 anti-reflective coating: reducing ambient light interference.

[0006] 2.2 Principle of Head Motion Trajectory Acquisition The high-frequency modulated light emitted by the edge micro LED light source is uniformly transmitted to the surface of the photoconductor film through the optical waveguide layer. After being modulated by the micro-nano grating array, it forms a reflected light pattern with a specific spatial frequency. This reflected light pattern specifically refers to the light signal reflected by the user's cornea, which is collected in real time by an image sensor (such as a front-facing camera) at a sampling frequency of ≥120Hz.

[0007] When the user's head rotates, the position of the eyeballs relative to the screen changes, causing a phase change in the reflected light pattern captured by the image sensor. This phase change is extracted by performing a Fourier transform and phase calculation on the reflected light pattern. The head trajectory calculation unit, combined with the system calibration coefficient K, calculates the head movement displacement Δx using the formula Δx = K × Δφ. Here, K is the system calibration coefficient, in millimeters per radian, calibrated using multi-position calibration objects before shipment and written into the firmware. It is related to the grating period, LED wavelength, and device optical parameters, and does not require repeated calibration by the user.

[0008] This system can achieve a resolution accuracy of ≤0.05 pixels and a viewing angle change delay of ≤20ms. Tests have shown that this delay can prevent users from experiencing dizziness.

[0009] 2.3 System Architecture A single-person first-person perspective interactive system based on a photoconductor film, wherein the signal connection relationship of each unit is as follows: the photoconductor film and the edge light source cooperate to form an optical detection structure, and its output light signal is acquired by an image sensor; the output of the image sensor is connected to the head trajectory calculation unit; the head trajectory calculation unit and the first-person perspective data acquisition unit jointly output to the time capsule generation unit; the time capsule generation unit outputs to the perspective transformation unit and subsequent auxiliary function units.

[0010] The system specifically includes: a light guide film: attached to the screen surface, comprising a micro / nano grating array, which modulates the light signal to form a reflected light pattern that can be captured by an image sensor; the phase change of the reflected light pattern has a definite correspondence with the head movement displacement; an edge light source: integrated at the edge of the light guide film, emitting high-frequency modulated light with a frequency ≥1MHz; an image sensor: located at the acquisition end device, used to acquire the reflected light pattern after reflection from the user's cornea at a sampling frequency ≥120Hz; and a head trajectory calculation unit: used to calculate the user's head movement trajectory based on the phase change of the reflected light pattern, with a calculation accuracy ≤0.05. The system comprises the following components: a first-person perspective data acquisition unit (FPZ) for real-time recording of the user's first-person perspective image via a camera; a time capsule generation unit for synchronizing the first-person perspective image with head movement trajectory data using a unified clock timestamp to generate first-person time capsule data at the millisecond level; a perspective transformation unit located on the playback device for receiving and parsing the time capsule data and adjusting the viewing angle of the displayed image in real-time based on the head movement trajectory; a gaze direction calibration unit for calculating the user's gaze direction based on corneal reflection characteristics combined with grating optical modulation characteristics; an AI clone generation unit for achieving real-time dialogue based on an offline trained model of the user; and an emotion feedback unit for outputting emotional states based on physiological data, voice features, or micro-motion features of reflected light patterns. The micro-motion features of reflected light patterns refer to high-frequency components with a frequency of 10-20Hz, extracted through bandpass filtering.

[0011] 2.4 Argument for Unavoidability The unavoidable nature of this system is reflected in the following three levels: Hardware structure level: The micro-nano grating array is hardware aligned with the screen pixels, which requires photolithography to achieve an alignment error of < 0.01 pixels, a precision that ordinary optical films cannot achieve; Precision constraint level: The head trajectory calculation precision is ≤ 0.05 pixels, which can be achieved under the thickness and power consumption constraints of mobile devices by using only grating-modulated light signals + high-frequency modulation (≥1MHz) + high-speed sampling (≥120Hz), while the precision of ordinary gyroscopes or facial AI tracking solutions is ≥ 0.1 pixels; Latency constraint level: The viewing angle change delay is ≤ 20ms, which is achieved by this system through hardware-level light signal modulation to achieve low-latency calculation, while ordinary solutions have a delay of ≥ 30ms under the power consumption constraints of mobile devices.

[0012] III. Beneficial Effects: High-precision head tracking: Achieves a resolution accuracy of ≤0.05 pixels and a latency of ≤20ms by modulating light signals with a photoconductor film and acquiring data with an image sensor; Single-person perspective change: Users can change the viewing angle of the playback device in real time based on head movements; Real-time dialogue: Users can have real-time dialogues with the AI ​​avatar of the person in the video; Emotional synchronization: The emotional state of the person shooting can be output synchronously; Irreplaceable hardware: The micro-nano grating array is the necessary hardware structure to achieve the above accuracy and latency under the constraints of mobile devices. Detailed Implementation

[0013] The present invention will be further described in detail below with reference to specific embodiments, but the scope of protection of the present invention is not limited thereto. This embodiment corresponds to the system and method described in claims 1-10.

[0014] Example 1: Medical Surgical Teaching Scene Data Acquisition Terminal: During surgery, a cardiac surgeon wears smart glasses (or a mobile phone holder) with the photoconductive film described in this invention attached to it. During the surgery: an edge-mounted micro-LED light source emits modulated light at a frequency of 1MHz; the micro-nano grating array of the photoconductive film modulates the light signal into a specific pattern, which is reflected by the surgeon's cornea and then collected by an image sensor (such as a miniature camera on the glasses) at a sampling frequency of 120Hz; the head trajectory calculation unit calculates the surgeon's head movement trajectory in real time based on the phase change of the reflected light pattern, with an accuracy of 0.05 pixels; the first-view data acquisition unit records the surgical scene seen by the surgeon through the glasses' camera; the time capsule generation unit synchronously encodes the surgical scene and head trajectory data at millisecond levels to generate a "cardiac bypass surgery" time capsule.

[0015] Playback Terminal: Medical students watch the time capsule on their mobile phones: The perspective change unit adjusts the screen perspective in real time according to the student's head rotation; when the student turns their head to the left, the screen automatically displays the perspective of the left side of the operating table; the gaze direction calibration unit marks "the doctor was looking at the vascular anastomosis site" in the screen; the AI ​​clone generation unit calls an AI model trained based on the doctor's preoperative interview data to answer the student's questions in the doctor's voice; the emotion feedback unit displays "attention level 95%" in the corner of the screen based on the heart rate data recorded by the doctor during the operation.

[0016] Example 2: Family Memory Preservation Scene Acquisition Terminal: The father uses a mobile phone with a light guide film to record a family dinner: the light guide film collects the father's head trajectory and gaze direction in real time; the camera records first-person perspective images (the dishes on the table, the smiling faces of family members); the voice acquisition module records the father's laughter and self-talk; the biosensor records heart rate data (the heart rate rises slightly when happy).

[0017] On the playback device: Years later, the son enters the time capsule via a tablet: He turns his head to look at the left side of the screen and sees the moment his mother cut the cake; the AI ​​clone replies in the father's voice: "Your mother's hands were shaking when she cut the cake"; the emotional feedback module displays the father's emotional curve at that time.

[0018] Example 3: Personal Vlog Creation Scene Blogger: An outdoor blogger uses an action camera with a light guide film to record the rock climbing process: the light guide film still stably captures the head trajectory under strong light; the temperature compensation layer ensures that the grating accuracy remains unchanged in an environment of -5℃; the first-person perspective video records the rock wall and the scenery below as seen by the blogger.

[0019] Fans: When watching on their phones: viewers can turn their heads to see the side view of the rock face; the blogger's AI clone provides real-time commentary: "This rock spot is very slippery, be careful"; emotional feedback shows that the blogger's heart rate was as high as 140 beats per minute at that time.

[0020] Example 4: Relatives and friends in different locations experience Grandma's cooking simultaneously: using a mobile phone with a light guide film attached to make dumplings in the kitchen and recording the whole process.

[0021] Grandson: In another city, he enters a time capsule via a tablet: He turns his head and sees the water vat next to his grandmother; the AI ​​clone says in his grandmother's voice: "When you were little, you loved eating the dumplings I made the most"; the emotional feedback shows the grandmother's happiness curve at that time.

Claims

1. Claim 1 A single-person first-person perspective interactive system based on a photoconductive film, characterized in that, include: A photoconductor film, attached to the screen surface, includes a micro / nano grating array. The micro / nano grating array modulates the light signal to form a reflected light pattern that can be captured by an image sensor. The phase change of the reflected light pattern has a definite correspondence with the head movement displacement. An edge light source, integrated at the edge of the photoconductor film, emits high-frequency modulated light with a frequency ≥1MHz. An image sensor, installed on the acquisition device, is used to acquire the pattern of reflected light after being reflected by the user's cornea at a sampling frequency of ≥120Hz; The head trajectory calculation unit is used to calculate the head movement trajectory of the collector based on the phase change of the reflected light pattern collected by the image sensor, with a calculation accuracy of ≤0.05 pixels; The first-person view data acquisition unit is used to record the first-person view of the collector in real time through a camera. The time capsule generation unit is used to synchronously encode the first-person view and head movement trajectory data at the millisecond level to generate first-person time capsule data. The perspective adjustment unit, located on the playback device, is used to receive and parse the time capsule data and adjust the perspective of the displayed image in real time based on the head movement trajectory.

2. Claim 2 The system according to claim 1, characterized in that, The photoconductive film has a micro-nano grating array period of 200-500nm. It is aligned with the screen pixels by photolithography and mask alignment, with an alignment error of < 0.01 pixels.

3. Claim 3 The system according to claim 1, characterized in that, The head trajectory calculation unit analyzes the phase change Δφ of the reflected light pattern and, in conjunction with the system calibration coefficient K, calculates the head motion displacement Δx according to the formula Δx = K×Δφ, where K is in millimeters per radian.

4. Claim 4 The system according to claim 1, characterized in that, The edge light source modulation frequency is ≥1MHz, and the image sensor sampling frequency is ≥120Hz.

5. Claim 5 The system according to claim 1, characterized in that, It also includes a gaze direction calibration unit, which is used to calculate the gaze direction of the user based on the corneal reflection characteristics in the reflected light pattern collected by the image sensor, combined with the optical modulation characteristics of the micro-nano grating array.

6. Claim 6 The system according to claim 1, characterized in that, It also includes an AI avatar generation unit, which is used to generate an AI avatar model based on offline training by the collector, and to have real-time conversations with the user.

7. Claim 7 The system according to claim 1, characterized in that, It also includes an emotional feedback unit, which is used to synchronously output emotional state information based on the collector's heart rate and breathing data, or based on the characteristics of voice tone and the 10-20Hz high-frequency micro-motion component in the reflected light pattern (extracted by bandpass filtering).

8. Claim 8 A single-person first-person perspective interaction method based on a photoconductive film, applied to the system described in any one of claims 1-7, characterized in that, The process includes the following steps: S1: An edge light source emits high-frequency modulated light at ≥1MHz, which is modulated by a micro / nano grating array of a photoconductor film to form a reflected light pattern; S2: An image sensor acquires the reflected light pattern after reflection from the user's cornea at a sampling frequency of ≥120Hz; S3: Based on the phase change of the reflected light pattern, the head movement trajectory is calculated with an accuracy of ≤0.05 pixels. S4 The camera records the first-person perspective of the data collector in real time. S5: Achieve millisecond-level synchronization through a unified timestamp, encoding the first-person perspective image and head trajectory data into a time capsule; S6: The playback end parses the data and adjusts the viewing angle of the displayed image in real time based on the head movement trajectory.

9. Claim 9 The method according to claim 8 is characterized in that, Step S3: Calculate the displacement using the formula Δx=K×Δφ, where Δφ is the phase change and K is the factory calibration coefficient, in millimeters / radians.

10. Claim 10 A computer-readable storage medium for storing a computer program, characterized in that, When the program is executed by the processor, it implements the method of any one of claims 8-9.