Visual training device based on dynamic light field regulation and control method

Abstract

The invention relates to the technical field of visual training, in particular to a visual training device based on dynamic light field regulation and control and a control method. The visual training device comprises a microcontroller unit, a multispectral sensor, a millimeter wave sensor, a driving module and a generation module, the microcontroller unit is in communication connection with the multispectral sensor, the millimeter wave sensor and the driving module, and the driving module is in communication connection with the generation module; the multispectral sensor is used for collecting a sequence eye image of a user after initial light field visual training, the millimeter wave sensor is used for collecting a head movement signal and a physiological signal of the user after initial light field visual training, and collecting multi-dimensional information, so that the microcontroller unit can generate a corresponding control instruction based on multi-dimensional data, and the control instruction is sent to the user. The accuracy of the control instruction is improved, and after the driving module drives the generation module to generate the target light field matched with the control instruction according to the received control instruction, the corresponding training effect can be improved in the visual training process.

Description

Technical Field

[0001] This invention relates to the field of visual training technology, and in particular to a visual training device and control method based on dynamic light field modulation. Background Technology

[0002] In today's society, due to excessive exposure to electronic screens and improper eye use without adequate rest, the rate of myopia in China is increasing year by year. This has made vision health a significant issue affecting the health of the Chinese people. To address these vision health problems, regular vision training is necessary. Currently, most vision training tools are still at a very basic level, such as fixed-pattern light boxes, which can only provide monochromatic or static multicolor light stimulation and cannot simulate the complex stimulation of the retina by natural dynamic light environments, resulting in limited training effectiveness. Therefore, improving the effectiveness of vision training is an urgent problem to be solved. Summary of the Invention

[0003] Therefore, it is necessary to provide a vision training device and control method based on dynamic light field modulation to address the above-mentioned technical problems and solve the problem of poor training effect during vision training.

[0004] A first aspect of this application provides a visual training device based on dynamic light field modulation. The visual training device includes a microcontroller unit, a multispectral sensor, a millimeter-wave sensor, a driving module, and a generation module. The microcontroller unit is communicatively connected to the multispectral sensor, the millimeter-wave sensor, and the driving module. The driving module is communicatively connected to the generation module. The multispectral sensor is used to acquire a sequence of eye images after the user has undergone visual training based on an initial light field, and to send the sequence of eye images to the microcontroller unit; The millimeter-wave sensor is used to collect head motion signals and physiological signals of the user after initial light field visual training, and to send the head motion signals and physiological signals to the microcontroller unit. The microcontroller unit is used to determine the user's eye movement information, head movement information, and physiological information based on the received sequence of eye images, head movement signals, and physiological signals; generate control commands based on the eye movement information, head movement information, and physiological information; and send the control commands to the drive module. The driving module is used to drive the generation module to generate a target light field that matches the control command according to the received control command, so as to use the target light field to perform visual training on the user.

[0005] A second aspect of this application provides a control method based on dynamic light field modulation, the control method being applied to the microcontroller unit described above, the control method comprising: The system receives sequential eye images of the user after initial light field visual training, collected by a multispectral sensor, and head motion and physiological signals of the user after initial light field visual training, collected by a millimeter-wave sensor. Based on the sequence of eye images, the head motion signal, and the physiological signal, the user's eye movement information, head movement information, and physiological information are determined. Control commands are generated based on the eye movement information, head movement information, and physiological information, so that the drive module drives the corresponding generation module to generate a target light field that matches the control commands.

[0006] The advantages of this application compared to the prior art are: This application provides a visual training device based on dynamic light field modulation, including a microcontroller unit, a multispectral sensor, a millimeter-wave sensor, a driving module, and a generation module. The microcontroller unit is communicatively connected to the multispectral sensor, the millimeter-wave sensor, and the driving module, and the driving module is communicatively connected to the generation module. The multispectral sensor is used to acquire sequential eye images of the user after visual training based on an initial light field. The millimeter-wave sensor is used to acquire head motion signals and physiological signals of the user after visual training based on the initial light field, acquiring multidimensional information so that the microcontroller unit can generate corresponding control commands based on multidimensional data, thereby improving the accuracy of the control commands. After the driving module drives the generation module to generate a target light field that matches the control commands according to the received control commands, the corresponding training effect can be improved during the visual training process. Attached Figure Description

[0007] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 This is a schematic diagram of the structure of a visual training device based on dynamic light field modulation provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of a visual training device based on dynamic light field modulation provided in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of a visual training device based on dynamic light field modulation provided in Embodiment 3 of the present invention; Figure 4This is a schematic diagram of the structure of a visual training device based on dynamic light field modulation provided in Embodiment 4 of the present invention; Figure 5 This is a schematic diagram of the structure of a visual training device based on dynamic light field modulation provided in Embodiment 5 of the present invention; Figure 6 This is a flowchart illustrating a control method based on dynamic light field modulation provided in Embodiment Six of the present invention; The system includes a microcontroller unit 11, a multispectral sensor 12, a millimeter-wave sensor 13, a drive module 14, a generation module 15, an audio drive module 21, a light source drive module 22, an audio player 31, an array light source 32, a light sensing module 41, and a communication module 51. Detailed Implementation

[0009] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0010] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0011] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0012] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0013] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0014] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0015] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0016] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0017] See Figure 1 This is a schematic diagram of a visual training device based on dynamic light field modulation provided in Embodiment 1 of the present invention. The visual training device includes a microcontroller unit 11, a multispectral sensor 12, a millimeter-wave sensor 13, a driving module 14, and a generation module 15. The microcontroller unit 11 is communicatively connected to the multispectral sensor 12, the millimeter-wave sensor 13, and the driving module 14. The driving module 14 is communicatively connected to the generation module 15.

[0018] The multispectral sensor 12 is used to acquire a sequence of eye images of the user after visual training based on an initial light field, and sends the sequence of eye images to the microcontroller unit 11. The multispectral sensor 12 can be a multispectral camera, which can capture light of different wavelengths to perform refined imaging of the user's eyes, obtaining a sequence of eye images. The microcontroller unit 11 then sends these images to the microcontroller unit 11, allowing it to determine corresponding eye movement information based on the sequence of eye images. This eye movement information includes changes in the user's pupil size, eye movement trajectory, blink frequency, etc.

[0019] The millimeter-wave sensor 13 is used to acquire the user's head motion signals and physiological signals after initial light field visual training, and transmits these signals to the microcontroller unit 11. The millimeter-wave sensor 13 can be an FMCW MIMO radar system, which has transmitting and receiving antennas for target detection. The FMCW MIMO radar system acquires the user's head motion signals and physiological signals, and transmits them to the microcontroller unit 11. This facilitates the microcontroller unit 11 in determining head movement information based on the head motion signals and physiological information based on the user's physiological signals. The head movement information may include head movement frequency and amplitude, and the physiological information may include the user's breathing and heart rate information.

[0020] It should be noted that the initial light field is determined based on the user's basic information, such as age and refractive error. The initial light field can be a rotating black and white striped light field, an RGB three-color ball rotating light field, a high-speed bouncing ball light field, a color gradient light field, or a light field where the letter "E" gradually changes size and flashes.

[0021] The microcontroller unit 11 is used to determine the user's eye movement information, head movement information and physiological information based on the received sequence of eye images, head movement signals and physiological signals, generate control commands based on the eye movement information, head movement information and physiological information, and send the control commands to the drive module 14.

[0022] The microcontroller unit 11 analyzes the received sequence of eye images to determine the user's eye movement information. When analyzing the sequence of eye images, the pupil region can be segmented based on Hough circle transform or deep learning (such as U-Net). Edge detection is performed on the pupil region of each eye image to determine the edge of the pupil region. Based on the edge of the pupil region in each eye image, the pupil size change information and eye movement trajectory information are determined. Key eye points are extracted from each eye image, and the upper and lower eyelids in the sequence of eye images are located. Based on the upper and lower eyelids in each eye image, the eyelid height change is determined, and based on the eyelid height change, the number of blinks per unit time, i.e., the blink frequency, is determined. Eye movement information is determined based on the pupil size change information, eye movement trajectory information, and blink frequency information. Other methods can also be used to determine eye movement information; this embodiment is not limited to any particular method.

[0023] The microcontroller unit 11 analyzes head motion signals and physiological signals to determine the user's head movement and physiological information. Specifically, when analyzing head motion signals, it analyzes the time difference between the transmitted and received signals from the millimeter-wave sensor 13 to determine the frequency and amplitude of the user's head movements. When analyzing physiological signals, it utilizes the changes in reflected signals caused by minute vibrations of the human chest cavity to extract physiological characteristics of respiration and heartbeat. Chest cavity vibrations cause phase changes in reflected signals; the phase changes of respiration and heartbeat are of different magnitudes and can be separated and extracted. Other methods can also be used to determine the user's head movement and physiological information; this embodiment does not limit this method.

[0024] Control commands are generated based on eye movement information, head movement information, and physiological information, and then sent to the drive module 14. The drive module 14 then drives the hardware module to complete the specified actions or functions according to the control commands, thereby enabling precise control of the entire visual training device.

[0025] When generating control commands based on eye movement information, head movement information, and physiological information, a deep learning model can be used to process the eye movement information, head movement information, and physiological information to obtain target light field parameters, and then generate the target light field based on the target light field parameters.

[0026] The driver module 14 drives the generation module 15 to generate a target light field matching the received control commands, which is then used for visual training of the user. The driver module 14 receives control commands via a communication interface (such as UART, SPI, I2C, CAN bus, etc.). These control commands are transmitted in a specific data format (such as binary, hexadecimal, or ASCII code) and contain all the information required to execute the action. The target light field is a dynamic light field, and the visual training is used to correct visual defects, improve visual skills, and alleviate visual fatigue. It should be noted that after receiving the control commands, the driver module 14 must ensure that the format of the control commands is consistent with the protocols supported by the driver module 14 to avoid data parsing errors. The integrity of the control command transmission must also be verified through methods such as checksum and CRC checksum to prevent command errors caused by noise or interference.

[0027] This application provides a visual training device based on dynamic light field modulation, including a microcontroller unit 11, a multispectral sensor 12, a millimeter-wave sensor 13, a driving module 14, and a generation module 15. The microcontroller unit 11 is communicatively connected to the multispectral sensor 12, the millimeter-wave sensor 13, and the driving module 14, and the driving module 14 is communicatively connected to the generation module 15. The multispectral sensor 12 is used to acquire sequential eye images of the user after visual training based on an initial light field, and the millimeter-wave sensor 13 is used to acquire head motion signals and physiological signals of the user after visual training based on the initial light field. By acquiring multidimensional information, the microcontroller unit 11 can generate corresponding control commands based on the multidimensional data, thereby improving the accuracy of the control commands. After the driving module 14 drives the generation module 15 to generate a target light field that matches the control commands according to the received control commands, the corresponding training effect can be improved during the visual training process.

[0028] Optionally, the control commands include audio control commands and light source control commands, and the drive module 14 includes an audio drive module 21 and a light source drive module 22; The audio driver module 21 is used to parse the received audio control commands, obtain the audio parsing results, and send the audio parsing results to the generation module 15; The light source driving module 22 is used to parse the received light source control command, obtain the light source parsing result, and send the light source parsing result to the generation module 15.

[0029] In this embodiment, see Figure 2 This is a schematic diagram of a visual training device based on dynamic light field modulation provided in Embodiment 2 of the present invention. The driving module 14 includes an audio driving module 21 and a light source driving module 22. The control commands include audio control commands and light source control commands. The audio control commands are used to adjust the output parameters of the audio device, such as volume, sound effect mode, playback status, etc., to meet the user's personalized needs for the sound environment. The light source control commands are used to adjust the brightness, color, on / off status, etc. of the lighting device to achieve personalized customization and automated management of lighting effects.

[0030] The audio driver module 21 parses the received audio control commands, obtains the audio parsing results, and sends the results to the generation module 15. The audio driver module 21 parses the audio control commands, including command header recognition, data field parsing, and command verification. The command header contains information such as command type, length, and target address. The audio driver module 21 determines the legality of the command and the subsequent processing method by recognizing the command header. The data field contains the specific parameters of the control command, such as action type, speed, and position. The audio driver module 21 parses the data field according to the control command format and extracts key parameters. The decoded control commands undergo legality verification, such as parameter range checks and action feasibility assessments, to ensure that the commands can be executed safely.

[0031] The light source driving module 22 is used to parse the received light source control commands, obtain the light source parsing results, and send the light source parsing results to the generation module 15. The method for parsing the light source control commands is the same as described above.

[0032] In this embodiment, an audio parsing module is used to parse audio control commands to support real-time adjustment of parameters such as volume and sound effects, meeting users' personalized needs. A light source driving module 22 is used to parse light source control commands to control the behavior of the light source according to the parsed commands. Simultaneously, both audio and light source control commands are parsed to adjust the audio and light source of the light field based on the parsing results, thereby improving training effectiveness through multi-sensory stimulation training.

[0033] Optionally, the generation module 15 includes an audio player 31 and an array of light sources 32; Audio player 31 is used to play the corresponding audio based on the audio parsing results; The array light source 32 is used to provide the corresponding light source based on the light source analysis results.

[0034] In this embodiment, see Figure 3 This is a schematic diagram of a visual training device based on dynamic light field modulation provided in Embodiment 3 of the present invention. The generation module 15 includes an audio player 31 and an array light source 32. The audio player 31 can be a corresponding speaker unit, used to play corresponding audio based on the audio analysis results, such as audio prompts, instructions, or background music, to enhance the user's experience and cooperation during training. The array light source 32 is composed of RGB three-color LEDs, which, under the control of the light source driving module 22, can emit light of different colors, brightness, and frequencies to form a dynamically changing light field. This allows for the training of the user's eye accommodation ability and visual sensitivity through specific pattern and color stimulation. Optionally, the visual training device also includes a light-sensing module 41, which is communicatively connected to the microcontroller unit 11; The light sensor module 41 is used to collect ambient light intensity and send the ambient light intensity to the microcontroller unit 11, so that the microcontroller unit 11 can generate control commands based on the ambient light intensity, eye movement information, head movement information and physiological information.

[0035] In this embodiment, see Figure 4 This is a schematic diagram of a visual training device based on dynamic light field modulation according to Embodiment 4 of the present invention. The visual training device also includes a light-sensing module 41, which is a sensor module for detecting light intensity. The light-sensing module 41 converts light signals into electrical signals through a photosensitive element to monitor the lighting environment. The light-sensing module 41 monitors the ambient light intensity in real time and sends the ambient light intensity to the microcontroller unit 11. This allows the microcontroller unit 11 to generate control commands based on the ambient light intensity, eye movement information, head movement information, and physiological information, so as to match the light field brightness with the ambient light brightness and improve the user's training comfort. Optionally, the visual training device also includes a communication module 51, which is communicatively connected to the microcontroller unit 11 and is used for communication between the microcontroller unit 11 and the terminal.

[0036] In this embodiment, see Figure 5 This is a schematic diagram of a visual training device based on dynamic light field modulation provided in Embodiment 5 of the present invention. The visual training device also includes a communication module 51, which is communicatively connected to the microcontroller unit 11. The communication module 51 is used for communication between the microcontroller unit 11 and the terminal. The communication module 51 may include a Bluetooth module and a Wi-Fi module to realize data transmission and communication between the system and the terminal device (such as a mobile APP). The system uploads the user's physiological data, training parameters, training records, etc., to the terminal via Bluetooth or Wi-Fi, and simultaneously receives control commands and user settings information sent by the terminal, ensuring real-time data interaction between the visual training device and the terminal.

[0037] See Figure 6 This is a flowchart illustrating a control method based on dynamic light field modulation provided in Embodiment Six of the present invention, as shown below. Figure 6 As shown, the control method based on dynamic light field modulation may include the following steps.

[0038] S601: Receives sequenced eye images of the user after initial light field visual training, acquired by a multispectral sensor, and head motion signals and physiological signals of the user after initial light field visual training, acquired by a millimeter-wave sensor. S602: Based on the sequence of eye images, head motion signals, and physiological signals, determine the user's eye movement information, head movement information, and physiological information, and generate control commands based on the eye movement information, head movement information, and physiological information, so that the drive module drives the corresponding generation module to generate a target light field that matches the control commands.

[0039] In this embodiment, the control method is applied to the aforementioned microcontroller unit. The microcontroller unit receives a sequence of eye images acquired by a multispectral sensor after visual training based on an initial light field, and head motion signals and physiological signals acquired by a millimeter-wave sensor after visual training based on the initial light field. The initial light field is determined based on the user's basic information, such as age and refractive error. The initial light field can be a rotating black-and-white stripe light field, an RGB three-color ball rotating light field, a high-speed bouncing ball light field, a color gradient light field, or a gradually changing flashing light field of the letter "E," etc.

[0040] Based on the sequence of eye images, head motion signals, and physiological signals, the user's eye movement information, head movement information, and physiological information are determined. Control commands are then generated based on these information, so that the drive module can drive the corresponding generation module to generate a target light field that matches the control commands.

[0041] In this embodiment, the microcontroller unit analyzes the received sequence of eye images to determine the user's eye movement information. When analyzing the sequence of eye images, the pupil region can be segmented based on Hough circle transform or deep learning (such as U-Net). Edge detection is performed on the pupil region of each eye image to determine the edge of the pupil region. Based on the edge of the pupil region in each eye image, pupil size change information and eye movement trajectory information are determined. Key eye points are extracted from each eye image, and the upper and lower eyelids in the sequence of eye images are located. Based on the upper and lower eyelids in each eye image, eyelid height changes are determined. Based on the eyelid height changes, the number of blinks per unit time, i.e., blink frequency, is determined. Eye movement information is determined based on pupil size change information, eye movement trajectory information, and blink frequency information. Other methods can also be used to determine eye movement information; this embodiment does not limit this method.

[0042] The microcontroller unit analyzes head motion signals and physiological signals to determine the user's head movement and physiological information. Specifically, when analyzing head motion signals, it analyzes the time difference between the transmitted and received signals from the millimeter-wave sensor to determine the frequency and amplitude of the user's head movements. When analyzing physiological signals, it utilizes the changes in reflected signals caused by minute vibrations of the chest cavity to extract physiological characteristics of respiration and heartbeat. Chest cavity vibrations cause phase changes in the reflected signals; the phase changes of respiration and heartbeat are of different magnitudes and can be separated and extracted. Other methods can also be used to determine the user's head movement and physiological information; this embodiment does not limit this method.

[0043] Control commands are generated based on eye movement information, head movement information, and physiological information. These commands are then sent to the drive module, which in turn drives the hardware module to perform the specified actions or functions. This enables precise control of the entire vision training device.

[0044] Optionally, control commands are generated based on eye movement information, head movement information, and physiological information, including: Determine the user's head movement trajectory based on eye movement information or head movement information; The initial light field trajectory is obtained, and the user's concentration level is determined based on the head movement trajectory and the light field trajectory. When the focus level does not meet the preset requirements, the light field trajectory is adjusted according to the head movement trajectory until the focus level meets the preset requirements. When the focus level meets the preset requirements, the target light field parameters are determined based on eye movement information, head movement information, and physiological information, and corresponding control commands are generated based on the target light field parameters.

[0045] In this embodiment, the user's head movement trajectory is determined based on eye movement information or head movement information. Specifically, when the range of head movement is small, eye movement information can be used to determine the user's head movement trajectory; when the range of head movement is large, head movement information can be used to determine the user's head movement trajectory.

[0046] It should be noted that when using eye movement information to determine a user's head movement trajectory, the head movement trajectory can be determined based on the eye movement trajectory. When using head movement information to determine a user's head movement trajectory, the head movement trajectory can be determined based on the movement frequency.

[0047] The initial light field trajectory is acquired. Based on the head movement trajectory and the light field trajectory, it is determined whether the user's focus meets the preset requirements. The light field trajectory represents the trajectory of objects within the corresponding light field, and the user's focus characterizes their ability to maintain concentration, suppress distractions, and efficiently process information during visual training. The preset requirement is that the user's head movement trajectory is similar to the light field trajectory. If the user's head movement trajectory is similar to the light field trajectory, the focus is considered not to meet the preset requirements. When the focus does not meet the preset requirements, the light field trajectory is adjusted based on the head movement trajectory until the focus meets the preset requirements. When the focus meets the preset requirements, the target light field parameters are determined based on eye movement information, head movement information, and physiological information. Based on the target light field parameters, corresponding control commands are generated. The target light field parameters may include audio parameters and light source parameters. For example, audio parameters may include specific audio prompts or background music, and light source parameters may include the brightness, color, and flicker frequency of the light field.

[0048] It should be noted that when adjusting the trajectory of the light field based on the head movement trajectory, the movement speed of the corresponding object in the light field can be adjusted. For example, if the user's head movement trajectory lags behind the trajectory of the light field, the movement speed of the corresponding object in the light field can be reduced; if the user's head movement trajectory is faster than the trajectory of the light field, the movement speed of the corresponding object in the light field can be increased.

[0049] In this embodiment, before generating the corresponding control command, it is determined whether the user's concentration meets the preset requirements. If the concentration does not meet the preset requirements, the light field trajectory is adjusted according to the head movement trajectory until the concentration meets the preset requirements, so that the corresponding light field's square running speed matches the user's head movement speed, thereby allowing the user to adapt to the corresponding light field and improve the corresponding training effect.

[0050] Optionally, determining whether the user's level of focus meets preset requirements includes: Determine the user's head movement speed based on the head movement trajectory; The switching speed of the initial light field is determined based on the trajectory of the light field. When the difference between the head movement speed and the switching speed of the initial light field is greater than a preset difference threshold, it is determined that the user's focus does not meet the preset requirements. When the difference between the head movement speed and the switching speed of the initial light field is not greater than a preset difference threshold, the user's focus is determined to meet the preset requirements.

[0051] In this embodiment, when determining whether a user's focus meets the preset requirements, the user's head movement speed can be determined based on the head movement trajectory. The head movement speed can be the average speed of head movement. The initial light field switching speed is determined based on the light field's trajectory; this initial switching speed can be the average switching speed. The difference between the user's head movement speed and the initial light field switching speed is calculated. If the difference is greater than a preset difference threshold, the user's focus is considered low, and the user's focus does not meet the preset requirements. If the difference is not greater than the preset difference threshold, the user's focus is considered high, and the user's focus meets the preset requirements. The preset difference threshold can be determined based on specific circumstances.

[0052] It should be noted that when determining the user's head movement trajectory based on eye movement information, and then determining the user's head movement speed based on the head movement trajectory, this can be calculated using the difference in eye movement information between adjacent images. Specifically, it is calculated based on the distance the eyes move in adjacent images and the time interval between adjacent images. When determining the user's head movement trajectory based on head movement information, and then determining the user's head movement speed based on the head movement trajectory, the head movement speed is calculated using Doppler frequency shift.

[0053] For example, a millimeter-wave sensor is an FMCW MIMO radar system. If the FMCW MIMO radar system has... One transmitting antenna and With multiple receiving antennas, and assuming the target is located in the far field of the array, specifically the user's head, the frequency of the chirp signal emitted by the radar increases linearly during the pulse repetition interval (PRI). The transmitted signal in a single linear frequency modulated pulse can be written as: in, For frequency modulation slope, The carrier frequency of the signal. Indicates duration, Represents time in the fast time dimension. ,in This represents time in the slow time dimension. Assuming the radial velocity of the target relative to the radar is v, the instantaneous distance between the target and the radar is: in, , Given the initial distance to the target, the delay between the received and transmitted signals is expressed as: in, At the speed of light, Considering the delay between the received and transmitted signals, based on the above derivation, the specific echo signal can be derived as follows: The difference frequency signal (IF) obtained after processing by the radar mixer is represented as: in, This represents the fundamental phase change caused by the target distance. This represents the second phase change caused by the distance to the target. It includes the distance to the target. Target speed And the influence of the Doppler effect, The main contents involved and The product of.

[0054] Since c takes a large value For smaller values, the formula for representing the difference frequency signal (IF) in the above equation can be rearranged to obtain: To obtain the Doppler signal generated by the target motion, a Fast Fourier Transform (FFT) is performed on the fast and slow time dimensions of the echo signal. in, Indicates the chirp duration, with the envelope term offset as... The corresponding Doppler frequency during the Chirp duration For slow time dimensions Performing an FFT yields: After completing the above FFT processing, the range-Doppler (RD) image of the radar echo signal can be obtained. The range and velocity information can be obtained from the above formula, derived from... The properties of the c(x) function show that it reaches its maximum value when x=0, therefore we can conclude that: in, Indicates fast time-dimension frequency shift, Representing the slow time-dimensional frequency shift, the magnitudes of these two parameters reflect the target's range and velocity, where the target's range is determined by the time delay. It is confirmed that the velocity can be determined by the Doppler frequency shift. It is confirmed that, among them, For the corresponding wavelength, This is the Doppler frequency shift, i.e., the slow time-dimensional frequency shift.

[0055] In this embodiment, the difference between the user's head movement speed and the switching speed of the initial light field can be used to determine the speed difference when the user's eyes chase a moving object in the corresponding light field. Thus, the user's focus can be determined based on the difference between the head movement speed and the switching speed of the initial light field, thereby improving the accuracy of determining whether the user's focus meets the preset requirements.

[0056] Optionally, determining the target light field parameters includes: A preset first deep learning model is obtained, and eye movement information, head movement information and physiological information are processed based on the first deep learning model to obtain the target light field parameters.

[0057] In this embodiment, a preset first deep learning model is obtained, wherein the first deep learning model is a neural network model. Eye movement information, head movement information, and physiological information are input into the preset first deep learning model, and target light field parameters are output. The target light field parameters may include parameters such as brightness, color, temporal sequence, and color temperature of the light field, so as to generate a corresponding target light field according to the corresponding target light field parameters.

[0058] In this embodiment, eye movement information, head movement information, and physiological information are processed according to the first deep learning model to obtain target light field parameters. By using reasonable model selection and parameter adjustment strategies, the prediction performance and efficiency of target light field parameters can be significantly improved.

[0059] Optionally, the microcontroller unit is also communicatively connected to the optical sensing module, and the control method further includes: The ambient light intensity is received by the light sensor module. A preset second deep learning model is obtained, and eye movement information, head movement information, physiological information and ambient light intensity are processed according to the second deep learning model to obtain the target light field parameters.

[0060] In this embodiment, the visual training device also includes a light-sensing module, which is a sensor module for detecting light intensity. The light-sensing module converts light signals into electrical signals through a photosensitive element to monitor the lighting environment. The light-sensing module monitors the ambient light intensity in real time and sends the ambient light intensity data to the microcontroller unit. This allows the microcontroller unit to generate control commands based on the ambient light intensity, eye movement information, head movement information, and physiological information, so as to match the light field brightness with the ambient light brightness and improve user training comfort.

[0061] After receiving the ambient light intensity collected by the light sensing module, a preset second deep learning model is obtained. This second deep learning model is a neural network model. Eye movement information, head movement information, physiological information, and ambient light intensity are input into the preset second deep learning model, and the target light field parameters are output. These target light field parameters may include parameters such as brightness, color, temporal sequence, and color temperature of the light field, so as to generate a corresponding target light field based on the corresponding target light field parameters.

[0062] In this embodiment, when determining the target light field parameters, the corresponding ambient light intensity is taken into account so that the brightness and color temperature of the generated target light field correspond to the corresponding ambient light intensity. This allows users to improve the training effect when using the corresponding target light field for visual training.

[0063] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A visual training device based on dynamic light field modulation, characterized in that, The visual training device includes a microcontroller unit, a multispectral sensor, a millimeter-wave sensor, a driving module, and a generation module. The microcontroller unit is communicatively connected to the multispectral sensor, the millimeter-wave sensor, and the driving module. The driving module is communicatively connected to the generation module. The multispectral sensor is used to acquire a sequence of eye images after the user has undergone visual training based on an initial light field, and to send the sequence of eye images to the microcontroller unit; The millimeter-wave sensor is used to collect head motion signals and physiological signals of the user after initial light field visual training, and to send the head motion signals and physiological signals to the microcontroller unit. The microcontroller unit is used to determine the user's eye movement information, head movement information, and physiological information based on the received sequence of eye images, head movement signals, and physiological signals; generate control commands based on the eye movement information, head movement information, and physiological information; and send the control commands to the drive module. The driving module is used to drive the generation module to generate a target light field that matches the control command according to the received control command, so as to use the target light field to perform visual training on the user.

2. The visual training device as described in claim 1, characterized in that, The control commands include audio control commands and light source control commands, and the driving module includes an audio driving module and a light source driving module; The audio driver module is used to parse the received audio control command, obtain the audio parsing result, and send the audio parsing result to the generation module; The light source driving module is used to parse the received light source control command, obtain the light source parsing result, and send the light source parsing result to the generation module.

3. The visual training device as described in claim 1, characterized in that, The generation module includes an audio player and an array of light sources; The audio player is used to play the corresponding audio based on the audio parsing result; The array of light sources is used to provide corresponding light sources based on the light source analysis results.

4. The visual training device as described in claim 1, characterized in that, It also includes a light-sensing module, which is communicatively connected to the microcontroller unit; The light-sensing module is used to collect ambient light intensity and send the ambient light intensity to the microcontroller unit, so that the microcontroller unit can generate control commands based on the ambient light intensity, the eye movement information, the head movement information and the physiological information.

5. The visual training device as described in claim 1, characterized in that, It also includes a communication module, which is connected to... The microcontroller unit is connected for communication, and the communication module is used for communication between the microcontroller unit and the terminal.

6. A control method based on dynamic optical field modulation, characterized in that, The control method is applied to the microcontroller unit as described in any one of claims 1 to 5, and the control method includes: The system receives sequential eye images of the user after initial light field visual training, collected by a multispectral sensor, and head motion and physiological signals of the user after initial light field visual training, collected by a millimeter-wave sensor. Based on the sequence of eye images, the head motion signal, and the physiological signal, the user's eye movement information, head movement information, and physiological information are determined. Control commands are generated based on the eye movement information, head movement information, and physiological information, so that the drive module drives the corresponding generation module to generate a target light field that matches the control commands.

7. The control method as described in claim 6, characterized in that, The step of generating control commands based on the eye movement information, the head movement information, and the physiological information includes: The user's head movement trajectory is determined based on the eye movement information or the head movement information; The light field trajectory of the initial light field is obtained, and the user's concentration level is determined to meet the preset requirements based on the head movement trajectory and the light field trajectory. When the level of focus does not meet the preset requirements, the trajectory of the light field is adjusted according to the head movement trajectory until the level of focus meets the preset requirements. When the focus level meets the preset requirements, the target light field parameters are determined based on the eye movement information, the head movement information, and the physiological information, and the corresponding control command is generated based on the target light field parameters.

8. The control method as described in claim 7, characterized in that, Determining whether the user's focus level meets preset requirements includes: Based on the head movement trajectory, determine the user's head movement speed; Based on the trajectory of the light field, determine the switching speed of the initial light field; When the difference between the head movement speed and the switching speed of the initial light field is greater than a preset difference threshold, it is determined that the user's focus does not meet the preset requirements. When the difference between the head movement speed and the switching speed of the initial light field is not greater than the preset difference threshold, it is determined that the user's focus meets the preset requirements.

9. The control method as described in claim 7, characterized in that, The determination of the target light field parameters includes: A preset first deep learning model is obtained, and the eye movement information, head movement information and physiological information are processed according to the first deep learning model to obtain the target light field parameters.

10. The control method as described in claim 9, characterized in that, The microcontroller unit is also communicatively connected to the optical sensing module, and the control method further includes: Receive the ambient light intensity collected by the light sensing module; A preset second deep learning model is obtained, and the eye movement information, facial movement information, physiological information and ambient light intensity are processed according to the second deep learning model to obtain target light field parameters.

Images