A weak vision training system based on multi-mode visual stimulation of Unity engine

Abstract

The application discloses a kind of weak vision training systems based on Unity engine multimode visual stimulation, it is related to visual acuity improvement training technical field.The system includes starting module, calling module, receiving module and recording module.Starting module receives the starting instruction of user to training module;Calling module generates the visual stimulation image of corresponding training mode according to the instruction calling corresponding functional component in Unity engine;Receiving module receives the interactive operation data of user;Recording module processes interactive data and stores training record.Wherein, the functional component called by calling module at least includes the combination of multiple kinds in Unity engine's graphic rendering system and physical collision detection system, stereoscopic vision camera system, shader system, particle system and animation system, and post-processing special effect system.The application generates accurate visual stimulation image by systematic calling multiple underlying functional components of Unity engine, solves the problem of single visual stimulation type and insufficient generation precision in the prior art, significantly improves the diversity and pertinence of visual stimulation in amblyopia training.

Description

Technical Field

[0001] This invention relates to the field of vision improvement training technology, and in particular to an amblyopia training system based on multi-modal visual stimulation using the Unity engine. Background Technology

[0002] With the development of computer-aided vision training technology, using electronic devices for amblyopia training is gradually becoming a new treatment approach. Unity Engine, as a real-time 3D development platform with powerful graphics rendering capabilities, a physics collision detection system, a shader system, a particle system, and post-processing effects, can theoretically provide diverse visual stimulus generation methods for amblyopia training.

[0003] However, existing amblyopia training software based on the Unity engine generally suffers from a critical problem: it fails to specifically utilize the relevant functional components within the Unity engine to generate corresponding visual stimulus images based on the clinical needs of different training modules. Specifically, for various training needs, such as random target generation and hit detection for fine visual acuity training, adjustable binocular disparity images for fusion and stereopsis training, chromatic aberration separation images for simultaneous vision training, dynamic background raster interference for desuppression training, and high-contrast controllable continuous images for afterimage training, existing software often relies solely on simple planar image displays. It fails to systematically utilize the underlying functional components of the Unity engine, such as stereoscopic cameras, shaders, particle systems, and post-processing effects, to achieve precise generation and control of multiple types of visual stimuli. This significantly limits the targetedness and effectiveness of amblyopia training. Summary of the Invention

[0004] In view of the above-mentioned shortcomings in the current field of vision improvement training technology, the present invention provides an amblyopia training system based on multi-mode visual stimulation of Unity engine, which can selectively call specific functional components of Unity engine to generate visual stimulation images according to different training needs, thereby improving the accuracy and targeting of amblyopia training.

[0005] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0006] A multi-modal visual stimulation amblyopia training system based on the Unity engine includes:

[0007] The startup module is used to receive user commands to start the training module;

[0008] The calling module is used to call the corresponding functional components in the Unity engine according to the startup instruction in order to generate visual stimulus images for the corresponding training mode;

[0009] The receiving module is used to receive interactive operation data generated by the user based on the visual stimulus image;

[0010] The recording module is used to process the interactive operation data according to preset rules, generate training feedback information, and store training records.

[0011] The functional components invoked by the calling module include at least several of the following:

[0012] The Unity engine's graphics rendering system and physics collision detection module are used to generate visual stimulus images that respond to click interactions;

[0013] The Unity engine's stereo vision camera module is used to generate stereo visual stimulus images with adjustable binocular parallax.

[0014] The shader module of the Unity engine is used to generate visual stimulus images with separate color channels;

[0015] The particle system and animation module of the Unity engine are used to generate visually stimulating images with dynamically changing background raster.

[0016] The Unity engine's post-processing effects module is used to generate visual stimulus images with a controllable duration.

[0017] According to one aspect of the present invention, the visual stimulus image generated by the graphics rendering system and the physical collision detection system includes multiple target objects that randomly appear in the training area. The target objects are randomly assigned different shapes, sizes or colors. The physical collision detection system identifies the user's click hit operation through ray detection.

[0018] According to one aspect of the present invention, the stereoscopic visual stimulus images generated by the stereoscopic vision camera system are used for fusion vision training and stereoscopic vision training; in the fusion vision training, binocular disparity images with different fusion difficulties are generated by adjusting the interocular distance parameter and convergence distance parameter of the stereoscopic vision camera system.

[0019] According to one aspect of the invention, the color channel-separated visual stimulus image generated by the shader system is a red-blue or red-green difference-separated image, and the contents of the left-eye view and the right-eye view are different from each other.

[0020] According to one aspect of the invention, the color combination of the dynamically changing background grating generated by the particle system and the animation system includes black and white, red and black, or red and blue, and the grating shape includes rotating stripes or squares.

[0021] According to one aspect of the invention, the high-contrast visual stimulus image generated by the post-processing effects system includes a black and white raster or rotating stripes, and the display duration is controlled by controlling the activation duration of the post-processing effects system.

[0022] According to one aspect of the invention, the calling module is further configured to call the Unity engine's graphics rendering system to generate quantized visual targets that conform to vision testing standards.

[0023] According to one aspect of the present invention, the system also dynamically adjusts the combination method and difficulty parameters of each training mode based on the training records to generate a personalized training plan.

[0024] According to one aspect of the invention, the calling module is further configured to call the Unity engine's graphics rendering system to generate a distant view scene, the distant view scene including slowly moving visual guide elements.

[0025] According to one aspect of the present invention, the calling module is further configured to call the Unity engine's graphics rendering system to generate a picture book story interface including text areas and image areas, wherein the text areas and image areas are arranged in an alternating layout with varying distances, and the Unity engine's animation system controls the text and images to be displayed in an alternating rhythm.

[0026] The advantages of this invention are as follows: Based on the clinical needs of different training modules, it selectively utilizes specific functional components of the Unity engine, such as the stereoscopic vision camera system, shader system, particle system, post-processing effects system, and physical collision detection system, to accurately generate visual stimulus images that match each training module. This solves the problem of existing technologies relying solely on simple planar image display and failing to systematically utilize the underlying functional components of the Unity engine. Consequently, it achieves a systematic integration of multi-dimensional visual function training, including fine visual acuity, fusion vision, stereopsis, simultaneous vision, dissuppression, and afterimage training. This improves the accuracy of visual stimulus generation and the relevance of training content in amblyopia training, ultimately effectively improving the visual function of amblyopic users. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating a method for amblyopia training based on multi-modal visual stimulation using the Unity engine, as described in this invention.

[0029] Figure 2This is a schematic view of the fine visual training described in this invention;

[0030] Figure 3 This is a schematic view of the fusion vision training described in this invention;

[0031] Figure 4 This is a schematic view of the stereoscopic vision training described in this invention;

[0032] Figure 5 This is a schematic view of the simultaneous visual training described in this invention;

[0033] Figure 6 This is a schematic view of the desuppression training described in this invention;

[0034] Figure 7 This is a schematic view of the afterimage training described in this invention;

[0035] Figure 8 This is a schematic view of the relaxation training described in this invention;

[0036] Figure 9 This is a schematic diagram of the farsightedness quantization training configuration interface described in this invention;

[0037] Figure 10 This is a schematic diagram of the farsightedness quantitative training and testing interface described in this invention.

[0038] Figure 11 This is a schematic diagram illustrating the process of amblyopia training based on multi-modal visual stimulation using the Unity engine, as described in this invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Example 1

[0041] like Figure 1 As shown, a multi-modal visual stimulation amblyopia training system based on the Unity engine includes:

[0042] The startup module M1 is used to receive the user's startup command for the training module.

[0043] Specifically, a vision training program developed based on the Unity engine runs on a computing device. This program presents a main training interface to the user, containing multiple training mode entry points. The user selects the desired training mode through clicking, touching, or other methods. The startup module M1 receives the startup command and notifies the calling module M2 to load the training scene corresponding to that training mode. The training mode entry points use cartoon-style icons and large touch areas for easy identification and operation by children. Each training mode can be started independently for single-item training, or the system can automatically recommend combinations of training modes based on the user's historical training records.

[0044] The module M2 is invoked to call the corresponding functional components in the Unity engine according to the startup instruction, so as to generate visual stimulus images for the corresponding training mode.

[0045] Specifically, the functional components invoked by the calling module M2 include at least one of the following:

[0046] The Unity engine's graphics rendering system and physics collision detection module M21 are used to generate visual stimulus images that respond to click interactions.

[0047] The graphics rendering system and the physical collision detection module M21 randomly generate multiple target objects within the training area. These target objects are randomly assigned different shapes (e.g., circles, squares, triangles, stars), sizes (e.g., diameters of 20 to 80 pixels), or colors (e.g., red, blue, green, yellow) by the graphics rendering system. Each target object is equipped with a collision detection component from the Unity engine and possesses identifiable physical attributes. When a user clicks on a target object within the training area, the physical collision detection system uses ray detection to determine if the click hits the target: if it does, the score increases, the target object is eliminated, and a new target object is immediately generated at a random location within the area; if it misses (clicking on a blank area), it is considered a false click. The training process lasts for a preset duration (e.g., 60 seconds), during which the score, accuracy, and remaining time are displayed in real time. The attributes of the target objects can be dynamically adjusted according to the user's training level, including gradually reducing the size of the target objects to increase the recognition difficulty, or increasing the color similarity between the target objects and the background to improve visual discrimination requirements. The training interface adopts a child-friendly visual style, with target objects designed as cartoon images of fruits, animals, etc., to enhance the training interest of child users.

[0048] The Unity engine's stereo vision camera module M22 is used to generate stereo visual stimulus images with adjustable binocular parallax.

[0049] The stereo vision camera module M22 is used for fusion vision training and stereo vision training, both of which are achieved through a regular VR glasses box in conjunction with a smartphone to achieve binocular split-view display.

[0050] In the fusion vision training, this module achieves binocular split-view display in the following ways: (1) Screen split-view layout: The smartphone screen is horizontally divided into two areas, left and right. The left area is used to display the left eye view, and the right area is used to display the right eye view. After the user inserts the phone into the VR glasses box, the left eye only sees the left side of the screen and the right eye only sees the right side of the screen through the lens group inside the box; (2) Interpupillary distance adjustment: The VR glasses box is equipped with an interpupillary distance adjustment knob. The user can rotate the knob to move the lens spacing to match their own interpupillary distance; (3) Binocular camera creation and configuration: The Unity engine creates two ordinary cameras in the scene - a left eye camera and a right eye camera. The viewport of the left eye camera is set to the left half of the screen (x=0, y=0, width=0.5, height=1), and the viewport of the right eye camera is set to the right half of the screen (x=0.5, y=0, width=0.5, height=1). (height=1), the left and right eye cameras maintain a certain distance in the horizontal direction to simulate the position of the left and right eyes of a person, respectively, and render the corresponding viewport areas, which are then output to the same mobile phone screen; (4) Parallax parameter adjustment: the size of the binocular parallax is controlled by adjusting the convergence distance parameter. The convergence distance ranges from 0.5 meters to 10 meters. The closer the convergence distance, the greater the binocular parallax and the higher the fusion difficulty. The farther the convergence distance, the smaller the parallax and the lower the fusion difficulty. The training difficulty starts from a large convergence distance (e.g., 10 meters, with a small parallax) and gradually decreases the convergence distance (e.g., 1 meter, with a large parallax) to transition to subtle fusion differences, gradually improving binocular fusion ability. During the training process, the user interacts by touching the mobile phone screen or using the Bluetooth controller to confirm the completion of fusion. The system records the completion status of each difficulty level.

[0051] In stereoscopic vision training, this module uses a stereoscopic vision camera system powered by the Unity engine to generate a 3D scene with depth information within the VR headset. Training content includes tasks such as determining the relative distances of multiple objects in the virtual scene, locating objects in 3D space, and recognizing 3D graphics with depth differences. The system controls the difficulty level of stereoscopic vision training by adjusting the depth differences of objects in the scene (e.g., a distance difference of 0.5 meters to 5 meters between objects), and records the user's reaction time and judgment accuracy, dynamically adjusting the degree of depth difference in the scene based on the training data.

[0052] The Unity engine's shader module M23 is used to generate visual stimulus images with separate color channels.

[0053] The shader module M23 is used for simultaneous vision training. It uses the Unity engine's shader system to separate the color channels of visual stimulus images, generating red-blue or red-green difference-separated images. The content of the left-eye and right-eye views is different, and they are not intended to be merged into a single image. The user wears color-coded glasses (e.g., red-blue glasses). The left eye sees only the blue channel image content through the red filter, and the right eye sees only the red channel image content through the blue filter, thus enabling both eyes to receive different visual information simultaneously. Training tasks include simultaneously tracking targets displayed in two different color channels on the screen and simultaneously recognizing information in the left and right eye images to enhance the collaborative ability of both eyes to work simultaneously. The color combinations for color difference separation can also include complementary color combinations such as red-green and red-cyan. The training difficulty can be adjusted by modifying parameters such as the degree of difference between the left and right eye image content, the target movement speed, and the target size.

[0054] The Unity engine's particle system and animation module M24 are used to generate visually stimulating images with dynamically changing background raster.

[0055] The particle system and animation module M24 are used for de-inhibition training. Hundreds to thousands of particles are emitted through the Unity engine's particle system, forming a large number of dynamically changing background particles. The animation system controls the rotation, movement, scaling, and other changes of these particles, creating a dynamically changing background raster. The background raster's color combinations include black and white, red and black, red, white and black, red and blue, red, white and blue, or red and green. The raster shape includes rotating stripes, squares, or spheres. A target object is superimposed on the dynamic raster background, and the user needs to complete target recognition or tracking tasks against this changing background interference. The changing speed, density, and color contrast of the background raster are adjusted according to the training difficulty. The target object's contrast starts at high contrast and gradually decreases the color difference between the target and the background to increase the difficulty of de-inhibition training. By alternately strengthening visual interference and stimulation, the suppressed visual pathways are activated.

[0056] The Unity engine's post-processing effects module M25 is used to generate visual stimulus images with a controllable duration.

[0057] The post-processing effects module M25 is used for afterimage training. It generates high-contrast stimulus patterns on the screen using the Unity engine's post-processing effects system. These stimulus patterns include black and white raster patterns, rotating stripes, square patterns, or light sphere patterns. The post-processing effects system overlays effects such as brightness enhancement, contrast enhancement, and edge sharpening to generate stimulus patterns of different intensity levels. The stimulus patterns are displayed for a preset duration (e.g., 30 seconds) before disappearing. The stimulus intensity is controlled by parameters of the post-processing effects system, and the display duration of the stimulus patterns is adjustable within the range of 10 to 60 seconds. After the stimulus patterns disappear, the screen switches to a dark background, allowing the user to observe and provide feedback on parameters such as the duration and clarity of the visual afterimage. The system dynamically adjusts the stimulus intensity and display duration based on user feedback to ensure the safety and effectiveness of the training.

[0058] The receiving module M3 is used to receive interactive operation data generated by the user based on the visual stimulus image.

[0059] Specifically, the receiving module M3 receives different types of interactive operation data generated by the user in each training mode, including: the number of click hits and the number of accidental touches in the first training mode, the fusion completion confirmation and stereo judgment results in the second training mode, the eye tracking completion rate in the third training mode, the target recognition accuracy in the fourth training mode, and the afterimage duration and user feedback data in the fifth training mode.

[0060] The recording module M4 is used to process the interactive operation data according to preset rules, generate training feedback information, and store training records.

[0061] Specifically, the recording module M4 processes the interactive operation data according to the preset scoring rules of each training mode: in the first training mode, the score and accuracy are calculated based on the number of hits and misclicks; in the second training mode, the fusion success rate and stereo vision accuracy are calculated; in the third training mode, the binocular tracking completion rate is calculated; in the fourth training mode, the target recognition accuracy is calculated; and in the fifth training mode, the afterimage duration and user feedback data are recorded. The recording module generates training feedback information from the processing results, which is displayed to the user in real time in the form of score, level, completion rate, etc. After training, the training record, which includes user information, training mode type, score, accuracy, training duration, and timestamp, is stored locally in an SQLite database and synchronized with the cloud server in JSON format via HTTPS. The recording module also dynamically adjusts the difficulty parameters and combinations of each training mode based on the user's training history data to generate personalized training plans. The data synchronized to the cloud is available for doctors to remotely monitor and adjust the training plan.

[0062] Example 2

[0063] like Figure 1 As shown, a method for amblyopia training based on multi-modal visual stimulation using the Unity engine is described, comprising the following steps:

[0064] Step S1: Receive the user's start command for the training mode, and start the corresponding training mode according to the start command.

[0065] A vision training program developed based on the Unity engine runs on a computing device. The program presents a main training interface to the user, which includes multiple training mode entry points. The user selects the desired training mode by clicking, touching, or other means. The vision training program receives the start command and loads the training scene corresponding to that training mode.

[0066] In practical applications, the training mode entry point can adopt a cartoonish icon and a large touch area design to facilitate recognition and operation by child users. Each training mode can be started independently to perform single training, or the system can automatically recommend combinations of training modes based on the user's historical training records.

[0067] Step S2: Receive interactive operation data generated by the user based on the training mode.

[0068] Step S2 specifically includes any one or a combination of the following training modes:

[0069] S21: First training mode, which calls the Unity engine's graphics rendering and physical collision detection system to generate and display visual stimulus images with recognizable physical properties and responsive click interactions.

[0070] like Figure 2 As shown, the training program randomly generates multiple target objects within the training area using the Unity engine's graphics rendering system. These target objects are randomly assigned different shapes (e.g., circles, squares, triangles, stars), sizes (e.g., diameters of 20 to 80 pixels), or colors (e.g., red, blue, green, yellow) by the graphics rendering system. Each target object is equipped with a collision detection component from the Unity engine and possesses identifiable physical properties. When a user clicks on a target object within the training area, the physical collision detection system uses raycasting to determine if the click hits the target. If it does, the score increases, the target object is eliminated, and a new target object is immediately generated at a random location within the area; if it misses (clicking on a blank area), it is considered a false hit. The training process lasts for a preset duration (e.g., 60 seconds), during which the score, accuracy, and remaining time are displayed in real time.

[0071] In practical applications, the attributes of the target object can be dynamically adjusted according to the user's training level. For example, the size of the target object can be gradually reduced to increase the recognition difficulty, or the color similarity between the target object and the background can be increased to improve the visual discrimination requirements. In addition, the training interface can adopt a child-friendly visual style, with the target object designed as cartoon images of fruits, animals, etc., to enhance the training interest of child users.

[0072] S22: Second training mode, which calls the stereo vision camera system of the Unity engine to generate and display stereo vision stimulus images with adjustable binocular parallax.

[0073] The second training mode includes fusion vision training and stereoscopic vision training, both of which use a regular VR glasses box in conjunction with a smartphone to achieve binocular split-view display.

[0074] S221: Fusion Vision Training. The training procedure achieves binocular split vision display through the following steps:

[0075] Step 2211: Split-view Layout. After the training program starts, the smartphone screen is horizontally divided into two areas: the left area displays the left-eye view, and the right area displays the right-eye view. After the user inserts the phone into the VR glasses box, through the lens assembly inside the box, the left eye only sees the left side of the screen, and the right eye only sees the right side of the screen, thus achieving binocular split-view display.

[0076] Step 2212: Interpupillary Distance Adjustment. The VR headset has an interpupillary distance adjustment knob. By rotating this knob, the user can move the lens spacing to match their own interpupillary distance, thereby obtaining the correct binocular vision effect. Interpupillary distance adjustment is entirely completed through the physical knob of the VR headset; the training program does not involve the acquisition or setting of interpupillary distance parameters.

[0077] Step 2213: Creating and Configuring the Two-Eye Camera. The Unity engine creates two regular cameras (non-VR-specific cameras) in the scene: a left-eye camera (LeftCamera) and a right-eye camera (RightCamera). The configuration of the two cameras is as follows:

[0078] (1) Viewport settings: The viewport of the left eye camera is set to the left half of the screen (x=0, y=0, width=0.5, height=1), and the viewport of the right eye camera is set to the right half of the screen (x=0.5, y=0, width=0.5, height=1).

[0079] (2) Position offset: The left eye camera and the right eye camera are kept at a certain distance in the horizontal direction so that the two cameras simulate the position of the left eye and the right eye of a person, respectively;

[0080] (3) Rendering target: The two cameras render their respective viewport areas and output them to the same mobile phone screen.

[0081] Step 2214: Disparity parameter adjustment. The binocular disparity is controlled by adjusting the following parameters to achieve fusion vision training of varying difficulty:

[0082] (1) Convergence Distance: Adjusts the distance between the target convergence points of the left and right eye cameras, ranging from 0.5 meters to 10 meters. The closer the convergence distance, the greater the binocular parallax and the more difficult the fusion; the farther the convergence distance, the smaller the parallax and the easier the fusion.

[0083] (2) Parallax formula: Parallax angle θ = 2 × arctan(d / (2 × D)), where d is the distance between the two cameras and D is the convergence distance;

[0084] (3) Scene object depth offset: While keeping the convergence distance unchanged, the depth position of the target object in the scene can be fine-tuned to generate training images with different fusion difficulties.

[0085] Step 2215: Binocular Fusion Training Process. After the user inserts their phone into the VR headset and wears it, two images with preset parallax are displayed on the left and right sides of the screen. The user adjusts their binocular visual axes to fuse the images received by the left and right eyes into a single stereoscopic image in their brain. The training difficulty starts with a large convergence distance (e.g., 10 meters, small parallax), gradually decreasing the convergence distance (e.g., 1 meter, large parallax), transitioning to subtle fusion differences, progressively improving binocular fusion ability. During training, the user interacts by touching the phone screen or using a Bluetooth controller to confirm the completion of fusion, and the system records the completion status for each difficulty level.

[0086] S222: Stereo Vision Training. The training program uses a stereo vision camera system powered by the Unity engine to generate a 3D scene with depth information within a VR headset. The implementation of stereo vision training is the same as fusion vision training, both using split-view display on the screen in conjunction with the VR headset to achieve binocular split-view display. Training content includes tasks such as judging the relative distances of multiple objects in a virtual scene, locating objects in 3D space, and recognizing 3D graphics with depth differences. The system controls the difficulty level of stereo vision training by adjusting the depth position differences of objects in the scene (e.g., a distance difference of 0.5 meters to 5 meters). The system records the user's reaction time and judgment accuracy, and dynamically adjusts the depth difference of the scene based on the training data.

[0087] S23: The third training mode, which calls the Unity engine's shader system to generate and display visual stimulus images with different color channel separations. For example... Figure 5As shown, this training mode is simultaneous visual training.

[0088] The training program uses the Unity engine's shader system to separate the color channels of visual stimulus images, generating red-blue or red-green difference images. The content of the left-eye and right-eye views is different, and they are not intended to be merged into a single image. Users wear color-coded glasses (such as red-blue glasses), so the left eye sees only the blue channel image content through the red filter, and the right eye sees only the red channel image content through the blue filter, thus enabling both eyes to receive different visual information simultaneously. Training tasks include simultaneously tracking targets displayed on two different color channels on the screen and simultaneously recognizing information in the left and right eye images, strengthening the collaborative ability of both eyes to work simultaneously.

[0089] In practical applications, the color combinations for color difference separation can also include complementary color combinations such as red-green and red-cyan. Training difficulty can be adjusted by modifying parameters such as the degree of difference between the left and right eye image content, the target movement speed, and the target size.

[0090] S24: The fourth training mode, which utilizes the Unity engine's particle and animation systems to generate and display visually stimulating images with dynamically changing background rasters and overlaid targets. For example... Figure 6 As shown, this training mode is desuppression training.

[0091] The training program generates a large number of dynamically changing background particles using the Unity engine's particle system. An animation system controls the rotation, movement, and scaling of these particles to create a dynamically changing background raster. The background raster's color combinations include black and white, red and black, red, white and black, red and blue, red, white and blue, or red and green, and its shapes include rotating stripes, squares, or spheres. Target objects are superimposed on this dynamic raster background, and users must complete target recognition or tracking tasks against this changing, distracting background. The background raster's changing speed, density, and color contrast can be adjusted according to the training difficulty, activating suppressed visual pathways through alternating forms of visual interference and stimulation.

[0092] In practical applications, the particle system can emit hundreds to thousands of particles to form a dense, dynamic interference background. The contrast of the target object can start at high contrast and gradually decrease the color difference between the target and the background to increase the difficulty of de-inhibition training.

[0093] S25: The fifth training mode, which utilizes the Unity engine's post-processing effects system to generate and display visual stimulus images with high contrast and controllable duration. For example... Figure 7 As shown, this training mode is post-image training.

[0094] The training program uses the Unity engine's post-processing effects system to generate high-contrast stimulus patterns on the screen. These patterns include black and white raster graphics, rotating stripes, squares, or light spheres. The stimulus patterns remain displayed for a preset duration (e.g., 30 seconds) before disappearing. The stimulus intensity is controlled by parameters from the post-processing effects system. After the stimulus patterns disappear, the screen switches to a dark background, allowing the user to observe and provide feedback on parameters such as the duration and clarity of the afterimage. The system dynamically adjusts the stimulus intensity and display duration based on user feedback to ensure the safety and effectiveness of the training.

[0095] In practical applications, the post-processing effects system can overlay effects such as brightness enhancement, contrast enhancement, and edge sharpening to generate stimulus patterns of different intensities. The display duration of the stimulus pattern can be adjusted from 10 to 60 seconds.

[0096] S26: The sixth training mode, which calls the Unity engine's graphics rendering system to generate and display quantized visual targets that meet vision test standards.

[0097] The training program uses the Unity engine's graphics rendering system to generate quantized optotypes based on visual acuity testing standards. These optotypes employ standard visual acuity chart patterns, such as the "E" optotype or the Landolt ring optotype. The specific calculation method for optotype size is as follows:

[0098] Step 261: Determine the correspondence between visual acuity and visual angle. According to the design principles of the international standard visual acuity chart, the following correspondence exists between visual acuity and the corresponding visual angle of the optotype:

[0099]

[0100] The formula for calculating the angle of view is: Angle of view (arc minutes) = 1 / visual acuity value × 1 arc minute;

[0101] Step 262: Calculate the physical size of the target based on the viewing angle and test distance. The formula for calculating the display size (height) of the target on the screen is:

[0102] The height of the target, h, is given by: 2 × D × tan(θ / 2);

[0103] in:

[0104] -h is the display height of the cursor on the screen (in meters);

[0105] -D represents the test distance, which is the vertical distance from the user's eyes to the screen (unit: meters).

[0106] -θ is the angle of view (unit: radians), θ = angle of view (ar minutes) × π / (60 × 180);

[0107] The simplified calculation formula is: sight height h≈D×θ (radians);

[0108] Specific examples:

[0109] When the test distance D = 3 meters and the target visual acuity value = 0.5:

[0110] The angle θ = 2 arcminutes = 2 × π / (60 × 180) ≈ 0.000582 radians;

[0111] The height of the target, h = 3 × 0.000582 ≈ 0.00175 meters = 1.75 millimeters;

[0112] Step 263: Pixel Size Conversion. Based on the calculated physical size of the target image, and combined with the screen's pixel density (PPI, PixelsPerInch), the physical size is converted into pixel size:

[0113] Target pixel height = Target height h × PPI / 0.0254;

[0114] Where 0.0254 is the conversion factor from inches to meters.

[0115] Step 264: Optical Target Generation and Display. The Unity engine's graphics rendering system generates an "E"-shaped or Landolt ring-shaped optical target of the corresponding size based on the calculated pixel dimensions. The opening direction of the optical target (top, bottom, left, right) is randomly generated. The system displays optical targets of different sizes sequentially according to the row requirements of a standard vision chart, and the user provides feedback on the opening direction of the optical targets through interactive operations.

[0116] Step 265: Visual Acuity Determination. The system uses the staircase method to determine the user's visual acuity threshold:

[0117] (1) Start displaying from the largest optotype (corresponding to a visual acuity of 0.1);

[0118] (2) Each time a user correctly identifies a target, they move to the next smaller target in the next row;

[0119] (3) For each incorrectly identified target by the user, return the larger target from the previous row;

[0120] (4) Correctly identify 5 optotypes of the same size in succession, and record this visual acuity as the effective visual acuity;

[0121] (5) If three identical optotypes are incorrectly identified consecutively, record the visual acuity value in the previous row as the final visual acuity;

[0122] (6) The final visual acuity value is the minimum visual acuity value at which the user can correctly identify more than 80% of the visual targets.

[0123] S27: The seventh training mode, which calls the Unity engine's graphics rendering system to generate and display visual stimulation images of distant viewing scenes to guide users to relax their ciliary muscles and extraocular muscles.

[0124] like Figure 8 As shown, the training program uses the Unity engine's graphics rendering system to generate simulated distant natural scenes, such as distant mountains, sea levels, and skylines, creating a sense of depth. The scene may include slowly moving visual guide elements (such as flocks of birds flying slowly or drifting clouds) to guide the user's gaze into the distance, allowing the ciliary muscles to fully relax. The training typically lasts 3-5 minutes, accompanied by soothing background music or voice guidance, prompting the user to take deep breaths and relax their eye muscles.

[0125] In practical applications, the distant viewing scene can be switched to different natural themes according to user preferences, such as forests, beaches, and grasslands. Simple eye-training animations can be incorporated into the scene to guide users in eye relaxation exercises such as moving their eyes up, down, left, and right.

[0126] S28: The eighth training mode, which calls the Unity engine's graphics rendering system to generate and display a picture book story interface containing text and images, guiding users to read visually stimulated images with alternating perspectives.

[0127] The training program uses the Unity engine's graphics rendering system to display text and image areas on the screen in separate sections. The text and image areas are arranged in a pre-defined alternating layout, guiding the user's gaze to frequently switch between close-up text and distant images during reading. The system can also control the alternation rhythm of text and image display through the Unity engine's animation system, such as displaying close-up text for 5 seconds, then switching to a distant image for 5 seconds, and so on, to train the ciliary muscle's adjustment ability and eye-hand coordination. The picture book stories cover scientific knowledge, fairy tales, etc., to enhance user reading interest and training compliance.

[0128] In practical applications, the display distance between text and images can be adjusted by changing the font size and image dimensions to simulate different viewing distances. Picture book stories can be categorized by theme, allowing users to select different story content based on their interests.

[0129] Step S3: Process the interactive operation data according to preset rules, generate training feedback information, and store the training records in the database.

[0130] The amblyopia training program processes user interaction data generated during training according to preset scoring rules for each training mode. Different types of interaction data include: the number of successful clicks and false clicks in the first training mode, used to calculate scores and accuracy; the fusion completion confirmation and stereoscopic judgment results in the second training mode, used to calculate fusion success rate and stereoscopic accuracy; the binocular tracking completion rate in the third training mode; the target recognition accuracy in the fourth training mode; the afterimage duration and user feedback data in the fifth training mode; and the optotype recognition accuracy and final visual acuity value in the sixth training mode. The system generates training feedback information from the processing results, displaying it to the user in real-time in the form of scores, levels, and completion rates. After training, the system stores training records containing user information, training mode type, score, accuracy, training duration, and timestamp locally in an SQLite database, and simultaneously synchronizes the data with a cloud server in JSON format via HTTPS.

[0131] In practical applications, the preset rules can be configured with different scoring standards for users of different age groups. Training feedback information can be displayed to users and caregivers through visual charts (such as trend graphs and radar charts) to show training progress. Data synchronized to the cloud can be used by doctors for remote monitoring and adjustment of training plans.

[0132] Advantages of this embodiment: Through the above technical solution, this embodiment can selectively call specific functional components of the Unity engine, such as the graphics rendering and physical collision detection system, stereo vision camera system, shader system, particle system and animation system, and post-processing effects system, according to the clinical needs of different training modes, to accurately generate visual stimulus images that match each training mode. This achieves a systematic integration of multi-dimensional visual function training, including fine visual acuity training, fusion vision training, stereo vision training, simultaneous vision training, desuppression training, afterimage training, and hyperopia quantification training. It solves the problem of existing technologies relying solely on simple planar image display and failing to systematically utilize the underlying functional components of the Unity engine, significantly improving the accuracy of visual stimulus generation and the relevance of training content in amblyopia training, ultimately effectively improving the visual function of amblyopic users.

[0133] Example 3

[0134] This embodiment describes the actual operation process of a user using the amblyopia training method based on multi-modal visual stimulation of the Unity engine. The method includes the following steps:

[0135] Step B1: User login and initial assessment.

[0136] Users (usually children or adolescents) launch an amblyopia training program developed using the Unity engine on a computing device (such as a smartphone or tablet) with the assistance of their guardians. Upon first use, users register an account and fill in basic information, including name, age, gender, date of birth, and attending physician. After completing registration, the system guides users through the initial vision assessment phase.

[0137] In practical applications, the login method can also adopt biometric authentication methods such as facial recognition or fingerprint recognition, which makes it easy for child users to log in quickly without having to remember complex passwords.

[0138] Step B2: Perform farsightedness quantification training to obtain an initial visual baseline.

[0139] The system automatically guides the user into the sixth training mode (farsightedness quantification training) to conduct a vision test. For example... Figure 9 As shown, the user first sets the test distance (e.g., 1 meter, 3 meters, 5 meters), and the system automatically calculates the target size based on the distance. Figure 10 As shown, the screen randomly displays "E" optotypes of different sizes and opening directions. Users can indicate the direction of the optotype opening by clicking the direction button. The system automatically adjusts the size of the optotype based on the accuracy of the user's answers until it determines the minimum resolving power of the user's eyes, generates initial visual baseline data, and stores it in the database.

[0140] Step B3: The system recommends a training plan, and the user selects a training mode.

[0141] The system reads the initial vision assessment results and user age, and automatically recommends suitable training programs. For example, for users with amblyopia who are found to have significant monocular suppression, the system prioritizes simultaneous vision training and dissuppression training; for users with weak fusion function, fusion vision training is prioritized. Users can view the recommended programs on the main training interface, or choose any training mode according to their own needs. The main training interface uses cartoon icons to display the entrances to each training mode, and each entrance is labeled with the training name and estimated duration.

[0142] In practical applications, the system can also lock the duration and sequence of specific training modes based on the training prescriptions set remotely by the attending physician, ensuring the professionalism and standardization of the training plan.

[0143] Step B4: Execute multiple training modes sequentially to complete a single training task.

[0144] Users can enter different training modes sequentially, either according to the recommended plan or by their own selection. Let's take a typical training combination as an example:

[0145] Users first enter the first training mode (fine visual acuity training), such as... Figure 2As shown, participants have 60 seconds to click on randomly distributed target objects on the screen, and the system displays the score and accuracy in real time. After completing the fine visual training, the system automatically records the training data.

[0146] Subsequently, the user enters the fusion vision training in the second training mode, such as... Figure 3 As shown, wearing red-blue glasses and facing a binocular disparity image on the screen, the user adjusts their binocular visual axes to fuse the images into a single stereoscopic image. The system determines the fusion completion status based on user confirmation and dynamically adjusts the disparity difficulty. After the fusion training is completed, the user can continue to stereoscopic vision training, such as... Figure 4 As shown, this demonstrates how to determine the distance between objects in a 3D scene.

[0147] Next, the user enters the fourth training mode (disinhibition training), such as... Figure 6 As shown, the user tracks and identifies target objects against a dynamically changing grating background, completing a disinhibition stimulation task. Finally, the user enters the seventh training mode (relaxation training), as shown... Figure 8 As shown, watch a distant scene for 3 minutes, accompanied by voice guidance to relax eye muscles and relieve eye strain during training.

[0148] In practical applications, users can pause or exit training at any time, and the system automatically saves the current training progress. The order and duration of each training mode can be dynamically adjusted based on the user's actual training feedback.

[0149] Step B5: Generate a training report and synchronize data to the cloud.

[0150] After a single training session is completed, the system processes the interactive operation data collected in each training mode according to preset rules and generates a training report for that session. The training report includes data such as the score, accuracy, completion rate, and training duration for each training mode, as well as training trend analysis displayed in visual charts (such as line graphs and radar charts). Training records are stored locally using SQLite and automatically synchronized to the cloud server in JSON format via HTTPS protocol when connected to the network.

[0151] Guardians can view training reports and understand the user's training progress through their own devices. Attending physicians can remotely access cloud data through their own devices, adjust subsequent training prescriptions based on the training reports, and the system automatically updates the user's training plan. The system also dynamically adjusts the difficulty parameters and combinations of various training modes based on the user's historical training data, enabling personalized training plan iterations.

[0152] The advantages of this embodiment are as follows: Through the above operation process, users only need to install an amblyopia training program developed based on the Unity engine on their computing devices to complete the entire training loop, from initial vision assessment, personalized program recommendation, multi-modal visual function training to data synchronization in the cloud. This embodiment fully leverages the cross-platform deployment advantages of the Unity engine, freeing amblyopia training from being limited to hospitals or professional institutions, and enabling convenient, systematic, and home-based daily training, significantly improving training compliance and treatment effectiveness for amblyopia users.

[0153] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-modal visual stimulation amblyopia training system based on the Unity engine, characterized in that, include: The startup module is used to receive user commands to start the training module; The calling module is used to call the corresponding functional components in the Unity engine according to the startup instruction in order to generate visual stimulus images for the corresponding training mode; The receiving module is used to receive interactive operation data generated by the user based on the visual stimulus image; The recording module is used to process the interactive operation data according to preset rules, generate training feedback information, and store training records. The functional components invoked by the calling module include at least several of the following: The Unity engine's graphics rendering system and physics collision detection module are used to generate visual stimulus images that respond to click interactions; The Unity engine's stereo vision camera module is used to generate stereo visual stimulus images with adjustable binocular parallax. The shader module of the Unity engine is used to generate visual stimulus images with separate color channels; The particle system and animation module of the Unity engine are used to generate visually stimulating images with dynamically changing background raster. The Unity engine's post-processing effects module is used to generate visual stimulus images with a controllable duration.

2. The amblyopia training system based on multi-modal visual stimulation using the Unity engine according to claim 1, characterized in that, The visual stimulus image generated by the graphics rendering system and the physical collision detection system contains multiple target objects that randomly appear in the training area. The target objects are randomly assigned different shapes, sizes or colors. The physical collision detection system identifies the user's click hit operation through ray detection.

3. The amblyopia training system based on multi-modal visual stimulation using the Unity engine according to claim 1, characterized in that, The stereoscopic visual stimulus images generated by the stereoscopic vision camera system are used for fusion vision training and stereoscopic vision training; in the fusion vision training, by adjusting the interocular distance parameter and convergence distance parameter of the stereoscopic vision camera system, binocular disparity images with different fusion difficulties are generated.

4. The amblyopia training system based on multi-modal visual stimulation using the Unity engine according to claim 1, characterized in that, The color channel-separated visual stimulus images generated by the shader system are red-blue or red-green difference-separated images, with the contents of the left-eye view and the right-eye view being different from each other.

5. The amblyopia training system based on multi-modal visual stimulation using the Unity engine according to claim 1, characterized in that, The color combinations of the dynamically changing background raster generated by the particle system and animation system include black and white, red and black, or red and blue, and the raster shape includes rotating stripes or squares.

6. The amblyopia training system based on multi-modal visual stimulation using the Unity engine according to claim 1, characterized in that, The high-contrast visual stimulation image generated by the post-processing effects system includes black and white raster or rotating stripes, and the display duration is controlled by controlling the activation duration of the post-processing effects system.

7. The amblyopia training system based on multi-modal visual stimulation using the Unity engine according to claim 1, characterized in that, The calling module is also used to call the Unity engine's graphics rendering system to generate quantitative visual targets that conform to vision testing standards.

8. The amblyopia training system based on multi-modal visual stimulation using the Unity engine according to claim 1, characterized in that, The system also dynamically adjusts the combination of training modes and difficulty parameters based on the training records to generate personalized training plans.

9. The amblyopia training system based on multi-modal visual stimulation using the Unity engine according to claim 1, characterized in that, The calling module is also used to call the Unity engine's graphics rendering system to generate a distant view scene, which includes slowly moving visual guide elements.

10. The amblyopia training system based on multi-modal visual stimulation using the Unity engine according to claim 1, characterized in that, The calling module is also used to call the Unity engine's graphics rendering system to generate a picture book story interface that includes text and image areas. The text and image areas are arranged in an alternating layout, and the Unity engine's animation system controls the text and images to be displayed in an alternating rhythm.

Images