Visual training method and device based on image recognition, equipment and medium

Abstract

The invention discloses a visual training method, device and equipment based on image recognition and a medium, and belongs to the field of visual training, and the method comprises the steps: collecting a retina image of a user through a fundus camera, and carrying out the preprocessing of the retina image, and obtaining a macular region image; performing image identification on the yellow spot area image to obtain a spatial position coordinate of the central recess; performing light stimulation on the retina of the user through a preset light source to achieve the purpose of visual training; wherein a preset illumination template is arranged at a position corresponding to the spatial position coordinates, so that the central fovea is not stimulated by illumination; in the illumination stimulation process, eyeball movement of the user is tracked in real time, and the illumination template is synchronously moved based on the eyeball movement, so that the illumination template continuously shields the fovea. Therefore, by implementing the visual training method and device, the accuracy of visual training can be improved, and closed-loop control can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of visual training, and in particular to a visual training method and device based on image recognition, equipment and medium. BACKGROUND

[0002] Visual training (such as afterimage therapy and parafoveal fixation training) is a key intervention method for improving parafoveal fixation amblyopia. The core principle is to block the macular fovea (also known as the fovea, a small depression in the center of the retina responsible for providing clear central vision, the most sensitive area of vision) and accurately stimulate the area around the macular fovea, guiding the user to return from parafoveal fixation to central fixation, thereby strengthening the function of the macular fovea, and ultimately achieving vision improvement and fixation stability improvement.

[0003] However, the current equipment for the above visual training has significant limitations, which are reflected in the following aspects: on the one hand, most of the existing equipment uses an empirical and human-dependent method for foveal positioning and stimulation setting: for example, by using a Haidinger brush to guide the user's fixation, the trainer determines the foveal position according to the feedback, or using a physical cover to avoid central area illumination. This kind of way is affected by the experience of the trainer and the cooperation degree of the user (such as the distraction of children or low-cooperation users, resulting in inaccurate feedback), which is easy to cause large foveal positioning error, and the operation process is complicated. On the other hand, although the current fundus camera has high-resolution imaging capability, it can capture the retina, macular area and other ocular posterior segment structures to provide basic data support for foveal positioning, but most of the equipment only stays at the static inspection level, lacking automatic recognition and target analysis capability of fundus images, and also lacking dynamic light adjustment and feedback control function. This leads to the fact that the existing technology cannot correct the stimulation position in real time according to the eye movement of the user during the training process, and it is difficult to form a closed-loop control system. SUMMARY

[0004] The present application provides a visual training method, device, equipment and medium based on image recognition, which can improve the accuracy of visual training and realize closed-loop control.

[0005] The present application provides a visual training method based on image recognition, comprising:

[0006] The fundus camera is used to collect the retinal image of the user, and the macular region image is obtained by preprocessing the retinal image;

[0007] The macular region image is subjected to image recognition to obtain the spatial position coordinates of the fovea;

[0008] The retina of the user is stimulated by a preset light source to achieve the purpose of visual training; wherein, a preset light stimulation template is arranged at a position corresponding to the spatial position coordinate, so that the fovea is not stimulated by light;

[0009] During the light stimulation process, the eye movement of the user is tracked in real time, and the light stimulation template is moved synchronously based on the eye movement, so that the light stimulation template continuously blocks the fovea.

[0010] The embodiment of the present application can ensure accurate coverage of the macular region before visual training by collecting the retinal image of the user through the fundus camera and preprocessing the retinal image, and can provide reliable data basis for subsequent fovea positioning and light stimulation; the spatial position coordinate of the fovea can be accurately extracted by image recognition of the macular region image, which provides a position basis for subsequent light stimulation template blocking; the fovea is not stimulated by light, and the retina around the fovea is stimulated by light, so that the negative afterimage is effectively induced, the user gradually returns to central fixation from peripheral central fixation, and the purpose of visual training is achieved, by presetting the light source to stimulate the retina of the user; and the light stimulation template is moved synchronously based on the eye movement during the light stimulation process, which can dynamically correct the blocking position of the light stimulation template, ensure the accuracy of visual training, and realize closed-loop control. Compared with the prior art, the present application can improve the accuracy of visual training and realize closed-loop control.

[0011] Further, before the collecting the retinal image of the user through the fundus camera and preprocessing the retinal image to obtain the macular region image, the method further comprises:

[0012] The head position of the user is fixed by the head support structure, so that the user can observe the observation window in a straight line of sight.

[0013] The fixation target is projected in the observation window, so that the user can fixate on the fixation target; wherein, the line of sight direction of the user when fixing on the fixation target is taken as the initial fixation direction.

[0014] The embodiment of the present application can help the user to stabilize the line of sight by guiding the user to fixate on the observation window through the head support structure, and provide a reference fixation direction for subsequent eye movement tracking.

[0015] Further, the collecting the retinal image of the user through the fundus camera and preprocessing the retinal image to obtain the macular region image comprises:

[0016] The retinal image is subjected to noise suppression to obtain a first retinal image;

[0017] The first retinal image is subjected to contrast enhancement to obtain a second retinal image;

[0018] The second retinal image is subjected to macular region cutting to obtain a macular region image.

[0019] The embodiment of the present application can ensure accurate coverage of the macular region before visual training by image preprocessing of the retinal image, and provide reliable data basis for subsequent fovea positioning and light stimulation.

[0020] Further, the image recognition of the macular region image obtains spatial position coordinates of the fovea, comprising:

[0021] The macular region image is subjected to feature extraction and spatial regression to obtain pixel coordinates of the fovea;

[0022] Based on the width of the observation window and the imaging resolution width of the fundus camera, the pixel coordinates are linearly converted into spatial position coordinates.

[0023] The embodiment of the present application can accurately extract the spatial position coordinates of the fovea by image recognition and coordinate conversion of the macular region image, and provide position basis for subsequent light template shielding.

[0024] Further, before the light stimulation of the user's retina by the preset light source to achieve the purpose of visual training, it further comprises:

[0025] Obtaining a preset illumination parameter of a trainer; wherein the illumination parameter comprises a light source wavelength, an energy density, a light spot diameter and an illumination time;

[0026] According to the light spot diameter, a corresponding light template is matched.

[0027] The embodiment of the present application can meet the needs of differentiated visual training by presetting the illumination parameter and the light template.

[0028] Further, the light stimulation of the user's retina by the preset light source to achieve the purpose of visual training comprises:

[0029] According to the light source wavelength, the energy density and the light spot diameter, a corresponding light source is set;

[0030] In the illumination time, the user's retina is subjected to light stimulation by the light source; wherein the preset light template is set at a position corresponding to the spatial position coordinates, so that the fovea is not subjected to light stimulation.

[0031] The retina of the user is stimulated by the preset light source, and the preset light stimulation template is arranged at the position corresponding to the spatial position coordinates, so that the fovea is not stimulated by light, and the retina around the fovea is stimulated by light, thereby effectively inducing the generation of a negative afterimage, guiding the user to gradually return to central fixation from peripheral central fixation, and achieving the purpose of visual training.

[0032] Further, in the process of light stimulation, the eye movement of the user is tracked in real time, and the light stimulation template is moved synchronously based on the eye movement, so that the light stimulation template continuously blocks the fovea, comprising:

[0033] Based on the eye movement data collected by the infrared eye tracker in real time, the offset amount of the real-time fixation direction of the user compared with the initial fixation direction is calculated;

[0034] Based on the offset amount, the light stimulation template is moved synchronously by controlling the deflector, so that the light stimulation template continuously blocks the fovea.

[0035] In the process of light stimulation, the eye movement of the user is tracked in real time, and the light stimulation template is moved synchronously based on the eye movement, so that the light stimulation template continuously blocks the fovea, comprising:

[0036] Another embodiment of the application also provides a visual training device based on image recognition, comprising: an image acquisition module, an image recognition module, a light stimulation module and an eye movement tracking module;

[0037] The image acquisition module is used to acquire the retina image of the user by an eye fundus camera, and pre-process the retina image to obtain a macular region image;

[0038] The image recognition module is used to perform image recognition on the macular region image to obtain the spatial position coordinates of the fovea;

[0039] The light stimulation module is used to stimulate the retina of the user by a preset light source to achieve the purpose of visual training; wherein a preset light stimulation template is arranged at the position corresponding to the spatial position coordinates, so that the fovea is not stimulated by light;

[0040] The eye movement tracking module is used to track the eye movement of the user in real time in the process of light stimulation, and move the light stimulation template synchronously based on the eye movement, so that the light stimulation template continuously blocks the fovea.

[0041] Another embodiment of the present application also provides a terminal device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, when the computer program is executed by the processor, steps of a visual training method based on image recognition are implemented.

[0042] Another embodiment of the present application also provides a computer readable storage medium, comprising a stored computer program, when the computer program is executed, the device where the computer readable storage medium is located executes steps of a visual training method based on image recognition. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A flowchart of an embodiment of the visual training method based on image recognition provided by the present application;

[0044] Figure 2 A structure diagram of an embodiment of the visual training device based on image recognition provided by the present application. DETAILED DESCRIPTION

[0045] For the purpose, technical scheme and advantages of the present application to be more clear, the technical scheme in the present application will be described clearly and completely in the following with the drawings in the embodiments of the present application, obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person of ordinary skill in the art without making creative labor, belong to the scope of protection of the present application.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs; the terms used herein are only for the purpose of describing specific embodiments, and are not intended to limit the present application; the terms in the specification and claims of the present application and the above description of drawings, "include" and "have", as well as any variation thereof, are intended to cover non-exclusive inclusion.

[0047] In the description of the embodiments of the present application, the technical terms "first", "second" and the like are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0048] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. As will be apparent to those of ordinary skill in the art, embodiments described herein can be combined with other embodiments.

[0049] In the description of the embodiments of the application, the term "and / or" is merely an association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0050] In the description of the embodiments of the application, the term "a plurality of" refers to two or more (including two), and similarly, "a plurality of groups" refers to two or more groups (including two groups), and "a plurality of pieces" refers to two or more pieces (including two pieces).

[0051] Reference Figure 1 To solve the problem of low fovea positioning accuracy and difficulty in meeting the visual training needs of low matching users in the prior art, an embodiment of the application provides a visual training method based on image recognition, which includes steps S101 to S104:

[0052] Step S101: acquiring a retinal image of a user by an eye fundus camera, and pre-processing the retinal image to obtain a macular region image.

[0053] Specifically, in order to keep the user's line of sight stable and reduce eye micro-movement, before the retinal image of the user is acquired by the eye fundus camera, it further includes: fixing the head position of the user through a head support structure to make the user look at the observation window; projecting a fixation target (such as a cross cursor, a dynamic animation or a product mascot image, etc.) in the observation window to make the user fixate on the fixation target; wherein the line of sight direction of the user when fixing the fixation target is taken as the initial fixation direction. When the positions of the user's eyes are stable, the retinal image of the user is acquired by the eye fundus camera.

[0054] In order to highlight the fovea structure feature, the retinal image needs to be pre-processed, including: noise suppression on the retinal image to obtain a first retinal image; contrast enhancement on the first retinal image to obtain a second retinal image; macular region cutting on the second retinal image to obtain a macular region image.

[0055] Step S102: performing image recognition on the macular region image to obtain the spatial position coordinates of the fovea.

[0056] Specifically, each eye has only one fovea, in order to accurately determine the position of the fovea on the fundus, a model with macular fovea positioning function (such as the fovea positioning module provided by Fundus Image Toolbox) can be called to perform image recognition on the macular region image, including: feature extraction and spatial regression on the macular region image to obtain the pixel coordinates of the fovea; based on the width of the observation window and the imaging resolution width of the fundus camera, linearly convert the pixel coordinates to spatial position coordinates. Wherein, let the pixel coordinates of the fovea be (fovea_x, fovea_y), the width of the observation window be (W_mm), and the imaging resolution width of the fundus camera be (W_px), then the spatial position coordinates of the fovea (fovea_x_space, fovea_y_space) can be linearly converted by the following formula:

[0057]

[0058] Step S103, the retina of the user is stimulated by a preset light source to achieve the purpose of visual training; wherein, a preset light stimulation template is set at a position corresponding to the spatial position coordinates, so that the fovea is not stimulated by light.

[0059] Specifically, in order to meet the differentiated visual training needs, before the retina of the user is stimulated by a preset light source, it further includes: obtaining the training instructor's preset illumination parameters; wherein, the illumination parameters include light source wavelength, energy density, spot diameter and illumination time; according to the spot diameter, a corresponding light stimulation template is matched. Wherein, the light stimulation template is an actually existing optical device, for example, a non-transparent blocking point is set in the center of a circular transparent lens or a light path modulation board, which is used to avoid the fovea from being stimulated by light; the size of the light stimulation template can be matched according to the spot diameter to realize accurate blocking of the fovea. In addition to the illumination parameters, the training instructor can also input the basic information of the user (such as name, number, age, diopter and fixation properties, etc.), and bind it with the visual training data of the user, so as to track and manage it later.

[0060] After the irradiation parameters and the light exposure template are set, visual training can be started. Specifically, a corresponding light source is set according to the wavelength, energy density and spot diameter of the light source. During the irradiation time, the retina of the user is stimulated by the light source. A preset light exposure template is set at a position corresponding to the spatial position coordinate, so that the fovea is not stimulated by light. Through the shielding of the light exposure template, the light source forms a light distribution pattern of "central shielding and peripheral irradiation" on the user's fundus, thereby effectively inducing the generation of negative afterimage, guiding the user to gradually return to central fixation from peripheral fixation, so as to achieve the purpose of visual training.

[0061] In step S104, the eye movement of the user is tracked in real time during the light stimulation, and the light exposure template is moved synchronously based on the eye movement, so that the light exposure template continuously shields the fovea.

[0062] Specifically, in order to overcome the interference of eye movement of children and low-compliance users on irradiation accuracy, closed-loop control can be realized through eye tracking during light stimulation. Specifically, based on the eye movement data (such as eye rotation and displacement data) collected by the infrared eye tracker in real time, the offset (which can be represented by angle or relative light path coordinate) of the real-time fixation direction of the user compared with the initial fixation direction is calculated. Based on the offset, the light exposure template is moved synchronously by controlling the deflector (such as a mechanical fine adjustment device), so that the light exposure template continuously shields the fovea.

[0063] Optionally, when it is detected that the user's line of sight frequently deviates or loses fixation for a long time, the irradiation can be automatically paused, and an acoustic and light prompt can be issued to remind the user to re-fixate the target. At the same time, compliance indicators (such as the proportion of effective fixation time and the number of deviations) are recorded, which provide quantitative basis for evaluating the quality of visual training. At the same time, the eye movement trajectory can be recorded synchronously during the visual training, and the eye movement trajectory, compliance indicators and irradiation parameters can be summarized as a data report after the visual training is completed, and saved locally or uploaded to the cloud for subsequent evaluation and optimization by the trainer.

[0064] The embodiment of the present application can ensure accurate coverage of the macular region before visual training by collecting a retinal image of a user through an ocular fundus camera and pre-processing the retinal image, and provide reliable data basis for subsequent fovea positioning and light stimulation; the spatial position coordinates of the fovea can be accurately extracted by image recognition on the macular region image, and the position basis for subsequent light template shielding is provided; the fovea is not stimulated by light, and the retina around the fovea is stimulated by light, so that the negative afterimage is effectively induced, the user gradually returns to central fixation from peripheral central fixation, and the purpose of visual training is achieved; the accuracy of visual training is ensured by dynamically correcting the shielding position of the light template through real-time tracking of the eye movement of the user during light stimulation and synchronously moving the light template based on the eye movement, and closed-loop control is realized.

[0065] Optionally, in the embodiment of the present application, before the retinal image of the user is collected through the ocular fundus camera and the retinal image is pre-processed to obtain the macular region image, the method further comprises:

[0066] The head position of the user is fixed by the head support structure, so that the user looks at the observation window;

[0067] The fixation target is projected in the observation window, so that the user fixates on the fixation target; wherein the line of sight direction of the user when the user fixates on the fixation target is taken as the initial fixation direction.

[0068] The embodiment of the present application guides the user to fixate through the head support structure and the observation window, which can help the user to stabilize the line of sight and provide a reference fixation direction for subsequent eye movement tracking.

[0069] Optionally, in the embodiment of the present application, the retinal image of the user is collected through the ocular fundus camera and the retinal image is pre-processed to obtain the macular region image, comprising:

[0070] The retinal image is subjected to noise suppression to obtain a first retinal image;

[0071] The first retinal image is subjected to contrast enhancement to obtain a second retinal image;

[0072] The second retinal image is subjected to macular region cutting to obtain a macular region image.

[0073] The embodiment of the present application can ensure accurate coverage of the macular region before visual training by pre-processing the retinal image, and provide reliable data basis for subsequent fovea positioning and light stimulation.

[0074] Optionally, in the embodiment of the present application, the image recognition on the macular region image obtains the spatial position coordinates of the fovea, comprising:

[0075] The feature extraction and spatial regression on the macular region image obtain the pixel coordinates of the fovea;

[0076] Based on the width of the observation window and the imaging resolution width of the fundus camera, the pixel coordinates are linearly converted into spatial position coordinates.

[0077] The embodiment of the present application can accurately extract the spatial position coordinates of the fovea by image recognition and coordinate conversion on the macular region image, and provide a position basis for subsequent light stimulation of the light template.

[0078] Optionally, in the embodiment of the present application, before the macular region image is irradiated by the preset light source to achieve the purpose of visual training, the method further comprises:

[0079] Obtaining the preset irradiation parameters of the trainer; wherein the irradiation parameters include the wavelength of the light source, the energy density, the spot diameter and the irradiation time;

[0080] According to the spot diameter, a corresponding light stimulation template is matched.

[0081] The embodiment of the present application can meet the needs of differentiated visual training by presetting the irradiation parameters and the light stimulation template.

[0082] Optionally, in the embodiment of the present application, the macular region image is irradiated by the preset light source to achieve the purpose of visual training, comprising:

[0083] According to the wavelength of the light source, the energy density and the spot diameter, a corresponding light source is set;

[0084] In the irradiation time, the macular region image is irradiated by the light source; wherein the preset light stimulation template is set at the position corresponding to the spatial position coordinates, so that the fovea is not irradiated.

[0085] The embodiment of the present application can ensure that the fovea is not irradiated and the macular region image is irradiated by the preset light source, and the preset light stimulation template is set at the position corresponding to the spatial position coordinates, so that the fovea is not irradiated and the macular region image is irradiated, thereby effectively inducing the generation of negative afterimage, guiding the user to gradually return to central fixation from peripheral central fixation, and achieving the purpose of visual training.

[0086] Optionally, in the embodiment of the present application, during the process of light stimulation, the eye movement of the user is tracked in real time, and the light template is moved synchronously based on the eye movement, so that the light template continuously blocks the fovea, comprising:

[0087] Based on the eye movement data collected by the infrared eye tracker in real time, the offset of the real-time gaze direction of the user compared to the initial gaze direction is calculated;

[0088] Based on the offset, the light template is moved synchronously by controlling the deflector, so that the light template continuously blocks the fovea.

[0089] In the embodiment of the present application, by tracking the eye movement of the user in real time during the process of light stimulation, and moving the light template synchronously based on the eye movement, the light template blocking position can be dynamically corrected, the accuracy of visual training is guaranteed, and closed-loop control is realized.

[0090] As shown in the above method embodiment, corresponding device embodiments are provided; Figure 2

[0091] An embodiment of the present application provides a visual training device based on image recognition, comprising: an image acquisition module 201, an image recognition module 202, a light stimulation module 203 and an eye movement tracking module 204.

[0092] The image acquisition module 201 is used to collect the retinal image of the user by the fundus camera, and pre-process the retinal image to obtain a macular region image;

[0093] The image recognition module 202 is used to perform image recognition on the macular region image to obtain the spatial position coordinates of the fovea;

[0094] The light stimulation module 203 is used to perform light stimulation on the retina of the user by a preset light source to achieve the purpose of visual training; wherein, a preset light template is arranged at a position corresponding to the spatial position coordinates, so that the fovea is not stimulated by light;

[0095] The eye movement tracking module 204 is used to track the eye movement of the user in real time during the process of light stimulation, and move the light template synchronously based on the eye movement, so that the light template continuously blocks the fovea.

[0096] Optionally, in the embodiment of the present application, before the image acquisition module 201, there are also a head support sub-module and an observation window sub-module;

[0097] ​The head support sub-module is configured to fix the head position of the user by the head support structure, so that the user looks straight at the observation window.

[0098] The observation window sub-module is configured to project a fixation target in the observation window, so that the user fixates on the fixation target; wherein the line of sight direction of the user when the user fixates on the fixation target is taken as an initial fixation direction.

[0099] The head support structure and the observation window guide the user to fixate in the embodiment of the present application, which can help the user to stabilize the line of sight and provide a reference fixation direction for subsequent eye movement tracking.

[0100] Optionally, in the embodiment of the present application, the image acquisition module 201 comprises a noise suppression sub-module, a contrast enhancement sub-module and a region cropping sub-module.

[0101] The noise suppression sub-module is configured to perform noise suppression on the retinal image to obtain a first retinal image.

[0102] The contrast enhancement sub-module is configured to perform contrast enhancement on the first retinal image to obtain a second retinal image.

[0103] The region cropping sub-module is configured to crop a macular region of the second retinal image to obtain a macular region image.

[0104] The embodiment of the present application can ensure accurate coverage of the macular region before visual training by performing image preprocessing on the retinal image, and provide reliable data basis for subsequent fovea positioning and light stimulation.

[0105] Optionally, in the embodiment of the present application, the image recognition module 202 comprises an image recognition sub-module and a coordinate conversion sub-module.

[0106] The image recognition sub-module is configured to perform feature extraction and spatial regression on the macular region image to obtain pixel coordinates of the fovea.

[0107] The coordinate conversion sub-module is configured to linearly convert the pixel coordinates into spatial position coordinates based on the width of the observation window and the imaging resolution width of the fundus camera.

[0108] The embodiment of the present application can accurately extract the spatial position coordinates of the fovea by performing image recognition and coordinate conversion on the macular region image, and provide a position basis for subsequent light template shielding.

[0109] Optionally, in the embodiment of the present application, the light stimulation module 203 further comprises an irradiation parameter sub-module and a light template sub-module.

[0110] The illumination parameter submodule is configured to obtain preset illumination parameters of a trainer; wherein the illumination parameters include a light source wavelength, an energy density, a light spot diameter, and an illumination time.

[0111] The light illumination template submodule is configured to obtain a corresponding light illumination template according to the light spot diameter.

[0112] The preset illumination parameters and the light illumination template can meet the needs of differentiated visual training.

[0113] Optionally, in the embodiment of the present application, the light illumination stimulating module 203 includes a light source setting submodule and a light illumination stimulating submodule.

[0114] The light source setting submodule is configured to set a corresponding light source according to the light source wavelength, the energy density, and the light spot diameter.

[0115] The light illumination stimulating submodule is configured to stimulate the retina of the user by the light source within the illumination time; wherein the preset light illumination template is set at a position corresponding to the spatial position coordinate, so that the fovea is not stimulated by light.

[0116] The embodiment of the present application stimulates the retina of the user by the preset light source, and sets the preset light illumination template at a position corresponding to the spatial position coordinate, so that the fovea is not stimulated by light, while the retina around the fovea is stimulated by light, thereby effectively inducing the generation of negative afterimage, guiding the user to gradually return to central fixation from peripheral fixation, and achieving the purpose of visual training.

[0117] Optionally, in the embodiment of the present application, the eye movement tracking module 204 includes an eye movement tracking submodule and a control deflection submodule.

[0118] The eye movement tracking submodule is configured to calculate an offset of a real-time fixation direction of the user compared to the initial fixation direction based on eye movement data collected by an infrared eye movement tracker in real time.

[0119] The control deflection submodule is configured to synchronously move the light illumination template by controlling a deflector based on the offset, so that the light illumination template continuously blocks the fovea.

[0120] The embodiment of the present application can dynamically correct the blocking position of the light illumination template by tracking the eye movement of the user in real time during the light stimulation and synchronously moving the light illumination template based on the eye movement, so as to guarantee the accuracy of visual training and realize closed-loop control.

[0121] It can be understood that the above device embodiment is corresponding to the method embodiment of the present application, and can realize the image recognition-based visual training method provided by any one of the above method embodiments of the present application.

[0122] The image acquisition module 201 collects the retinal image of the user, and pre-processes the retinal image, so as to ensure accurate coverage of the macular region before visual training, and provide reliable data basis for subsequent fovea positioning and light stimulation. The image recognition module 202 performs image recognition on the macular region image, so as to accurately extract the spatial position coordinates of the fovea, and provide the position basis for subsequent light template shielding. The light stimulation module 203 performs light stimulation on the retina of the user, and sets the preset light template on the position corresponding to the spatial position coordinates, so as to ensure that the fovea is not stimulated by light, while the retina around the fovea is stimulated by light, thereby effectively inducing the generation of negative afterimage, guiding the user to gradually return to central fixation from peripheral central fixation, and achieving the purpose of visual training. The eye movement tracking module 204 tracks the eye movement of the user in real time during the light stimulation, and synchronously moves the light template based on the eye movement, so as to dynamically correct the shielding position of the light template, guarantee the accuracy of visual training, and realize closed-loop control.

[0123] It should be noted that the device embodiments described above are only schematic, and part or all of the modules can be selected to achieve the purpose of the present embodiment. In addition, in the device embodiment provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which can be realized as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0124] On the basis of the above-mentioned embodiment of the image recognition-based visual training method, another embodiment of the present application provides a terminal device, which comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the image recognition-based visual training method of any one of the embodiments of the present application is realized.

[0125] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present application. The one or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the terminal device.

[0126] The terminal device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The terminal device can include, but is not limited to, a processor and a memory.

[0127] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like. The processor is a control center of the terminal device, and is connected to various parts of the terminal device through various interfaces and lines.

[0128] On the basis of the above-mentioned method embodiment, another embodiment of the present application provides a computer readable storage medium, including a stored computer program, wherein when the computer program runs, the device where the computer readable storage medium is located performs the image recognition-based visual training method in any one of the above-mentioned method embodiments of the present application.

[0129] The modules / units integrated in the apparatus / terminal device can be stored in a computer readable storage medium if they are realized in the form of software function units and sold or used as independent products. Based on this understanding, all or part of the processes in the above-mentioned embodiments can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0130] The above is the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can also make a number of improvements and refinements, these improvements and refinements are also considered to be within the scope of the present application.

Claims

1. A visual training method based on image recognition, characterized in that, include: Images of the user's retina are acquired using a fundus camera, and the retinal images are preprocessed to obtain images of the macular region. Image recognition is performed on the image of the macular region to obtain the spatial coordinates of the fovea. The user's retina is stimulated by light from a preset light source to achieve the purpose of visual training; wherein, the preset light template is set at the position corresponding to the spatial coordinates so that the fovea is not stimulated by light. During the illumination stimulation, the user's eye movements are tracked in real time, and the illumination template is moved synchronously based on the eye movements so that the illumination template continuously blocks the fovea.

2. The visual training method based on image recognition as described in claim 1, characterized in that, Before acquiring the user's retinal image via a fundus camera and preprocessing the retinal image to obtain the macular region image, the method further includes: The user's head position is fixed by a head support structure so that the user can look directly at the viewing window. A gaze target is projected within the viewing window so that the user gazes at the gaze target; wherein the direction of the user's gaze while maintaining the gaze on the gaze target is taken as the initial gaze direction.

3. The visual training method based on image recognition as described in claim 1, characterized in that, The process of acquiring retinal images of the user via a fundus camera and preprocessing the retinal images to obtain macular region images includes: The retinal image is subjected to noise suppression to obtain a first retinal image; The first retinal image is contrast-enhanced to obtain a second retinal image; The macular region is cropped from the second retinal image to obtain a macular region image.

4. The visual training method based on image recognition as described in claim 2, characterized in that, The step of performing image recognition on the macular region image to obtain the spatial coordinates of the fovea includes: Feature extraction and spatial regression are performed on the macular region image to obtain the pixel coordinates of the concave center; Based on the width of the observation window and the imaging resolution width of the fundus camera, the pixel coordinates are linearly converted into spatial position coordinates.

5. The visual training method based on image recognition as described in claim 1, characterized in that, Before stimulating the user's retina with light through a preset light source to achieve the purpose of visual training, the method further includes: Obtain the irradiation parameters preset by the trainer; wherein, the irradiation parameters include the light source wavelength, energy density, light spot diameter, and irradiation time; The corresponding illumination template is obtained by matching the light spot diameter.

6. The visual training method based on image recognition as described in claim 5, characterized in that, The step of stimulating the user's retina with light through a preset light source to achieve visual training includes: The corresponding light source is set according to the light source wavelength, energy density, and spot diameter; During the irradiation period, the user's retina is stimulated by light from the light source; wherein a preset light template is set at the position corresponding to the spatial coordinates so that the fovea is not stimulated by light.

7. The image recognition-based visual training method as described in claim 2, characterized in that, During the illumination stimulation process, the user's eye movements are tracked in real time, and the illumination template is moved synchronously based on the eye movements to continuously block the fovea, including: Based on real-time eye movement data collected by an infrared eye tracker, the offset of the user's real-time gaze direction compared to the initial gaze direction is calculated; Based on the offset, the light template is moved synchronously by controlling the deflector so that the light template continuously blocks the central concave area.

8. A visual training device based on image recognition, characterized in that, include: Image acquisition module, image recognition module, illumination stimulation module, and eye tracking module; The image acquisition module is used to acquire retinal images of the user through a fundus camera and preprocess the retinal images to obtain macular region images; The image recognition module is used to perform image recognition on the macular region image to obtain the spatial coordinates of the fovea. The light stimulation module is used to stimulate the user's retina with light through a preset light source to achieve the purpose of visual training; wherein, the preset light template is set at the position corresponding to the spatial coordinates so that the fovea is not stimulated by light. The eye-tracking module is used to track the user's eye movements in real time during light stimulation, and to move the light template synchronously based on the eye movements so that the light template continuously blocks the fovea.

9. A terminal device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements a visual training method based on image recognition as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform a visual training method based on image recognition as described in any one of claims 1-7.

Images