Eye position image acquisition and management method and system

Abstract

The application belongs to the technical field of eye detection, and provides an eye position image acquisition and management method and system. The method comprises the following steps: grouping eye rotation images according to eye positions; predicting a first key parameter based on user's illness record data and each historical eye rotation image corresponding to any position; obtaining a second key parameter from each eye rotation image in the same position group; screening a target eye rotation image from the group and obtaining a third key parameter of the eye; synthesizing each target eye rotation image into a panoramic image according to the position corresponding to the group, and storing the panoramic image in association with the third key parameter. The application can automatically collect eye position images and corresponding eye feature parameters of a user, and the data after association storage can be used for analysis of strabismus detection, screening and the like at present, and comparison analysis of recovery degree in the future, so that the efficiency of strabismus illness analysis can be significantly improved.

Description

Technical Field

[0001] This invention relates to the field of eye detection technology, and more specifically, to a method and system for acquiring and managing eye position images. Background Technology

[0002] In clinical ophthalmology, strabismus and amblyopia are common diseases that seriously affect patients' visual function and quality of life. Accurately determining the eye position of patients with strabismus and amblyopia plays a crucial role in the diagnosis, treatment planning, and evaluation of treatment effectiveness.

[0003] Currently, in ophthalmological clinical practice, the assessment of eye position in patients with strabismus and amblyopia mainly relies on manual visual methods, such as the occlusion test and corneal reflex test. Taking the occlusion test as an example, its procedure is cumbersome, requiring the examiner to frequently and alternately cover both eyes and carefully observe the movement direction of the uncovered eye to determine eye position. This not only demands a high level of experience and attention from the examiner but is also highly susceptible to subjective interference, leading to unreliable results. While the corneal reflex test is relatively simple, it can only provide a rough assessment based on the position of the corneal reflective point, often failing to accurately detect latent strabismus and intermittent exotropia. Furthermore, these traditional methods generally lack quantitative processing of examination data, failing to provide precise numerical indicators to intuitively reflect eye position, making it difficult to objectively and accurately compare results from different periods, severely hindering the effective monitoring and analysis of the patient's condition progression.

[0004] A more prominent problem is that traditional visual methods are insufficient for preserving images of the eye. In the medical process, imaging data is irreplaceable for doctors to comprehensively understand a patient's condition, develop personalized treatment plans, and track treatment effectiveness. However, due to the lack of image retention, doctors can only rely on written records and memory to recall a patient's condition, which undoubtedly increases the difficulty of comparing medical records and hinders the establishment of complete and systematic medical records for patients. Summary of the Invention

[0005] To address this, the present invention provides a method and system for acquiring and managing eye position images, thereby solving the aforementioned technical problems.

[0006] This invention provides a method for acquiring and managing eye position images, comprising the following steps:

[0007] The system rapidly acquires a set of eye movement images of the user within the target acquisition frame, and groups the eye movement images that meet the preset image quality according to the eye orientation.

[0008] Based on the user's medical record data and historical eye movement images corresponding to any position, a first key parameter is predicted to characterize the expected recovery degree of strabismus, and a second key parameter is extracted from each group of eye movement images in the same position. The first key parameter and the second key parameter both include at least the binocular visual axis angle, the eye deviation direction angle, and the binocular movement coordination parameter.

[0009] Based on the first key parameter and the second key parameter, the target eye movement image is obtained by filtering from the group;

[0010] The third key parameter of the eyeball is extracted from the eyeball rotation images of each target, and the eyeball rotation images of each target are combined into a panoramic image according to the orientation corresponding to their respective groups. The third key parameter is then associated with and stored in the panoramic image.

[0011] Furthermore, eye movement images in the eye movement image set that meet the preset image quality are grouped according to eye orientation, including:

[0012] The eye orientation is determined based on the gaze direction of the eye in the eye movement image. The eye orientation includes at least the frontal orientation, leftward orientation, rightward orientation, upward orientation, downward orientation, upper leftward orientation, upper rightward orientation, lower leftward orientation, and lower rightward orientation.

[0013] According to each eye position, eye movement images that meet the preset image quality are respectively grouped into the corresponding position group; the eye movement images grouped into the same position have a deviation angle between the gaze direction of the corresponding eye and the eye position of the group not exceeding a set number of degrees.

[0014] Furthermore, based on the user's medical record data, a first key parameter for characterizing the expected recovery rate of strabismus is predicted based on historical eye movement images corresponding to any position, including:

[0015] Extract treatment cycles, previous treatment plans, historical strabismus degree changes, and basic eye parameters from the user's medical record data;

[0016] For each historical eye movement image corresponding to any position, feature alignment is performed, and historical parameter values ​​of binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters are extracted for each historical eye movement image in that position.

[0017] The treatment cycle, previous treatment plan, historical strabismus degree change trend and historical parameter values ​​are input into the trained prediction model, and the predicted values ​​of the binocular visual axis angle, the predicted value of the eyeball deflection direction angle and the predicted value of the binocular motion coordination parameter, which represent the expected recovery degree of strabismus in this orientation, are output as the first key parameter.

[0018] Furthermore, the treatment cycle, previous treatment plans, historical strabismus degree change trends, and historical parameter values ​​are input into the trained prediction model. The model outputs predicted values ​​for the binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters, representing the expected recovery degree of strabismus in that orientation. These include:

[0019] The historical strabismus degree change trend and historical parameter values ​​are input into the first sub-model of the prediction model to predict the preliminary binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameter.

[0020] Through the attention mechanism of the prediction model, an attention weight matrix is ​​generated based on the treatment cycle, previous treatment plan, and historical strabismus degree change trend. The attention weight matrix is ​​multiplied with a random noise matrix to generate a random signal. The attention weight matrix is ​​used to characterize the degree of influence of each input feature on the degree of strabismus recovery.

[0021] The preliminary predicted values ​​of the binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters, along with the random signal, are input into the second sub-model of the prediction model to generate the final predicted values ​​of the binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters, which serve as the first key parameters.

[0022] Furthermore, through the attention mechanism of the prediction model, an attention weight matrix is ​​generated based on the treatment cycle, previous treatment plans, and historical strabismus degree change trends, including:

[0023] The user's performance characteristics when voluntarily rotating their eyes are extracted from the collected eye rotation images. These performance characteristics include eye rotation rate, rotation range compliance rate, dwell time in each direction, and rotation stability.

[0024] The treatment cycle, previous treatment plan, historical strabismus degree change trend and manifestation characteristics were fused to obtain a fused feature matrix;

[0025] The fused feature matrix is ​​input into a fully connected network with an attention mechanism, and then processed by the ReLU activation function and normalized by the softmax function to generate an attention weight matrix, wherein the weight ratio corresponding to the performance feature is not less than a preset ratio.

[0026] The present invention also provides an eye position image acquisition and management system, including an acquisition and grouping unit, an extraction unit, a filtering unit, and a management unit;

[0027] The acquisition and grouping unit acquires a set of eye movement images of the user in the target acquisition frame at high speed, and groups the eye movement images in the set that meet the preset image quality according to the eye orientation.

[0028] The extraction unit predicts a first key parameter to characterize the expected recovery degree of strabismus based on the user's medical record data and historical eye movement images corresponding to any position, and extracts a second key parameter from each group of eye movement images in the same position. The first key parameter and the second key parameter both include at least the binocular visual axis angle, the eye deviation direction angle, and the binocular movement coordination parameter.

[0029] The filtering unit filters out target eye movement images from the group based on the first key parameter and the second key parameter;

[0030] The management unit extracts the third key parameter of the eyeball based on the eyeball rotation images of each target, synthesizes the eyeball rotation images of each target according to the orientation corresponding to their respective groups into a panoramic image, and stores the third key parameter in association with the panoramic image.

[0031] Furthermore, the acquisition and grouping unit is used for:

[0032] The eye orientation is determined based on the gaze direction of the eye in the eye movement image. The eye orientation includes at least the frontal orientation, leftward orientation, rightward orientation, upward orientation, downward orientation, upper leftward orientation, upper rightward orientation, lower leftward orientation, and lower rightward orientation.

[0033] According to each eye position, eye movement images that meet the preset image quality are respectively grouped into the corresponding position group; the eye movement images grouped into the same position have a deviation angle between the gaze direction of the corresponding eye and the eye position of the group not exceeding a set number of degrees.

[0034] The present invention also provides an electronic device comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the method as described in any of the preceding claims.

[0035] The present invention also provides a computer storage medium storing a computer program that can be executed by a processor to implement the methods described in any of the preceding claims.

[0036] The present invention also provides a computer program product comprising a computer program that can be executed by a processor to implement the methods as described in any of the preceding claims.

[0037] The solution of this invention, on the one hand, can automatically collect the user's eye position images and corresponding eye feature parameters. This data, after being linked and stored, can be conveniently used for current strabismus detection, screening, and subsequent comparative analysis of recovery levels, significantly improving the efficiency of strabismus condition analysis. On the other hand, this invention can also quickly filter out the most suitable target eye movement images for subsequent comparative analysis based on the user's medical record data, thereby ensuring the practical application value of the final panoramic image and improving the accuracy of subsequent strabismus comparative analysis. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating a method for acquiring and managing eye position images disclosed in an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of a panoramic image according to an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the prediction model in an embodiment of the present invention.

[0042] Figure 4 This is a schematic diagram of the structure of an eye position image acquisition and management system disclosed in an embodiment of the present invention. Detailed Implementation

[0043] The following specific embodiments illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0045] like Figure 1 As shown in the figure, this embodiment discloses a method for acquiring and managing eye position images, including the following steps:

[0046] S10: High-speed acquisition of a set of eye movement images of the user within the target acquisition frame; grouping eye movement images that meet the preset image quality according to eye orientation.

[0047] This step is used to acquire and initially classify high-quality images of eye movement.

[0048] The user's face is aligned with the view frame of the high-speed acquisition device, with the eye area placed within the target acquisition frame. The user then moves their eyes voluntarily. The high-speed acquisition device captures a set of images of the user's eye movements within the target acquisition frame. High-speed acquisition ensures that every detail of eye movement is captured, avoiding the loss of crucial moments due to insufficient acquisition speed.

[0049] The acquired images undergo quality screening to select eye movement images that meet preset image quality standards (such as sharpness, contrast, and no occlusion) to exclude blurry, interfering, or other invalid images. The screened images are then grouped according to common eye orientations (such as frontal, left, right, upward, and downward gaze).

[0050] S20, based on the user's medical record data, a first key parameter for characterizing the expected recovery degree of strabismus is predicted based on each historical eye movement image corresponding to any position, and a second key parameter is extracted from each eye movement image grouped in the same position. The first key parameter and the second key parameter both include at least the angle between the visual axes of the two eyes, the direction angle of eye deviation, and the coordination parameter of the two eye movements.

[0051] This step is based on the user's medical record data, which includes information such as the user's medical history, treatment process, and previous diagnoses, and can be used to predict the expected recovery degree of the user's strabismus. Based on historical eye movement images corresponding to any position, a specific algorithm model is used to predict the first key parameter. The first key parameter is used to characterize the expected recovery degree of strabismus in that position, and it includes at least the binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters, which can reflect the expected recovery status from core dimensions such as strabismus degree, type, and binocular coordination.

[0052] Simultaneously, for groups in the same orientation, a second key parameter corresponding to the first key parameter is extracted from each eye movement image acquired during this session. This second key parameter also includes at least the three core parameters mentioned above, reflecting the actual state of the eye in that orientation. For example, the second key parameter is extracted using a near-infrared camera array, a polarized light module, and high-frame-rate eye-tracking technology. The polarized light module can eliminate interference from corneal reflections, while high-frame-rate eye-tracking accurately captures corneal reflection points and pupil displacement. Combined with the eye's three-dimensional coordinates (XYZ axis displacement), the strabismus angle is calculated, ensuring the accuracy of key parameters such as the angle between the visual axes of the two eyes.

[0053] S30, based on the first key parameter and the second key parameter, the target eyeball rotation image is obtained by filtering from the group.

[0054] By comparing the first and second key parameters, the differences between the actual state reflected in the acquired eye movement images and the expected degree of recovery are analyzed. For example, the degree of deviation in the visual axis angle between the two eyes, the consistency of the eye deviation direction angle, and the matching degree of binocular motion coordination parameters are compared. Through this comparative analysis, the image most closely related to the expected degree of recovery and best reflecting the key state of the current treatment stage is selected as the target eye movement image. This target image serves as the most representative sample for subsequent analysis of the user's strabismus recovery by the physician.

[0055] S40, based on the eye rotation images of each target, the third key parameter of the eye is extracted, the eye rotation images of each target are combined into a panoramic image according to the orientation corresponding to their respective groups, and the third key parameter is associated with and stored in the panoramic image.

[0056] First, a third key parameter is extracted from the eye movement images of each target eye. This third key parameter, in addition to the aforementioned first and second key parameters, can further include other eye features, such as iris texture details, pupil size changes, and eyelid position—more detailed feature data—to supplement and improve the description of the eye's state. Understandably, this more detailed feature data can be manually entered by the user or doctor before this data collection and analysis, or automatically determined based on the user's strabismus recovery stage; further details will not be elaborated upon here.

[0057] Then, as Figure 2 As shown, images of each target eye movement are grouped according to their corresponding orientation and then combined into a panoramic image. The panoramic image can intuitively present the user's overall eye position in various orientations. For example, nine or twelve photos from different orientations can be automatically stitched together into a complete panoramic image. Users or doctors can freely rotate, zoom, and view each image through the interactive interface, making it easy to quickly and comprehensively understand the user's eye condition.

[0058] Finally, the third key parameter is associated with the panoramic image for storage. This associated storage method allows for the simultaneous acquisition of intuitive image information and quantitative data information during subsequent queries and retrievals, providing complete data support for medical record establishment, disease analysis, and treatment effect evaluation, and also facilitating subsequent data traceability and reuse.

[0059] The solution of this invention, on the one hand, can automatically collect the user's eye position images and corresponding eye feature parameters. This data, after being linked and stored, can be conveniently used for current strabismus detection, screening, and subsequent comparative analysis of recovery levels, significantly improving the efficiency of strabismus condition analysis. On the other hand, this invention can also quickly filter out the most suitable target eye movement images for subsequent comparative analysis based on the user's medical record data, thereby ensuring the practical application value of the final panoramic image and improving the accuracy of subsequent strabismus comparative analysis.

[0060] It should be noted that other eye movement images that meet the preset image quality can also be retained for doctors or users to review and even filter later. For example, doctors can manually select eye movement images that they deem more suitable to generate the panoramic images mentioned above. The specifics will not be elaborated further.

[0061] As an example, the grouping of eye movement images in the eye movement image set that meet a preset image quality according to eye orientation includes:

[0062] The eye orientation is determined based on the gaze direction of the eye in the eye movement image. The eye orientation includes at least the frontal orientation, leftward orientation, rightward orientation, upward orientation, downward orientation, upper leftward orientation, upper rightward orientation, lower leftward orientation, and lower rightward orientation.

[0063] According to each eye position, eye movement images that meet the preset image quality are respectively grouped into the corresponding position group; the eye movement images grouped into the same position have a deviation angle between the gaze direction of the corresponding eye and the eye position of the group not exceeding a set number of degrees.

[0064] Users can move their eyes in the corresponding direction based on the output prompts, which naturally groups the eye movement images. However, this method requires accurate matching with the prompts (e.g., "move your eyes to the upper left"), otherwise the eye movement images in the group may not match the group, for example, if the user misunderstands the prompts.

[0065] To address this, this invention allows users to freely and autonomously rotate their eyes within the target acquisition frame, without requiring any prompts. The high-speed acquisition device performs high-speed, redundant image acquisition. Then, the gaze direction of the eyes in each of the high-speed acquired eye-rotation images is determined, thereby identifying the eye's orientation. This includes not only basic orientations such as frontal, left, right, upward, and downward gaze, but also further refinements to orientations such as upper left, upper right, lower left, and lower right, comprehensively covering the common range of eye rotations and enabling more detailed capture of eye positions in different directions.

[0066] Based on the determined eye orientations, eye movement images that meet the preset image quality are grouped into their corresponding orientation groups. To ensure consistency within the same group, the angle between the eye's gaze direction and the corresponding eye orientation is limited to a set degree, such as 5°. This reduces interference caused by orientation deviations, making the grouping results more accurate and providing a reliable classification basis for subsequent group-based parameter extraction and image filtering.

[0067] As an example, the first key parameter for characterizing the expected recovery degree of strabismus, predicted based on the user's medical record data and historical eye movement images corresponding to any position, includes:

[0068] Extract treatment cycles, previous treatment plans, historical strabismus degree changes, and basic eye parameters from the user's medical record data;

[0069] For each historical eye movement image corresponding to any position, feature alignment is performed, and historical parameter values ​​of binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters are extracted for each historical eye movement image in that position.

[0070] The treatment cycle, previous treatment plan, historical strabismus degree change trend and historical parameter values ​​are input into the trained prediction model, and the predicted values ​​of the binocular visual axis angle, the predicted value of the eyeball deflection direction angle and the predicted value of the binocular motion coordination parameter, which represent the expected recovery degree of strabismus in this orientation, are output as the first key parameter.

[0071] Key information is extracted from the user's medical records, including treatment cycles (such as treatment duration and intervals), previous treatment plans (such as the type of glasses worn and training methods), historical strabismus degree change trends (such as the rate of decrease and fluctuations), and basic eye parameters (such as axial length and corneal curvature). This information reflects the user's treatment background and basic eye condition, providing multi-dimensional references for prediction.

[0072] For each historical eye movement image corresponding to any orientation, feature alignment processing is performed to ensure that images acquired at different times are comparable in the same orientation. Based on this, historical parameter values ​​of binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters are extracted for each historical image in that orientation. These historical parameter values ​​directly reflect the user's past strabismus state in that orientation.

[0073] The extracted treatment cycle, previous treatment plan, historical parameter change trend and historical parameter value are input into the trained prediction model. The prediction model has learned and mastered the correlation between these data through pre-training and outputs the predicted value of the binocular visual axis angle, the predicted value of the eyeball deviation direction angle and the predicted value of the binocular motion coordination parameter in this position. These predicted values ​​together constitute the first key parameter, which is used to characterize the expected recovery degree of strabismus in this position.

[0074] As an example, such as Figure 3 As shown, the treatment cycle, previous treatment plans, historical strabismus degree change trends, and historical parameter values ​​are input into the trained prediction model. The model outputs predicted values ​​for the binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters, representing the expected recovery degree of strabismus in that orientation. These include:

[0075] The historical strabismus degree change trend and historical parameter values ​​are input into the first sub-model of the prediction model to predict the preliminary binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameter.

[0076] Through the attention mechanism of the prediction model, an attention weight matrix is ​​generated based on the treatment cycle, previous treatment plan, and historical strabismus degree change trend. The attention weight matrix is ​​multiplied with a random noise matrix to generate a random signal. The attention weight matrix is ​​used to characterize the degree of influence of each input feature on the degree of strabismus recovery.

[0077] The preliminary predicted values ​​of the binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters, along with the random signal, are input into the second sub-model of the prediction model to generate the final predicted values ​​of the binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters, which serve as the first key parameters.

[0078] First, the historical strabismus degree change trend and historical parameter values ​​are input into the first sub-model of the prediction model. Based on these data that directly reflect the strabismus state and change pattern, the first sub-model outputs preliminary predicted values ​​of the binocular visual axis angle, eyeball deviation direction angle, and binocular motion coordination parameters by fitting the historical evolution pattern, providing a basic reference for subsequent optimization.

[0079] Simultaneously, an attention mechanism is introduced to weight key influencing factors. Specifically, an attention weight matrix is ​​generated based on the treatment cycle, previous treatment plans, and historical strabismus degree change trends. Each element in the matrix corresponds to the influence weight of different input features on the degree of strabismus recovery (e.g., the weight of a specific treatment plan is higher than that of other factors). This attention weight matrix is ​​multiplied by a random noise matrix of the same dimension to generate a random signal, which not only retains the influence weight of key factors but also enhances the model's adaptability to complex variables through random noise.

[0080] Examples are given below:

[0081] The treatment cycle (denoted as the feature vector) ,in This indicates quantitative values ​​such as treatment duration and intervals. Indicates the number of characteristics of the treatment cycle. Previous treatment plans (denoted as feature vectors) ,in Binary identifiers or efficacy scores for different treatment methods. This indicates the number of characteristics of previous treatment plans. Historical strabismus degree variation trend (denoted as feature vector) ,in Indicates characteristics such as the rate of change of degree and the amplitude of fluctuation. It is the number of these features. The input feature matrix is ​​standardized to obtain a uniform dimension. The semicolon indicates vertical splicing.

[0082] Weights are calculated using a two-layer fully connected network:

[0083] First layer: ,in This is the weight matrix of the first fully connected layer. For bias terms, It is the ReLU activation function. It is an intermediate feature representation obtained after processing by the first fully connected network layer;

[0084] Second layer: The attention weight matrix is ​​obtained. ,in, This is the weight matrix of the second fully connected layer. For bias terms, , , The influence weights of the binocular visual axis angle, the ocular deviation direction angle, and the binocular motion coordination parameters are respectively defined, and the following conditions are met: .

[0085] Generate a random noise matrix with the same dimension as the weight matrix. ,in , , It follows a normal distribution. This refers to the noise intensity hyperparameter.

[0086] The formula for calculating random signals is: ,in This represents element-wise multiplication, ensuring that the signal retains the weight trend while introducing randomness.

[0087] Perform a second normalization on the random signal: The final output This refers to the random signal used to correct the initial predicted value, where... Reflecting the dynamic adjustment weights of each parameter in the prediction, .

[0088] Finally, the preliminary predictions and random signals are input into the second sub-model. The second sub-model adjusts and optimizes the parameters by fusing the static preliminary predictions with the dynamic weight signals. The final output is the predicted values ​​of the binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters, which can more accurately characterize the expected recovery degree of strabismus, and serves as the first key parameter.

[0089] Understandably, the first sub-model preferably uses a Long Short-Term Memory (LSTM) network or a Temporal Convolutional Network (TCN), while the second sub-model preferably uses a Multilayer Perceptron (MLP) or an Attention-Enhanced Fully Connected Network.

[0090] This embodiment uses hierarchical processing and attention mechanisms to ensure that the prediction results take into account both historical data patterns and the influence of personalized treatment factors.

[0091] As an example, the generation of an attention weight matrix based on the treatment cycle, previous treatment plans, and historical strabismus degree change trends through the attention mechanism of the prediction model includes:

[0092] The user's performance characteristics when voluntarily rotating their eyes are extracted from the collected eye rotation images. These performance characteristics include eye rotation rate, rotation range compliance rate, dwell time in each direction, and rotation stability.

[0093] The treatment cycle, previous treatment plan, historical strabismus degree change trend and manifestation characteristics were fused to obtain a fused feature matrix;

[0094] The fused feature matrix is ​​input into a fully connected network with an attention mechanism, and then processed by the ReLU activation function and normalized by the softmax function to generate an attention weight matrix, wherein the weight ratio corresponding to the performance feature is not less than a preset ratio.

[0095] The characteristics of a user's voluntary eye movements (such as rotation rate, stability, and range achievement rate) directly reflect their eye movement control ability, treatment compliance, and current eye function status. These factors are closely related to the potential for strabismus recovery and treatment effectiveness. For example, a low range achievement rate may indicate weak eye muscle accommodation and a slower recovery speed; poor rotation stability may suggest unstable factors during treatment, affecting the recovery trend. The aforementioned embodiments did not incorporate these characteristics into the attention mechanism, resulting in an attention weight matrix that relies solely on historical treatment data and trends. This fails to capture the dynamic impact of the user's current eye movement characteristics on recovery, making it easy for the weight allocation to deviate from the user's actual treatment status. Consequently, the generated random signals and the final prediction of the first key parameter are difficult to accurately adapt to individual user differences (such as compliance and current eye function status), which to some extent reduces the predictive model's adaptability to complex treatment scenarios and the personalized accuracy of the first key parameter.

[0096] To address this, the present invention further integrates the aforementioned performance characteristics into the attention mechanism, enabling the weight matrix to better align with the user's actual treatment state and improving the personalization and accuracy of the prediction of the first key parameter. Specifically:

[0097] The core performance of users' voluntary eye movements is analyzed from a collection of eye movement images, including eye movement rate (the change in angle of rotation per unit time), rotation range compliance rate (the percentage of actual rotation angle that matches the target angle), dwell time in each direction (the duration of image acquisition in directions such as frontal and leftward gaze), and rotation stability (the standard deviation of angle fluctuation during rotation).

[0098] The treatment cycle, previous treatment plans, historical strabismus degree changes, and the aforementioned characteristics are unified and combined to form a fusion feature matrix. Understandably, the fusion process also requires standardization (such as Z-score normalization) to eliminate dimensional differences between different features.

[0099] The fully connected network with the fused feature matrix is ​​input into the attention mechanism. It is first processed by the ReLU activation function to enhance the nonlinear feature representation, and then normalized by the softmax function to obtain the attention weight matrix. The weight percentage corresponding to the representative features is no less than a preset percentage (e.g., 20%) to ensure its substantial impact on the weight allocation. Ultimately, the matrix reflects both treatment history and trends, as well as the user's current eye movement ability, providing a more comprehensive feature basis for subsequent random signal generation.

[0100] like Figure 4 As shown, this embodiment of the invention also provides an eye position image acquisition and management system, including an acquisition and grouping unit 101, an extraction unit 102, a filtering unit 103, and a management unit 104;

[0101] The acquisition and grouping unit 101 acquires a set of eye movement images of the user in the target acquisition frame at high speed, and groups the eye movement images in the set that meet the preset image quality according to the eye orientation.

[0102] The extraction unit 102 predicts a first key parameter to characterize the expected recovery degree of strabismus based on the user's medical record data and historical eye movement images corresponding to any position, and extracts a second key parameter from each group of eye movement images in the same position. The first key parameter and the second key parameter both include at least the angle between the visual axes of the two eyes, the direction angle of eye deviation, and the coordination parameter of the two eyes' movements.

[0103] The filtering unit 103 filters out target eye movement images from the group based on the first key parameter and the second key parameter;

[0104] The management unit 104 extracts the third key parameter of the eyeball based on the eyeball rotation images of each target, synthesizes the eyeball rotation images of each target according to the orientation corresponding to their respective groups into a panoramic image, and stores the third key parameter in association with the panoramic image.

[0105] As an example, the acquisition and grouping unit 101 is used for:

[0106] The eye movement images in the set that meet the preset image quality are grouped according to eye orientation, including:

[0107] The eye orientation is determined based on the gaze direction of the eye in the eye movement image. The eye orientation includes at least the frontal orientation, leftward orientation, rightward orientation, upward orientation, downward orientation, upper leftward orientation, upper rightward orientation, lower leftward orientation, and lower rightward orientation.

[0108] According to each eye position, eye movement images that meet the preset image quality are respectively grouped into the corresponding position group; the eye movement images grouped into the same position have a deviation angle between the gaze direction of the corresponding eye and the eye position of the group not exceeding a set number of degrees.

[0109] As an example, the extraction unit 102 is used for:

[0110] Extract treatment cycles, previous treatment plans, historical strabismus degree changes, and basic eye parameters from the user's medical record data;

[0111] For each historical eye movement image corresponding to any position, feature alignment is performed, and historical parameter values ​​of binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters are extracted for each historical eye movement image in that position.

[0112] The treatment cycle, previous treatment plan, historical strabismus degree change trend and historical parameter values ​​are input into the trained prediction model, and the predicted values ​​of the binocular visual axis angle, the predicted value of the eyeball deflection direction angle and the predicted value of the binocular motion coordination parameter, which represent the expected recovery degree of strabismus in this orientation, are output as the first key parameter.

[0113] As an example, the extraction unit 102 is used for:

[0114] The historical strabismus degree change trend and historical parameter values ​​are input into the first sub-model of the prediction model to predict the preliminary binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameter.

[0115] Through the attention mechanism of the prediction model, an attention weight matrix is ​​generated based on the treatment cycle, previous treatment plan, and historical strabismus degree change trend. The attention weight matrix is ​​multiplied with a random noise matrix to generate a random signal. The attention weight matrix is ​​used to characterize the degree of influence of each input feature on the degree of strabismus recovery.

[0116] The preliminary predicted values ​​of the binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters, along with the random signal, are input into the second sub-model of the prediction model to generate the final predicted values ​​of the binocular visual axis angle, eye deviation direction angle, and binocular motion coordination parameters, which serve as the first key parameters.

[0117] As an example, the extraction unit 102 is used for:

[0118] The user's performance characteristics when voluntarily rotating their eyes are extracted from the collected eye rotation images. These performance characteristics include eye rotation rate, rotation range compliance rate, dwell time in each direction, and rotation stability.

[0119] The treatment cycle, previous treatment plan, historical strabismus degree change trend and manifestation characteristics were fused to obtain a fused feature matrix;

[0120] The fused feature matrix is ​​input into a fully connected network with an attention mechanism, and then processed by the ReLU activation function and normalized by the softmax function to generate an attention weight matrix, wherein the weight ratio corresponding to the performance feature is not less than a preset ratio.

[0121] This invention also provides an electronic device comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the method as described in any of the foregoing embodiments.

[0122] This invention also provides a computer storage medium storing a computer program that can be executed by a processor to implement the methods described in any of the foregoing claims.

[0123] This invention also provides a computer program product comprising a computer program that can be executed by a processor to implement the method as described in any of the foregoing embodiments.

[0124] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0125] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method of ocular topography acquisition and management, characterized by: The method comprises the following steps: high-speed collection of a set of eye rotation images of a user in a target collection frame, grouping eye rotation images in the set of eye rotation images that meet a preset image quality according to eye directions; based on the user's medical record data and each historical eye rotation image corresponding to any direction, predicting a first key parameter for representing the expected recovery degree of strabismus, and from each eye rotation image in the same direction group, respectively, predicting a second key parameter for reflecting the actual state of the current eye in that direction, wherein the first key parameter and the second key parameter each include at least a binocular visual axis angle, an eye deviation direction angle, and a binocular movement coordination parameter; the second key parameter is extracted by means of a near-infrared camera group, a polarized light module, and a high-frame-rate eye movement tracking technology; based on the first key parameter and the second key parameter, screening a target eye rotation image from the group; based on each target eye rotation image, extracting a third key parameter of the eye, synthesizing a panoramic image according to the direction corresponding to the group to which each target eye rotation image belongs, and storing the third key parameter in association with the panoramic image; based on the user's medical record data, predicting a first key parameter for representing the expected recovery degree of strabismus based on each historical eye rotation image corresponding to any direction, comprising: extracting a treatment period, a previous treatment plan, a historical strabismus degree change trend, and eye basic parameters from the user's medical record data; aligning the features of each historical eye rotation image corresponding to any direction, and extracting historical parameter values of the binocular visual axis angle, the eye deviation direction angle, and the binocular movement coordination parameter of each historical eye rotation image in that direction; inputting the treatment period, the previous treatment plan, the historical strabismus degree change trend, and the historical parameter values into a trained prediction model, and outputting binocular visual axis angle prediction values, eye deviation direction angle prediction values, and binocular movement coordination parameter prediction values in that direction as the first key parameter, which represent the expected recovery degree of strabismus; inputting the treatment period, the previous treatment plan, the historical strabismus degree change trend, and the historical parameter values into a trained prediction model, and outputting binocular visual axis angle prediction values, eye deviation direction angle prediction values, and binocular movement coordination parameter prediction values in that direction, which represent the expected recovery degree of strabismus, comprising: inputting the historical strabismus degree change trend and the historical parameter values into a first sub-model of the prediction model to predict preliminary binocular visual axis angle prediction values, eye deviation direction angle prediction values, and binocular movement coordination parameter prediction values; generating an attention weight matrix based on the treatment period, the previous treatment plan, and the historical strabismus degree change trend through an attention mechanism of the prediction model, multiplying the attention weight matrix by a random noise matrix to generate a random signal; wherein the attention weight matrix is used to represent the influence of each input feature on the recovery degree of strabismus. The preliminary binocular visual axis angle prediction value, the eye deviation direction angle prediction value, and the binocular movement coordination parameter prediction value are input into a second sub-model of the prediction model, and final binocular visual axis angle prediction value, eye deviation direction angle prediction value, and binocular movement coordination parameter prediction value are generated as the first key parameters.

2. The eye position image acquisition and management method of claim 1, wherein: The eye movement images meeting the preset image quality are grouped according to the eye positions, including: The eye positions are determined based on the gaze directions of the eyes in the eye movement images, and the eye positions at least include the orthovision position, the left vision position, the right vision position, the up vision position, the down vision position, the left-up vision position, the right-up vision position, the left-down vision position, and the right-down vision position; The eye movement images meeting the preset image quality are respectively classified into the groups corresponding to the positions according to the positions of the eyes; the eye movement images classified into the same group have a deviation angle of the gaze direction of the corresponding eye from the corresponding eye position of the group not more than a set number of degrees.

3. The eye position image acquisition and management method of claim 1, wherein: Through the attention mechanism of the prediction model, an attention weight matrix is generated based on the treatment cycle, the previous treatment scheme, and the historical strabismus degree change trend, including: Performance characteristics of the user when the eyes are voluntarily rotated are extracted from the collected eye movement image set, and the performance characteristics include the eye rotation rate, the rotation range compliance rate, the position staying time, and the rotation smoothness; The treatment cycle, the previous treatment scheme, the historical strabismus degree change trend, and the performance characteristics are fused to obtain a fusion feature matrix; The fusion feature matrix is input into a fully connected network of the attention mechanism, and is sequentially processed by a ReLU activation function and normalized by a softmax function to generate an attention weight matrix, wherein the weight proportion corresponding to the performance characteristics is not less than a preset proportion.

4. An ocular image acquisition and management system, characterized by It includes an acquisition and grouping unit, an extraction unit, a screening unit, and a management unit. The acquisition and grouping unit acquires a set of eye movement images of the user in a target acquisition frame at a high speed, and groups the eye movement images meeting the preset image quality in the set of eye movement images according to the positions of the eyes. The extraction unit obtains a first key parameter for representing the expected recovery degree of strabismus based on the disease record data of the user and each historical eye movement image corresponding to any position, and obtains a second key parameter for reflecting the actual state of the current eye in the position from each eye movement image in the same position group, wherein the first key parameter and the second key parameter at least include the binocular visual axis angle, the eye deviation direction angle, and the binocular movement coordination parameter; the second key parameter is extracted by means of a near-infrared camera group, a polarized light module, and a high-frame-rate eye tracking technology; The screening unit screens target eye movement images from the group based on the first key parameter and the second key parameter; The management unit obtains a third key parameter of the eye based on each target eye movement image, synthesizes a panoramic image from each target eye movement image according to the position corresponding to the group to which the target eye movement image belongs, and stores the third key parameter and the panoramic image in association; The extraction unit is configured to: extracting a treatment cycle, a previous treatment scheme, a historical strabismus degree change trend and eye basic parameters from user's illness record data; aligning features of each historical eye rotation image corresponding to any orientation, and extracting historical parameter values of a binocular visual axis included angle, an eye deviation direction angle and a binocular movement coordination parameter of each historical eye rotation image in the orientation; inputting the treatment cycle, the previous treatment scheme, the historical strabismus degree change trend and the historical parameter values into the trained prediction model, and outputting binocular visual axis included angle prediction values, eye deviation direction angle prediction values and binocular movement coordination parameter prediction values in the orientation as the first key parameters, which represent an expected recovery degree of strabismus; the extraction unit is configured to: input the historical strabismus degree change trend and the historical parameter values into a first sub-model of the prediction model, and predict preliminary binocular visual axis included angle prediction values, eye deviation direction angle prediction values and binocular movement coordination parameter prediction values; generate an attention weight matrix based on the treatment cycle, the previous treatment scheme and the historical strabismus degree change trend through an attention mechanism of the prediction model, multiply the attention weight matrix by a random noise matrix to generate a random signal, and use the attention weight matrix to represent an influence degree of each input feature on the strabismus recovery degree; input the preliminary binocular visual axis included angle prediction values, the eye deviation direction angle prediction values and the binocular movement coordination parameter prediction values and the random signal into a second sub-model of the prediction model, and generate final binocular visual axis included angle prediction values, eye deviation direction angle prediction values and binocular movement coordination parameter prediction values as the first key parameters.

5. The ocular position image acquisition and management system of claim 4, wherein: the acquisition and grouping unit is configured to: determine eye orientations based on gaze directions of eyes in the eye rotation images, wherein the eye orientations at least include a direct vision orientation, a left vision orientation, a right vision orientation, an up vision orientation, a down vision orientation, a left-up vision orientation, a right-up vision orientation, a left-down vision orientation and a right-down vision orientation; group eye rotation images meeting a preset image quality into groups corresponding to the eye orientations according to the eye orientations, and the deviation angle of the gaze direction of the corresponding eye of the eye rotation images grouped into the same group from the eye orientation corresponding to the group is not more than a set degree.

6. An electronic device, the electronic device comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program is executed by the processor to implement the method of any one of claims 1-3.

7. A computer storage medium, characterized in that: The computer storage medium stores a computer program executable by the processor to implement the method of any one of claims 1-3.

8. A computer program product, characterised in that: The computer program product includes a computer program executable by the processor to implement the method of any one of claims 1-3.

Images