An eye fatigue monitoring system for ophthalmology, an eye image processing method, an apparatus, and a storage medium

Abstract

The application provides an eye fatigue monitoring system for ophthalmology, an eye image processing method, equipment and a storage medium. A monitoring image and an optical coherence tomography image of a user's eye are obtained, wherein the optical coherence tomography image comprises a plurality of anatomical partitions; a tear boundary position in each image frame is extracted based on a gradient feature of a region edge in the monitoring image, differential analysis is performed on all tear boundary positions, and a regional disturbance feature of a corneal region when the user uses the eye is obtained; a cross-modal feature map network is constructed through the regional disturbance feature, and then a visual fatigue starting frame of the user is identified; after the visual fatigue starting frame is identified, a functional correlation degree of different anatomical partitions is determined, a dynamic modularity is determined through all functional correlation degrees; and a dominant fatigue partition is identified from different anatomical partitions according to all dynamic modularity. The scheme of the application can target detection of eye fatigue based on multi-modal images.

Description

Technical Field

[0001] The present application relates to the technical field of eye fatigue monitoring, and more specifically, to an eye fatigue monitoring system, an eye image processing method, a device and a storage medium for ophthalmology. Background Art

[0002] Eye fatigue monitoring used in ophthalmology is aimed at the decline in visual system function caused by long-term use of eyes, and realizes early warning and health management of eyes through multi-dimensional detection; in daily life, long-term use of electronic products will lead to high incidence of eye fatigue, and its monitoring relies on a variety of technical means; in terms of physiological index detection, with the help of electroophthalmology, retinal electrical activity is captured, and by analyzing the waveform changes of graphic visual evoked potentials, electroretinograms, etc., the response ability of retinal cells to light stimulation is quantified to judge the loss of visual function; at the same time, the eye tracker tracks the movement trajectory of eye scanning, gaze, blinking, etc. When the blinking frequency is lower than the normal 15-20 times per minute and the scanning speed slows down, it indicates eye muscle fatigue, etc.; eye fatigue monitoring used in ophthalmology usually involves eye image processing of the monitoring target, However, in existing eye image processing methods, monitoring images of the monitoring target (such as eyelid movement videos) and optical coherence tomography images are often fused through simple feature splicing methods. This method has the problem of insufficient depth of multimodal data association modeling (that is, the coordinated disturbance patterns of multiple eye areas such as the cornea, ciliary body, and retina during fatigue are not modeled). As a result, existing eye image processing methods are unable to dynamically characterize the functional correlation evolution of different eye areas when fatigue of the monitoring target is triggered (such as the dynamic correlation changes between the decreased stability of the corneal tear film and the reduced retinal blood flow after long-term eye use), making it difficult to accurately locate the dominant fatigue partition of the monitoring target; therefore, how to perform targeted detection of eye fatigue based on multimodal images has become a difficult problem faced by the industry. Summary of the Invention

[0003] The present application provides an eye fatigue monitoring system, an eye image processing method, a device and a storage medium for ophthalmology, which can perform targeted detection of eye fatigue based on multimodal images.

[0004] In a first aspect, the present application provides an eye image processing method, comprising the following steps:

[0005] acquiring a monitoring image and an optical coherence tomography image of a user's eye, wherein the optical coherence tomography image includes a plurality of anatomical regions of the user's eye;

[0006] Extracting the tear boundary position in each image frame based on the gradient features of the regional edge in the monitoring image, performing differential analysis on all tear boundary positions, and obtaining regional disturbance features of the corneal region when the user uses the eyes;

[0007] A cross-modal feature map network between the monitoring image and the optical coherence tomography image is constructed by using the feature coupling relationship between the regional perturbation feature and each anatomical partition, and then a user's visual fatigue onset frame is identified based on the temporal perturbation patterns of all anatomical partitions in the cross-modal feature map network;

[0008] After identifying the visual fatigue onset frame, determining the functional correlations of different anatomical partitions in the cross-modal feature map network, and then determining the dynamic modularity of different anatomical partitions in the user's eye through all the functional correlations;

[0009] The dominant fatigue zones of users' eyes are identified from different anatomical zones based on all dynamic modularity.

[0010] In some embodiments, extracting the tear boundary position in each image frame based on the gradient feature of the region edge in the monitoring image specifically includes:

[0011] Extracting gradient features of the edge of the region in each image frame of the monitoring image based on an image gradient algorithm;

[0012] Determine multiple boundary candidate points of each image frame according to all gradient features;

[0013] The tear boundary position in each image frame is generated through all boundary candidate points.

[0014] In some embodiments, differential analysis is performed on all tear boundary positions to obtain regional disturbance features of the cornea region when the user uses the eyes, specifically including:

[0015] Generate a boundary change sequence of the user's corneal region through all tear boundary positions;

[0016] Performing time-series differentiation on the tear boundary position in the vector image frame according to the boundary change sequence to obtain a disturbance amplitude sequence of the corneal region;

[0017] The regional disturbance characteristics of the cornea region when the user uses the eyes are determined according to the disturbance amplitude sequence.

[0018] In some embodiments, constructing a cross-modal feature map network between the monitoring image and the optical coherence tomography image through the feature coupling relationship between the regional disturbance feature and each anatomical partition specifically includes:

[0019] Acquiring image feature parameters of a plurality of preset anatomical partitions in the optical coherence tomography image;

[0020] Performing correlation analysis on the regional disturbance feature and the image feature parameters of each anatomical partition to obtain a feature coupling relationship between the regional disturbance feature and each anatomical partition;

[0021] An image feature node mapping is established between the monitoring image and the optical coherence tomography image based on the feature coupling relationship to obtain a cross-modal feature graph network between the monitoring image and the optical coherence tomography image.

[0022] In some embodiments, identifying the user's visual fatigue onset frame based on the temporal perturbation patterns of all anatomical partitions in the cross-modal feature map network specifically includes:

[0023] Obtaining an evolution sequence of regional perturbation features corresponding to all anatomical partitions in the cross-modal feature map network;

[0024] performing a perturbation analysis on the evolution sequence to obtain a perturbation trend of each anatomical partition in each image frame;

[0025] Identify the user's visual fatigue starting frame through all disturbance trends.

[0026] In some embodiments, determining the functional association of different anatomical regions in the cross-modal feature map network specifically includes:

[0027] Generate a functional description vector through the image features and regional perturbation features of each anatomical partition;

[0028] The functional correlation between different anatomical partitions in the cross-modal feature map network is determined based on the functional description vectors between the various anatomical partitions.

[0029] In some embodiments, identifying the dominant zone of eye fatigue of the user from different anatomical zones based on all dynamic modularities specifically includes:

[0030] Constructing dynamic correlation fields between different anatomical partitions based on all dynamic modularity;

[0031] determining the user's eye fatigue degree in each anatomical region according to the dynamic correlation field;

[0032] The dominant eye fatigue zone of the user is identified among all anatomical zones with different eye fatigue levels.

[0033] In a second aspect, the present application provides an ophthalmic eye fatigue monitoring system, comprising an eye image processing unit, wherein the eye image processing unit comprises:

[0034] an acquisition module, configured to acquire a monitoring image and an optical coherence tomography image of a user's eye, wherein the optical coherence tomography image includes a plurality of anatomical regions of the user's eye;

[0035] a processing module, configured to extract the tear boundary position in each image frame based on the gradient features of the regional edges in the monitoring image, perform differential analysis on all tear boundary positions, and obtain regional disturbance features of the corneal region when the user uses the eyes;

[0036] The processing module is further configured to construct a cross-modal feature map network between the monitoring image and the optical coherence tomography image based on the feature coupling relationship between the regional disturbance feature and each anatomical partition, and further identify the user's visual fatigue onset frame based on the temporal disturbance patterns of all anatomical partitions in the cross-modal feature map network;

[0037] The processing module is further configured to, after identifying the visual fatigue onset frame, determine the functional correlations of different anatomical partitions in the cross-modal feature map network, and further determine the dynamic modularity of different anatomical partitions in the user's eye through all the functional correlations;

[0038] An execution module is used to identify the dominant fatigue zone when the user uses his eyes from different anatomical zones according to all dynamic modularities.

[0039] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned eye image processing method when executing the computer program.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned eye image processing method are implemented.

[0041] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0042] The eye fatigue monitoring system, eye image processing method, device and storage medium provided in the present application obtain a monitoring image and an optical coherence tomography image of a user's eye, wherein the optical coherence tomography image includes multiple anatomical partitions of the user's eye; based on the gradient characteristics of the regional edges in the monitoring image, the tear boundary position in each image frame is extracted, and differential analysis is performed on all tear boundary positions to obtain the regional disturbance characteristics of the corneal region when the user uses the eye; a cross-modal feature map network between the monitoring image and the optical coherence tomography image is constructed through the feature coupling relationship between the regional disturbance characteristics and each anatomical partition, and then the user's visual fatigue start frame is identified based on the temporal disturbance pattern of all anatomical partitions in the cross-modal feature map network; after the visual fatigue start frame is identified, the functional correlation of different anatomical partitions in the cross-modal feature map network is determined, and then the dynamic modularity of different anatomical partitions in the user's eye is determined through all functional correlations; and the dominant fatigue partition when the user uses the eye is identified from the different anatomical partitions based on all dynamic modularities.

[0043] It can be seen that in the present application, after obtaining the monitoring image and optical coherence tomography image of the user's eyes, first, the tear boundary position in each image frame of the monitoring image is extracted for differential analysis to obtain the regional disturbance feature of the corneal area when the user uses the eyes, that is: the dynamic feature (monitoring image) based on a single modality is converted into a numerical index reflecting the stability of the cornea. This index can not only independently characterize the fatigue-related changes in the corneal area, but also serve as a bridge for cross-modal fusion - forming a "dynamic"-"static" association with the structural features such as corneal thickness and curvature in the optical coherence tomography image (for example, when the regional disturbance feature decreases, if the optical coherence tomography image shows an increase in micro-wrinkles in the corneal epithelium, fatigue judgment can be enhanced), so that eye fatigue detection breaks through the limitations of a single indicator, and provides underlying feature support for the subsequent identification of fatigue trigger frames in the cross-modal network, and then locks the key time point of fatigue occurrence through time series analysis, for the target. A time window is defined to locate the fatigue area; then, at the starting frame of visual fatigue, the functional correlation of each anatomical partition such as the cornea, ciliary body, and retina in the cross-modal network is first calculated to quantify the synergistic or inhibitory relationship of different regions in the fatigue mechanism - for example, an increase in the ciliary body association strength may indicate that regulatory fatigue is dominant, while abnormal retinal association strength may indicate neural conduction fatigue. Based on this, the dynamic modularity is determined to reflect the dynamic association characteristics of the functional modules of the user's eye anatomical partitions under fatigue state, so as to accurately identify the "hub node" (dominant fatigue partition) with the highest contribution to fatigue. For example, if the edge weight of the ciliary body and the lens is significantly higher than that of other regions, and the association strength between the two shows a synergistic upward trend, it can be directly determined that the path is the dominant mechanism of the current fatigue, realizing traceability analysis from multimodal data to specific fatigue areas; in summary, this scheme can perform targeted detection of eye fatigue based on multimodal images. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flowchart of an eye image processing method according to some embodiments of the present application;

[0045] Figure 2 is a schematic diagram of a process for determining regional disturbance characteristics according to some embodiments of the present application;

[0046] Figure 3 is a schematic diagram of a process for identifying a dominant fatigue zone according to some embodiments of the present application;

[0047] Figure 4 is a schematic structural diagram of an eye image processing unit according to some embodiments of the present application;

[0048] Figure 5 This is a diagram of the internal structure of a computer device for implementing an eye image processing method according to some embodiments of the present application. DETAILED DESCRIPTION

[0049] In order to better understand the technical solution in this embodiment, the technical solution in this embodiment will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0050] refer to Figure 1 , which is a flowchart of an eye image processing method according to some embodiments of the present application. The eye image processing method 100 mainly includes the following steps:

[0051] In step 101, a monitoring image and an optical coherence tomography image of a user's eye are acquired, wherein the optical coherence tomography image includes a plurality of anatomical regions of the user's eye.

[0052] In specific implementation, an infrared imaging unit (such as a high-definition industrial camera, an infrared camera or an integrated eye movement detection module) can be deployed in front of the user's eyes to continuously capture video of the user's eye use process and obtain real-time monitoring images of eye areas such as the cornea, iris, and pupil; at the same time, an optical coherence tomography imaging module configured on the same detection platform or in a nearby position can be used to perform periodic structural scanning of the user's eyes to obtain a tomographic image in the depth direction as the optical coherence tomography image in this application.

[0053] It should be noted that the monitoring image can obtain a visible light image sequence or infrared image sequence of the user during the user's eye use process through continuous frame acquisition. The image sequence can reflect external visual characteristics such as optical boundary changes, tear edge distribution, and palpebral fissure changes in the corneal area; the optical coherence tomography image obtains multi-layer structural images at different depths of the user's eyes through the principle of low-coherence interference, including the stroma, anterior chamber, iris edge, tear film distribution area and other anatomical partitions with spatial organizational characteristics.

[0054] In step 102, the tear boundary position in each image frame is extracted based on the gradient characteristics of the regional edge in the monitoring image, and differential analysis is performed on all tear boundary positions to obtain the regional disturbance characteristics of the corneal area when the user uses his eyes.

[0055] In some embodiments, extracting the tear boundary position in each image frame based on the gradient features of the region edge in the monitoring image can be achieved by using the following steps:

[0056] Extracting gradient features of the edge of the region in each image frame of the monitoring image based on an image gradient algorithm;

[0057] Determine multiple boundary candidate points of each image frame according to all gradient features;

[0058] The tear boundary position in each image frame is generated through all boundary candidate points.

[0059] In specific implementation, the gradient features of the regional edges in each image frame in the monitoring image are extracted based on the image gradient algorithm. The following method is adopted, namely: first, each image frame in the monitoring image is converted into a grayscale image by the grayscale method in the prior art, and noise suppression and contrast enhancement preprocessing is performed; then, the preprocessed grayscale image is convolved by the image gradient operator to calculate the gradient components in the horizontal and vertical directions respectively, thereby obtaining the gradient amplitude and gradient direction of each pixel; finally, the gradient amplitude is normalized, and non-maximum suppression and threshold filtering can be optionally used to optimize the edge response, and the optimized gradient amplitude map and gradient direction map are used as the gradient features of the regional edges in each image frame in the monitoring image; in other embodiments, the gradient calculation step can also be implemented by using the gradient calculation module of the Canny edge detection algorithm or the edge detection network based on deep learning, which is not limited in this application.

[0060] It should be noted that the gradient features of the edge of the area refer to the feature set composed of the gradient components of each pixel in the horizontal and vertical directions obtained by calculating the image gradient operator after preprocessing the monitoring image such as grayscale and noise suppression; the gradient features can be formally expressed as a gradient amplitude map and a gradient direction map, where the gradient amplitude reflects the intensity of the edge and the gradient direction indicates the normal direction of the edge. After normalization or threshold screening, they are used to characterize the tear boundary position and change trend of target areas such as the cornea and iris in the monitoring image.

[0061] In specific implementation, multiple boundary candidate points of each image frame are determined based on all gradient features in the following manner, namely: first, based on the gradient amplitude and gradient direction data of the gradient feature, a non-maximum suppression operation is performed on the gradient amplitude map, the local maximum pixel points along the gradient direction are retained, and a single-pixel width edge response map is generated to eliminate the pseudo-gradient influence of the non-edge area; then, by setting a gradient amplitude threshold (which can be determined by 1.5 times the global mean), pixels with gradient amplitudes greater than the threshold are screened out from the edge response map to form an initial boundary candidate point set; finally, morphological operations are performed on the initial boundary candidate point set, including closing operations to fill small gaps. Hole and open operations are used to remove discrete noise points, and connected domain analysis is used to remove isolated points and small areas that do not conform to the structural characteristics of the eye, retaining pixel points that meet the continuity and area conditions as multiple boundary candidate points of each image frame; wherein, as a preferred embodiment, the screening process of the boundary candidate points can be combined with prior knowledge of the eye structure (such as the circular characteristics of the corneal area, the concentric structure of the iris and pupil), and the distribution of candidate points can be further optimized by ellipse fitting or Hough transform to improve the accuracy of boundary positioning; in other embodiments, a key point detection network based on deep learning can also be used to directly identify boundary candidate points from gradient features, which is not limited in this application.

[0062] In specific implementation, the tear boundary position in each image frame is generated by all boundary candidate points in the following way: first, coordinate clustering is performed on all boundary candidate points, and the density clustering algorithm or K-means clustering algorithm is used to divide the candidate points with similar spatial positions into different sets, and the discrete isolated noise point sets are removed; then, for each clustered candidate point set, the least squares method is used to fit the curve. For the boundary candidate point set of the corneal area, an elliptical curve can be fitted, and for areas such as the iris, a circular curve or a polygonal curve can be fitted to obtain a preliminary boundary curve model; finally, based on the grayscale distribution information of the image and the gradient direction consistency constraint, The preliminary boundary curve model is iteratively optimized, the grayscale difference between the inner and outer areas of the curve is calculated, and the position and shape of the boundary curve are corrected by screening the angle threshold between the gradient direction and the curve normal vector, thereby obtaining the tear boundary position in each image frame; among them, as a preferred embodiment, an active contour model or a level set method can be introduced, and the boundary candidate points are used as the initial contour. The contour position is iteratively adjusted by minimizing the energy function to achieve accurate extraction of the boundary of complex eye structures; in other implementations, a semantic segmentation network based on deep learning can also be used to post-process the boundary candidate points and directly output the precise tear boundary position, which is not limited in this application.

[0063] In some embodiments, reference Figure 2As shown in FIG, this figure is a schematic diagram of a process for determining regional disturbance features according to some embodiments of the present application. Differential analysis is performed on all tear boundary positions to obtain regional disturbance features of the cornea region when the user uses the eyes. The following steps can be used to achieve this:

[0064] First, in step 1021 , a boundary change sequence of the user's corneal region is generated through all tear boundary positions;

[0065] Then, in 1022, a time-series difference is performed on the tear boundary position in the vector image frame according to the boundary change sequence to obtain a disturbance amplitude sequence of the corneal region;

[0066] Finally, in 1023, the regional disturbance characteristics of the cornea region when the user uses his eyes are determined according to the disturbance amplitude sequence.

[0067] In a specific implementation, generating a boundary change sequence of the user's corneal area through all tear boundary positions can be achieved in the following manner: first, extracting key features of the tear boundary position in each image frame, such as a coordinate set of boundary points and parameters of the boundary curve (such as the major axis, minor axis, and center coordinates of an ellipse), and encoding these features into a computer-processable digital vector form, for example, normalizing the boundary point coordinates to the interval [0, 1] and standardizing the ellipse parameters; then, arranging the encoded corneal boundary feature vectors in sequence according to the time sequence of the image frames to form a boundary change sequence representing the boundary changes of the user's corneal area at different times. As a preferred embodiment, the physiological characteristics of the eye can be combined during feature extraction to more accurately select and process boundary features and improve the quality of the boundary change sequence; in other implementations, the tear boundary position data can also be analyzed and processed in the frequency domain using discrete Fourier transform or other methods to assist in generating the boundary change sequence, which is not limited in this application.

[0068] In a specific implementation, the tear boundary position in the vector image frame is time-series differentiated according to the boundary change sequence to obtain the disturbance amplitude sequence of the corneal region, which can be achieved in the following manner: first, a differential operation is performed on the tear boundary position vectors of adjacent image frames in the boundary change sequence in the time domain, and an initial differential vector sequence is generated by calculating the Euclidean distance or Mahalanobis distance of the corresponding boundary points; then, the amplitude of the initial differential vector sequence is calculated, and it is converted into a scalar value by a vector modulus operation to obtain the disturbance amplitude sequence of the corneal region; among them, as a preferred embodiment, a Kalman filter algorithm can be introduced, and the tear boundary position can be predicted in combination with a priori model of corneal movement, so as to optimize the differential calculation process and improve the calculation accuracy; in other implementations, the frequency characteristics of the boundary change can also be extracted by frequency domain analysis means, such as short-time Fourier transform, to construct a multi-scale disturbance amplitude sequence, which is not limited in this application.

[0069] It should be noted that the disturbance amplitude sequence of the corneal area described in the present application refers to a numerical sequence formed by performing a time-series difference operation on the corneal tear boundary position vectors of adjacent image frames in the boundary change sequence, calculating the corresponding boundary point distances and converting the differential vector modulo into a scalar value, and then arranging them in chronological order; the disturbance amplitude sequence is represented as a set of ordered scalar values, each value corresponds to the degree of change of the corneal area boundary in a frame of image relative to the previous frame, and its numerical value directly reflects the structural fluctuation of the corneal area in the time dimension, which can be used for subsequent evaluation of the structural stability of the corneal area when the user uses the eyes.

[0070] In specific implementation, the regional disturbance characteristics of the corneal area when the user uses the eyes can be determined according to the disturbance amplitude sequence in the following manner, namely: first, the statistical characteristics of the disturbance amplitude sequence are extracted, the standard deviation, coefficient of variation or root mean square value of the sequence are calculated, and the statistical parameters reflecting the degree of fluctuation of the corneal regional structure are obtained; then, the statistical parameters are slidingly calculated based on a preset time window to generate a local stability index sequence, wherein the length of the time window can be dynamically adjusted according to the physiological characteristics of the human eye or the eye behavior cycle; finally, the local stability index sequence is weightedly fused, the weight coefficient is set in combination with the prior knowledge of the corneal structure, and the fusion result is mapped to [0, 1] interval, obtain the regional disturbance characteristics of the corneal area when the user uses the eyes; wherein, as a preferred embodiment, a support vector machine or a random forest algorithm can be introduced to construct a stability evaluation model, and historical data can be used for training to achieve adaptive learning of the complex mapping relationship between the disturbance amplitude sequence and the regional disturbance characteristics; in other embodiments, a fuzzy comprehensive evaluation model can also be constructed through fuzzy mathematics theory to conduct a comprehensive evaluation of multi-dimensional stability indicators, which is not limited in this application; it should be noted that the regional disturbance characteristics are quantitative indicators used to reflect the degree of structural fluctuation of the corneal area, and the fatigue state or functional abnormality of the corneal area can be evaluated through the regional disturbance characteristics.

[0071] In step 103, a cross-modal feature map network between the monitoring image and the optical coherence tomography image is constructed through the feature coupling relationship between the regional disturbance features and each anatomical partition, and then the user's visual fatigue starting frame is identified based on the temporal disturbance patterns of all anatomical partitions in the cross-modal feature map network.

[0072] It should be noted that the regional disturbance feature is a dynamic indicator calculated based on the boundary change characteristics of the corneal region in the monitoring image, which is used to reflect the structural stability of the cornea in continuous image frames; in the image space, the corneal region in the monitoring image and the multiple anatomical partitions in the optical coherence tomography image have positional overlap and tissue structure correspondence, so the regional disturbance feature can be associated with the anatomical partition containing corneal information in the optical coherence tomography image; that is: through image registration and tear boundary position mapping, the regional disturbance feature can be spatially aligned between the two types of images, and then the dynamic boundary information of the monitoring image can be mapped to the static anatomical structure in the optical coherence tomography image, providing a basis for constructing a cross-modal feature map network.

[0073] In some embodiments, constructing a cross-modal feature map network between the monitoring image and the optical coherence tomography image through the feature coupling relationship between the regional disturbance feature and each anatomical partition can be achieved by the following steps:

[0074] Acquiring image feature parameters of a plurality of preset anatomical partitions in the optical coherence tomography image;

[0075] Performing correlation analysis on the regional disturbance feature and the image feature parameters of each anatomical partition to obtain a feature coupling relationship between the regional disturbance feature and each anatomical partition;

[0076] An image feature node mapping is established between the monitoring image and the optical coherence tomography image based on the feature coupling relationship to obtain a cross-modal feature graph network between the monitoring image and the optical coherence tomography image.

[0077] It should be noted that the image feature parameters of the multiple anatomical partitions preset in the optical coherence tomography image refer to a multi-dimensional feature data set extracted from preset anatomical partitions such as the corneal epithelium, stroma, and endothelial cell layer after preprocessing and regional segmentation of the optical coherence tomography image; the image feature parameters are composed of morphological feature parameters (such as regional area, perimeter, and roundness index), grayscale statistical feature parameters (such as mean, variance, and kurtosis value), texture feature parameters (such as energy, entropy, and contrast parameters based on grayscale co-occurrence matrix, or local binary pattern feature vectors) and geometric feature parameters (such as structural thickness and curvature parameters). The image feature parameters are used to quantitatively describe the morphology, grayscale distribution, texture pattern, and geometric morphological characteristics of each anatomical partition, providing a data basis for the subsequent analysis of the coupling relationship between regional disturbance features and each region; as a preferred embodiment, U-Net or DeepLabv3 can be introduced. + and other deep learning semantic segmentation networks, which are trained using labeled medical image datasets to achieve automatic and accurate segmentation and feature extraction of each anatomical partition, thereby obtaining image feature parameters of each anatomical partition; in other embodiments, the feature parameters of a specific anatomical partition can also be obtained through manual interaction combined with computer-aided measurement tools, which is not limited in this application.

[0078] In specific implementation, correlation analysis is performed on the regional disturbance features and the image feature parameters of each anatomical partition to obtain the feature coupling relationship of the regional disturbance features to each anatomical partition, which can be achieved in the following manner: first, the linear correlation degree of the regional disturbance features and the morphological, grayscale, texture, geometry and other features in the image feature parameters of each anatomical partition is calculated using the Pearson correlation coefficient, and the nonlinear relationship features are measured using the Spearman rank correlation coefficient or the Kendall τ correlation coefficient to generate an initial correlation coefficient matrix; then, the initial correlation coefficient matrix is ​​weighted and the coefficients are mapped to the [0, 1] interval by the Softmax function to obtain the feature coupling relationship of the regional disturbance features to each anatomical partition; as a preferred embodiment, a random forest regression algorithm can be introduced, with the regional disturbance features as the dependent variable and the image feature parameters as the independent variable, and the feature importance score is calculated by the node splitting situation during model training, as the feature coupling relationship of the regional disturbance features to each anatomical partition; in other embodiments, other methods can also be used for determination, which are not limited here.

[0079] It should be noted that the initial correlation coefficient matrix refers to a matrix formed by arranging the obtained correlation coefficients in a specific order after calculating the correlation between the regional disturbance features and the morphological, grayscale, texture, geometric and other features in the image feature parameters of each anatomical partition by using the Pearson correlation coefficient, the Spearman rank correlation coefficient or the Kendall τ correlation coefficient respectively. The rows of the initial correlation coefficient matrix correspond to different anatomical partitions, and the columns correspond to different types of image feature parameters. The numerical value of each element in the matrix reflects the degree of association between the corresponding anatomical partition feature parameters and the regional disturbance features. The closer the absolute value is to 1, the stronger the correlation is, and the closer it is to 0, the weaker the correlation is. In addition, the feature coupling relationship refers to the weighted association relationship between the regional disturbance features and each anatomical partition determined after weight calculation of the initial correlation coefficient matrix and mapping the coefficients to the interval [0, 1] by the Softmax function. The weight relationship quantifies the degree of influence of the image feature parameters of each anatomical partition on the regional disturbance features in numerical form. The larger the value, the closer the association between the corresponding anatomical partition and corneal stability is.

[0080] In a specific implementation, an image feature node mapping is established between the monitoring image and the optical coherence tomography image based on the feature coupling relationship, and a cross-modal feature graph network between the monitoring image and the optical coherence tomography image is obtained, which can be implemented in the following manner, namely: first, the corneal boundary feature points in the monitoring image and the image feature parameters of each anatomical partition in the optical coherence tomography image are respectively used as nodes in the graph structure of the prior art, wherein the nodes of the monitoring image cover features such as tear boundary position and curvature, and the nodes of the optical coherence tomography image include image feature parameters such as morphology, grayscale, and texture; subsequently, the coupling weight value of the regional perturbation feature to each anatomical partition is mapped to the weight of the edge between nodes to obtain an initial weighted undirected graph structure; then, the feature nodes of the monitoring image and the optical coherence tomography image are projected to the graph structure of the prior art through the linear transformation method in the prior art. The same feature space, secondly, determine the structural similarity and feature consistency between nodes, and then define a reconstruction error function based on the similarity measure, and then use optimization methods such as gradient descent to adjust the mapping function parameters by minimizing the reconstruction error. When the reconstruction error change of consecutive iterations is less than the set threshold, stop the optimization, and finally, use the optimized weighted undirected graph as the cross-modal feature graph network between the monitoring image and the optical coherence tomography image; wherein, as a preferred embodiment, a graph neural network model can be introduced, with the weighted undirected graph as input, and the high-order correlation features between nodes are learned through graph convolution operations, and the node mapping relationship is automatically optimized to obtain the cross-modal feature graph network; in other implementations, the canonical correlation analysis method can also be used to find the linear projection direction between the two image modalities, maximize the correlation of the projected features, and realize feature node mapping, which is not limited in this application.

[0081] In some embodiments, identifying the user's visual fatigue onset frame based on the temporal perturbation patterns of all anatomical partitions in the cross-modal feature map network can be achieved by using the following steps:

[0082] Obtaining an evolution sequence of regional perturbation features corresponding to all anatomical partitions in the cross-modal feature map network;

[0083] performing a perturbation analysis on the evolution sequence to obtain a perturbation trend of each anatomical partition in each image frame;

[0084] Identify the user's visual fatigue starting frame through all disturbance trends.

[0085] In a specific implementation, obtaining the evolution sequence of the regional perturbation features corresponding to all anatomical partitions in the cross-modal feature map network can be achieved in the following manner: first, traversing the cross-modal feature map network to determine the nodes of all anatomical partitions therein; then, for each node of each anatomical partition, obtaining the corresponding regional perturbation features; then, collecting the regional perturbation features of each anatomical partition in each image frame in chronological order to form time series data corresponding to the regional perturbation features of each anatomical partition; finally, integrating the time series data of all anatomical partitions to construct an evolution sequence of the regional perturbation features corresponding to all anatomical partitions; as a preferred embodiment, integrating the time series data of all anatomical partitions specifically includes: associating the regional perturbation features of different anatomical partitions at the same time point; for example, collecting the regional perturbation features of all anatomical partitions in the frame together to form a vector containing multiple anatomical partition indices as the evolution sequence of the regional perturbation features corresponding to the anatomical partitions; in other embodiments, other methods can also be used for implementation, which are not limited here; it should be noted that the evolution sequence of the regional perturbation features is a sequence that describes the law of change of the regional perturbation features of all anatomical partitions over time.

[0086] In a specific implementation, perturbation analysis is performed on the evolution sequence to obtain the perturbation trend of each anatomical partition in each image frame. This can be achieved in the following manner: first, the regional perturbation feature difference between adjacent image frames of each anatomical partition in the evolution sequence is calculated; then, a perturbation threshold is set, and each regional perturbation feature difference is numerically compared with the perturbation threshold; when the regional perturbation feature difference is greater than the perturbation threshold, it is determined that the perturbation trend of the corresponding anatomical partition in the corresponding image frame is increasing; when the regional perturbation feature difference is less than the perturbation threshold, it is determined that the perturbation trend of the corresponding anatomical partition in the corresponding image frame is decreasing; when the regional perturbation feature difference is equal to the perturbation threshold, it is determined that the anatomical partition is in a stable state in the corresponding image frame, thereby obtaining the perturbation trend of each anatomical partition in each image frame.

[0087] It should be noted that the disturbance trend refers to the state change tendency of each anatomical partition in each image frame determined by analyzing the evolution sequence of regional disturbance features; in addition, the disturbance threshold can be preset according to the requirements of the application scenario. For example, in scenarios with high requirements for visual fatigue detection accuracy, if it is hoped to capture more subtle changes, a smaller disturbance threshold is set; in scenarios with relatively low requirements for accuracy and more emphasis on overall trend judgment, a larger disturbance threshold is set.

[0088] In specific implementation, identifying the user's visual fatigue start frame through all disturbance trends can be achieved in the following manner, namely: first, constructing a mapping model between the disturbance trends of each anatomical partition and the visual fatigue state, training a machine learning model (such as support vector machine, random forest, convolutional neural network) based on prior knowledge in the medical field or through a large amount of image data containing visual fatigue annotations, and determining the correspondence between different disturbance trend combinations and visual fatigue levels (such as no fatigue, mild fatigue, moderate fatigue, and severe fatigue); then setting trigger judgment rules, including assigning different weights to anatomical partitions (such as assigning higher weights to key areas such as the corneal epithelium and retina), defining trigger thresholds for each fatigue level (such as mild fatigue requires that the disturbance trends of at least 50% of the high-weight areas meet preset conditions and last for more than 3 frames), and setting a multi-area collaborative judgment mechanism; finally, traversing all disturbance trends of each image frame, inputting the disturbance trends into the trained model to obtain a prediction result, and when the prediction result meets the threshold condition and the number of frames required in the trigger judgment rule, the current image frame is determined to be the visual fatigue start frame of the corresponding fatigue level.

[0089] Preferably, a mapping model between the disturbance trend of each anatomical partition and the visual fatigue state can be constructed by collecting a large amount of image data related to visual fatigue, and labeling the data with visual fatigue states through a visual fatigue assessment method (such as dividing them into categories such as no fatigue, mild fatigue, moderate fatigue, severe fatigue, etc.). Subsequently, the disturbance trend of each anatomical partition in each image frame is used as a feature vector, and together with the corresponding visual fatigue state label, it constitutes a training data set. Using the training data set, a machine learning algorithm is used to construct a visual fatigue feature association model; for example, when using a support vector machine algorithm, by selecting a suitable kernel function (such as a radial basis kernel function), the optimal hyperplane is found in the feature space to achieve a classification mapping between the disturbance trend feature vector and the visual fatigue state; or a convolutional neural network in deep learning is used to automatically extract high-order features in the disturbance trend through multi-layer convolution and pooling operations, and then the prediction result of the visual fatigue state is output through a fully connected layer, thereby constructing a mapping model between the disturbance trend of each anatomical partition and the visual fatigue state. In other embodiments, other methods can also be used for determination, which are not limited here.

[0090] In step 104, after the visual fatigue starting frame is identified, the functional correlation of different anatomical partitions in the cross-modal feature map network is determined, and then the dynamic modularity of different anatomical partitions in the user's eyes is determined through all functional correlations.

[0091] In some embodiments, determining the functional correlation of different anatomical regions in the cross-modal feature map network may be achieved by the following steps:

[0092] Generate a functional description vector through the image features and regional perturbation features of each anatomical partition;

[0093] The functional correlation between different anatomical partitions in the cross-modal feature map network is determined based on the functional description vectors between the various anatomical partitions.

[0094] In specific implementation, the function description vector generated by the image features and regional disturbance features of each anatomical partition can be achieved in the following manner, namely: first, the multi-dimensional image features such as morphology, grayscale, texture and spatial position of each anatomical partition are extracted, and the regional disturbance features of the corresponding anatomical partition are obtained, and then the weighted summation is performed by the weighted fusion method adopted in the prior art to obtain the function description vector of the anatomical partition; it should be noted that the weighted fusion method is to generate a vector by setting a weight coefficient to perform a weighted operation on the image feature vector and the regional disturbance feature, and the weight coefficient can be optimized by cross-validation; as a preferred embodiment, the attention mechanism can be used to weight the image features, highlight the feature dimensions that are sensitive to visual fatigue, and then fuse them with the regional disturbance features; in other implementations, the kernel method can also be used to map the image features and the regional disturbance features to a high-dimensional space and then fuse them, and this application does not limit this.

[0095] In specific implementation, the functional correlation of different anatomical partitions in the cross-modal feature map network is determined based on the functional description vectors between each anatomical partition, that is: the similarity of the functional description vectors between each anatomical partition is used as the functional correlation of different anatomical partitions in the cross-modal feature map network; for example: the cosine similarity, Euclidean distance and other measurement methods in the prior art are used to calculate the similarity value between the functional description vectors of any two anatomical partitions, and the similarity value is used as the functional correlation of the corresponding two anatomical partitions in the cross-modal feature map network.

[0096] In some embodiments, determining the dynamic modularity of different anatomical regions in the user's eye using all functional associations may be accomplished by:

[0097] Evaluate the fatigue evolution trend of each anatomical partition based on all functional correlations and the coupling relationship between anatomical partitions;

[0098] The dynamic modularity of different anatomical partitions in the user's eye is determined according to the fatigue evolution trend of all anatomical partitions.

[0099] It should be noted that the coupling relationship refers to the interdependent and synergistic correlation characteristics of each anatomical partition at the physiological function level; for example, in the visual system, the photoelectric conversion function of the retina depends on the stable blood supply of the choroid to maintain metabolism, while the optical properties of the cornea are closely related to the moistening and nutritional effects of the tear film; as a preferred embodiment, the functional coupling strength between each two anatomical partitions can be determined by expert scoring (for example, for anatomical partitions with direct material exchange and energy transfer relationships, a higher coupling strength value (such as 0.8-1.0) is assigned; for anatomical partitions with weaker functional associations, a lower value (such as 0-0.2) is assigned), and then the functional coupling strengths between all anatomical partitions are presented in matrix form to form a functional coupling relationship matrix. Finally, the functional coupling relationship matrix is ​​used as the functional coupling relationship between anatomical partitions; therefore, in specific implementation, the fatigue evolution trend of each anatomical partition can be evaluated based on all functional correlations combined with the functional coupling relationship between anatomical partitions. This can be achieved in the following way, namely: first, all functional correlations are combined. The degree of connection is constructed as a functional association matrix, and then the functional association matrix is ​​tensor-producted with the functional coupling relationship matrix to obtain a comprehensive association matrix. Then, a graph structure is constructed with the comprehensive association matrix, with anatomical partitions as nodes and comprehensive association strengths as edge weights, and the graph structure is processed by a graph convolutional network. The functional description vector of each node is input, and a fatigue feature vector is output through a multi-layer graph convolution operation. Subsequently, the rate of change of the fatigue feature vector in the time dimension (fatigue evolution trend value) is calculated as the fatigue evolution trend of each anatomical partition. Preferably, the functional association matrix can be obtained by organizing and arranging in a two-dimensional matrix form. The row index and column index of the functional association matrix correspond to different anatomical partitions, and the element value of the functional association matrix is ​​the functional association value between the two anatomical partitions, thereby forming a functional association matrix that completely characterizes the functional association relationship between all anatomical partitions. In addition, it should be noted that the fatigue evolution trend refers to a quantitative indicator of the fatigue state of each anatomical partition over time.

[0100] In specific implementations, the dynamic modularity of different anatomical regions within a user's eye can be determined based on the fatigue evolution trends of all anatomical regions. The following approach is employed: First, a graph structure is constructed using the anatomical regions as nodes and the values ​​of the integrated association matrix as edge weights. The fatigue evolution trend value of each node is used as a node attribute. Subsequently, a modularity optimization algorithm is used to partition the graph into communities and calculate the initial modularity. This process incorporates a fatigue evolution trend similarity constraint (e.g., nodes with trend value differences less than a preset threshold are preferentially assigned to the same module). Finally, the edge weights within the module are weighted by the fatigue evolution trend value (nodes with higher trend values ​​contribute more weight to the connection strength within the module). The dynamic modularity, which accounts for fatigue status, is then recalculated and output. It should be noted that the preset threshold and weighting coefficient can be calibrated using anatomical functional clustering data. As a preferred embodiment, a dynamic graph neural network model can be constructed to update node features in real time based on the fatigue evolution trend and optimize modularity calculation. In other implementations, a modularity metric based on information entropy can also be used, combined with information gain from fatigue trends to assess the rationality of module partitioning. This is not a limitation of this application.

[0101] It should be noted that the dynamic modularity is a quantitative indicator used to reflect the dynamic correlation characteristics of functional modules in the user's eye anatomical partitions under fatigue state. The dynamic modularity realizes the dynamic optimization of the eye functional module division by integrating the functional correlation, coupling relationship and fatigue evolution trend of the anatomical partitions, and provides a quantitative basis for analyzing the synergistic effect pattern of various eye areas during visual fatigue.

[0102] In step 105 , the dominant fatigue partition when the user uses his eyes is identified from different anatomical partitions according to all dynamic modularity.

[0103] In some embodiments, identifying the dominant fatigue zone of the user's eyes from different anatomical zones based on all dynamic modularity can be achieved by using the following steps:

[0104] Constructing dynamic correlation fields between different anatomical partitions based on all dynamic modularity;

[0105] determining the user's eye fatigue degree in each anatomical region according to the dynamic correlation field;

[0106] The dominant eye fatigue zone of the user is identified among all anatomical zones with different eye fatigue levels.

[0107] In specific implementation, the dynamic association field between different anatomical partitions based on all dynamic modularity can be achieved in the following manner, namely: first, a basic graph structure is constructed with anatomical partitions as nodes and the element values ​​of the comprehensive association matrix obtained by dynamic modularity calculation as edge weights; then, the fatigue evolution trend value of each node (the time change rate of the fatigue feature vector obtained by processing the function description vector through the graph convolutional network) is embedded into the graph as a node attribute; then, according to the community division result of the modularity optimization algorithm (such as the Louvain algorithm), module labels are added to the nodes and module-level association features are calculated (such as the sum of the edge weights between modules and the average fatigue trend within the module); finally, the association strength of nodes with high fatigue trends is enhanced through a dynamic weight adjustment mechanism (such as for every 0.1 increase in the trend value, the corresponding node edge weight increases by 5%) to form a dynamic association field; as a preferred embodiment, weight updates can be performed directly based on the graph structure calculated by dynamic modularity; in other implementations, association fields with state characteristics can also be quickly constructed through module affiliation, which is not limited in this application.

[0108] It should be noted that the dynamic association field is a graph structure constructed with anatomical partitions as nodes and the element values ​​of the comprehensive association matrix obtained by dynamic modularity calculation as edge weights, in which the fatigue evolution trend value of each node (the time change rate of the fatigue characteristic vector obtained by processing the functional description vector through the graph convolutional network) is embedded as a node attribute, and module labels are added to the nodes according to the community division results of the modularity optimization algorithm.

[0109] In specific implementation, the following method can be used to determine the user's eye fatigue degree in each anatomical partition based on the dynamic association field, namely: first, extract the real-time association strength value, dynamic evolution trend (such as association enhancement rate, weakening rate) and other attribute information of each node in the dynamic association field, and simultaneously obtain the edge weight (current frame association strength) information of the edge between the nodes; then, use the real-time association strength value as the basic measure of eye fatigue, and modify the basic measure according to the dynamic evolution trend (such as association enhancement rate indicates strengthening of association, weakening rate indicates weakening of association), and further increase the eye fatigue degree of the anatomical partition connected to the edge with higher edge weight (current frame association strength) and corresponding node with higher real-time association strength value; thereby calculating to the degree of eye fatigue of each anatomical partition; as a preferred embodiment, correction coefficients can be set for different dynamic evolution trends (such as association enhancement rate and attenuation rate), for example, the coefficient range for the association enhancement rate is 1.2-1.5, the coefficient range for the association attenuation rate is 0.5-0.8, and the coefficient for no significant dynamic evolution trend is 1.0; then, the real-time association strength value is used as the basic metric and multiplied by the correction coefficient corresponding to the dynamic evolution trend to obtain the degree of eye fatigue of the anatomical partition; in other embodiments, the hierarchical analysis method can also be used, and the real-time association strength value, dynamic evolution trend, edge weight (current frame association strength) and other influencing factors are weighted in combination with expert experience to calculate the degree of eye fatigue, and this application does not limit this.

[0110] For specific implementation, refer to Figure 3As shown in the figure, this figure is a schematic diagram of the process of identifying the dominant fatigue partition shown in some embodiments of the present application. The following method can be used to identify the dominant fatigue partition when the user uses his eyes among all anatomical partitions with different eye fatigue levels, namely: first, the eye fatigue levels of each anatomical partition are arranged in descending order; then, a dominant fatigue partition screening threshold is set (the screening threshold can be determined based on historical data statistics or experimental test results), and the anatomical partition with an eye fatigue level greater than the screening threshold is used as a candidate dominant fatigue partition; if the number of candidate areas is 1, the area is directly identified as the dominant fatigue partition when the user uses his eyes; if the number of candidate areas is greater than 1, the ratio of the eye fatigue level of each candidate area to the total eye fatigue level of all candidate areas is calculated. For example, the area with the highest proportion is determined as the dominant fatigue zone; if the eye fatigue levels of all anatomical zones are lower than the screening threshold, it is determined that there is no obvious dominant fatigue zone at present; in addition, it should be noted that the screening threshold can be dynamically adjusted according to the characteristics of different user groups or eye usage scenarios; as a preferred embodiment, a clustering algorithm can be used to perform cluster analysis on the eye fatigue data of all anatomical zones, and the area with the highest eye fatigue level in the largest cluster after clustering is identified as the dominant fatigue zone; in other implementations, a classification model based on a decision tree can also be constructed, with anatomical zone attributes such as eye fatigue level and real-time correlation strength value, dynamic evolution trend (such as correlation enhancement rate, weakening rate) as input, and output the dominant fatigue zone, which is not limited in this application.

[0111] In addition, in another aspect of the present application, in some embodiments, the present application provides an eye fatigue monitoring system for ophthalmology, the system also includes an eye image processing unit, referring to Figure 4 , which is a schematic diagram of the structure of an eye image processing unit according to some embodiments of the present application. The eye image processing unit 200 includes: an acquisition module 201, a processing module 202, and an execution module 203, which are described as follows:

[0112] Acquisition module 201, in this application, acquisition module 201 is mainly used to acquire a monitoring image and an optical coherence tomography image of the user's eye, wherein the optical coherence tomography image includes multiple anatomical regions of the user's eye;

[0113] Processing module 202, in this application, is mainly used to extract the tear boundary position in each image frame based on the gradient characteristics of the regional edge in the monitoring image, perform differential analysis on all tear boundary positions, and obtain regional disturbance characteristics of the corneal area when the user uses the eyes;

[0114] In addition, the processing module 202 in the present application is further configured to construct a cross-modal feature map network between the monitoring image and the optical coherence tomography image based on the feature coupling relationship between the regional disturbance feature and each anatomical partition, and then identify the user's visual fatigue onset frame based on the temporal disturbance pattern of all anatomical partitions in the cross-modal feature map network;

[0115] In addition, the processing module 202 in the present application is further configured to determine the functional correlation of different anatomical partitions in the cross-modal feature map network after identifying the visual fatigue onset frame, and then determine the dynamic modularity of different anatomical partitions in the user's eye through all functional correlations;

[0116] The execution module 203 in this application is mainly used to identify the dominant fatigue partition when the user uses his eyes from different anatomical partitions based on all dynamic modularity.

[0117] In addition, the present application also provides a computer device, which includes a memory and a processor, wherein the memory stores code, and the processor is configured to obtain the code and execute the above-mentioned eye image processing method.

[0118] In some embodiments, reference Figure 5 , which is an internal structure diagram of a computer device for implementing an eye image processing method according to some embodiments of the present application. The eye image processing method in the above embodiment can be Figure 5 The computer device 300 shown in FIG. 1 is implemented as shown in FIG. 1 , and the computer device 300 includes at least one processor 301 , a communication bus 302 , a memory 303 , and at least one communication interface 304 .

[0119] The processor 301 may be a general-purpose central processing unit (CPU), or an application-specific integrated circuit (ASIC) or one or more processors for controlling the execution of the eye image processing method of the present application.

[0120] The communication bus 302 is used to transmit information between the above components.

[0121] Memory 303 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, an optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer. Memory 303 may exist independently and be connected to processor 301 via communication bus 302. Memory 303 may also be integrated with processor 301.

[0122] Memory 303 is used to store program code for executing the present invention, and is controlled by processor 301 for execution. Processor 301 is used to execute the program code stored in memory 303. The program code may include one or more software modules. The eye image processing method in the above embodiment can be implemented by processor 301 and one or more software modules in the program code stored in memory 303.

[0123] The communication interface 304 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0124] In a specific implementation, as an example, a computer device may include multiple processors, each of which may be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0125] The aforementioned computer device may be a general-purpose computer device or a dedicated computer device. In a specific implementation, the computer device may be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of this application do not limit the type of computer device.

[0126] In addition, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned eye image processing method is implemented.

[0127] In summary, the eye fatigue monitoring system, eye image processing method, device, and storage medium disclosed in the embodiments of the present application obtain a monitoring image and an optical coherence tomography image of a user's eye, wherein the optical coherence tomography image includes multiple anatomical partitions of the user's eye; based on the gradient features of the regional edges in the monitoring image, the tear boundary position in each image frame is extracted, and differential analysis is performed on all tear boundary positions to obtain the regional disturbance features of the corneal region when the user uses the eye; a cross-modal feature map network between the monitoring image and the optical coherence tomography image is constructed through the feature coupling relationship between the regional disturbance features and each anatomical partition, and then the user's visual fatigue start frame is identified based on the temporal disturbance pattern of all anatomical partitions in the cross-modal feature map network; after the visual fatigue start frame is identified, the functional correlation of different anatomical partitions in the cross-modal feature map network is determined, and then the dynamic modularity of different anatomical partitions in the user's eye is determined based on all functional correlations; the dominant fatigue partition when the user uses the eye is identified from the different anatomical partitions based on all dynamic modularities; and targeted detection of eye fatigue can be performed based on multimodal images.

[0128] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0129] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if such changes and modifications fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such changes and modifications.

Claims

1. A method for processing eye images, characterized in that: The steps include: acquiring a monitoring image and an optical coherence tomography image of a user's eye, wherein the optical coherence tomography image includes a plurality of anatomical regions of the user's eye; Extracting the tear boundary position in each image frame based on the gradient features of the regional edge in the monitoring image, performing differential analysis on all tear boundary positions, and obtaining regional disturbance features of the corneal region when the user uses the eyes; A cross-modal feature map network between the monitoring image and the optical coherence tomography image is constructed by using the feature coupling relationship between the regional perturbation feature and each anatomical partition, and then a user's visual fatigue onset frame is identified based on the temporal perturbation patterns of all anatomical partitions in the cross-modal feature map network; After identifying the visual fatigue onset frame, determining the functional correlations of different anatomical partitions in the cross-modal feature map network, and then determining the dynamic modularity of different anatomical partitions in the user's eye through all the functional correlations; Identify the dominant fatigue zones of users during eye use from different anatomical zones based on all dynamic modularity; Determining the functional correlation of different anatomical partitions in the cross-modal feature map network specifically includes: Generate a functional description vector through the image features and regional perturbation features of each anatomical partition; determining the functional correlation between different anatomical partitions in the cross-modal feature map network based on the functional description vectors between the anatomical partitions; The dynamic modularity of different anatomical partitions in the user's eye is determined by all functional associations, specifically including: Evaluate the fatigue evolution trend of each anatomical partition based on all functional correlations and the coupling relationship between anatomical partitions; Determining the dynamic modularity of different anatomical regions of the user's eye based on the fatigue evolution trend of all anatomical regions; Among them, the dominant fatigue zones of the user's eyes are identified from different anatomical zones based on all dynamic modularity, specifically including: Constructing dynamic correlation fields between different anatomical partitions based on all dynamic modularity; determining the user's eye fatigue degree in each anatomical region according to the dynamic correlation field; The dominant eye fatigue zone of the user is identified among all anatomical zones with different eye fatigue levels.

2. The method according to claim 1, wherein Extracting the tear boundary position in each image frame based on the gradient feature of the region edge in the monitoring image specifically includes: Extracting gradient features of the edge of the region in each image frame of the monitoring image based on an image gradient algorithm; Determine multiple boundary candidate points of each image frame according to all gradient features; The tear boundary position in each image frame is generated through all boundary candidate points.

3. The method according to claim 1, wherein Perform differential analysis on all tear boundary positions to obtain the regional disturbance characteristics of the cornea area when the user uses the eyes, including: Generate a boundary change sequence of the user's corneal region through all tear boundary positions; Performing time-series differentiation on the tear boundary position in the vector image frame according to the boundary change sequence to obtain a disturbance amplitude sequence of the corneal region; The regional disturbance characteristics of the cornea region when the user uses the eyes are determined according to the disturbance amplitude sequence.

4. The method according to claim 1, wherein Constructing a cross-modal feature map network between the monitoring image and the optical coherence tomography image through the feature coupling relationship between the regional disturbance feature and each anatomical partition specifically includes: Acquiring image feature parameters of a plurality of preset anatomical partitions in the optical coherence tomography image; Performing correlation analysis on the regional disturbance feature and the image feature parameters of each anatomical partition to obtain a feature coupling relationship between the regional disturbance feature and each anatomical partition; An image feature node mapping is established between the monitoring image and the optical coherence tomography image based on the feature coupling relationship to obtain a cross-modal feature graph network between the monitoring image and the optical coherence tomography image.

5. The method according to claim 1, wherein Identifying the user's visual fatigue onset frame based on the temporal perturbation patterns of all anatomical partitions in the cross-modal feature map network specifically includes: Obtaining an evolution sequence of regional perturbation features corresponding to all anatomical partitions in the cross-modal feature map network; performing a perturbation analysis on the evolution sequence to obtain a perturbation trend of each anatomical partition in each image frame; Identify the user's visual fatigue starting frame through all disturbance trends.

6. An ophthalmological eye fatigue monitoring system, comprising an eye image processing unit, which processes eye images using the method according to any one of claims 1 to 5, characterized in that: The eye image processing unit includes: an acquisition module, configured to acquire a monitoring image and an optical coherence tomography image of a user's eye, wherein the optical coherence tomography image includes a plurality of anatomical regions of the user's eye; a processing module, configured to extract the tear boundary position in each image frame based on the gradient features of the regional edges in the monitoring image, perform differential analysis on all tear boundary positions, and obtain regional disturbance features of the corneal region when the user uses the eyes; The processing module is further configured to construct a cross-modal feature map network between the monitoring image and the optical coherence tomography image based on the feature coupling relationship between the regional disturbance feature and each anatomical partition, and further identify the user's visual fatigue onset frame based on the temporal disturbance patterns of all anatomical partitions in the cross-modal feature map network; The processing module is further configured to, after identifying the visual fatigue onset frame, determine the functional correlations of different anatomical partitions in the cross-modal feature map network, and further determine the dynamic modularity of different anatomical partitions in the user's eye through all the functional correlations; An execution module is used to identify the dominant fatigue zone when the user uses his eyes from different anatomical zones according to all dynamic modularities.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the eye image processing method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the eye image processing method according to any one of claims 1 to 5 are implemented.

Images