Eye fatigue monitoring system for ophthalmology department, eye image processing method and device and storage medium

Abstract

The invention provides an eye fatigue monitoring system for ophthalmology, an eye image processing method and device and a storage medium, and the method comprises the steps: obtaining a monitoring image and an optical coherence tomography image of the eyes of a user, and enabling the optical coherence tomography image to comprise a plurality of anatomical partitions; extracting a tear boundary position in each image frame based on the gradient feature of the region edge in the monitoring image, and carrying out differential analysis on all tear boundary positions to obtain a region disturbance feature of a cornea region when the user uses eyes; constructing a cross-modal feature map network through the regional disturbance features, and further identifying a visual fatigue start frame of the user; after the visual fatigue starting frame is recognized, determining function association degrees of different anatomical partitions, and determining a dynamic modularity through all the function association degrees; a dominant fatigue partition is identified from the different anatomical partitions according to all the dynamic modularity. By adopting the scheme of the invention, targeted detection can be carried out on eyestrain based on the multi-modal image.

Description

Technical Field

[0001] This application relates to the technical field of eye fatigue monitoring. More specifically, this application relates to an eye fatigue monitoring system for ophthalmology, an eye image processing method, a device, and a storage medium. Background Art

[0002] Eye fatigue monitoring in ophthalmology aims to detect the decline of the visual system function caused by long-term eye use and achieve early warning and health management of the eyes through multi-dimensional detection. In daily life, long-term use of electronic products can lead to a high incidence of eye fatigue, and its monitoring relies on a variety of technical means. In terms of physiological index detection, an electro-oculogram is used to capture the electroretinal activity, and by analyzing the waveform changes of pattern visual evoked potential, electroretinogram, etc., the response ability of retinal cells to light stimulation is quantified to judge the loss of visual function. At the same time, an eye tracker is used to track the movement trajectories of eye saccades, fixations, blinks, etc. When the blink frequency is lower than the normal 15-20 times per minute and the saccade speed slows down, it indicates eye muscle fatigue, etc. In eye fatigue monitoring in ophthalmology, image processing of the eyes of the monitoring target is usually involved. However, in the existing eye image processing methods, the monitoring images (such as eyelid movement videos) of the monitoring target and optical coherence tomography images are often fused by a simple feature splicing method. This method has the problem of insufficient depth of multi-modal data association modeling (that is, the co-disturbance patterns of multiple eye regions such as the cornea, ciliary body, and retina during the fatigue process are not modeled), resulting in the inability of the existing eye image processing methods to dynamically depict the functional association evolution of different eye regions when the monitoring target's fatigue is triggered (such as the dynamic association changes between the decrease in corneal tear film stability and the reduction in retinal blood flow during long-term eye use), making it difficult to accurately locate the dominant fatigue partition of the monitoring target. Therefore, how to perform targeted detection of eye fatigue based on multi-modal images has become a difficult problem faced by the industry. Summary of the Invention

[0003] This application provides an eye fatigue monitoring system for ophthalmology, an eye image processing method, a device, and a storage medium, which can perform targeted detection of eye fatigue based on multi-modal images.

[0004] In a first aspect, this application provides an eye image processing method, including the following steps: Obtain a monitoring image and an optical coherence tomography image of the user's eyes, where the optical coherence tomography image includes multiple anatomical partitions of the user's eyes; Extract the tear boundary positions in each image frame based on the gradient features of the region edges in the monitoring image, and perform differential analysis on all the tear boundary positions to obtain the regional disturbance features of the corneal region when the user is using the eyes; Construct a cross-modal feature map network between the monitoring image and the optical coherence tomography image through the feature coupling relationship between the regional perturbation feature and each anatomical partition, and then identify the starting frame of the user's visual fatigue according to the temporal perturbation pattern of all anatomical partitions in the cross-modal feature map network; After identifying the starting frame of the visual fatigue, determine the functional association degree of different anatomical partitions in the cross-modal feature map network, and then determine the dynamic modularity of different anatomical partitions in the user's eyes through all the functional association degrees; Identify the dominant fatigue partition when the user is using eyes from different anatomical partitions according to all the dynamic modularity.

[0005] In some embodiments, extracting the tear boundary position in each image frame based on the gradient feature of the region edge in the monitoring image specifically includes: Extract the gradient feature of the region edge in each image frame of the monitoring image based on the image gradient algorithm; Determine multiple boundary candidate points of each image frame according to all the gradient features; Generate the tear boundary position in each image frame through all the boundary candidate points.

[0006] In some embodiments, performing differential analysis on all the tear boundary positions to obtain the regional perturbation feature of the corneal region when the user is using eyes specifically includes: Generate a boundary change sequence of the user's corneal region through all the tear boundary positions; Perform temporal difference on the tear boundary position in the vector image frame according to the boundary change sequence to obtain the perturbation amplitude sequence of the corneal region; Determine the regional perturbation feature of the corneal region when the user is using eyes according to the perturbation amplitude sequence.

[0007] In some embodiments, constructing the cross-modal feature map network between the monitoring image and the optical coherence tomography image through the feature coupling relationship between the regional perturbation feature and each anatomical partition specifically includes: Obtain the image feature parameters of multiple preset anatomical partitions in the optical coherence tomography image; Perform correlation analysis on the regional perturbation feature and the image feature parameters of each anatomical partition to obtain the feature coupling relationship of the regional perturbation feature to each anatomical partition; Establish an image feature node mapping between the monitoring image and the optical coherence tomography image based on the feature coupling relationship to obtain the cross-modal feature map network between the monitoring image and the optical coherence tomography image.

[0008] In some embodiments, identifying the starting frame of the user's visual fatigue according to the temporal perturbation patterns of all anatomical partitions in the cross-modal feature map network specifically includes: Obtaining the evolution sequence of the regional perturbation features corresponding to all anatomical partitions in the cross-modal feature map network; Performing perturbation analysis on the evolution sequence to obtain the perturbation trends of each anatomical partition in each image frame; Identifying the starting frame of the user's visual fatigue through all the perturbation trends.

[0009] In some embodiments, determining the functional correlation degree of different anatomical partitions in the cross-modal feature map network specifically includes: Generating a functional description vector through the image features and regional perturbation features of each anatomical partition; Determining the functional correlation degree of different anatomical partitions in the cross-modal feature map network according to the functional description vectors between each anatomical partition.

[0010] In some embodiments, determining the dynamic modularity of different anatomical partitions in the user's eyes through all the functional correlation degrees specifically includes: Constructing a dynamic association field between different anatomical partitions based on all the dynamic modularities; Determining the eye fatigue degree of the user in each anatomical partition according to the dynamic association field; Identifying the dominant fatigue partition when the user is using the eyes in different anatomical partitions through all the eye fatigue degrees.

[0011] In a second aspect, the present application provides an eye fatigue monitoring system for ophthalmology, including an eye image processing unit, and the eye image processing unit includes: An acquisition module, configured to acquire a monitoring image and an optical coherence tomography image of the user's eyes, wherein the optical coherence tomography image includes a plurality of anatomical partitions of the user's eyes; A processing module, configured to extract the tear boundary position in each image frame based on the gradient feature of the region edge in the monitoring image, perform differential analysis on all the tear boundary positions to obtain the regional perturbation features of the corneal region when the user is using the eyes; The processing module is further configured to construct a cross-modal feature map network between the monitoring image and the optical coherence tomography image through the feature coupling relationship between the regional perturbation features and each anatomical partition, and then identify the starting frame of the user's visual fatigue according to the temporal perturbation patterns of all anatomical partitions in the cross-modal feature map network; The processing module is further configured to, after identifying the starting frame of the visual fatigue, determine the functional correlation degree of different anatomical partitions in the cross-modal feature map network, and then determine the dynamic modularity of different anatomical partitions in the user's eyes through all the functional correlation degrees; An execution module, configured to identify a dominant fatigue area when the user is using eyes from different anatomical partitions according to all dynamic modularities.

[0012] In a third aspect, the present application provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned eye image processing method are implemented.

[0013] In a fourth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned eye image processing method are implemented.

[0014] The technical solutions provided by the disclosed embodiments of the present application have the following beneficial effects: In the eye fatigue monitoring system, eye image processing method, device and storage medium provided by the present application, by obtaining a monitoring image and an optical coherence tomography image of the user's eyes, wherein the optical coherence tomography image includes a plurality of anatomical partitions of the user's eyes; extracting the tear boundary position in each image frame based on the gradient feature of the region edge in the monitoring image, performing differential analysis on all tear boundary positions to obtain the region perturbation feature of the corneal region when the user is using eyes; constructing a cross-modal feature map network between the monitoring image and the optical coherence tomography image through the feature coupling relationship between the region perturbation feature and each anatomical partition, and then identifying the starting frame of the user's visual fatigue according to the temporal perturbation pattern of all anatomical partitions in the cross-modal feature map network; after identifying the starting frame of the visual fatigue, determining the functional association degree of different anatomical partitions in the cross-modal feature map network, and then determining the dynamic modularity of different anatomical partitions in the user's eyes through all functional association degrees; identifying the dominant fatigue area when the user is using eyes from different anatomical partitions according to all dynamic modularities.

[0015] It can be seen that in this application, after obtaining the monitoring image and optical coherence tomography (OCT) image of the user's eyes, first, the tear boundary positions in each image frame of the monitoring image are extracted for differential analysis to obtain the regional perturbation characteristics of the corneal area when the user is using their eyes, that is: the dynamic characteristics based on a single modality (monitoring image) are transformed into a numerical index reflecting corneal stability. This index can not only independently characterize the fatigue-related changes in the corneal area but also serve as a bridge for cross-modal fusion - forming a "dynamic" - "static" association with the structural characteristics such as corneal thickness and curvature in the OCT image (for example, when the regional perturbation characteristics decline, if the OCT image shows an increase in microfolds in the corneal epithelium layer, it can strengthen the fatigue judgment). This enables the detection of eye fatigue to break through the limitation of a single index, provides underlying feature support for identifying fatigue-triggering frames in the subsequent cross-modal network, and then locks the key time points of fatigue occurrence through time series analysis, delimiting a time window for targeted positioning of the fatigue area; Subsequently, at the moment of the starting frame of visual fatigue, first, the functional correlation degrees of each anatomical region such as the cornea, ciliary body, and retina in the cross-modal network are calculated to quantify the cooperative or inhibitory relationships of different regions in the fatigue mechanism - for example, an increase in the correlation intensity of the ciliary body may indicate that accommodative fatigue is dominant, and abnormal correlation intensity of the retina may suggest neural conduction fatigue. Based on this, the dynamic modularity is determined to reflect the dynamic association characteristics of the functional modules of the user's eye anatomical regions in the fatigue state, thereby accurately identifying the "hub node" (dominant fatigue region) with the highest contribution to fatigue. For example, if the edge weight of the connection between the ciliary body and the lens is significantly higher than that of other regions, and the correlation intensity between the two shows a synergistic upward trend, it can be directly determined that this path is the dominant mechanism of the current fatigue, realizing the traceability analysis from multi-modal data to the specific fatigue region; In summary, this solution can perform targeted detection of eye fatigue based on multi-modal images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic flowchart of an eye image processing method according to some embodiments of the present application; Figure 2 is a schematic flowchart of determining regional perturbation characteristics according to some embodiments of the present application; Figure 3 is a schematic flowchart of identifying the dominant fatigue region according to some embodiments of the present application; Figure 4 is a schematic structural diagram of an eye image processing unit according to some embodiments of the present application; Figure 5 is an internal structural diagram of a computer device for implementing the eye image processing method according to some embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] To better understand the technical solution in this embodiment, the technical solution in this embodiment will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0018] Referring to Figure 1 , this figure is a schematic flowchart of an eye image processing method shown according to some embodiments of the present application. The eye image processing method 100 mainly includes the following steps: In step 101, a monitoring image and an optical coherence tomography image of the user's eye are acquired. Among them, the optical coherence tomography image includes multiple anatomical regions of the user's eye.

[0019] Specifically, when implemented, an infrared imaging unit (such as a high-definition industrial camera, an infrared camera, or an integrated eye movement detection module) can be deployed in front of the user's eyes to continuously collect video of the user's eye usage process, and a monitoring image including eye regions such as the cornea, iris, and pupil is obtained in real time; at the same time, through an optical coherence tomography imaging module configured on the same detection platform or in a nearby location, the user's eye is scanned periodically for its structure, and a tomographic image in the depth direction is obtained as the optical coherence tomography image in the present application.

[0020] It should be noted that the monitoring image can obtain a visible light image sequence or an infrared image sequence during the user's eye usage process through continuous frame acquisition. This image sequence can reflect external visual characteristics such as the optical boundary change of the cornea region, the edge distribution of tears, and the change of the palpebral fissure; the optical coherence tomography image is a multi-layer structure image of different depths of the user's eye obtained through the principle of low-coherence interference, including multiple anatomical regions with spatial tissue characteristics such as the stromal layer, the anterior chamber cavity, the iris edge, and the tear film distribution area.

[0021] In step 102, based on the gradient feature of the region edge in the monitoring image, the tear boundary position in each image frame is extracted, and differential analysis is performed on all tear boundary positions to obtain the region perturbation feature of the cornea region when the user uses the eye.

[0022] In some embodiments, the extraction of the tear boundary position in each image frame based on the gradient feature of the region edge in the monitoring image can be implemented by the following steps: Based on the image gradient algorithm, the gradient feature of the region edge in each image frame of the monitoring image is extracted; According to all gradient features, multiple boundary candidate points of each image frame are determined; The tear boundary position in each image frame is generated through all boundary candidate points.

[0023] In specific implementation, the gradient features of the region edges in each image frame of the monitoring image can be extracted based on the image gradient algorithm in the following manner: First, each image frame in the monitoring image is converted into a grayscale image by using the grayscale method in the prior art, and preprocessing such as noise suppression and contrast enhancement is performed. Then, the preprocessed grayscale image is subjected to a convolution operation by using an image gradient operator to calculate the gradient components in the horizontal and vertical directions respectively, and thus the gradient magnitude and gradient direction of each pixel are obtained. Finally, the gradient magnitude is normalized, and non-maximum suppression and threshold filtering can be selectively used to optimize the edge response, and the optimized gradient magnitude map and gradient direction map are used as the gradient features of the region edges in each image frame of the monitoring image. In other embodiments, the gradient calculation step can also be implemented by using the gradient calculation module of the Canny edge detection algorithm or an edge detection network based on deep learning, and the present application does not limit this.

[0024] It should be noted that the gradient features of the region edges refer to the feature set composed of the gradient components of each pixel point in the horizontal and vertical directions calculated by using an image gradient operator after preprocessing such as grayscale conversion and noise suppression of the monitoring image. The gradient features can be formally represented as a gradient magnitude map and a gradient direction map, where the gradient magnitude reflects the intensity of the edge, and the gradient direction indicates the normal direction of the edge. After being normalized or threshold-screened, etc., they are used to characterize the tear boundary positions and change trends of target regions such as the cornea and iris in the monitoring image.

[0025] In specific implementation, the multiple boundary candidate points of each image frame can be determined according to all the gradient features in the following manner: First, based on the gradient magnitude and gradient direction data of the gradient features, a non-maximum suppression operation is performed on the gradient magnitude map to retain the local maximum pixel points along the gradient direction, and a single-pixel-width edge response map is generated to eliminate the influence of pseudo-gradients in non-edge regions. Then, by setting a gradient magnitude threshold (which can be determined by 1.5 times the global mean), the pixel points with a gradient magnitude greater than the threshold are screened out from the edge response map to form an initial boundary candidate point set. Finally, morphological operations are performed on the initial boundary candidate point set, including closing operation to fill small holes and opening operation to remove discrete noise points, and isolated points and small regions that do not conform to the eye structure characteristics are removed through connected component analysis, and the pixel points that meet the continuity and area conditions are retained as the multiple boundary candidate points of each image frame. Among them, as a preferred embodiment, the screening process of the boundary candidate points can be combined with the prior knowledge of the eye structure (such as the circular feature of the cornea region and the concentric structure of the iris and pupil), and the distribution of the candidate points can be further optimized through ellipse fitting or Hough transform to improve the accuracy of boundary positioning. In other embodiments, a key point detection network based on deep learning can also be used to directly identify the boundary candidate points from the gradient features, and the present application does not limit this. In specific implementation, generating the tear boundary position in each image frame from all boundary candidate points can be achieved in the following manner: First, perform coordinate clustering on all boundary candidate points. Use the density clustering algorithm or the K-means clustering algorithm to divide candidate points with similar spatial positions into different sets, and remove discrete and isolated noise point sets. Then, for each clustered candidate point set, use the least squares method to fit a curve. For the candidate point set of the corneal region boundary, it can be fitted into an elliptical curve. For regions such as the iris, it can be fitted into a circular curve or a polygonal curve to obtain a preliminary boundary curve model. Finally, based on the gray distribution information of the image and the gradient direction consistency constraint, iteratively optimize the preliminary boundary curve model, calculate the gray difference degree between the inner and outer regions of the curve, and screen through the angle threshold between the gradient direction and the normal vector of the curve to correct the position and shape of the boundary curve, thereby obtaining the tear boundary position in each image frame. Among them, as a preferred embodiment, an active contour model or a level set method can be introduced. Using the boundary candidate points as the initial contour, iteratively adjust the contour position by minimizing the energy function to achieve accurate extraction of the boundaries of complex eye structures. In other embodiments, a semantic segmentation network based on deep learning can also be used to post-process the boundary candidate points and directly output the accurate tear boundary position. This application does not limit this.

[0026] In some embodiments, referring to Figure 2 As shown, this figure is a schematic flowchart of determining the regional perturbation characteristics in some embodiments of this application. Performing differential analysis on all tear boundary positions to obtain the regional perturbation characteristics of the corneal region when the user is using the eyes can be achieved through the following steps: First, in 1021, generate a boundary change sequence of the user's corneal region from all tear boundary positions; Then, in 1022, perform temporal difference on the tear boundary positions in the vector image frame according to the boundary change sequence to obtain a perturbation amplitude sequence of the corneal region; Finally, in 1023, determine the regional perturbation characteristics of the corneal region when the user is using the eyes according to the perturbation amplitude sequence.

[0027] In specific implementation, generating a boundary change sequence of the user's corneal region based on all tear boundary positions can be achieved in the following manner: First, extract the key features of the tear boundary positions in each image frame, such as the coordinate set of boundary points and the parameters of the boundary curve (such as the major axis, minor axis, and center coordinates of an ellipse), and encode these features into a digital vector form that can be processed by a computer. For example, normalize the boundary point coordinates to the interval [0, 1], and standardize the ellipse parameters; Subsequently, arrange the encoded corneal boundary feature vectors in the chronological order of the image frames to form a boundary change sequence representing the boundary changes of the user's corneal region at different moments. Among them, as a preferred embodiment, the eye physiological features can be combined during feature extraction to more accurately select and process the boundary features and improve the quality of the boundary change sequence; In other implementation manners, the tear boundary position data can also be analyzed and processed in the frequency domain by means such as discrete Fourier transform to assist in generating the boundary change sequence, and the present application does not limit this.

[0028] In specific implementation, performing temporal difference on the tear boundary positions in the vector image frames according to the boundary change sequence to obtain a perturbation amplitude sequence of the corneal region can be achieved in the following manner: First, perform a difference operation on the tear boundary position vectors of adjacent image frames in the boundary change sequence in the time domain, and generate an initial difference vector sequence by calculating the Euclidean distance or Mahalanobis distance of the corresponding boundary points; Then, perform amplitude calculation on the initial difference vector sequence, and convert it into a scalar value through vector modulus operation to obtain a perturbation amplitude sequence of the corneal region; Among them, as a preferred embodiment, the Kalman filter algorithm can be introduced to predict the tear boundary positions in combination with the prior model of corneal movement, optimize the difference calculation process, and improve the calculation accuracy; In other implementation manners, frequency domain analysis means such as short-time Fourier transform can also be used to extract the frequency features of boundary changes and construct a multi-scale perturbation amplitude sequence, and the present application does not limit this.

[0029] It should be noted that the perturbation amplitude sequence of the corneal region in the present application refers to a numerical sequence formed by arranging in chronological order after performing temporal difference operation on the corneal tear boundary position vectors of adjacent image frames in the boundary change sequence, calculating the distance of the corresponding boundary points, and converting the difference vector into a scalar value by taking the modulus; The perturbation amplitude sequence is represented as a set of ordered scalar values in form, and each value corresponds to the degree of change of the corneal region boundary in one frame of image relative to the previous frame. Its numerical size directly reflects the structural fluctuation of the corneal region in the time dimension and can be used to evaluate the structural stability of the corneal region when the user is using the eyes subsequently.

[0030] In specific implementation, determining the regional perturbation feature of the corneal region when the user is using eyes according to the perturbation amplitude sequence can be achieved by the following method: First, extract statistical features from the perturbation amplitude sequence, calculate the standard deviation, coefficient of variation or root mean square value of the sequence, and obtain statistical parameters reflecting the structural fluctuation degree of the corneal region. Then, perform sliding calculation on the statistical parameters based on a preset time window to generate a local stability index sequence, where the length of the time window can be dynamically adjusted according to the physiological characteristics of the human eye or the usage behavior cycle. Finally, perform weighted fusion on the local stability index sequence, set weight coefficients in combination with the prior knowledge of the corneal structure, and map the fusion result to the interval [0, 1] through a non-linear mapping function to obtain the regional perturbation feature of the corneal region when the user is using eyes. Among them, as a preferred embodiment, a stability evaluation model can be constructed by introducing a support vector machine or a random forest algorithm, and trained with historical data to achieve adaptive learning of the complex mapping relationship between the perturbation amplitude sequence and the regional perturbation feature. In other implementation manners, a fuzzy comprehensive evaluation model can also be constructed through the fuzzy mathematics theory to comprehensively evaluate multi-dimensional stability indicators, and the present application does not limit this. It should be noted that the regional perturbation feature is a quantitative index used to reflect the structural fluctuation degree of the corneal region, and the fatigue state or functional abnormality of the corneal region can be evaluated through the regional perturbation feature.

[0031] In step 103, a cross-modal feature map network between the monitoring image and the optical coherence tomography image is constructed through the feature coupling relationship between the regional perturbation feature and each anatomical partition, and then the starting frame of the user's visual fatigue is recognized according to the temporal perturbation pattern of all anatomical partitions in the cross-modal feature map network.

[0032] It should be noted that the regional perturbation feature is a dynamic index calculated based on the boundary change feature of the corneal region in the monitoring image, and is used to reflect the structural stability of the cornea in consecutive image frames. In the image space, there is a position overlap and organizational structure correspondence relationship between the corneal region in the monitoring image and multiple anatomical partitions in the optical coherence tomography image. Therefore, the regional perturbation feature can be associated with the anatomical partitions containing corneal information in the optical coherence tomography image. That is: Through image registration and mapping of the tear fluid boundary position, spatial alignment of the regional perturbation feature between the two types of images can be achieved, and then the dynamic boundary information of the monitoring image can be mapped to the static anatomical structure in the optical coherence tomography image, providing a basis for constructing a cross-modal feature map network.

[0033] In some embodiments, constructing the cross-modal feature map network between the monitoring image and the optical coherence tomography image through the feature coupling relationship between the regional perturbation feature and each anatomical partition can be achieved by the following steps: Obtain the image feature parameters of a plurality of preset anatomical regions in the optical coherence tomography image; Perform a correlation analysis on the regional perturbation feature and the image feature parameters of each anatomical region to obtain the feature coupling relationship of the regional perturbation feature to each anatomical region; Based on the feature coupling relationship, establish an image feature node mapping between the monitoring image and the optical coherence tomography image to obtain a cross-modal feature map network between the monitoring image and the optical coherence tomography image.

[0034] It should be noted that the image feature parameters of a plurality of preset anatomical regions in the optical coherence tomography image refer to a multi-dimensional feature data set extracted for preset anatomical regions such as the corneal epithelium layer, stromal layer, and endothelial cell layer after preprocessing and region segmentation of the optical coherence tomography image; the image feature parameters include morphological feature parameters (such as regional area, perimeter, roundness index), gray-scale statistical feature parameters (such as mean, variance, kurtosis value), texture feature parameters (such as energy, entropy, contrast parameters based on the gray-level co-occurrence matrix, or local binary pattern feature vectors), and geometric feature parameters (such as structural thickness, curvature parameters). The image feature parameters are used to quantitatively describe the morphology, gray-scale distribution, texture pattern, and geometric morphology features of each anatomical region, providing a data basis for subsequent analysis of the coupling relationship between the regional perturbation feature and each region; as a preferred embodiment, a deep learning semantic segmentation network such as U-Net or DeepLabv3+ can be introduced and trained using a labeled medical image data set to achieve automatic and accurate segmentation and feature extraction of each anatomical region, thereby obtaining the image feature parameters of each anatomical region; in other embodiments, the feature parameters of specific anatomical regions can also be obtained through a manual interaction method combined with computer-aided measurement tools, and the present application does not limit this.

[0035] In specific implementation, the correlation analysis is performed between the regional perturbation features and the image feature parameters of each anatomical region to obtain the feature coupling relationship of the regional perturbation features to each anatomical region. The following method can be used to achieve this, that is: First, for the morphological, grayscale, texture, geometric and other features in the regional perturbation features and the image feature parameters of each anatomical region, the Pearson correlation coefficient is used to calculate the degree of linear correlation respectively, and for the non-linear relationship features, the Spearman rank correlation coefficient or Kendall's tau correlation coefficient is used to measure, generating an initial correlation coefficient matrix; Then, the weight calculation is performed on the initial correlation coefficient matrix, and the coefficients are mapped to the [0, 1] interval through the Softmax function to obtain the feature coupling relationship of the regional perturbation features to each anatomical region; As a preferred embodiment, the random forest regression algorithm can also be introduced, taking the regional perturbation features as the dependent variable and the image feature parameters as the independent variable, and calculating the feature importance score through the node splitting situation during model training as the feature coupling relationship of the regional perturbation features to each anatomical region; In other embodiments, other methods can also be used to determine, which is not limited here.

[0036] It should be noted that the initial correlation coefficient matrix refers to the matrix formed by arranging the obtained correlation coefficients in a specific order after performing correlation calculations on the morphological, grayscale, texture, geometric and other features in the regional perturbation features and the image feature parameters of each anatomical region by using the Pearson correlation coefficient, Spearman rank correlation coefficient or Kendall's tau correlation coefficient respectively. The rows of the initial correlation coefficient matrix correspond to different anatomical regions, and the columns correspond to different types of image feature parameters. The numerical size of each element in the matrix reflects the degree of association between the feature parameters of the corresponding anatomical region and the regional perturbation features. The closer the absolute value of the numerical value is to 1, the stronger the correlation, and the closer it is to 0, the weaker the correlation; In addition, the feature coupling relationship refers to the weight association relationship between the regional perturbation features and each anatomical region determined after performing weight calculation on the initial correlation coefficient matrix and mapping the coefficients to the [0, 1] interval through the Softmax function; This weight relationship quantifies the influence degree of the image feature parameters of each anatomical region on the regional perturbation features in numerical form. The larger the numerical value, the closer the corresponding anatomical region is associated with corneal stability.

[0037] In specific implementation, establishing an image feature node mapping between the monitoring image and the optical coherence tomography (OCT) image based on the feature coupling relationship to obtain the cross-modal feature map network between the monitoring image and the OCT image can be achieved in the following manner, that is: First, the corneal boundary feature points in the monitoring image and the image feature parameters of each anatomical region in the OCT image are respectively used as nodes in the graph structure of the prior art. Among them, the nodes of the monitoring image cover features such as the position and curvature of the tear film boundary, and the nodes of the OCT image include image feature parameters such as morphology, grayscale, and texture. Subsequently, the coupling weight values of the regional perturbation features to each anatomical region are mapped as the weights of the edges between the nodes to obtain an initial weighted undirected graph structure. Then, the feature nodes of the monitoring image and the OCT image are projected into the same feature space through the linear transformation method in the prior art. Secondly, the structural similarity and feature consistency between the nodes are determined, and then a reconstruction error function based on the similarity metric is defined. Furthermore, optimization methods such as gradient descent are used to adjust the mapping function parameters by minimizing the reconstruction error. When the change in the reconstruction error of consecutive iterations is less than the set threshold, the optimization is stopped. Finally, the optimized weighted undirected graph is used as the cross-modal feature map network between the monitoring image and the OCT image. Among them, as a preferred embodiment, a graph neural network model can be introduced. Taking the weighted undirected graph as the input, the high-order correlation features between the nodes are learned through graph convolution operations, and the node mapping relationship is automatically optimized to obtain the cross-modal feature map network. In other embodiments, the canonical correlation analysis method can also be used to find the linear projection direction between the two image modalities, maximize the correlation of the features after projection, and realize the feature node mapping. This application does not make any limitations in this regard.

[0038] In some embodiments, identifying the starting frame of the user's visual fatigue based on the temporal perturbation patterns of all anatomical regions in the cross-modal feature map network can be achieved through the following steps: Obtain the evolution sequence of the regional perturbation features corresponding to all anatomical regions in the cross-modal feature map network; Perform perturbation analysis on the evolution sequence to obtain the perturbation trends of each anatomical region in each image frame; Identify the starting frame of the user's visual fatigue through all the perturbation trends.

[0039] In specific implementation, the evolution sequence of the regional perturbation features corresponding to all anatomical partitions in the cross-modal feature map network can be obtained in the following manner: First, traverse the cross-modal feature map network to determine the nodes of all anatomical partitions. Then, for each node of an anatomical partition, obtain the corresponding regional perturbation feature. Subsequently, collect the regional perturbation features of each anatomical partition in each image frame in chronological order to form time series data corresponding to the regional perturbation features of each anatomical partition. Finally, integrate the time series data of all anatomical partitions to construct the evolution sequence of the regional perturbation features corresponding to all anatomical partitions. As a preferred embodiment, integrating the time series data of all anatomical partitions specifically includes: correlating the regional perturbation features of different anatomical partitions at the same time point. For example, collect the regional perturbation features of all anatomical partitions in this frame together to form a vector containing multiple anatomical partition indices as the evolution sequence of the regional perturbation features corresponding to the anatomical partition. In other embodiments, other methods can also be used for implementation, which is not limited here. It should be noted that the evolution sequence of the regional perturbation features is a sequence describing the variation law of the regional perturbation features of all anatomical partitions over time.

[0040] In specific implementation, the perturbation trend of each anatomical partition in each image frame can be obtained by performing perturbation analysis on the evolution sequence in the following manner: First, calculate the difference in regional perturbation features between adjacent image frames of each anatomical partition in the evolution sequence. Then, set a perturbation threshold and numerically compare each regional perturbation feature difference with the perturbation threshold. When the regional perturbation feature difference is greater than the perturbation threshold, it is determined that the perturbation trend of the corresponding anatomical partition in the corresponding image frame is upward; when the regional perturbation feature difference is less than the perturbation threshold, it is determined that the perturbation trend of the corresponding anatomical partition in the corresponding image frame is downward; when the regional perturbation feature difference is equal to the perturbation threshold, it is determined that the anatomical partition is in a stable state in the corresponding image frame, thereby obtaining the perturbation trend of each anatomical partition in each image frame.

[0041] It should be noted that the perturbation trend refers to the tendency of the state change of each anatomical partition in each image frame determined by analyzing the evolution sequence of the regional perturbation features. In addition, the perturbation threshold can be preset according to the differences in application scenarios. For example, in a scenario with high requirements for visual fatigue detection accuracy, where more subtle changes are expected to be captured, a smaller perturbation threshold is set; in a scenario with relatively lower accuracy requirements and a greater focus on overall trend judgment, a larger perturbation threshold is set.

[0042] In specific implementation, the starting frame of visual fatigue of the user can be identified by all perturbation trends through the following method, that is: First, construct a mapping model between the perturbation trends of each anatomical region and the visual fatigue state. Based on prior knowledge in the medical field or by training a machine learning model (such as support vector machine, random forest, convolutional neural network) with a large amount of image data containing visual fatigue annotations, determine the corresponding relationship between different combinations of perturbation trends and visual fatigue levels (such as no fatigue, mild fatigue, moderate fatigue, severe fatigue); then set the trigger judgment rules, including assigning different weights to anatomical regions (such as higher weights to key regions such as corneal epithelium and retina), defining the trigger thresholds for each fatigue level (such as mild fatigue requires that the perturbation trends of at least 50% of the high-weight regions meet the preset conditions and last for more than 3 frames), and setting a multi-region collaborative judgment mechanism; finally, traverse all the perturbation trends of each image frame, input the perturbation trends into the trained model to obtain the prediction result. When the prediction result meets the threshold conditions and the requirement of the continuous frame number in the trigger judgment rules, the current image frame is determined as the starting frame of visual fatigue corresponding to the fatigue level.

[0043] Preferably, to construct a mapping model between the perturbation trends of each anatomical region and the visual fatigue state, a large amount of image data related to visual fatigue can be collected, and the visual fatigue state of the data can be annotated through a visual fatigue evaluation method (such as classified into categories such as no fatigue, mild fatigue, moderate fatigue, severe fatigue, etc.). Subsequently, the perturbation trends of each anatomical region in each image frame are used as feature vectors, and together with the corresponding visual fatigue state annotations, a training data set is formed. Using this training data set, a machine learning algorithm is used to construct a visual fatigue feature association model; for example, when using the support vector machine algorithm, by selecting an appropriate kernel function (such as the radial basis kernel function), the optimal hyperplane is found in the feature space to achieve the classification mapping between the perturbation trend feature vectors and the visual fatigue state; or using a convolutional neural network in deep learning, high-order features in the perturbation trends are automatically extracted through multiple convolutional and pooling operations, and then the prediction result of the visual fatigue state is output through the fully connected layer, so as to construct a mapping model between the perturbation trends of each anatomical region and the visual fatigue state. In other embodiments, other methods can also be used to determine, which is not limited here.

[0044] In step 104, after identifying the starting frame of visual fatigue, determine the functional correlation degree of different anatomical regions in the cross-modal feature map network, and then determine the dynamic modularity of different anatomical regions in the user's eye through all the functional correlation degrees.

[0045] In some embodiments, the following steps can be used to determine the functional correlation degree of different anatomical regions in the cross-modal feature map network: Generate a functional description vector through the image features and regional perturbation features of each anatomical region; Determine the functional correlation degree of different anatomical regions in the cross-modal feature map network according to the functional description vectors between the anatomical regions.

[0046] When specifically implemented, generating the functional description vector through the image features and regional perturbation features of each anatomical region can be achieved by the following method, that is: First, extract multi-dimensional image features such as the morphology, grayscale, texture, and spatial position of each anatomical region, and obtain the regional perturbation features of the corresponding anatomical region. Subsequently, perform weighted summation by using the weighted fusion method in the prior art to obtain the functional description vector of the anatomical region; it should be noted that the weighted fusion method is to perform weighted operations on the image feature vector and the regional perturbation feature by setting weight coefficients to generate a vector, and the weight coefficients can be optimized through cross-validation; as a preferred embodiment, the attention mechanism can be used to weight the image features, highlight the feature dimensions sensitive to visual fatigue, and then fuse with the regional perturbation features; in other implementation manners, the kernel method can also be used to map the image features and regional perturbation features to a high-dimensional space and then fuse them, and the present application does not limit this.

[0047] When specifically implemented, determine the functional correlation degree of different anatomical regions in the cross-modal feature map network according to the functional description vectors between the anatomical regions, that is: regard the similarity degree of the functional description vectors between the anatomical regions as the functional correlation degree of different anatomical regions in the cross-modal feature map network; for example: adopt measurement methods such as cosine similarity and Euclidean distance in the prior art to calculate the similarity value between the functional description vectors of any two anatomical regions, and use the similarity value as the functional correlation degree of the corresponding two anatomical regions in the cross-modal feature map network.

[0048] In some embodiments, determining the dynamic modularity of different anatomical regions in the user's eye through all functional correlation degrees can be achieved by the following steps: Evaluate the fatigue evolution trend of each anatomical region based on all functional correlation degrees in combination with the coupling relationship between the anatomical regions; Determine the dynamic modularity of different anatomical regions in the user's eye according to the fatigue evolution trend of all anatomical regions.

[0049] It should be noted that the coupling relationship refers to the interdependent and cooperative association characteristics presented by each anatomical region at the physiological function level; for example, in the visual system, the photoelectric conversion function of the retina depends on the stable blood supply of the choroid to maintain metabolism, while the optical performance of the cornea is closely related to the moistening and nutritional effects of the tear film; as a preferred embodiment, the functional coupling strength between every two anatomical regions can be determined by the method of expert scoring (for example, for anatomical regions with a direct material exchange and energy transfer relationship, a higher coupling strength value (such as 0.8 - 1.0) is assigned; for anatomical regions with a weaker functional association, a lower value (such as 0 - 0.2) is assigned), and then the functional coupling strengths between all anatomical regions are presented in matrix form to form a functional coupling relationship matrix. Finally, the functional coupling relationship matrix is used as the functional coupling relationship between anatomical regions; therefore, in specific implementation, based on all functional association degrees and the functional coupling relationship between anatomical regions, the fatigue evolution trend of each anatomical region can be evaluated in the following way, that is: First, all functional association degrees are constructed into a functional association degree matrix. Subsequently, the tensor product operation is performed on the functional association degree matrix and the functional coupling relationship matrix to obtain a comprehensive association matrix. Then, a graph structure is constructed with the comprehensive association matrix, with anatomical regions as nodes and comprehensive association strengths as edge weights, and a graph convolutional network is used to process the graph structure. The functional description vectors of each node are input, and the fatigue feature vectors are output through multiple graph convolutional operations; subsequently, the change rate (fatigue evolution trend value) of the fatigue feature vectors in the time dimension is calculated as the fatigue evolution trend of each anatomical region; preferably, the functional association degree matrix can be obtained by organizing and arranging in a two-dimensional matrix form; wherein, the row index and column index of the functional association degree matrix respectively correspond to different anatomical regions, and the element value of the functional association degree matrix is the functional association degree value between the corresponding two anatomical regions, thus forming a functional association degree matrix that completely represents the functional association relationship between all anatomical regions; in addition, it should be noted that the fatigue evolution trend is a quantitative index of the change trend of the fatigue state of each anatomical region over time.

[0050] In specific implementation, the dynamic modularity of different anatomical regions in the user's eyes can be determined according to the fatigue evolution trends of all anatomical regions by the following method, that is: First, construct a graph structure with anatomical regions as nodes and the comprehensive correlation matrix element values as edge weights, and use the fatigue evolution trend values of each node as node attributes; Subsequently, use the modularity optimization algorithm to perform community partitioning on the graph structure and calculate the initial modularity. During the process, introduce the fatigue evolution trend similarity constraint (for example, nodes with trend value differences less than a preset threshold are preferentially partitioned into the same module); Finally, weight the edge weights within the module by the fatigue evolution trend values (the higher the trend value of a node, the greater the contribution weight to the connection strength within the module), and recalculate and output the dynamic modularity considering the fatigue state. It should be noted that the preset threshold and the weighting coefficient can be calibrated through anatomical function clustering data. As a preferred embodiment, a dynamic graph neural network model can be constructed to update the node features in real time according to the fatigue evolution trend and optimize the modularity calculation; In other implementation manners, a modularity measurement method based on information entropy can also be used to evaluate the rationality of module partitioning in combination with the information gain of the fatigue trend. This application does not limit this.

[0051] It should be noted that the dynamic modularity is a quantitative index used to reflect the dynamic association characteristics of functional modules in the user's eye anatomical regions under fatigue conditions. The dynamic modularity realizes the dynamic optimization of the division of eye functional modules by integrating the functional association degree, coupling relationship, and fatigue evolution trend of anatomical regions, and provides a quantitative basis for analyzing the collaborative action mode of each region of the eye during visual fatigue.

[0052] In step 105, the dominant fatigue region during the user's eye use is identified from different anatomical regions according to all the dynamic modularities.

[0053] In some embodiments, the dominant fatigue region during the user's eye use can be identified from different anatomical regions according to all the dynamic modularities by the following steps: Construct a dynamic association field between different anatomical regions based on all the dynamic modularities; Determine the eye fatigue degree of the user in each anatomical region according to the dynamic association field; Identify the dominant fatigue region during the user's eye use from different anatomical regions through all the eye fatigue degrees.

[0054] In specific implementation, the dynamic association field between different anatomical regions can be constructed based on all dynamic modularities in the following manner: First, a basic graph structure is constructed with anatomical regions as nodes and the element values of the comprehensive association matrix obtained by calculating dynamic modularity as edge weights; subsequently, the fatigue evolution trend values of each node (the time change rate of the fatigue feature vector obtained by processing the functional description vector through a graph convolutional network) are embedded into the graph as node attributes; then, according to the community partition results of the modularity optimization algorithm (such as the Louvain algorithm), module labels are added to the nodes and module-level association features are calculated (such as the sum of the weights of the edges between modules and the average fatigue trend within the module); finally, the association strength of the nodes with high fatigue trends is enhanced through a dynamic weight adjustment mechanism (such as when the trend value increases by 0.1, the weight of the corresponding node's edge is increased by 5%) to form a dynamic association field; As a preferred embodiment, the weight update can be directly performed based on the graph structure calculated by dynamic modularity; in other implementation manners, an association field with state features can also be quickly constructed through module membership, and the present application does not limit this.

[0055] It should be noted that the dynamic association field is a graph structure constructed with anatomical regions as nodes and the element values of the comprehensive association matrix obtained by calculating dynamic modularity as edge weights, in which the fatigue evolution trend values of each node (the time change rate of the fatigue feature vector obtained by processing the functional description vector through a graph convolutional network) are embedded as node attributes, and module labels are added to the nodes according to the community partition results of the modularity optimization algorithm.

[0056] When specifically implemented, determining the eye fatigue degree of the user in each anatomical region according to the dynamic association field can be achieved by the following method, that is: First, extract the attribute information such as the real-time association strength value and the dynamic evolution trend (such as the association enhancement rate, weakening rate) of each node in the dynamic association field, and at the same time obtain the edge weight (current frame association strength) information of the edges connecting the nodes; Subsequently, use the real-time association strength value as the basic measure of eye fatigue degree, and correct the basic measure according to the dynamic evolution trend (for example, the association enhancement rate indicates association strengthening, and the weakening rate indicates association weakening). For the anatomical regions connected to the edges with higher edge weights (current frame association strength) and with high real-time association strength values of the corresponding nodes, further increase their eye fatigue degrees; thus, calculate the eye fatigue degrees of each anatomical region; As a preferred embodiment, correction coefficients can be set for different dynamic evolution trends (such as the association enhancement rate, weakening rate). For example, the coefficient range corresponding to the association enhancement rate is 1.2-1.5, the coefficient range corresponding to the weakening rate is 0.5-0.8, and the coefficient corresponding to no significant dynamic evolution trend is 1.0; Subsequently, use the real-time association strength value as the basic measure and multiply it by the correction coefficient of the corresponding dynamic evolution trend to obtain the eye fatigue degree of the anatomical region; In other embodiments, the analytic hierarchy process can also be used to calculate the eye fatigue degree after assigning weights to the influencing factors such as the real-time association strength value, dynamic evolution trend, and edge weight (current frame association strength) in combination with expert experience. The present application does not limit this.

[0057] When specifically implemented, refer to Figure 3As shown in the figure, this is a schematic flowchart of identifying the dominant fatigue area according to some embodiments of the present application. The dominant fatigue area when the user is using eyes can be identified from all anatomical areas with different eye fatigue degrees by the following method, that is: First, arrange the eye fatigue degrees of each anatomical area in descending order; Subsequently, set a screening threshold for the dominant fatigue area (the screening threshold can be determined according to historical data statistics or experimental test results), and regard the anatomical areas with eye fatigue degrees greater than the screening threshold as candidate dominant fatigue areas; If the number of candidate areas is 1, directly determine this area as the dominant fatigue area when the user is using eyes; If the number of candidate areas is greater than 1, calculate the proportion of the eye fatigue degree of each candidate area in the total eye fatigue degree of all candidate areas, and determine the area with the highest proportion as the dominant fatigue area; If the eye fatigue degrees of all anatomical areas are lower than the screening threshold, it is determined that there is no obvious dominant fatigue area at present; In addition, it should be noted that the screening threshold can be dynamically adjusted according to different user group characteristics or eye use scenarios; As a preferred embodiment, a clustering algorithm can be used to perform clustering analysis on the eye fatigue degree data of all anatomical areas, and the area with the highest eye fatigue degree in the largest cluster after clustering is regarded as the dominant fatigue area; In other embodiments, a classification model based on a decision tree can also be constructed, using anatomical area attributes such as eye fatigue degree, real-time association strength value, and dynamic evolution trend (such as association enhancement rate, weakening rate) as inputs, and outputting the dominant fatigue area. The present application does not limit this.

[0058] In addition, on the other hand of the present application, in some embodiments, the present application provides an eye fatigue monitoring system for ophthalmology. The system further includes an eye image processing unit. Refer to Figure 4 , this figure is a schematic structural diagram of the eye image processing unit according to some embodiments of the present application. The eye image processing unit 200 includes: The acquisition module 201, the processing module 202, and the execution module 203 are described as follows: The acquisition module 201. In the present application, the acquisition module 201 is mainly used to acquire the monitoring image and optical coherence tomography (OCT) image of the user's eyes. Among them, the optical coherence tomography image includes multiple anatomical areas of the user's eyes; The processing module 202. In the present application, the processing module 202 is mainly used to extract the tear boundary position in each image frame based on the gradient feature of the region edge in the monitoring image, perform differential analysis on all tear boundary positions, and obtain the region perturbation feature of the corneal region when the user is using eyes; In addition, in the present application, the processing module 202 is further used to construct a cross-modal feature map network between the monitoring image and the optical coherence tomography image through the feature coupling relationship between the region perturbation feature and each anatomical area, and then identify the starting frame of the user's visual fatigue according to the temporal perturbation pattern of all anatomical areas in the cross-modal feature map network; In addition, the processing module 202 in the present application is further configured to determine the functional association degrees of different anatomical regions in the cross-modal feature map network after identifying the visual fatigue start frame, and further determine the dynamic modularity of different anatomical regions in the user's eyes based on all the functional association degrees; An execution module 203. In the present application, the execution module 203 is mainly configured to identify the dominant fatigue region when the user uses the eyes from different anatomical regions according to all the dynamic modularity.

[0059] In addition, the present application further provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to obtain the code and execute the above-mentioned eye image processing method.

[0060] In some embodiments, refer to Figure 5 , this figure is the internal structure diagram of a computer device for implementing the eye image processing method according to some embodiments of the present application. The eye image processing method in the above embodiments can be implemented by Figure 5 the computer device shown. The computer device 300 includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0061] The processor 301 can be a general-purpose central processing unit (CPU), or can be an application-specific integrated circuit (ASIC), or one or more are used to control the execution of the eye image processing method in the present application.

[0062] The communication bus 302 is used to transmit information between the above components.

[0063] The memory 303 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), or other type of dynamic storage device that can store information and instructions. It may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk, or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 303 may exist independently and be connected to the processor 301 through the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0064] Among them, the memory 303 is used to store the program code for executing the solution of this application, and is controlled by the processor 301 to execute. The processor 301 is used to execute the program code stored in the memory 303. The program code may include one or more software modules. The above-mentioned eye image processing method in the embodiment may be implemented by one or more software modules in the program code in the processor 301 and the memory 303.

[0065] The communication interface 304 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0066] In a specific implementation, as an embodiment, the computer device may include multiple processors, and each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, the processor may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0067] The above computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of the present application do not limit the type of the computer device.

[0068] In addition, the present application also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above eye image processing method is implemented.

[0069] In summary, in the eye fatigue monitoring system, eye image processing method, device, and storage medium disclosed in the embodiments of the present application, by obtaining a monitoring image and an optical coherence tomography (OCT) image of a user's eye, where the OCT image includes multiple anatomical partitions of the user's eye; extracting the position of the tear boundary in each image frame based on the gradient feature of the region edge in the monitoring image, and performing differential analysis on all tear boundary positions to obtain the regional perturbation feature of the corneal region when the user is using the eyes; constructing a cross-modal feature map network between the monitoring image and the OCT image through the feature coupling relationship between the regional perturbation feature and each anatomical partition, and then identifying the starting frame of the user's visual fatigue based on the temporal perturbation pattern of all anatomical partitions in the cross-modal feature map network; after identifying the starting frame of the visual fatigue, determining the functional association degree of different anatomical partitions in the cross-modal feature map network, and then determining the dynamic modularity of different anatomical partitions in the user's eye through all the functional association degrees; identifying the dominant fatigue partition when the user is using the eyes from different anatomical partitions according to all the dynamic modularity; and target detection of eye fatigue can be performed based on multi-modal images.

[0070] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present application.

[0071] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. An eye image processing method, characterized in that, It includes the following steps: Obtain a monitoring image and an optical coherence tomography (OCT) image of the user's eyes. Among them, the OCT image includes multiple anatomical regions of the user's eye; Extract the tear boundary position in each image frame based on the gradient feature of the region edge in the monitoring image, perform differential analysis on all tear boundary positions, and obtain the regional perturbation feature of the corneal region when the user is using their eyes; Construct a cross-modal feature map network between the monitoring image and the OCT image through the feature coupling relationship between the regional perturbation feature and each anatomical region. Furthermore, identify the starting frame of the user's visual fatigue based on the temporal perturbation pattern of all anatomical regions in the cross-modal feature map network; After identifying the starting frame of visual fatigue, determine the functional association degree of different anatomical regions in the cross-modal feature map network. Furthermore, determine the dynamic modularity of different anatomical regions in the user's eye through all functional association degrees; Identify the dominant fatigue region when the user is using their eyes from different anatomical regions according to all dynamic modularities; 2. The method according to claim 1, characterized in that Specifically, extracting the tear boundary position in each image frame based on the gradient feature of the region edge in the monitoring image includes: Extract the gradient feature of the region edge in each image frame of the monitoring image based on the image gradient algorithm; Determine multiple boundary candidate points for each image frame according to all gradient features; Generate the tear boundary position in each image frame through all boundary candidate points; 3. The method according to claim 1, characterized in that Specifically, performing differential analysis on all tear boundary positions to obtain the regional perturbation feature of the corneal region when the user is using their eyes includes: Generate a boundary change sequence of the user's corneal region through all tear boundary positions; Perform temporal difference on the tear boundary positions in the vector image frame according to the boundary change sequence to obtain a perturbation amplitude sequence of the corneal region; Determine the regional perturbation feature of the corneal region when the user is using their eyes according to the perturbation amplitude sequence; 4. The method according to claim 1, wherein Specifically, constructing a cross-modal feature map network between the monitoring image and the OCT image through the feature coupling relationship between the regional perturbation feature and each anatomical region includes: Obtain the image feature parameters of multiple preset anatomical regions in the OCT image; Perform correlation analysis on the regional perturbation feature and the image feature parameters of each anatomical region to obtain the feature coupling relationship of the regional perturbation feature to each anatomical region; Establish an image feature node mapping between the monitoring image and the OCT image based on the feature coupling relationship to obtain a cross-modal feature map network between the monitoring image and the OCT image; 5. The method according to claim 1, characterized in that Specifically, identifying the starting frame of the user's visual fatigue based on the temporal perturbation pattern of all anatomical regions in the cross-modal feature map network includes: Obtain the evolution sequence of the regional perturbation features corresponding to all anatomical regions in the cross-modal feature map network; Perform perturbation analysis on the evolution sequence to obtain the perturbation trend of each anatomical region in each image frame; Identify the starting frame of the user's visual fatigue through all perturbation trends; 6. The method according to claim 1, wherein Specifically, determining the functional association degree of different anatomical regions in the cross-modal feature map network includes: Generate a functional description vector based on the image features and regional perturbation features of each anatomical region; Determine the functional association degree of different anatomical regions in the cross-modal feature map network according to the functional description vectors between the anatomical regions.

7. The method according to claim 1, wherein Determining the dynamic modularity of different anatomical regions in the user's eye through all functional association degrees specifically includes: Construct a dynamic association field between different anatomical regions based on all dynamic modularities; Determine the eye fatigue degree of the user in each anatomical region according to the dynamic association field; Identify the dominant fatigue region when the user is using the eyes in different anatomical regions through all eye fatigue degrees.

8. An eye fatigue monitoring system for ophthalmology, comprising an eye image processing unit, characterized in that, The eye image processing unit includes: An acquisition module, configured to acquire a monitoring image and an optical coherence tomography image of the user's eye, wherein the optical coherence tomography image includes a plurality of anatomical regions of the user's eye; A processing module, configured to extract the tear boundary position in each image frame based on the gradient feature of the region edge in the monitoring image, perform differential analysis on all tear boundary positions, and obtain the regional perturbation feature of the corneal region when the user is using the eyes; The processing module is further configured to construct a cross-modal feature map network between the monitoring image and the optical coherence tomography image through the feature coupling relationship between the regional perturbation feature and each anatomical region, and then identify the starting frame of the user's visual fatigue according to the temporal perturbation pattern of all anatomical regions in the cross-modal feature map network; The processing module is further configured to, after identifying the starting frame of the visual fatigue, determine the functional association degree of different anatomical regions in the cross-modal feature map network, and then determine the dynamic modularity of different anatomical regions in the user's eye through all functional association degrees; An execution module, configured to identify the dominant fatigue region when the user is using the eyes from different anatomical regions according to all dynamic modularities.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the eye image processing method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the steps of the eye image processing method according to any one of claims 1 to 7 are implemented.

Images