Retina detachment grading method and system based on multi-mode eye fundus image

Abstract

The invention relates to the technical field of image analysis, in particular to a retina detachment grading method and system based on a multi-mode fundus image, and the method comprises the following steps: obtaining a fundus image gray boundary and extracting a mutation point, calculating the gray difference of adjacent pixels and comparing a trend baseline to screen the mutation point, calculating a path direction and screening a consistent path segment, extracting an unstable path through an ST-CRF model to generate a separation preliminary path set, extending the unstable path to screen a candidate area, calculating a direction change rate and mutation times to extract a separation candidate path segment, labeling a grade label through a support vector machine model, and fusing an expansion trend and a grading result to construct the multi-modal visual network. According to the method, a real path is tracked and a false separation path is eliminated by extracting a sudden change point in combination with gray difference and trend judgment, an unstable path section is analyzed and candidate area detection is enhanced, grade judgment is carried out through an SVM, an intervention map is generated, and visual guidance and intervention are carried out.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and in particular to a method and system for grading retinal detachment based on multimodal fundus images. Background Technology

[0002] Image analysis technology encompasses various methods for processing and understanding image data, aiming to extract useful feature information from images and perform operations such as classification, recognition, detection, and segmentation. The core content of this technology includes image acquisition, image preprocessing, feature extraction, image classification and discrimination, and image semantic understanding. Medical image analysis, as an important branch, is mainly used to assist in diagnosis and pathological analysis. By analyzing data acquired by medical imaging equipment, it extracts the structural and lesion features of tissues and organs, improving the efficiency and accuracy of clinical diagnosis and treatment. In ophthalmology, image analysis technology is mainly applied to the quality assessment of fundus images, the regional localization and structural identification of retinal lesions, and plays a crucial role, especially in the screening and classification of diseases such as diabetic retinopathy, glaucoma, macular degeneration, and retinal detachment.

[0003] The retinal detachment grading method and system based on multimodal fundus images refers to the use of fundus image data from different imaging modalities, including color fundus photography and optical coherence tomography (OCT) images, to automatically determine the severity of retinal detachment by constructing and training a deep convolutional neural network model for image grading tasks. This method employs image registration and fusion techniques to achieve spatial alignment of images from different modalities, combines multi-scale image feature extraction to describe retinal structural integrity, and uses supervised learning for image label training to establish mapping relationships for grading recognition. Furthermore, this method constructs a sample database based on labeled data and employs loss function construction and optimization strategies to improve the model's discriminative ability in multi-level classification tasks.

[0004] Current technologies for analyzing multimodal fundus images often rely on single feature extraction and fixed threshold discrimination, easily overlooking subtle changes in pixel grayscale over time or spatial location. This makes it difficult to identify some minor structural mutations in a timely manner, reducing sensitivity to early signs of retinal detachment. In the complex path identification of retinal lesions, existing methods mostly employ single-scale or single-discrimination strategies, lacking dynamic analysis of path trends, which easily leads to interference from false detachment paths, resulting in missed or false detections of the final lesion area. For the severity of different detachment areas, traditional grading often relies on manual experience or single feature value classification, failing to objectively and continuously determine the structural level status across regions, thus limiting the accuracy and repeatability of grading results. In actual clinical assistance, the results generated by existing technologies are mostly static grading labels, failing to combine with the dynamic trajectory of retinal structural changes. This results in clinicians having insufficient grasp of the lesion extent and development trend when formulating intervention plans, easily leading to one-sided diagnostic and treatment decisions. In cases where the boundaries between a localized, mildly detached area and the surrounding tissue are unclear, traditional analysis fails to provide traceable coordinate trajectories, resulting in a lack of structured support for diagnosis and delaying timely intervention in the progression of the lesion. Summary of the Invention

[0005] To address the technical problems existing in the prior art, embodiments of the present invention provide a method and system for retinal detachment classification using multimodal fundus images. The technical solution is as follows:

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for grading retinal detachment using multimodal fundus images, comprising the following steps:

[0007] S1: Obtain the boundary gray lines in the multimodal fundus image and calculate the gray difference between adjacent pixels. Compare the difference with the time trend baseline and extract the coordinates of abrupt change points that exceed the jump threshold to generate structural abrupt change coordinate points.

[0008] S2: Calculate the coordinate point direction based on the structural mutation coordinate point and select path segments with consistent direction, and calculate the directional trend through the ST-CRF model and select unstable path segments as detachment paths to generate a preliminary set of retinal detachment paths.

[0009] S3: Based on the preliminary set of retinal detachment paths, extract the path extension segments of unstable path segments, calculate the direction change rate and the number of direction abrupt changes of the path extension segments, and screen detachment candidate path segments to generate potential detachment structure candidate regions.

[0010] S4: Use a support vector machine model to determine the level transition status of the detachment candidate path in the potential detachment structure candidate region and label the level to generate the retinal detachment level classification result.

[0011] S5: Obtain the trajectory of node coordinate changes in the candidate region of the potential detachment structure, calculate the direction of expansion trend, and construct a guiding direction line by combining the path segment corresponding to the retinal detachment level classification result and fuse the level label to generate a multimodal retinal detachment classification intervention map.

[0012] As a further aspect of the present invention, the structural mutation coordinate points include the gray-level difference between adjacent pixels, the time trend baseline, and the coordinates of the mutation point; the preliminary set of retinal detachment paths includes unstable path segments, detachment paths, path segment directions, and consistent path segments; the potential detachment structure candidate regions include path extension segments, direction change rates, number of direction mutations, and detachment candidate path segments; the retinal detachment grade classification results include detachment candidate paths, grade jump states, and grade labels; and the multimodal retinal detachment grade intervention atlas includes node coordinate change trajectories, expansion trend directions, and guiding direction lines.

[0013] As a further aspect of the present invention, step S1 specifically comprises:

[0014] S101: Based on multimodal fundus images, continuous pixels within the boundary regions of multiple images are identified by grayscale scanning, and the grayscale difference between adjacent pixels is calculated to generate a grayscale difference sequence of the boundary regions.

[0015] The multimodal fundus images include color fundus photography, optical coherence tomography (OCT) scans, and infrared / near-infrared imaging.

[0016] S102: Based on the gray-level difference sequence of the boundary region, extract the gray-level difference curve under continuous time frames, call the image acquisition time series data to construct the trend baseline curve of the same frame number, compare the vertical difference of the pixel position of the two curves and extract the jump amplitude of the pixel difference in the image to obtain the jump difference comparison result.

[0017] S103: Based on the jump difference comparison results, extract the coordinates of all grayscale points in the jump amplitude sequence that exceed the jump threshold as structural change points and summarize them to generate structural change coordinate points;

[0018] The jump threshold is set by analyzing the statistical regularity of changes in retinal structure or grayscale features in multimodal images.

[0019] As a further aspect of the present invention, step S2 specifically comprises:

[0020] S201: Based on the coordinate points of the structural mutation, obtain the spatial distribution in the coordinate system of the original fundus image, combine the local gray-level gradient direction, calculate the angle of the line connecting adjacent coordinate points and determine whether the direction change of adjacent coordinate points is continuous, and generate the direction continuity angle.

[0021] S202: Obtain the coordinates and pixel grayscale changes of the path segment according to the directional coherence angle, calculate the internal state transition trend of the path segment through the ST-CRF model, determine whether the state change of the path segment maintains a single trend, and obtain the state trend evolution value.

[0022] The ST-CRF model consists of two parts: a spatial domain and a time domain.

[0023] S203: Based on the state transition trend value and the direction continuity angle, determine whether the direction and state changes in the path segment simultaneously present a discontinuous pattern, filter the set of path segments with unstable change characteristics, and generate a preliminary path set for retinal detachment.

[0024] As a further aspect of the present invention, the method of calculating the internal state transition trend of a path segment using the ST-CRF model is described. The formula used is:

[0025] ;

[0026] in, This represents the pixel grayscale value of the i-th coordinate point in the current path segment. This represents the pixel grayscale value of the (i-1)th coordinate point in the preceding path segment. This represents the directional continuity weight of the i-th coordinate point. Let i be the spatiotemporal correlation factor of the i-th coordinate point. It is the noise suppression constant. This is the dynamic balance coefficient. This represents the gradient magnitude at the i-th coordinate point. This represents the total number of coordinate points included in the current path segment.

[0027] As a further aspect of the present invention, step S3 specifically comprises:

[0028] S301: Based on the preliminary path set of retinal detachment, obtain the coordinate points of unstable path segments, extend the edge structure of adjacent regions along the direction, detect the continuity of pixel grayscale and construct a coordinate chain for outward extension of the path, extract the path extension segments with uninterrupted continuity, and generate an extended path sequence.

[0029] S302: Based on the extended path sequence, calculate the direction vectors of continuous points within multiple path segments, statistically analyze the direction difference and change frequency between adjacent points, construct a direction stability index, and obtain the direction fluctuation characteristics.

[0030] S303: Based on the directional fluctuation characteristics, filter out path segments with large directional rate fluctuations and frequent abrupt changes, and extract the spatial coordinate point set of the corresponding coverage area to generate potential detachment structure candidate regions.

[0031] As a further aspect of the present invention, step S4 specifically comprises:

[0032] S401: Based on the potential detachment candidate region, obtain the corresponding spatial coordinate points and detachment candidate path segments, calculate the pixel density change, path extension span and structural boundary curvature gradient of multiple paths in the candidate region and combine them to generate path hierarchical input features.

[0033] S402: Based on the path hierarchical input features, calculate the class boundary distance of the path segment in the feature space through the support vector machine model, determine the corresponding level jump state according to the decision function, classify the judgment result corresponding to each feature vector as a level identifier, and generate a jump level discrimination result.

[0034] The support vector machine model is mainly composed of an input layer, a kernel function mapping layer, and an optimal classification hyperplane construction module.

[0035] S403: Based on the jump level discrimination result, label the original path segment set and map it to the pixel label map under the image region coordinates to generate the retinal detachment level classification result.

[0036] As a further aspect of the present invention, the method of calculating the class boundary distance of the path segment in the feature space using a support vector machine model is described. The formula used is:

[0037] ;

[0038] in, This represents the original weight coefficient of the k-th feature dimension in the support vector machine. Represents the maximum value of all weight coefficients. This represents the original observation value of the k-th feature component of the current path segment. This represents the baseline mean value of the k-th feature component in the training set. This represents the standard deviation of the k-th feature component in the historical data. The regularization coefficient represents the dimension of the m-th feature space. This represents a numerical stability constant.

[0039] As a further aspect of the present invention, step S5 specifically comprises:

[0040] S501: Obtain the distribution position of the potential detached structure candidate region in consecutive frames of the image, calculate the displacement vector direction and change magnitude of the coordinate points between adjacent frames, and establish a direction change sequence based on the continuity of the coordinate trend direction change and the offset magnitude to generate the expansion trend direction result.

[0041] S502: Based on the expansion trend direction result and the retinal detachment level classification result, extract the boundary coordinate points and extension direction of the corresponding level path segment to construct the path guidance vector, and perform level identification mapping on multiple path segments to generate guidance direction label mapping.

[0042] S503: Based on the guidance direction label mapping, the original image area is fused with the guidance vector map and the grade label map, and the spatial positioning index and visual distribution output under the unified coordinate system are generated to produce a multimodal retinal detachment graded intervention map.

[0043] On the other hand, a retinal detachment grading system based on multimodal fundus images is provided. This system is applied to a retinal detachment grading method based on multimodal fundus images. The system includes:

[0044] The boundary detection module acquires the boundary gray lines in the multimodal fundus image and calculates the gray difference between adjacent pixels. It compares the difference with the time trend baseline and extracts the coordinates of abrupt change points that exceed the jump threshold. It generates structural abrupt change coordinate points and passes them to the decoupling extraction module.

[0045] The detachment extraction module calculates the direction of the coordinate point based on the structural mutation coordinate point and filters the path segments with the same direction. It also calculates the directional trend through the ST-CRF model and filters unstable path segments as detachment paths, generating a preliminary set of retinal detachment paths and passing it to the path extension module.

[0046] The path extension module extracts path extension segments of unstable path segments based on the preliminary path set of retinal detachment, calculates the direction change rate and the number of direction abrupt changes of the path extension segments, filters detachment candidate path segments, generates potential detachment structure candidate regions, and passes them to the hierarchical discrimination module.

[0047] The grading and discrimination module uses a support vector machine model to determine the grade transition status of the detachment candidate path in the candidate region of the potential detachment structure and marks the grade label, generating the retinal detachment grade grading result and transmitting it to the intervention atlas module.

[0048] The intervention atlas module obtains the trajectory of node coordinate changes in the candidate region of the potential detachment structure, calculates the direction of expansion trend, and constructs a guide direction line by combining the path segment corresponding to the retinal detachment level classification result and fusing the level label to generate a multimodal retinal detachment classification intervention atlas.

[0049] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0050] By extracting abrupt change points in the boundary grayscale lines of multimodal fundus images and comparing the grayscale difference between adjacent pixels with the time trend baseline, subtle grayscale fluctuations in retinal structures can be effectively captured. This differentiated grayscale distribution analysis enhances the ability to identify minute structural abrupt changes. Directional calculation of abrupt change coordinates and consistent path selection enable more accurate tracking of the true trajectory of structural detachment against complex retinal backgrounds. Furthermore, the dynamic assessment of directional trend stability using a trend conditional random field further eliminates false detachment paths, effectively enhancing the accuracy of path selection. The extension analysis of unstable path segments and the superposition judgment of directional change rates and abrupt change counts strengthen the complete delineation of potential detached structural regions, improving the comprehensiveness of candidate region detection. Based on this, a support vector machine is used to discriminate and label the level transition states of path segments in different regions, ensuring the objectivity and consistency of retinal detachment severity grading. By analyzing the correlation between node coordinate changes and expansion trends, and guiding the grading results to corresponding path segments to generate intervention maps, this approach fully utilizes multimodal image information and provides visual guidance on structural grading status, enhancing the specificity of clinical interpretation and the guidance for subsequent interventions. This comprehensive processing method, through multi-stage correlation judgment, path direction trend tracking, and label generation, effectively improves the accuracy of retinal detachment grading and the operability of intervention planning. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of the workflow of the present invention;

[0053] Figure 2 This is a system flowchart of the present invention. Detailed Implementation

[0054] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0055] In embodiments of the present invention, words such as "exemplarily" and "comprising" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0056] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0057] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0058] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0059] Please see Figure 1 This invention provides a technical solution: a method for grading retinal detachment using multimodal fundus images, comprising the following steps:

[0060] S1: Obtain the boundary gray lines in the multimodal fundus image and calculate the gray difference between adjacent pixels. Compare the difference with the time trend baseline and extract the coordinates of abrupt change points that exceed the jump threshold to generate structural abrupt change coordinate points.

[0061] S2: Calculate the direction of the coordinate points based on the coordinate points of structural mutation and select path segments with the same direction. Then, calculate the directional trend through the ST-CRF model and select unstable path segments as detachment paths to generate a preliminary set of retinal detachment paths.

[0062] S3: Based on the preliminary set of retinal detachment paths, extract the path extension segments of unstable path segments, calculate the rate of change of direction and the number of abrupt changes in direction of the path extension segments, screen candidate detachment path segments, and generate candidate regions of potential detachment structures.

[0063] S4: Use a support vector machine model to determine the level transition status of the detachment candidate path in the potential detachment structure candidate region and label the level to generate the retinal detachment level classification result.

[0064] S5: Obtain the trajectory of node coordinate changes in the candidate region of potential detachment structures, calculate the direction of expansion trend, and construct the guiding direction line by combining the path segment corresponding to the retinal detachment level classification result and integrate the level label to generate a multimodal retinal detachment classification intervention map.

[0065] The coordinates of structural mutation points include the gray-level difference between adjacent pixels, the time trend baseline, and the coordinates of the mutation point. The preliminary set of retinal detachment paths includes unstable path segments, detachment paths, path segment directions, and consistent path segments. The potential detachment structure candidate areas include path extension segments, direction change rates, number of direction mutations, and detachment candidate path segments. The retinal detachment grade classification results include detachment candidate paths, grade jump states, and grade labels. The multimodal retinal detachment grading intervention atlas includes the trajectory of node coordinate changes, the direction of expansion trends, and the guiding direction line.

[0066] Please see Figure 1 The specific steps of S1 are as follows:

[0067] S101: Based on multimodal fundus images, continuous pixels within the boundary regions of multiple images are identified by grayscale scanning, and the grayscale difference between adjacent pixels is calculated to generate a grayscale difference sequence of the boundary regions.

[0068] Multimodal fundus images include color fundus photography, optical coherence tomography (OCT) scans, and infrared / near-infrared imaging.

[0069] Based on multimodal fundus images, a 10×10 pixel region was delineated along the optic disc boundary. A grayscale scanner was used to acquire pixel values ​​line by line with a 0.1mm step precision. In color fundus photography, a region with coordinates (100, 200) to (150, 250) was selected to obtain the grayscale values ​​of 120 consecutive pixels. The Manhattan distance method was used to calculate the grayscale value of the nth pixel. With the gray value of the (n+1)th pixel Perform the calculation to obtain the difference. including when , At that time, the calculation yielded The calculation was repeated 119 times to generate the sequence. In the optical coherence tomography (OCT) image, the fovea region of the macula (200, 300) to (250, 350) was selected, and the same operation was performed to obtain 80 difference data. Table 1 shows 5 typical difference data examples. As shown in Table 1, a sequence including 200 difference data was generated.

[0070] Table 1 Examples of Gray-Scale Difference in Boundary Regions

[0071]

[0072] S102: Based on the gray-level difference sequence of the boundary region, extract the gray-level difference curve under continuous time frames, call the image acquisition time series data to construct the trend baseline curve of the same frame number, compare the vertical difference of the pixel position of the two curves, extract the jump amplitude of the pixel difference value in each image, and obtain the jump difference comparison result.

[0073] Within a 10-second time window, one frame of the image is captured every 0.5 seconds, for a total of 20 time frames. The average grayscale difference value is calculated for the (100, 200) region of each frame. The average difference value is measured in the first frame. Frame 2 When constructing the trend baseline, the moving average is calculated using the data from the first 5 frames. When t=5, Longitudinal difference calculation adopts When the 6th frame , hour, The threshold for determining the jump amplitude is set to 30% of the baseline value, when the moving average... The threshold is 1.77, and 2.4 > 1.77 is considered a valid transition.

[0074] S103: Based on the jump difference comparison results, extract the coordinates of all grayscale points in the jump amplitude sequence that exceed the jump threshold as structural change points and summarize them to generate structural change coordinate points;

[0075] The jump threshold is set by analyzing the statistical regularities of changes in retinal structure or grayscale features in multimodal images;

[0076] When determining structural mutations, jump data were collected from 100 normal fundus images, and the mean was calculated. Standard deviation Set threshold When the jump values ​​of a single pixel are 15.6, 16.2 and 17.1 for three consecutive frames, they all exceed the threshold. The point coordinates (122, 208) are extracted. The density of points exceeding the threshold is counted during the summary. When there are more than 50 mutation points in an area (1 mm²), it is marked as a lesion area. Table 2 lists the data of three typical mutation points and generates a structured dataset including the coordinates of the lesion area and the corresponding jump values.

[0077] Please see Figure 1 The specific steps of S2 are as follows:

[0078] S201: Based on the coordinate points of structural mutation, obtain the spatial distribution in the coordinate system of the original fundus image, combine the local gray-level gradient direction, calculate the angle of the line connecting adjacent coordinate points and determine whether the direction change of adjacent coordinate points is continuous, and generate the direction coherence angle.

[0079] Based on the structural abrupt change coordinates (122, 208), (135, 222), and (140, 235), a rectangular coordinate system is established in the original fundus image. Taking (122, 208) as the starting point, the direction angle of the line connecting (135, 222) is calculated. The gradient direction between the two points is obtained using the gray-level gradient method. First, the gray-level values ​​of the 3×3 neighboring pixels at (122, 208) are extracted, and the derivative in the x-direction is calculated. y-direction derivative The gradient direction angle is obtained. Similarly, calculate the gradient direction angle at (135, 222). The direction angle of the line connecting the two coordinate points The condition for determining directional continuity is: and In the example Marked as discontinuous, Table 2 shows the direction angle data of 3 sets of coordinate points. As shown in Table 2, a coherence evaluation result including angle deviation values ​​is generated.

[0080] Table 2 Examples of Directional Consistency Angles

[0081] coordinate point pair Gradient angle 1 Gradient angle 2 Connecting Angle Deviation value (122，208)-(135，222) 63.43° 58.21° 47.12° 16.31° (135，222)-(140，235) 58.21° 61.34° 55.82° 2.39° (140，235)-(148，240) 61.34° 59.87° 50.19° 11.15°

[0082] S202: Obtain the coordinates and pixel grayscale changes of the path segment based on the directional coherence angle, calculate the internal state transition trend of the path segment through the ST-CRF model, determine whether the state change of the path segment maintains a single trend, and obtain the state trend evolution value.

[0083] The ST-CRF model consists of two parts: the spatial domain and the time domain.

[0084] Within the path segment (135, 222)-(140, 235)-(148, 240), the second coordinate point (140, 235) is selected for state transition calculation. Represents the current path segment number The pixel grayscale value at each coordinate point is converted to grayscale using the RGB-to-grayscale formula. Calculations, including the red area at point (140, 235). ,green ,blue , ,but (Rounded to 173) Represents the preceding path segment The grayscale value of the point is obtained in the same way. Including the previous point (135, 222) , The weight representing directional continuity is quantified using the following formula: ( (The difference between the gradient direction angle of adjacent points and the direction angle of the connecting line), when hour, , Representing the spatiotemporal correlation factor, including the coordinate difference between points (140, 235) and (135, 222). ,but , This represents the noise suppression constant, set based on the signal-to-noise ratio test data of the fundus imaging equipment (E=2.5 when the signal-to-noise ratio is 30dB). The dynamic equilibrium coefficient, derived from regression analysis of 50 normal fundus data, ensures that the two terms in the formula are of similar magnitude. Represents the gradient magnitude, calculated as follows: Including point (140, 235) ,but Substitute into the formula to calculate: Calculate the coordinates of 3 points consecutively. The state trend evolution value is obtained through adjacent Value difference calculation, Represents the internal state transition trend, when In the case of determining a sudden trend change, in the example The critical value has been reached.

[0085] S203: Based on the state transition trend value and the direction coherence angle, determine whether the direction and state changes in the path segment simultaneously present a discontinuous pattern, filter the set of path segments with unstable change characteristics, and generate a preliminary set of paths for retinal detachment.

[0086] The directional coherence angle deviation value [16.31°, 2.39°, 11.15°] and the state trend evolution difference value [1.5, 0.8] are set with dual threshold conditions: angle deviation > 10° and trend difference > 1.2. In the example, the first path segment (122, 208) - (135, 222) has an angle deviation of 16.31° > 10° and a trend difference of 1.5 > 1.2. The third path segment (140, 235) - (148, 240) has an angle deviation of 11.15° > 10° but a trend difference of 0.8 < 1.2. Only the first path segment is filtered, generating a preliminary set including two unstable path segments.

[0087] Please see Figure 1 The specific steps of S3 are as follows:

[0088] S301: Based on the preliminary path set of retinal detachment, obtain the coordinate points of unstable path segments, extend the edge structure of adjacent regions along the direction, detect the continuity of pixel grayscale and construct the coordinate chain of the path extending outward, extract the path extension segments with uninterrupted continuity, and generate the extended path sequence.

[0089] Based on the initial path set for retinal detachment, the path turning points P1(120, 80) and P2(135, 95) are selected as starting points. The gray values ​​of adjacent pixels are detected within an 8-neighborhood. The gray value difference threshold Δ=15 is set. When the gray value fluctuation range of three consecutive pixels is within the interval (I-Δ, I+Δ], it is determined to be a continuous region, including the pixel sequence (158, 162, 155] which satisfies the Δ=15 condition. The coordinate increment of the extension point is calculated along the 45-degree direction: Δx=cos(45°)*2≈1.414, Δy=sin(45°)*2≈1.414, generating the extension coordinate chain {(121.4, 81.4), (122.8, 82.8)}. When the gray value change point (130, 88) is encountered (the gray value suddenly drops from 160 to 120), the current extension path is terminated, and the first 5 consecutive points are retained to construct the extended path sequence, as shown in Table 3 for the path extension coordinate data.

[0090] Table 3 Path Extension Coordinate Data Table

[0091] Serial Number X coordinate Y coordinate grayscale value 1 120 80 158 2 121 81 162 3 122 82 155 4 123 83 160 5 124 84 157

[0092] As shown in Table 3, the grayscale value fluctuation range of serial numbers 1-5 is between 155 and 162, which meets the continuity judgment standard of Δ=15. When the grayscale value of serial number 6 (125, 85) suddenly changes to 122, the path extension is terminated, and an extended path sequence including 5 coordinate points is generated.

[0093] S302: Based on the extended path sequence, calculate the direction vectors of continuous points within multiple path segments, statistically analyze the direction difference and change frequency between adjacent points, construct a direction stability index, and obtain the direction fluctuation characteristics.

[0094] Take three consecutive points A(120, 80), B(121, 81), and C(122, 82) from the extended path sequence, and calculate the vector. and The orientation angles are all 45 degrees, and the orientation difference δ = 0 degrees. A fluctuation sequence (0°, 2°, 5°, 0°, 3°, 8°, 1°, 0°, 4°, 2°) is obtained by statistically analyzing the orientation differences of 10 adjacent points. A mutation threshold θ = 5° is set. When the orientation difference exceeds θ three times consecutively, it is marked as a high-frequency mutation (including the 5th-7th data group where 3°→8°→1° did not meet the consecutive threshold exceeding condition). The orientation stability index is calculated. ,when When the directional stability index S is obtained after 3 cycles, it is 0.7. When S < 0.8, it is judged to be a directional fluctuation characteristic.

[0095] S303: Based on the directional fluctuation characteristics, filter out path segments with large directional rate fluctuations and high mutation frequency, and extract the spatial coordinate point set of the corresponding coverage area to generate potential detachment structure candidate areas.

[0096] For path segments with a directional stability index S=0.7, a directional change rate threshold α=3° was set. Three single-step directional changes exceeding α were detected in the path segment (e.g., a sudden change from 45° to 60°, a change of 15°). The coordinate set of the abrupt change points {(128, 90), (133, 88), (140, 85)} was extracted. A 5×5 pixel detection window was established with multiple points as the center. The average gray-level gradient magnitude G within the window was calculated to be 58 (higher than the baseline value G0=40). When three consecutive windows satisfy G>G0, a candidate region coordinate set {(128-130, 90-92), (133-135, 88-90), (140-142, 85-87)} was generated. The region covers 6.5% of the retinal area, which is consistent with the typical characteristic of 5-8% of the detached lesion area in clinical medicine.

[0097] Please see Figure 1 The specific steps of S4 are as follows:

[0098] S401: Based on the potential detachment structure candidate region, obtain the corresponding spatial coordinate points and detachment candidate path segments, calculate the pixel density change, path extension span and structural boundary curvature gradient of multiple paths in the candidate region and combine them to generate path hierarchical input features.

[0099] Based on the candidate region coordinate set {(128-130, 90-92), (133-135, 88-90)}, the number of pixels N=85 is extracted from the path segment L1 (128, 90 to 130, 92). The density per unit area D=N / Area=85 / (3×3)=9.44 points / pixel² is calculated. Compared with the adjacent region L2 (125-127, 88-90), D=6.33 points / pixel². The density change rate threshold ΔD=2.0 is set. When |D1-D2|=3.11>ΔD, it is judged as a drastic change. The path span L= =2.828 pixels, calculate the curvature gradient C=|60-45| / 2 pixels=7.5, combine the feature vectors 9.44, 2.828, 7.5, as shown in Table 4, the feature data combination process of the three candidate regions.

[0100] Table 4. Path Hierarchy Feature Data Table

[0101]

[0102] As shown in Table 4, the density of candidate region 1 is 9.44 points / pixel², which exceeds the baseline value of 7.0. The extension span of 2.828 pixels is within the effective range of 2.0 and 3.5. The curvature gradient of 7.5° / pixel is higher than the curvature mutation threshold of 5.0. The combined feature vectors 9.44, 2.828, and 7.5 are used as hierarchical input features.

[0103] S402: Based on the path hierarchical input features, the class boundary distance of the path segment in the feature space is calculated through the support vector machine model. The corresponding level jump state is determined according to the decision function. The judgment result corresponding to each feature vector is classified as a level identifier, and a jump level discrimination result is generated.

[0104] The key components of the support vector machine model are the input layer, the kernel function mapping layer, and the optimal classification hyperplane construction module;

[0105] The feature vectors 9.44, 2.828, and 7.5 are obtained from the training set. =7.2, μ2=3.1, μ3=4.8, σ1=1.5, σ2=0.9, σ3=1.2, Represents the number of feature dimensions (density / span / curvature). The representative was obtained through SVM weight analysis of 500 training samples, among which... (Dense weights) are determined by the ranking of feature importance. Represents taking the weight set The maximum value, The mean of the features in the training set ( pass calculate), Representative standard deviation ( pass calculate), Representing the The density feature value of each sample This represents the L2 regularization coefficient, which is set based on the cross-validation results. This represents a correction to prevent the denominator from being zero; the example calculation represents taking the features of the current sample. Substitute into the formula to calculate the representative The results showed that the sample was 3.42 units away from the classification hyperplane, which exceeded the level jump threshold of 2.5, triggering a high-risk judgment.

[0106] S403: Based on the jump level discrimination result, label the original path segment set and map it to the pixel label map under the image region coordinates to generate the retinal detachment level classification result;

[0107] The risk level label of path segment L1 was removed and mapped to the image coordinates (128, 90)-(130, 92), and the pixel label value was set to 255 (corresponding to red warning). The adjacent area L2 (125, 88)-(127, 90) was labeled with the medium risk label value of 128. In the generated classification result map, the high risk area accounted for 12.7% of the coverage area, the medium risk area accounted for 8.3%, and the remaining areas were marked with the normal value of 0.

[0108] Please see Figure 1 The specific steps of S5 are as follows:

[0109] S501: Obtain the distribution position of the potential decoupled structure candidate region in consecutive frames of the image, calculate the displacement vector direction and change magnitude of the coordinate points between adjacent frames, and establish a direction change sequence based on the continuity of the coordinate trend direction change and the offset magnitude to generate the expansion trend direction result.

[0110] Based on the candidate region coordinate set {(128, 90), (133, 88), (140, 85)}, the displacement data of coordinate points in three consecutive frames of images are extracted, with a frame interval Δt = 0.1 seconds. The displacement vector of point P1(128, 90) to (130, 92) in the second frame is calculated. The direction angle θ1 = 45°, the third frame shifts to (133, 95), and the vector... With direction angle θ2 = 45° and a continuous direction threshold Δθ = 10°, a continuous trend is determined when |θ2 - θ1| = 0° < Δθ. Three sets of displacement amplitudes are statistically analyzed. Pixels Represents the displacement vector. The threshold for the rate of change of pixels is Δr = 1.5 pixels / frame. When |4.242-2.828| = 1.414 < Δr, the sequence is preserved, generating the direction sequence 45°, 45°, 45° and the amplitude sequence 2.828, 4.242, 5.657, as shown in Table 5, which contains the trend data of three sets of coordinate points.

[0111] Table 5. Expansion Trend Direction Data Table

[0112]

[0113] As shown in Table 5, the direction of point P1 has a continuous deviation of 0°, and the amplitude changes by 1.414 pixels / frame, generating a stable expansion trend of 45°. The direction of point P2 has a cumulative change of 40°, exceeding Δθ=10°. Δθ represents the continuous threshold of direction and is marked as an abnormal fluctuation trend.

[0114] S502: Based on the expansion trend direction results and the retinal detachment level classification results, extract the boundary coordinates and extension direction of the corresponding level path segments to construct the path guidance vector, and perform level identification mapping on multiple path segments to generate guidance direction label mapping.

[0115] Take the coordinate set of the high-risk area {(128, 90)-(130, 92)}, extract the boundary points B1(128, 90) and B2(130, 92), and calculate the extension direction vector. With a direction angle φ = 45°, construct the guiding vector. When mapping to the risk level label, the vector strength coefficient is set to 1.5 for high-risk areas and 1.0 for medium-risk areas. The current guiding vector is then calculated. Generate a label mapping relationship: the vector (1.06, 1.06) at coordinates (129, 91) is associated with the level label L3.

[0116] S503: Based on the guidance direction label mapping, the original image area is fused with the guidance vector map and the grade label map, and the spatial positioning index and visual distribution output under the unified coordinate system are generated to produce a multimodal retinal detachment graded intervention map.

[0117] The guide vector image and the level label image are merged in a 512×512 pixel coordinate system. The high-risk area coordinates (128, 90)-(130, 92) are mapped to a red channel value of 255 and a vector arrow length of 10 pixels. The medium-risk area coordinates (125, 88)-(127, 90) are mapped to a yellow channel value of 128 and an arrow length of 6 pixels. In the generated merged image, the vector direction of the high-risk area is concentrated in the 45°±5° range, accounting for 73.5%, while the direction distribution of the medium-risk area has a dispersion of 32.6%. The output image has a DPI of 300 and is stored in RGB three-channel TIFF format.

[0118] Please see Figure 2 A retinal detachment grading system based on multimodal fundus images, the system being used to execute the aforementioned retinal detachment grading method based on multimodal fundus images, the system comprising:

[0119] The boundary detection module acquires the boundary gray lines in the multimodal fundus image and calculates the gray difference between adjacent pixels. It compares the difference with the time trend baseline and extracts the coordinates of abrupt change points that exceed the jump threshold. It generates structural abrupt change coordinate points and passes them to the decoupling extraction module.

[0120] The detachment extraction module calculates the direction of the coordinate points based on the coordinate points of structural mutation and filters the path segments with the same direction. It also calculates the directional trend through the ST-CRF model and filters unstable path segments as detachment paths, generating a preliminary set of retinal detachment paths and passing them to the path extension module.

[0121] The path extension module extracts path extension segments from the initial path set of retinal detachment, calculates the direction change rate and the number of direction abrupt changes of the path extension segments, filters detachment candidate path segments, generates potential detachment structure candidate regions, and passes them to the hierarchical discrimination module.

[0122] The grading and discrimination module uses a support vector machine model to determine the grade transition status of the detachment candidate path in the potential detachment structure candidate area and marks the grade label, generating the retinal detachment grade grading result and transmitting it to the intervention atlas module.

[0123] The intervention map module obtains the trajectory of node coordinate changes in the candidate region of potential detachment structures, calculates the direction of expansion trend, and constructs guiding direction lines by combining the path segments corresponding to the retinal detachment grade classification results and integrating grade labels to generate a multimodal retinal detachment grade intervention map.

[0124] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist: A and / or B, which can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it may also indicate an "and / or" relationship; please refer to the context for a more accurate understanding.

[0125] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. "At least one of a, b, or c" can be represented as: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0126] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0127] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0128] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0129] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. This includes the fact that the apparatus embodiments described above are merely illustrative, and that the division of units is only a logical functional division; in actual implementation, there may be other division methods, including the possibility that multiple units or components can be combined or integrated into another device, or that some features can be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed can be through some interface; the indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0132] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A retinal detachment grading method based on multimodal fundus images, characterized in that, The method includes: S1: Obtain the boundary gray lines in the multimodal fundus image and calculate the gray difference between adjacent pixels. Compare the difference with the time trend baseline and extract the coordinates of abrupt change points that exceed the jump threshold to generate structural abrupt change coordinate points. S2: Calculate the coordinate point direction based on the structural mutation coordinate point and select path segments with consistent direction, and calculate the directional trend through the ST-CRF model and select unstable path segments as detachment paths to generate a preliminary set of retinal detachment paths. S3: Based on the preliminary set of retinal detachment paths, extract the path extension segments of unstable path segments, calculate the direction change rate and the number of direction abrupt changes of the path extension segments, and screen detachment candidate path segments to generate potential detachment structure candidate regions. S4: Use a support vector machine model to determine the level transition status of the detachment candidate path in the potential detachment structure candidate region and label the level to generate the retinal detachment level classification result. S5: Obtain the trajectory of node coordinate changes in the candidate region of the potential detachment structure, calculate the direction of expansion trend, and construct a guiding direction line by combining the path segment corresponding to the retinal detachment level classification result and fuse the level label to generate a multimodal retinal detachment classification intervention map.

2. The method for grading retinal detachment using multimodal fundus images according to claim 1, characterized in that, The structural mutation coordinates include the gray-level difference between adjacent pixels, the time trend baseline, and the coordinates of the mutation point. The preliminary set of retinal detachment paths includes unstable path segments, detachment paths, path segment directions, and consistent path segments. The potential detachment structure candidate regions include path extension segments, direction change rates, number of direction mutations, and detachment candidate path segments. The retinal detachment grade classification results include detachment candidate paths, grade jump states, and grade labels. The multimodal retinal detachment grading intervention atlas includes node coordinate change trajectories, expansion trend directions, and guiding direction lines.

3. The method for grading retinal detachment using multimodal fundus images according to claim 1, characterized in that, The specific steps of S1 are as follows: S101: Based on multimodal fundus images, continuous pixels within the boundary regions of multiple images are identified by grayscale scanning, and the grayscale difference between adjacent pixels is calculated to generate a grayscale difference sequence of the boundary regions. The multimodal fundus images include color fundus photography, optical coherence tomography (OCT) scans, and infrared / near-infrared imaging. S102: Based on the gray-level difference sequence of the boundary region, extract the gray-level difference curve under continuous time frames, call the image acquisition time series data to construct the trend baseline curve of the same frame number, compare the vertical difference of the pixel position of the two curves and extract the jump amplitude of the pixel difference in the image to obtain the jump difference comparison result. S103: Based on the jump difference comparison results, extract the coordinates of all grayscale points in the jump amplitude sequence that exceed the jump threshold as structural change points and summarize them to generate structural change coordinate points; The jump threshold is set by analyzing the statistical regularity of changes in retinal structure or grayscale features in multimodal images.

4. The method for grading retinal detachment using multimodal fundus images according to claim 1, characterized in that, The specific steps of S2 are as follows: S201: Based on the coordinate points of the structural mutation, obtain the spatial distribution in the coordinate system of the original fundus image, combine the local gray-level gradient direction, calculate the angle of the line connecting adjacent coordinate points and determine whether the direction change of adjacent coordinate points is continuous, and generate the direction continuity angle. S202: Obtain the coordinates and pixel grayscale changes of the path segment according to the directional coherence angle, calculate the internal state transition trend of the path segment through the ST-CRF model, determine whether the state change of the path segment maintains a single trend, and obtain the state trend evolution value. The ST-CRF model consists of two parts: a spatial domain and a time domain. S203: Based on the state transition trend value and the direction continuity angle, determine whether the direction and state changes in the path segment simultaneously present a discontinuous pattern, filter the set of path segments with unstable change characteristics, and generate a preliminary path set for retinal detachment.

5. The method for grading retinal detachment using multimodal fundus images according to claim 4, characterized in that, The ST-CRF model is used to calculate the internal state transition trend of the path segment. The formula used is: ； in, This represents the pixel grayscale value of the i-th coordinate point in the current path segment. This represents the pixel grayscale value of the (i-1)th coordinate point in the preceding path segment. This represents the directional continuity weight of the i-th coordinate point. Let i be the spatiotemporal correlation factor of the i-th coordinate point. It is the noise suppression constant. This is the dynamic balance coefficient. This represents the gradient magnitude at the i-th coordinate point. This represents the total number of coordinate points included in the current path segment.

6. The method for grading retinal detachment using multimodal fundus images according to claim 1, characterized in that, The specific steps in S3 are as follows: S301: Based on the preliminary path set of retinal detachment, obtain the coordinate points of unstable path segments, extend the edge structure of adjacent regions along the direction, detect the continuity of pixel grayscale and construct a coordinate chain for outward extension of the path, extract the path extension segments with uninterrupted continuity, and generate an extended path sequence. S302: Based on the extended path sequence, calculate the direction vectors of continuous points within multiple path segments, statistically analyze the direction difference and change frequency between adjacent points, construct a direction stability index, and obtain the direction fluctuation characteristics. S303: Based on the directional fluctuation characteristics, filter out path segments with large directional rate fluctuations and frequent abrupt changes, and extract the spatial coordinate point set of the corresponding coverage area to generate potential detachment structure candidate regions.

7. The method for grading retinal detachment using multimodal fundus images according to claim 1, characterized in that, The specific steps of S4 are as follows: S401: Based on the potential detachment candidate region, obtain the corresponding spatial coordinate points and detachment candidate path segments, calculate the pixel density change, path extension span and structural boundary curvature gradient of multiple paths in the candidate region and combine them to generate path hierarchical input features. S402: Based on the path hierarchical input features, calculate the class boundary distance of the path segment in the feature space through the support vector machine model, determine the corresponding level jump state according to the decision function, classify the judgment result corresponding to each feature vector as a level identifier, and generate a jump level discrimination result. The support vector machine model is mainly composed of an input layer, a kernel function mapping layer, and an optimal classification hyperplane construction module. S403: Based on the jump level discrimination result, label the original path segment set and map it to the pixel label map under the image region coordinates to generate the retinal detachment level classification result.

8. The method for grading retinal detachment using multimodal fundus images according to claim 7, characterized in that, The path segment's class boundary distance in the feature space is calculated using a support vector machine model. The formula used is: ； in, This represents the original weight coefficient of the k-th feature dimension in the support vector machine. This represents the maximum value of all weight coefficients. This represents the original observation value of the k-th feature component of the current path segment. This represents the baseline mean value of the k-th feature component in the training set. This represents the standard deviation of the k-th feature component in the historical data. The regularization coefficient represents the dimension of the m-th feature space. This represents a numerical stability constant.

9. The method for grading retinal detachment using multimodal fundus images according to claim 1, characterized in that, The specific steps of S5 are as follows: S501: Obtain the distribution position of the potential detached structure candidate region in consecutive frames of the image, calculate the displacement vector direction and change magnitude of the coordinate points between adjacent frames, and establish a direction change sequence based on the continuity of the coordinate trend direction change and the offset magnitude to generate the expansion trend direction result. S502: Based on the expansion trend direction result and the retinal detachment level classification result, extract the boundary coordinate points and extension direction of the corresponding level path segment to construct the path guidance vector, and perform level identification mapping on multiple path segments to generate guidance direction label mapping. S503: Based on the guidance direction label mapping, the original image area is fused with the guidance vector map and the grade label map, and the spatial positioning index and visual distribution output under the unified coordinate system are generated to produce a multimodal retinal detachment graded intervention map.

10. A retinal detachment grading system based on multimodal fundus images, characterized in that, The system is used to implement the retinal detachment grading method based on multimodal fundus images as described in any one of claims 1-9, and the system comprises: The boundary detection module acquires the boundary gray lines in the multimodal fundus image and calculates the gray difference between adjacent pixels. It compares the difference with the time trend baseline and extracts the coordinates of abrupt change points that exceed the jump threshold. It generates structural abrupt change coordinate points and passes them to the decoupling extraction module. The detachment extraction module calculates the direction of the coordinate point based on the structural mutation coordinate point and filters the path segments with the same direction. It also calculates the directional trend through the ST-CRF model and filters unstable path segments as detachment paths, generating a preliminary set of retinal detachment paths and passing it to the path extension module. The path extension module extracts path extension segments of unstable path segments based on the preliminary path set of retinal detachment, calculates the direction change rate and the number of direction abrupt changes of the path extension segments, filters detachment candidate path segments, generates potential detachment structure candidate regions, and passes them to the hierarchical discrimination module. The grading and discrimination module uses a support vector machine model to determine the grade transition status of the detachment candidate path in the candidate region of the potential detachment structure and marks the grade label, generating the retinal detachment grade grading result and transmitting it to the intervention atlas module. The intervention atlas module obtains the trajectory of node coordinate changes in the candidate region of the potential detachment structure, calculates the direction of expansion trend, and constructs a guide direction line by combining the path segment corresponding to the retinal detachment level classification result and fusing the level label to generate a multimodal retinal detachment classification intervention atlas.

Images