Method and system for segmenting fundus lesion images

Abstract

The application relates to the technical field of medical image processing, and provides a fundus lesion image segmentation method and system, which solves the problem of insufficient spatial continuity and anatomical rationality of three-dimensional segmentation results in the prior art. The method comprises the following steps: acquiring optical coherence data and structural light data of a retina, and registering and fusing the data into enhanced point clouds; extracting deep features of each point of the enhanced point clouds, and determining global dependency relationships; inputting a segmentation model to perform three-dimensional instance segmentation, and obtaining an initial segmentation result; introducing a geometric constraint function corresponding to a prior model of a normal anatomical structure of the retina, adjusting the initial result so that a lesion boundary conforms to the anatomical structure, and obtaining a target segmentation result; identifying key points of the lesion in the enhanced point clouds according to the target segmentation result; outputting a three-dimensional mask of the key points, and calculating the volume, surface area and spatial position of the lesion relative to the retinal layer according to the three-dimensional mask. The application can realize high-precision three-dimensional segmentation and quantitative analysis of fundus lesions.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method and system for segmenting images of fundus lesions. Background Technology

[0002] Segmentation of fundus lesion images is a key step in the diagnosis and treatment planning of ophthalmic diseases. It provides quantitative analysis for the diagnosis of ophthalmic diseases and has broad application prospects in improving the efficiency and accuracy of diagnosis and treatment.

[0003] In existing technologies, three-dimensional ophthalmic images are decomposed into independent two-dimensional slice sequences, and convolutional neural networks are used to perform pixel-level classification on each slice to identify lesion areas. Finally, the classification results are stacked to form three-dimensional segmentation data.

[0004] This approach, which treats three-dimensional data as a collection of two-dimensional slices, severs the inherent continuity and overall structural information of lesions in three-dimensional space. This can easily lead to breaks or inconsistencies between layers of the segmented lesions, lacks effective utilization of the layered anatomical structure of the eyeball itself, and may produce segmentation results that do not conform to the physiological position of the retina. This can affect the accuracy of subsequent calculations of key clinical indicators such as volume and depth. Therefore, the existing technology has technical problems with insufficient spatial coherence and anatomical rationality of the three-dimensional segmentation results. Summary of the Invention

[0005] This application provides a method and system for segmenting fundus lesion images to solve the problems of insufficient spatial coherence and anatomical rationality of the three-dimensional segmentation results in the prior art.

[0006] In a first aspect, this application provides a method for segmenting fundus lesion images, comprising:

[0007] Acquire optical coherence data and structured light data from the retina, and register and fuse the optical coherence data and the structured light data into an enhanced point cloud;

[0008] From the enhanced point cloud, deep features of each point are extracted by learning the interaction information between each point and all remaining points in the enhanced point cloud, and the global dependency relationship of the enhanced point cloud is determined based on the deep features of each point.

[0009] The global dependency is input into a pre-built segmentation model, which then performs 3D instance segmentation based on the global dependency to obtain an initial segmentation result.

[0010] During the segmentation process, a geometric constraint function corresponding to a priori model of the normal retinal anatomy is introduced. The geometric constraint function uses the geometric topology of the priori model as a reference boundary to adjust the initial segmentation result, so that the spatial relationship between the optimized lesion boundary and the normal retinal tissue conforms to the anatomical structure, and the target segmentation result is obtained.

[0011] Based on the target segmentation results, key points belonging to fundus lesions are identified and located in the enhanced point cloud;

[0012] Output the 3D mask for each key point, and calculate the volume, surface area, and spatial position of the fundus lesion relative to the retinal layer based on the 3D mask.

[0013] Optionally, the deep features are fused with positional features, geometric features, and grayscale features;

[0014] From the enhanced point cloud, deep features of each point are extracted by learning the interaction information between each point and all remaining points in the enhanced point cloud, and the global dependencies of the enhanced point cloud are determined based on the deep features of each point, including:

[0015] Extract the initial position features, initial geometric features, and initial grayscale features of each point from the enhanced point cloud;

[0016] The initial position features, the initial geometric features, and the initial grayscale features are concatenated to generate an initial feature vector;

[0017] The initial feature vector is mapped to a high-dimensional feature space through a mapping network to obtain the high-dimensional feature value of each point.

[0018] Calculate the association strength value between each point in the enhanced point cloud and all remaining points, and perform weighted aggregation of the high-dimensional feature values ​​of all points based on the association strength value to generate deep features for each point;

[0019] The deep features of all points in the enhanced point cloud are globally aggregated to generate global dependencies.

[0020] Optionally, the global dependency is input into a pre-built segmentation model, which then performs 3D instance segmentation based on the global dependency to obtain an initial segmentation result, including:

[0021] The global dependencies are received through a pre-built segmentation model;

[0022] The global dependencies are fused with the deep features of each point to generate enhanced point features;

[0023] Based on the enhanced point features, an initial segmentation label is assigned to each point, and the initial segmentation label represents the probability value that the point belongs to the lesion area;

[0024] Based on the spatial continuity of points, points with spatial distances less than a preset threshold and similar probability values ​​are classified to obtain candidate lesion regions;

[0025] The candidate lesion regions that are spatially adjacent and have similar enhancement point features are merged to obtain the initial segmentation result.

[0026] Optionally, during the segmentation process, a geometric constraint function corresponding to a priori model of the normal retinal anatomy is introduced. This geometric constraint function uses the geometric topology of the priori model as a reference boundary to adjust the initial segmentation result, ensuring that the spatial relationship between the optimized lesion boundary and normal retinal tissue conforms to the anatomical structure, thus obtaining the target segmentation result, including:

[0027] Construct a priori model of the normal anatomical structure of the retina, the priori model including smooth curved surface models of each layer of the retina;

[0028] Based on the geometric topology of the prior model, a geometric constraint function is constructed, which is used to calculate the spatial distance between the lesion boundary point and the corresponding retinal layer of the prior model.

[0029] The geometric constraint function is used to correct each lesion boundary point in the initial segmentation result, and the correction offset of each lesion boundary point is obtained.

[0030] Based on the corrected offset, the lesion boundary points are projected and smoothed to obtain the intermediate lesion boundary.

[0031] The geometric difference between the intermediate lesion boundary and the prior model is minimized, while the feature consistency of points inside the lesion is preserved, in order to obtain the target segmentation result.

[0032] Optionally, minimizing the geometric difference between the intermediate lesion boundary and the prior model while preserving the feature consistency of points within the lesion to obtain the target segmentation result includes:

[0033] The distance difference is calculated based on the spatial distance from each point on the boundary of the intermediate lesion to the prior model;

[0034] The feature similarity is calculated based on the deep features of points within the area enclosed by the boundary of the intermediate lesion.

[0035] The distance difference and the feature similarity are weighted and summed to construct a joint optimization objective;

[0036] The three-dimensional position of each point on the boundary of the intermediate lesion is adjusted so that the value of the joint optimization objective is continuously reduced until the preset convergence condition is reached;

[0037] The region formed by all points on the boundary of the intermediate lesion when the preset convergence condition is met is determined as the target segmentation result.

[0038] Optionally, based on the target segmentation result, key points belonging to fundus lesions are identified and located in the enhanced point cloud, including:

[0039] Extract all points marked as lesions from the target segmentation results to form a candidate lesion point set;

[0040] Calculate the feature similarity and spatial proximity between any two candidate lesion points. The feature similarity is determined by the closeness of the deep feature representations of the two points in the feature space, and the spatial proximity is determined by the Euclidean distance between the two points in three-dimensional space.

[0041] The feature similarity and spatial proximity are weighted and fused to generate a comprehensive affinity.

[0042] Points with a comprehensive affinity higher than a set threshold are grouped into the same region to generate a lesion region.

[0043] Within each lesion region, points with high average comprehensive affinity to other candidate lesions in the same region are selected as key points for fundus lesions.

[0044] Optionally, a three-dimensional mask for each key point is output, and the volume, surface area, and spatial position relative to the retinal layer of the fundus lesion are calculated based on the three-dimensional mask, including:

[0045] A closed three-dimensional surface mesh is generated by connecting each key point, and the points inside the three-dimensional surface mesh are marked as lesion areas to obtain a three-dimensional mask;

[0046] Based on the grid vertex coordinates of the three-dimensional mask, the sum of the areas of all grid faces is calculated as the surface area of ​​the fundus lesion;

[0047] The internal space of the three-dimensional mask is discretized into multiple small cubes, and the total volume of the small cubes is counted as the volume of the fundus lesion;

[0048] Using the internal limiting membrane of the retina as a reference plane, the three-dimensional spatial offset of the centroid of the three-dimensional mask from the reference plane is calculated to determine the spatial position of the fundus lesion relative to the retinal layer.

[0049] Secondly, this application provides a segmentation system for fundus lesion images, comprising:

[0050] The acquisition module is used to acquire optical coherence data and structured light data of the retina, and to register and fuse the optical coherence data and the structured light data into an enhanced point cloud;

[0051] The extraction module is used to extract deep features of each point from the enhanced point cloud by learning the interaction information between each point in the enhanced point cloud and all remaining points, and to determine the global dependency relationship of the enhanced point cloud based on the deep features of each point.

[0052] The input module is used to input the global dependency relationship into a pre-built segmentation model, and the segmentation model performs 3D instance segmentation based on the global dependency relationship to obtain an initial segmentation result;

[0053] An adjustment module is used to introduce a geometric constraint function corresponding to a prior model of the normal retinal anatomy during the segmentation process. The geometric constraint function uses the geometric topology of the prior model as a reference boundary to adjust the initial segmentation result so that the spatial relationship between the optimized lesion boundary and the normal retinal tissue conforms to the anatomical structure, thereby obtaining the target segmentation result.

[0054] The identification module is used to identify and locate key points belonging to fundus lesions in the enhanced point cloud based on the target segmentation results.

[0055] The calculation module is used to output a three-dimensional mask for each key point, and to calculate the volume, surface area and spatial position of the fundus lesion relative to the retinal layer based on the three-dimensional mask.

[0056] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a segmentation method for fundus lesion images as described in the first aspect above.

[0057] Fourthly, this application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a method for segmenting fundus lesion images as described in the first aspect.

[0058] The beneficial effects of this application are:

[0059] This application fused optical coherence data with structured light data to form an enhanced point cloud containing rich three-dimensional spatial and textural information, thereby constructing a complete three-dimensional context for lesion analysis. Based on this, three-dimensional instance segmentation was performed by capturing the global interactive dependencies between all points in the point cloud. Furthermore, an innovative retinal anatomical prior model was introduced as a geometric constraint to optimize the segmentation boundary. This resulted in the final target segmentation result not only fully preserving the three-dimensional morphological structure of the lesion, but also ensuring that the spatial relationship between its boundary and the surrounding normal retinal tissue strictly conformed to anatomical common sense. This fundamentally solved the problems of three-dimensional structural breakage and segmentation results deviating from the actual anatomical location caused by existing two-dimensional slicing methods.

[0060] Furthermore, based on the segmentation results, key points of the lesion are identified and corresponding three-dimensional masks are generated. The surface area, volume, and spatial position of the lesion relative to the retinal layer are directly calculated based on the three-dimensional mask. This achieves integrated, high-fidelity three-dimensional quantitative analysis of lesion morphology and spatial distribution, ensuring that the quantitative indicators are directly derived from the real and continuous three-dimensional lesion morphology. This avoids the cumulative error introduced by the approximation of two-dimensional slice stacking, and provides a reliable and consistent data foundation for clinical assessment of lesion progression and formulation of treatment plans.

[0061] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 A flowchart of a method for segmenting fundus lesion images provided in this application is shown;

[0064] Figure 2 A schematic diagram of the structure of a segmentation system for fundus lesion images provided in this application is shown;

[0065] Figure 3 A schematic diagram of the structure of a computing device provided in this application is shown. Detailed Implementation

[0066] To enable those skilled in the art to better understand the present application, the technical solution of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0067] Figure 1A flowchart of a method for segmenting fundus lesion images provided in this application is shown below. Figure 1 As shown, the method includes:

[0068] Step 101: Acquire optical coherence data and structured light data of the retina, and register and fuse the optical coherence data and structured light data into an enhanced point cloud.

[0069] In the above steps, optical coherence data refers to a set of three-dimensional image data obtained by performing a three-dimensional scan of the patient's fundus using optical coherence tomography (OCT) technology. This data reflects the structure of different depth layers within the retina and is used to provide cross-sectional details of the lesion. Structured light data refers to data obtained by projecting a specific pattern of light onto the surface of the patient's retina and analyzing the deformation of the specific light pattern projected onto the retinal surface. This data is used to reconstruct the three-dimensional contour of the retinal surface.

[0070] Augmented point cloud is a unified dataset formed by fusing the first two types of data after high-precision alignment. Its data points contain both accurate spatial coordinates and internal structural information, forming a three-dimensional model with both accurate geometric shape and rich internal texture.

[0071] In this embodiment, optical coherence volume (OCT) data and structured light data are combined into a single dataset, namely, an "enhanced point cloud". The entire process can be divided into four main stages: data acquisition, feature extraction, registration transformation, and fusion generation.

[0072] The first stage of this method involves acquiring two complementary types of fundus data: optical coherence data and structured light data. In clinical practice, a composite fundus imaging device scans the target retinal region of the patient. First, the device's optical coherence tomography (OCT) module rapidly constructs a three-dimensional optical coherence data set by analyzing near-infrared light reflection information. This data is morphologically a three-dimensional volumetric data block, resembling a stack of hundreds of high-resolution longitudinal slices of the retina, with the grayscale values ​​of each voxel revealing the structural information of the deep retinal tissues. Next, the device's structured light scanning module projects a specific pattern of light onto the same area, accurately reconstructing the structured light data by capturing the deformation of the pattern on the retinal surface. This data is morphologically a three-dimensional point cloud, composed of tens of thousands of precise three-dimensional coordinate points, accurately depicting the surface contour of the retina.

[0073] Having obtained the two independent datasets, the next crucial step is to establish a common reference system between them. This requires extracting stable and mutually corresponding feature "landmarks" from both datasets. For the optical coherence data acquired in the previous step, the system generates a clear two-dimensional "bird's-eye view" of the blood vessels by projecting its three-dimensional volume data along the depth direction with maximum intensity. On this image, unique blood vessel bifurcation points are automatically identified as the first set of feature points, and a "digital fingerprint" describing the local image texture is generated for each point. Simultaneously, for the structured light data acquired in the previous step, which is in the form of a three-dimensional point cloud, the system analyzes its three-dimensional geometry to locate geometrically significant points with drastic curvature changes, such as the edge of the optic disc, as the second set of feature points, and similarly generates "digital fingerprints" describing their local three-dimensional shapes for them.

[0074] The subsequent registration process aims to precisely align the two sets of feature points found from different data sources. The algorithm first establishes preliminary matching pairs by comparing the "digital fingerprints" of the two sets of feature points. Since these preliminary matches inevitably contain errors, the system employs the Random Sample Consensus (RANSAC) algorithm for "refinement," that is, by repeatedly sampling and verifying, finding the optimal transformation scheme that achieves the broadest consensus on matching pairs. After selecting this set of highly reliable matching pairs, a global optimization is performed using the least squares method to calculate a high-precision spatial transformation matrix that minimizes all matching errors.

[0075] The final step is deep data fusion to generate the final enhanced point cloud. The system applies the high-precision spatial transformation matrix obtained in the previous step to the entire structured light data, ensuring that the coordinate system of its 3D point cloud is completely unified with the coordinate system of the optical coherence volume data. Next, for each point in the transformed structured light data, the system uses its 3D coordinates to query the 3D volume data block of the optical coherence volume data, finding the voxel whose spatial location perfectly corresponds to it and reading its grayscale value. This grayscale value, representing the optical properties of deep tissue, is attached as a new attribute to that point. Through this process, the original structured light data is "enhanced," forming a unified 3D model that combines high-precision spatial positioning with rich internal structural information—the enhanced point cloud.

[0076] Through this series of processes, a unified 3D model with both accurate geometry and rich internal texture was obtained, providing a high-quality data foundation for subsequent lesion segmentation and analysis.

[0077] Step 102: Extract deep features of each point from the augmented point cloud by learning the interaction information between each point in the augmented point cloud and all remaining points, and determine the global dependency relationship of the augmented point cloud based on the deep features of each point.

[0078] In this application, deep features refer to a high-level information description generated for each point in the enhanced point cloud. It not only includes the point's own coordinates, grayscale, and local geometry, but more importantly, it incorporates, through learning, the spatial and feature relationships between that point and all other points within the retinal scan range. For example, a point's deep features can reflect whether it is located on the edge of an isolated aberrant reflective point or a large area of ​​fluid; this contextual information is crucial for distinguishing lesions from artifacts.

[0079] Global dependencies, on the other hand, are the final summary of the deep features of all points in the entire retinal scan area, forming a feature vector that describes the overall structural characteristics of the 3D scene. This feature vector can macroscopically characterize whether the current retina is in a "healthy state with clear structural layers" or a "pathological state with large-scale structural disorder".

[0080] Optionally, step 102 may specifically include the following steps:

[0081] Step 1021: Extract the initial position features, initial geometric features, and initial grayscale features of each point from the enhanced point cloud.

[0082] The initial positional features are the (x, y, z) coordinates of each point in the three-dimensional space of the fundus. These coordinates are inherited from the structured light data that constitutes the enhanced point cloud, precisely defining its physical location within the retinal structure. The initial geometric features also originate from the three-dimensional point cloud contours provided by the structured light data, describing the morphology of the local micro-surface where the point is located. The initial grayscale features are the optical reflectance intensity values ​​fused from the optical coherence data during the generation of the enhanced point cloud; these are the core criteria for distinguishing different tissue types.

[0083] Step 1022: Concatenate the initial position features, initial geometric features, and initial grayscale features to generate an initial feature vector.

[0084] The initial feature vector is a digital archive formed by integrating various basic information of a single point in the enhanced point cloud.

[0085] Step 1023: Map the initial feature vector to a high-dimensional feature space through a mapping network to obtain the high-dimensional feature value of each point.

[0086] The mapping network is a specially trained neural network that learns to transform the "digital archive" of each point in a point cloud into a more abstract and classification-friendly representation. For example, in the context of fundus lesion analysis, this network can learn to map points with similar pathological significance (even if the original feature values ​​differ significantly) to neighboring locations in the feature space. The high-dimensional feature space is an abstract mathematical space in which the features of different types of retinal tissue can be more clearly distinguished. High-dimensional feature values ​​are the new, more discriminative feature representations obtained for each point in this high-dimensional space after processing by the mapping network.

[0087] Step 1024: Calculate the association strength value between each point in the enhanced point cloud and all remaining points. Based on the association strength value, perform weighted aggregation on the high-dimensional feature values ​​of all points to generate deep features for each point.

[0088] The association strength value is a dynamically calculated weight used to quantify the mutual importance between any two points in the entire augmented point cloud. Weighted aggregation is a key step in generating deep features. It fuses the features of a point with the features of its associated points. The higher the association strength, the greater the weight of the feature information in the aggregation process.

[0089] Step 1025: Globally aggregate the deep features of all points in the enhanced point cloud to generate global dependencies.

[0090] The global aggregation operation aims to summarize the deep feature information of all points and compress it into a single, fixed-length global feature vector, which captures the core structural features of the entire point cloud scene from a macroscopic perspective.

[0091] In this embodiment, a comprehensive judgment is made by integrating the local information of each data point and its global environment within the overall structure. The overall process is as follows: First, an initial feature vector containing its spatial location, local geometry, and tissue reflectivity is established for each point in the enhanced point cloud. Then, this vector is mapped into a more abstract and discriminative high-dimensional feature value through a mapping network. Next, a self-attention mechanism is used to calculate the correlation strength value between each point and all other points within the scanning range, and this value is used to weight and aggregate the high-dimensional feature values ​​of all points to form a deep feature containing global contextual information. Finally, the deep features of all points are globally aggregated to extract a global dependency that can represent the current macroscopic state of the entire retinal scanning area.

[0092] As an example, taking a local retinal scan area containing neovascular lesions as an example, three representative points {P1, P2, P3} are selected from the enhanced point cloud to illustrate the process. Here, P1 and P2 are two adjacent points located within the same subretinal fluid accumulation area caused by neovascular exudation, while P3 is a point on a distant healthy retinal photoreceptor layer.

[0093] First, steps 1021 and 1022 are performed to extract and stitch together the initial features of each point. Points in the fluid accumulation region (P1, P2) typically have high initial grayscale features, similar initial position features, and consistent initial geometric features with the same orientation. Points in the healthy photosensitive layer (P3) exhibit different characteristics in these features. Specific features are shown in Table 1 below:

[0094] Table 1 Feature extraction table for points P1-P3

[0095] point Scene Description Initial position features (x, y, z) Initial geometric features (nx, ny, nz) Initial grayscale features (g) P1 Inside the fluid accumulation area (0.1,0.2,0.3) (0,0,1) 205 P2 Inside the fluid accumulation area, adjacent to P1 (0.1,0.3,0.3) (0,0,1) 210 P3 Healthy photosensitive layer (0.8,0.7,0.6) (0,1,0) 50

[0096] Next, step 1023 is executed, in which a mapping network pre-trained on a large amount of labeled fundus lesion data is used to map these 7-dimensional initial feature vectors to a 64-dimensional high-dimensional feature space to obtain the high-dimensional feature value of each point.

[0097] Next, step 1024 is executed to generate deep features for each point. This process is implemented through a self-attention mechanism, specifically including:

[0098] The first step is to calculate the association strength value: The model learns to generate three different role vectors for each point: query vector, key vector, and value vector. When calculating the association strength of point P1 with other points, the model calculates the matching degree between P1's query vector and the key vectors of all points, including P1 itself. The higher the matching degree, the more related the two points are in terms of structure and features. In the example above, the query vector of P1 (fluid accumulation area) has a high matching degree with the key vector of P2 (also a fluid accumulation area), while the matching degree with the key vector of P3 (healthy tissue) is extremely low. By normalizing all matching degree scores, the association strength value of P1 with all other points can be obtained.

[0099] The second step is weighted aggregation to generate deep features: The model collects the "value" vectors of all points and performs a weighted sum based on the association strength values ​​calculated in the previous step. For P1, since its association strength with P2 is very high, the value vector information of P2 will have a large weight in the generation of new features for P1; while since its association strength with P3 is almost zero, the value vector information of P3 will be ignored. The result of this weighted sum becomes the new deep feature of P1. This newly generated deep feature not only describes point P1 itself, but more importantly, it numerically encodes the relationship information that point P1 belongs to the same category as point P2, but to a different category than point P3.

[0100] Finally, step 1025 is executed to globally aggregate the deep features of all points, generating global dependencies. This step aims to perform a final aggregation of the deep features of all points to form a macroscopic description of the entire scene. A commonly used aggregation method is max-pooling, which iterates through each dimension of all deep feature vectors and selects the maximum value among all vectors for that dimension of the new vector. Through this operation, a completely new feature vector of fixed length is finally constructed, namely the "global dependency". This vector serves as a global descriptor, and its value can characterize the overall structural properties of the scene, such as whether there is large-scale structural disorder and abnormally high reflectivity signals, thus providing a basis for subsequent segmentation models.

[0101] The above example is only one example of this application. In practical applications, the dimension of features, the specific structure of the network, and the aggregation method can all be adjusted according to the requirements. This application does not limit them.

[0102] This application starts with the raw, isolated features of each data point and utilizes a deep learning network to learn the complex interactions between points, thereby generating deep features rich in contextual information for each point and ultimately extracting the global dependencies describing the entire point cloud. This method enables the model to transcend the limitations of local information, understand the retinal structure from a global perspective, and significantly enhances the ability to identify lesions with diverse shapes and blurred boundaries.

[0103] Step 103: Input the global dependency relationship into the pre-built segmentation model, and the segmentation model performs 3D instance segmentation based on the global dependency relationship to obtain the initial segmentation result.

[0104] In this application's context, the segmentation model is a deep learning network module trained on a large amount of fundus lesion data. Its core task is to assign a category label to each point in the enhanced point cloud, i.e., to determine whether the point belongs to normal retinal tissue or a specific lesion. 3D instance segmentation is a segmentation task that not only identifies all points belonging to lesions (semantic segmentation) but also further distinguishes which independent lesion instance these points belong to. For example, if there are two unconnected fluid accumulation areas on the retina, instance segmentation needs to label them as "lesion instance 1" and "lesion instance 2," rather than simply labeling them as "lesion." The initial segmentation result is a preliminary lesion identification map output by the segmentation model, which labels one or more regions considered to be independent lesions in 3D form, but this result has not yet been finely calibrated for anatomical structures.

[0105] Optionally, step 103 may specifically include the following steps:

[0106] Step 1031: Receive global dependencies through a pre-built segmentation model.

[0107] Step 1032: The global dependency relationship is fused with the deep features of each point to generate enhanced point features.

[0108] The enhanced point features are generated by combining global dependencies, representing global structural information, with deep features representing local context information. This fusion operation ensures that the final feature representation of each point includes both its local environment information and the macroscopic state information of the entire scanning scene.

[0109] Step 1033: Based on the enhanced point features, assign an initial segmentation label to each point. The initial segmentation label represents the probability value that the point belongs to the lesion area.

[0110] The initial segmentation label is not a final "yes" or "no" judgment, but a probability value between 0 and 1. This probability value quantifies the model's confidence in "how likely this point is part of a lesion." For example, a label of 0.98 means that the model is 98% confident that it belongs to a lesion.

[0111] Step 1034: Based on the spatial continuity of points, classify points whose spatial distance is less than a preset threshold and whose probability values ​​are similar to obtain candidate lesion regions.

[0112] The candidate lesion regions are formed through a preliminary clustering process, consisting of an initial set of high-probability points. This process is based on the fundamental assumption that points belonging to the same lesion should be spatially proximate and have similar probabilities of belonging to the lesion.

[0113] Step 1035: Merge spatially adjacent candidate lesion regions with similar enhancement point features to obtain the initial segmentation result.

[0114] The purpose of this step is to refine the candidate lesion regions generated in the previous step, in order to address the problem that a single large lesion may be incorrectly segmented into multiple smaller regions. By comparing the intrinsic feature similarity of adjacent candidate regions, the model can decide whether to merge them to reconstruct the complete lesion structure.

[0115] In this embodiment, the segmentation process follows an analysis flow from global to local, and then to instance. First, the model combines global dependencies representing the global situation with the deep features of each point to generate more complete enhanced point features for each point. Then, based on these enhanced point features, the model calculates a probability that each point belongs to a lesion. Next, points that are spatially adjacent and have a probability value higher than a preset threshold are clustered to form several candidate lesion regions. Finally, the model evaluates these adjacent candidate lesion regions; if their intrinsic features are highly similar, they are merged to form more accurate lesion instances, thus obtaining the initial segmentation result.

[0116] As an example, we continue using the retinal scan data that previously contained neovascularization lesions. This data shows a large subretinal fluid accumulation area (lesion A) and another separate, smaller drusen (lesion B).

[0117] First, by executing steps 1031 and 1032, the model summarizes the global dependency of "existing large-scale structural disorder" and fuses it with the deep features of each point. For points within the fluid accumulation area, their deep features already contain information about "similar clustering," and combined with the global "abnormal" signal, their final enhanced point features will strongly point to the "lesion" category.

[0118] Next, step 1033 is executed, where the model calculates probabilities based on the enhanced point features. It is assumed that points within the effusion area (lesion A) may obtain a probability value of 0.9 to 0.98, points on drusen (lesion B) may obtain a probability value of 0.85 to 0.95, while the probability of points in healthy areas is less than 0.01.

[0119] Next, step 1034 is executed, where the model performs classification. All points that are spatially close to each other and have a probability value higher than a certain threshold (e.g., 0.8) are grouped together. Because the fluid accumulation region (lesion A) is relatively large, it may be incorrectly segmented into two spatially adjacent sets at this step due to minor internal feature differences; these sets are called "candidate regions A1" and "candidate regions A2". Meanwhile, the isolated drusen (lesion B) is correctly classified into a single set, called "candidate region B".

[0120] Finally, step 1035 is executed, and the model performs a merging check. The model detects that "candidate region A1" and "candidate region A2" are spatially close. Subsequently, the model compares the average enhancement features of all points in these two regions. If the features are found to be highly similar, it indicates that they are likely to belong to the same pathological structure. Therefore, the model merges A1 and A2 into one large region, namely the complete lesion A. At the same time, the model checks candidate region B, which remains independent as it has no other candidate regions around it, becoming lesion B. Thus, the initial segmentation result is generated, successfully identifying two independent lesion instances: a large effusion area A and a small drusen B.

[0121] The above example is only one example of this application. In practical applications, probability thresholds, spatial distance thresholds, and criteria for judging feature similarity can all be set according to requirements. This application does not limit them.

[0122] This application can not only mark the location of lesions, but also accurately distinguish each individual lesion instance within the field of view, providing a reliable basis for subsequent detailed analysis and quantitative evaluation.

[0123] Step 104: During the segmentation process, a geometric constraint function corresponding to the prior model of the normal retinal anatomy is introduced. The geometric constraint function uses the geometric topology of the prior model as a reference boundary to adjust the initial segmentation result, so that the spatial relationship between the optimized lesion boundary and the normal retinal tissue conforms to the anatomical structure, and the target segmentation result is obtained.

[0124] In this application's context, the prior model is a pre-constructed, standardized three-dimensional digital model of a healthy retina. This model accurately depicts the standard morphology and relative positions of various physiological layers of the retina (such as the internal limiting membrane, ganglion cell layer, and retinal pigment epithelium) under normal conditions. The geometric constraint function is a mathematical function designed based on this prior model, which quantifies the spatial deviation between the initially segmented lesion boundary and the corresponding anatomical structure. The target segmentation result is the final lesion contour after geometric constraint optimization, with smoother and more regular boundaries, and its spatial relationship with the surrounding normal tissue conforms to the anatomical prior.

[0125] Optionally, step 104 may specifically include the following steps:

[0126] Step 1041: Construct a priori model of the normal anatomical structure of the retina. The priori model includes smooth curved surface models of each layer of the retina.

[0127] By statistically analyzing and modeling fundus scan data from a large number of healthy individuals, a set of smooth three-dimensional surfaces representing the various layers of the standard retina were generated.

[0128] Step 1042: Construct a geometric constraint function based on the geometric topology of the prior model. The geometric constraint function is used to calculate the spatial distance between the lesion boundary point and the corresponding retinal layer of the prior model.

[0129] This function can receive any point on the initial lesion boundary and calculate the shortest spatial distance from that point to the retinal layer surface that it should be closest to in the healthy prior model.

[0130] Step 1043: Correct each lesion boundary point in the initial segmentation result using a geometric constraint function to obtain the corrected offset of each lesion boundary point.

[0131] The model iterates through every point on the initial lesion boundary and uses geometric constraint functions to calculate the distance and direction of the point's deviation from the normal anatomical position. The result of this calculation is the corrected offset.

[0132] Step 1044: Project and smooth the lesion boundary points based on the corrected offset to obtain the intermediate lesion boundary.

[0133] The intermediate lesion boundary is the lesion outline after preliminary correction. It is obtained by moving each point on the boundary according to its own correction offset and smoothing it to eliminate irregular mutations, thus obtaining a transitional boundary that is closer to the anatomical morphology.

[0134] Step 1045: Minimize the geometric difference between the intermediate lesion boundary and the prior model, and preserve the feature consistency of points inside the lesion to obtain the target segmentation result.

[0135] This step aims to achieve a joint optimization process that minimizes geometric differences while maintaining internal feature consistency. It involves a joint optimization objective that considers two aspects: first, the boundary should closely match the prior model; second, the region defined by the boundary must still contain highly consistent lesion tissue. The boundary position is iteratively fine-tuned until the joint optimization objective is optimal; this boundary represents the final target segmentation result.

[0136] Step 1045 may specifically include the following steps:

[0137] The distance difference is calculated based on the spatial distance from each point on the boundary of the intermediate lesion to the prior model; the feature similarity is calculated based on the deep features of the points within the region enclosed by the boundary of the intermediate lesion; the distance difference and the feature similarity are weighted and summed to construct a joint optimization objective; the three-dimensional position of each point on the boundary of the intermediate lesion is adjusted so that the value of the joint optimization objective continuously decreases until the preset convergence condition is reached; the region formed by all points on the boundary of the intermediate lesion when the preset convergence condition is reached is determined as the target segmentation result.

[0138] As an example, let's continue analyzing the larger subretinal fluid effusion area (lesion A) identified earlier. The initial segmentation results given in step 103 may have rough boundaries, and may even have erroneously invaded the retinal pigment epithelium (RPE) layer in some places.

[0139] First, steps 1041 and 1042 are executed. The model loads a smooth surface representing a healthy RPE layer as a prior model and constructs a geometric constraint function that can calculate the distance from any point to this surface.

[0140] Next, steps 1043 and 1044 are performed, and the model checks the initial boundary of lesion A. For a boundary point that is incorrectly recessed into the RPE layer, the geometric constraint function calculates an outward-pointing correction offset. Conversely, for an excessively convex boundary point, an inward-pointing offset is calculated. After applying these corrections to all boundary points and smoothing, a more regular lesion boundary with no obvious intersection with the RPE layer is obtained.

[0141] Finally, step 1045 is executed to perform joint optimization. This is an iterative process to find the optimal boundary position.

[0142] The first step is to construct a joint optimization objective: the model establishes an objective function consisting of two weighted terms. The first term is the "geometric difference term," which measures the degree of fit between the current boundary and the prior model of the RPE layer. The second term is the "feature consistency term," which measures whether the features of all points within the region enclosed by the current boundary still consistently represent "liquid accumulation."

[0143] The second step is iterative boundary adjustment: the model iteratively adjusts the position of the intermediate lesion boundary to minimize the value of the joint optimization objective function. For example, it attempts to move a small segment on the boundary inward to make it closer to the RPE prior model.

[0144] The third step is to evaluate the effect of the adjustment: After the shift, the model recalculates the joint optimization objective function value. If the adjustment significantly reduces the geometric difference term while the feature consistency term decreases only slightly, the total objective function value decreases, and the adjustment is accepted. Conversely, if the adjustment results in a large number of points with strong effusion characteristics being excluded from the boundary, causing a sharp decrease in the feature consistency term and an increase in the total objective function value, the adjustment will be rejected or the adjustment magnitude will be reduced.

[0145] The fourth step is to achieve convergence: the model continuously makes such fine adjustments at various points on the boundary until the objective function value no longer decreases significantly, thus meeting the convergence condition.

[0146] The final stable boundary at this point is the "target segmentation result." It is smooth and conforms to the anatomical structure, accurately encompassing the actual lesion area, thus achieving an effective combination of data-driven segmentation results and prior anatomical knowledge.

[0147] The above example is only one example of this application. In practical applications, the specific construction method of the prior model, the weight of each item in the optimization objective, etc. can be set according to the requirements. This application does not limit this.

[0148] This application optimizes segmentation boundaries by introducing prior anatomical knowledge as a constraint, effectively solving the problems of irregular boundaries and non-biological consistency that may arise from data-driven segmentation methods alone. This method can generate smoother, more accurate, and anatomically meaningful lesion boundaries while ensuring that the main lesion is not erroneously removed, greatly improving the quality and clinical interpretability of the segmentation results.

[0149] Step 105: Based on the target segmentation results, identify and locate key points belonging to fundus lesions in the enhanced point cloud.

[0150] In the context of this application, a key point refers to one or more points that best represent the core characteristics of the lesion in terms of features and space. The features and location of this point can highly summarize the condition of the entire lesion and can serve as a representative anchor point for the lesion.

[0151] Optionally, step 105 may specifically include the following steps:

[0152] Step 1051: Extract all points marked as lesions from the target segmentation results as a candidate lesion point set.

[0153] All points within all lesion regions that are finally determined in step 104, have smooth boundaries, and conform to anatomical structures are extracted to form a candidate point set for identifying key points.

[0154] Step 1052: Calculate the feature similarity and spatial proximity between any two candidate lesion points. The feature similarity is determined by the closeness of the deep feature representations of the two points in the feature space, and the spatial proximity is determined by the Euclidean distance between the two points in three-dimensional space.

[0155] Feature similarity measures the degree of similarity between two candidate points in terms of pathological properties, and is achieved by calculating the distance between the deep features of the two points in the feature space. Spatial proximity measures the distance between two points in three-dimensional physical space, usually represented by Euclidean distance.

[0156] Step 1053: Weighted fusion of feature similarity and spatial proximity is performed to generate comprehensive affinity.

[0157] Among them, the comprehensive affinity is a comprehensive measure that evaluates the correlation between two points in terms of features and space by weighted summation of feature similarity and spatial proximity.

[0158] Step 1054: Points with a comprehensive affinity higher than the set threshold are grouped into the same region to generate lesion regions.

[0159] By performing an internal clustering of the candidate lesion set, the core region of the lesion can be identified. It groups together points with sufficiently high overall affinity, thereby helping to filter out some points located at the edge of the lesion that have less typical characteristics.

[0160] Step 1055: Within each lesion area, select points with high average comprehensive affinity to other candidate lesion points in the same area as key points of fundus lesions.

[0161] This step is the final keypoint selection process. For each core lesion region generated in the previous step, the model calculates the average comprehensive affinity of each point within it with all other points in the region. The point with the highest average comprehensive affinity is selected as the keypoint for that lesion because it has the strongest overall correlation with other points in the region, both in terms of features and spatiality.

[0162] In this embodiment, the process of identifying key points aims to locate the most representative core points within a segmented, complete lesion region. This process comprehensively considers the spatial location and characteristic attributes of the points to ensure that the selected key points accurately reflect the core characteristics of the entire lesion.

[0163] As an example, let's continue with that refined subretinal fluid accumulation area (lesion A).

[0164] First, step 1051 is executed, in which the model extracts all points within the lesion A region to form a candidate lesion point set.

[0165] Next, steps 1052 and 1053 are executed, and the model calculates the overall affinity between any two points within the point set. For example, two points located deep within the fluid accumulation area will have a very high overall affinity due to their close physical distance and similar deep features. Conversely, a point located at the edge of the fluid accumulation, close to normal tissue, will have a relatively low overall affinity compared to a point located deep within the fluid accumulation area.

[0166] Next, step 1054 is executed, where the model sets a high affinity threshold and performs internal clustering. Only those points that maintain a high overall affinity with a large number of neighboring points are retained, forming a "core effusion region" that is slightly smaller in size but has highly consistent internal characteristics.

[0167] Finally, step 1055 is performed to select key points within the "core effusion area". The model iterates through every point within the core area, calculating its average comprehensive affinity with all other points within the core area. A point located near the geometric center of the core area and with highly typical characteristics will typically have the highest average comprehensive affinity. The point that obtains the highest average comprehensive affinity score is identified as the key point for this effusion lesion (lesion A).

[0168] The above example is only one example of this application. In practical applications, the weights and screening thresholds for affinity calculation can be set according to requirements, and this application does not limit them.

[0169] This application employs a clustering and selection mechanism that comprehensively considers spatial location and feature similarity, enabling robust and accurate localization of key points that best represent the core characteristics of a complete lesion region. This method simplifies complex area analysis into point analysis, significantly improving the efficiency of subsequent quantitative calculations and providing a stable and representative anchor point for long-term lesion tracking and efficacy evaluation.

[0170] Step 106: Output the 3D mask for each key point, and calculate the volume, surface area, and spatial position of the fundus lesion relative to the retinal layer based on the 3D mask.

[0171] In this application's context, the 3D mask refers to the final target segmentation result produced in step 104, representing a single complete lesion region. This 3D mask precisely defines the spatial extent of the lesion. The 3D curved mesh is a geometric representation created for the surface of this 3D mask, composed of multiple tiny polygons (such as triangles). It accurately describes the surface morphology of the lesion and serves as the basis for geometric measurements.

[0172] Optionally, step 106 may specifically include the following steps:

[0173] Step 1061: A closed three-dimensional surface mesh is generated by connecting each key point, and the points inside the three-dimensional surface mesh are marked as lesion areas to obtain a three-dimensional mask.

[0174] It is important to clarify that the "3D mask" here refers to the target segmentation result obtained in step 104. The specific execution process of this step is as follows: a closed 3D surface mesh is generated by connecting all the boundary points of the target segmentation result. This mesh completely encloses the lesion area, and all points within it are ultimately identified as lesions.

[0175] Step 1062: Based on the grid vertex coordinates of the 3D mask, calculate the sum of the areas of all grid faces as the surface area of ​​the fundus lesion.

[0176] The surface area of ​​a lesion is the total area of ​​its interface with surrounding tissues. After generating a three-dimensional curved mesh of the lesion, the model can accurately calculate the area of ​​each tiny polygon that makes up the mesh, and then sum the areas of all these polygons to obtain the total surface area of ​​the lesion.

[0177] Step 1063: Discretize the internal space of the three-dimensional mask into multiple small cubes, and count the total volume of the small cubes as the volume of the fundus lesion.

[0178] The volume of a lesion is the size of the space it occupies. The volume calculation process involves discretizing the internal space enclosed by the three-dimensional curved mesh into multiple voxels (i.e., small cubes) of standard size. By counting the total number of voxels required to fill the space and then multiplying it by the volume of a single voxel, the total volume of the entire lesion can be accurately calculated.

[0179] Step 1064: Using the internal limiting membrane of the retina as a reference plane, calculate the three-dimensional spatial offset of the centroid of the three-dimensional mask from the reference plane to determine the spatial position of the fundus lesion relative to the retinal layer.

[0180] To provide an anatomically meaningful location description of the lesion, a stable anatomical structure (such as the inner limiting membrane of the retina) is needed as a reference plane. The model then uses the key point identified in step 105 that best represents the core of the lesion as the lesion's location point and precisely calculates the three-dimensional spatial offset of this location point relative to the reference plane. This offset provides a precise description of the lesion's location, such as "located below the fovea, 150 micrometers below the retina."

[0181] In this embodiment, the final quantification process involves comprehensive geometric parameter measurement and spatial localization of the identified lesions. This process includes: generating an accurate three-dimensional surface model (three-dimensional curved surface mesh) for the lesion, calculating its surface area and internal volume, and determining the precise coordinates of its core representative point relative to the anatomical reference plane.

[0182] As an example, let's continue with the subretinal fluid accumulation area (lesion A) that was eventually precisely segmented.

[0183] First, step 1061 is executed, and the model retrieves the target segmentation result of lesion A. Then, all the points constituting the boundary of lesion A are connected to generate a closed three-dimensional curved surface mesh composed of thousands of tiny triangles that completely encloses the entire fluid accumulation area.

[0184] Next, step 1062 is executed. Based on the newly generated curved mesh, the model calculates the area of ​​each triangle one by one, and sums up all the area values ​​to finally obtain a numerical value. For example, the surface area of ​​lesion A is 1.2 square millimeters. This value can reflect the extent of the lesion's influence.

[0185] Next, step 1063 is executed, where the model calculates the space inside the curved mesh and, through voxel summation, determines that the total volume of lesion A is 0.5 cubic millimeters. This value is a key indicator for assessing the severity of fluid accumulation.

[0186] Finally, step 1064 is executed, where the model retrieves the coordinates of the unique keypoint found for lesion A in step 105. Simultaneously, the location of the internal limiting membrane (ILM) is obtained from the prior model used in step 104 as a reference plane. By calculating the three-dimensional vector from the keypoint to the ILM reference plane, the model can output a precise localization report: "Lesion A, with its core representative point located at reference coordinates (x, y, z), at a depth of 150 micrometers below the internal limiting membrane."

[0187] Through the above steps, a complete quantitative report on lesion A is generated, which includes the lesion's surface area, volume, and precise spatial location, providing objective data support for clinical diagnosis and treatment.

[0188] The above example is only one example of this application. In practical applications, the selection of the reference plane, the specific area and volume calculation algorithms, etc. can be set according to the requirements, and this application does not limit them.

[0189] This application successfully transforms qualitative, image-based lesion identification into a series of quantitative, clinically significant objective indicators through precise geometric measurement and localization of the final segmentation results. This not only provides doctors with a reliable basis for assessing the severity and extent of lesions but also lays a solid data foundation for subsequent automated disease grading, efficacy comparison, and long-term disease progression tracking.

[0190] Figure 2 This application provides a schematic diagram of the structure of a segmentation system for fundus lesion images, as shown below. Figure 2 As shown, the system includes:

[0191] The acquisition module 21 is used to acquire optical coherence data and structured light data of the retina, and to register and fuse the optical coherence data and structured light data into an enhanced point cloud;

[0192] Extraction module 22 is used to extract deep features of each point from the augmented point cloud by learning the interaction information between each point in the augmented point cloud and all remaining points, and to determine the global dependency relationship of the augmented point cloud based on the deep features of each point.

[0193] Input module 23 is used to input global dependencies into a pre-built segmentation model, which then performs 3D instance segmentation based on the global dependencies to obtain initial segmentation results.

[0194] The adjustment module 24 is used to introduce the geometric constraint function corresponding to the prior model of the normal retinal anatomy during the segmentation process. The geometric constraint function uses the geometric topology of the prior model as the reference boundary to adjust the initial segmentation result so that the spatial relationship between the optimized lesion boundary and the normal retinal tissue conforms to the anatomical structure, and the target segmentation result is obtained.

[0195] The identification module 25 is used to identify and locate key points belonging to fundus lesions in the enhanced point cloud based on the target segmentation results;

[0196] The calculation module 26 is used to output a three-dimensional mask for each key point, and to calculate the volume, surface area and spatial position of the fundus lesion relative to the retinal layer based on the three-dimensional mask.

[0197] Figure 2 A segmentation system for fundus lesion images can perform... Figure 1 The implementation principle and technical effects of the fundus lesion image segmentation method shown in the embodiment will not be elaborated further. The specific methods by which each module and unit performs operations in the fundus lesion image segmentation system of the above embodiment have been described in detail in the embodiments related to this method, and will not be elaborated further here.

[0198] In one possible design, Figure 2 The segmentation system for fundus lesion images shown in the embodiment can be implemented as a computing device, such as... Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0199] Storage component 31 stores one or more computer instructions, wherein one or more computer instructions are invoked and executed by processing component 32.

[0200] Processing component 32 is used for the above Figure 1 An embodiment of a method for segmenting fundus lesion images.

[0201] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0202] Storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0203] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.

[0204] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.

[0205] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.

[0206] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0207] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The illustrated embodiment provides a method for segmenting fundus lesion images.

[0208] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0209] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0210] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for segmenting images of fundus lesions, characterized in that, include: Acquire optical coherence data and structured light data from the retina, and register and fuse the optical coherence data and the structured light data into an enhanced point cloud; From the enhanced point cloud, deep features of each point are extracted by learning the interaction information between each point and all remaining points in the enhanced point cloud, and the global dependency relationship of the enhanced point cloud is determined based on the deep features of each point. The global dependency is input into a pre-built segmentation model, which then performs 3D instance segmentation based on the global dependency to obtain an initial segmentation result. During the segmentation process, a geometric constraint function corresponding to a priori model of the normal retinal anatomy is introduced. The geometric constraint function uses the geometric topology of the priori model as a reference boundary to adjust the initial segmentation result, so that the spatial relationship between the optimized lesion boundary and the normal retinal tissue conforms to the anatomical structure, and the target segmentation result is obtained. Based on the target segmentation results, key points belonging to fundus lesions are identified and located in the enhanced point cloud; Output the 3D mask for each key point, and calculate the volume, surface area, and spatial position of the fundus lesion relative to the retinal layer based on the 3D mask; From the enhanced point cloud, deep features of each point are extracted by learning the interaction information between each point and all remaining points in the enhanced point cloud, and the global dependencies of the enhanced point cloud are determined based on the deep features of each point, including: Extract the initial position features, initial geometric features, and initial grayscale features of each point from the enhanced point cloud; The initial position features, the initial geometric features, and the initial grayscale features are concatenated to generate an initial feature vector; The initial feature vector is mapped to a high-dimensional feature space through a mapping network to obtain the high-dimensional feature value of each point. Calculate the association strength value between each point in the enhanced point cloud and all remaining points, and perform weighted aggregation of the high-dimensional feature values ​​of all points based on the association strength value to generate deep features for each point; The deep features of all points in the enhanced point cloud are globally aggregated to generate global dependencies.

2. The method for segmenting fundus lesion images according to claim 1, characterized in that, The global dependencies are input into a pre-built segmentation model, which then performs 3D instance segmentation based on these global dependencies to obtain initial segmentation results, including: The global dependencies are received through a pre-built segmentation model; The global dependencies are fused with the deep features of each point to generate enhanced point features; Based on the enhanced point features, an initial segmentation label is assigned to each point, and each initial segmentation label represents the probability value that the corresponding point belongs to the lesion area. Based on the spatial continuity of points, points with spatial distances less than a preset threshold and similar probability values ​​are classified to obtain candidate lesion regions; The candidate lesion regions that are spatially adjacent and have similar enhancement point features are merged to obtain the initial segmentation result.

3. The method for segmenting fundus lesion images according to claim 1, characterized in that, During the segmentation process, a geometric constraint function corresponding to a priori model of the normal retinal anatomy is introduced. This geometric constraint function uses the geometric topology of the priori model as a reference boundary to adjust the initial segmentation result, ensuring that the spatial relationship between the optimized lesion boundary and normal retinal tissue conforms to the anatomical structure, thus obtaining the target segmentation result, including: Construct a priori model of the normal anatomical structure of the retina, the priori model including smooth curved surface models of each layer of the retina; Based on the geometric topology of the prior model, a geometric constraint function is constructed, which is used to calculate the spatial distance between the lesion boundary point and the corresponding retinal layer of the prior model. The geometric constraint function is used to correct each lesion boundary point in the initial segmentation result, and the correction offset of each lesion boundary point is obtained. Based on the corrected offset, the lesion boundary points are projected and smoothed to obtain the intermediate lesion boundary. The geometric difference between the intermediate lesion boundary and the prior model is minimized, while the feature consistency of points inside the lesion is preserved, in order to obtain the target segmentation result.

4. The method for segmenting fundus lesion images according to claim 3, characterized in that, Minimize the geometric difference between the intermediate lesion boundary and the prior model, while preserving the feature consistency of points within the lesion, to obtain the target segmentation result, including: The distance difference is calculated based on the spatial distance from each point on the boundary of the intermediate lesion to the prior model; The feature similarity is calculated based on the deep features of points within the area enclosed by the boundary of the intermediate lesion. The distance difference and the feature similarity are weighted and summed to construct a joint optimization objective; The three-dimensional position of each point on the boundary of the intermediate lesion is adjusted so that the value of the joint optimization objective is continuously reduced until the preset convergence condition is reached; The region formed by all points on the boundary of the intermediate lesion when the preset convergence condition is met is determined as the target segmentation result.

5. The method for segmenting fundus lesion images according to claim 1, characterized in that, Based on the target segmentation results, key points belonging to fundus lesions are identified and located in the enhanced point cloud, including: Extract all points marked as lesions from the target segmentation results, and use them as a candidate lesion point set; Calculate the feature similarity and spatial proximity between any two candidate lesion points. The feature similarity is determined by the closeness of the deep feature representations of the two points in the feature space, and the spatial proximity is determined by the Euclidean distance between the two points in three-dimensional space. The feature similarity and spatial proximity are weighted and fused to generate a comprehensive affinity. Points with a comprehensive affinity higher than a set threshold are grouped into the same region to generate lesion regions; Within each lesion region, points with high average comprehensive affinity to other candidate lesions in the same region are selected as key points for fundus lesions.

6. The method for segmenting fundus lesion images according to claim 1, characterized in that, Output a 3D mask for each key point, and calculate the volume, surface area, and spatial position relative to the retinal layer of the fundus lesion based on the 3D mask, including: A closed three-dimensional surface mesh is generated by connecting each key point, and the points inside the three-dimensional surface mesh are marked as lesion areas to obtain a three-dimensional mask; Based on the grid vertex coordinates of the three-dimensional mask, the sum of the areas of all grid faces is calculated as the surface area of ​​the fundus lesion; The internal space of the three-dimensional mask is discretized into multiple small cubes, and the total volume of the small cubes is counted as the volume of the fundus lesion; Using the internal limiting membrane of the retina as a reference plane, the three-dimensional spatial offset of the centroid of the three-dimensional mask from the reference plane is calculated to determine the spatial position of the fundus lesion relative to the retinal layer.

7. A segmentation system for fundus lesion images, used in the segmentation method for fundus lesion images according to any one of claims 1 to 6, characterized in that, include: The acquisition module is used to acquire optical coherence data and structured light data of the retina, and to register and fuse the optical coherence data and the structured light data into an enhanced point cloud; The extraction module is used to extract deep features of each point from the enhanced point cloud by learning the interaction information between each point in the enhanced point cloud and all remaining points, and to determine the global dependency relationship of the enhanced point cloud based on the deep features of each point. The input module is used to input the global dependency relationship into a pre-built segmentation model, and the segmentation model performs 3D instance segmentation based on the global dependency relationship to obtain an initial segmentation result; An adjustment module is used to introduce a geometric constraint function corresponding to a prior model of the normal retinal anatomy during the segmentation process. The geometric constraint function uses the geometric topology of the prior model as a reference boundary to adjust the initial segmentation result so that the spatial relationship between the optimized lesion boundary and the normal retinal tissue conforms to the anatomical structure, thereby obtaining the target segmentation result. The identification module is used to identify and locate key points belonging to fundus lesions in the enhanced point cloud based on the target segmentation results. The calculation module is used to output a three-dimensional mask for each key point, and to calculate the volume, surface area and spatial position of the fundus lesion relative to the retinal layer based on the three-dimensional mask.

8. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a method for segmenting fundus lesion images as described in any one of claims 1 to 6.

9. A computer storage medium, characterized in that, The device contains a computer program that, when executed by a computer, implements a method for segmenting fundus lesion images as described in any one of claims 1 to 6.

Images