OCT lesion simulation image generation method and device, equipment and storage medium

Abstract

The OCT lesion simulation image generation method provided by the application is based on the gray difference of different physiological levels of the lesion fundus, each image in the B-scan image sequence generated by the OCT point cloud data of the lesion fundus is segmented, the volume data of each physiological level of the lesion fundus is obtained, the volume data of each physiological level is respectively subjected to three-dimensional reconstruction, the three-dimensional model of the lesion fundus in the hierarchical level is obtained, the thickness change distribution graph of the physiological level where the lesion occurs is extracted from the three-dimensional model, the thickness change distribution graph is segmented, the lesion position is extracted and is subjected to down-sampling, the lesion weight graph used for describing the lesion shape and range is obtained, and the lesion database is constructed by using the lesion weight graph; the lesion weight graph is applied to the healthy fundus data, and the OCT lesion simulation image is generated. The lesion data saved by the application has a small volume, is easy to construct a large-scale local database, the generated lesion OCT image is intuitive and real, can be used for teaching and scientific research activities, and serves as a training sample for deep learning.

Description

Technical Field

[0001] This invention belongs to the field of medical imaging, and specifically proposes a method, device, equipment and storage medium for generating simulated images of fundus optical coherence tomography (OCT) lesions. Background Technology

[0002] OCT (Optical Coherence Toxicity) is a technology that uses infrared light to scan the surface of human tissues, acquiring depth information several millimeters below the tissue surface and further achieving three-dimensional imaging. As a non-contact, non-invasive optical detection method, OCT technology plays an important role in the diagnosis of ophthalmic diseases. In clinical ophthalmology practice, OCT technology is widely used to detect various retinal diseases, such as macular holes, serous retinal detachment, and age-related macular degeneration. By imaging different layers of the retina, observing disease-specific imaging characteristics, and evaluating biomarkers for diagnosis and prognosis, doctors can identify early lesions and develop treatment plans. However, despite the expanding clinical application of OCT, the scarcity of medical imaging data has become a bottleneck for education and research.

[0003] For medical education, abundant case data is crucial for mastering disease characteristics and diagnostic skills. However, some retinal diseases are relatively rare in clinical practice, making it difficult for medical students to access a sufficient number of real-world cases. Furthermore, the acquisition and sharing of medical images are limited by ethical approvals, privacy protection, and acquisition costs, further exacerbating the scarcity of OCT images. For deep learning research, the size and quantity of datasets are significant factors influencing model training. However, existing OCT image datasets for lesions are generally small in size and cover relatively limited disease types. For example, the Retouch dataset primarily focuses on the segmentation of intraretinal fluid, while the Retinal OCT-C8 dataset covers a limited range of lesion types, with relatively few samples of certain rare lesions. This scarcity of OCT image data directly limits the application of deep learning models in ophthalmic disease identification. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the purpose of this invention is to address the scarcity of ophthalmic OCT image data by providing a method, apparatus, device, and storage medium for generating simulated OCT lesion images. This invention can extract lesion features from OCT scan data of diseased fundus and synthesize them with OCT data of healthy fundus to generate simulated OCT lesion images, allowing for intuitive observation of the physiological changes caused by the lesions. This invention can also be used for medical training and generating training datasets.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The first aspect of this invention provides a method for generating OCT simulated images of fundus lesions, comprising the following steps:

[0008] OCT point cloud data of the diseased fundus is acquired. Based on the OCT point cloud data, a B-scan image sequence of the diseased fundus is generated. According to the gray-level differences of different physiological levels of the diseased fundus, each B-scan image in the B-scan image sequence is segmented to obtain volume data of each physiological level of the diseased fundus. Three-dimensional reconstruction is performed on the volume data of each physiological level to obtain a hierarchical three-dimensional model of the diseased fundus. The thickness variation distribution map of the physiological level where the lesion occurs is extracted from the hierarchical three-dimensional model of the diseased fundus, segmented, and the lesion location is extracted and downsampled to obtain a lesion weight map used to describe the shape and extent of the lesion. A lesion database is constructed using the lesion weight maps of different lesion types.

[0009] Obtain OCT point cloud data of healthy fundus, and obtain a hierarchical three-dimensional model of healthy fundus by processing it in the same way as the OCT point cloud data of diseased fundus. Select the lesion weight map corresponding to the target lesion from the lesion database, select the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical three-dimensional model of healthy fundus, and generate corresponding fundus tissue deformation in the hierarchical three-dimensional model of healthy fundus according to the lesion weight map corresponding to the target lesion to obtain OCT simulation image of fundus lesion.

[0010] In some embodiments, the step of segmenting each B-scan image in the B-scan image sequence based on the grayscale differences of different physiological levels of the diseased fundus to obtain volumetric data of each physiological level of the diseased fundus specifically includes:

[0011] Based on the grayscale differences of different physiological levels of the diseased fundus, each B-scan image in the B-scan image sequence is segmented to obtain the set of layer line coordinates between each physiological structure.

[0012] The layer lines of stable fundus physiological structures are selected as reference markers. The first B-scan image in the B-scan image sequence is used as the reference image. The average offset between the reference image and the reference markers in the remaining B-scan images is calculated. The remaining B-scan images are translated and adjusted using the average offset, and the coordinate set of the layer lines is updated.

[0013] For each B-scan image in the B-scan image sequence, the points between two adjacent layer lines are stored in the data structure of the corresponding fundus physiological structure to obtain the volume data of each physiological layer of the diseased fundus.

[0014] In some embodiments, the step of performing three-dimensional reconstruction on the volume data of each physiological level to obtain a hierarchical three-dimensional model of the diseased fundus specifically includes:

[0015] Transparency mapping parameters were set using the ray-throwing method for body data at each physiological level.

[0016] Set color mapping parameters to map physiological structures with grayscale differences to different colors;

[0017] All volume data are added to the same renderer and rendered to obtain a hierarchical 3D model of the diseased fundus.

[0018] In some embodiments, the step of extracting the thickness variation distribution map of the physiological level of the lesion from the hierarchical three-dimensional model of the diseased fundus, segmenting it, extracting the lesion location, and downsampling it to obtain a lesion weight map for describing the shape and extent of the lesion specifically includes:

[0019] The thickness of each physiological structure level is compared with prior knowledge to determine the level of lesion occurrence;

[0020] A thickness variation distribution matrix is ​​initialized, with its size consistent with the lateral plane resolution used in OCT scanning, to store the thickness variation distribution of each point in the lesion occurrence layer; the thickness of the lesion occurrence layer is extracted from the upper and lower layer lines of the lesion occurrence layer, and the thickness of the unaffected area in the lesion occurrence layer is used as the thickness threshold. When the thickness at a certain point is less than or equal to the thickness threshold, the value of the element corresponding to that point in the thickness variation distribution matrix is ​​set to 0; otherwise, the value of the element corresponding to that point in the thickness variation distribution matrix is ​​the difference between the thickness at that point and the thickness threshold, thereby obtaining the thickness variation distribution matrix of the lesion occurrence layer;

[0021] The thickness variation distribution matrix of the lesion occurrence level is regarded as a grayscale image and denoted as the thickness variation distribution image. It is binarized, and all connected components are checked. Connected components corresponding to healthy areas are removed, and connected components corresponding to lesion areas are retained. The minimum bounding rectangle surrounding the connected components corresponding to the lesion areas is determined. The value of the minimum bounding rectangle is set to 1, and the value of the remaining areas is set to 0. In this way, a lesion area mask is generated.

[0022] The thickness variation distribution map of the lesion layer is segmented using the lesion area mask to obtain the lesion distribution map;

[0023] The lesion distribution map is downsampled and grayscale normalized to obtain the lesion weight map. The normalized grayscale value of each point in the lesion weight map is recorded as the weight of each point.

[0024] In some embodiments, the pixel size of the lesion weight map is 128 pixels * 128 pixels.

[0025] In some embodiments, selecting the center location of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus 3D model, and generating corresponding fundus tissue deformation in the hierarchical healthy fundus 3D model according to the lesion weight map corresponding to the target lesion, specifically includes:

[0026] The selected lesion weight map is rotated and scaled based on the scanning parameters and the characteristics of the target lesion.

[0027] The center location of the lesion is selected from the depth plane corresponding to the target lesion in the hierarchical healthy fundus 3D model. The influence range of the simulated lesion is determined by combining the parameters of rotation and scaling operations on the selected lesion weight map.

[0028] The deformation intensity is set, and the deformation intensity is multiplied by the weight of each point in the rotated and scaled deformation weight map to obtain the displacement of each voxel within the simulated lesion influence range. The displacement is performed on each voxel within the simulated lesion influence range to achieve fundus tissue deformation.

[0029] In some embodiments, the process of deforming the fundus tissue further includes any one or more of the following post-processing steps:

[0030] ① Adjust the pixel color of the displacement point to simulate the change in signal reflection intensity of the lesion area in the B-scan image;

[0031] ② Voxel filling was performed on the cavities caused by displacement to simulate the fluid accumulation that may be caused by the lesion;

[0032] ③ The deformation intensity is multiplied by the Gaussian decay function to adjust the deformation intensity according to the distance from the lesion center, simulating the decay or displacement that may occur in healthy tissue when it is squeezed or deformed by the lesion.

[0033] A second aspect of the present invention provides an apparatus for generating OCT simulated images of fundus lesions based on any embodiment of the first aspect of the present invention, comprising:

[0034] The lesion database construction module is used to acquire OCT point cloud data of the fundus of the lesion, generate a B-scan image sequence of the fundus of the lesion based on the OCT point cloud data, segment each B-scan image in the B-scan image sequence according to the gray-level differences of different physiological levels of the fundus of the lesion, obtain volume data of each physiological level of the fundus of the lesion, and perform three-dimensional reconstruction on the volume data of each physiological level to obtain a hierarchical three-dimensional model of the fundus of the lesion; extract the thickness change distribution map of the physiological level of the lesion from the hierarchical three-dimensional model of the fundus of the lesion, segment it, extract the lesion location and downsample it to obtain a lesion weight map for describing the shape and extent of the lesion; and construct a lesion database using the lesion weight maps of different lesion types.

[0035] The OCT simulation image generation module for fundus lesions is used to obtain a hierarchical three-dimensional model of healthy fundus according to the same processing method as the OCT point cloud data of the diseased fundus. It selects the lesion weight map corresponding to the target lesion from the lesion database, selects the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical three-dimensional model of healthy fundus, and generates corresponding fundus tissue deformation in the hierarchical three-dimensional model of healthy fundus according to the lesion weight map corresponding to the target lesion, thereby obtaining an OCT simulation image of fundus lesions.

[0036] A third aspect of the present invention provides an electronic device comprising:

[0037] At least one processor, and a memory communicatively connected to said at least one processor;

[0038] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to perform the above-described method for generating OCT simulated images of fundus lesions.

[0039] The fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute the above-described method for generating simulated OCT images of fundus lesions.

[0040] Features and beneficial effects of the present invention:

[0041] This invention is based on high-definition OCT B-scan image segmentation, transforming point cloud data acquired by OCT equipment into fundus physiological structures and lesion entities. Based on this, lesions are extracted, and lesion simulations are performed on healthy images. By peeling away the physiological layers of the retina layer by layer, lesions are exposed and sampled, and the lesion distribution data is saved in grayscale image form. The lesion data can be applied to healthy fundus data through a program interface to generate simulated lesion OCT images. The saved lesion data is small in size, making it easy to build a large-scale local database; the generated lesion OCT images are intuitive and realistic, suitable for teaching and research activities, and can be used as training samples for deep learning. Attached Figure Description

[0042] Figure 1 This is a flowchart of an OCT lesion simulation image generation method provided in the first aspect embodiment of the present invention;

[0043] Figure 2 This is a flowchart illustrating the construction of a lesion database in an OCT lesion simulation image generation method provided in the first aspect embodiment of the present invention;

[0044] Figure 3 This is an OCT B-scan image of the fundus of a diseased eye generated from point cloud data of the diseased fundus in the first aspect embodiment of the present invention;

[0045] Figure 4 This is a flowchart illustrating the generation of OCT lesion simulation images based on a constructed lesion database in an OCT lesion simulation image generation method provided in the first aspect embodiment of the present invention.

[0046] Figure 5(a) is a three-dimensional model of a healthy fundus obtained in the first aspect embodiment;

[0047] Figure 5(b) is a three-dimensional model of the lesion fundus corresponding to Figure 5(a) generated by the first aspect embodiment of the present invention and stripped to the corresponding lesion occurrence level.

[0048] Figure 6 This is a schematic diagram of the structure of an electronic device provided in a third aspect embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for explaining this application and are not intended to limit this application.

[0050] Conversely, this application covers any alternatives, modifications, equivalent methods, and schemes made within the spirit and scope of this application as defined by the claims. Furthermore, to provide the public with a better understanding of this application, certain specific details are described in detail below. However, this application can be fully understood by those skilled in the art even without these detailed descriptions.

[0051] The first aspect of this invention provides a method for generating OCT images of fundus lesions, which will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] The overall flow of an OCT lesion simulation image generation method provided in the first aspect embodiment of the present invention is as follows: Figure 1 As shown, fundus lesions can be viewed as deformations of healthy physiological structures, which can be described using a deformation weight distribution. This distribution reflects the compressive intensity of the lesion on different parts of healthy tissue, and the weight map is a data structure that stores the deformation weight distribution. This method extracts the weight map from OCT scan data of diseased fundus lesions, constructs a lesion database, and applies the weight map to OCT scan data of healthy fundus lesions to generate OCT three-dimensional simulation images of the lesions.

[0053] See Figure 1 The first aspect of the present invention provides a method for generating OCT simulated images of fundus lesions, comprising the following steps:

[0054] Step S1: Lesion Database Construction: Obtain OCT point cloud data of the diseased fundus. Based on this OCT point cloud data, generate a B-scan image sequence of the diseased fundus. The gray-level layers in the B-scan image sequence correspond to different physiological levels of the fundus retina. According to the gray-level differences of different physiological levels of the diseased fundus, segment each B-scan image in the B-scan image sequence to obtain volume data of each physiological level of the diseased fundus. Perform three-dimensional reconstruction on the volume data of each physiological level to obtain a hierarchical three-dimensional model of the diseased fundus. Extract the thickness variation distribution map of the physiological level where the lesion occurs from the three-dimensional model of the diseased fundus, segment it, extract the lesion location and downsample it to obtain a lesion weight map used to describe the shape and extent of the lesion. Construct a lesion database using the lesion weight maps of different lesion types.

[0055] Step S2: Generation of OCT simulation images of fundus lesions: Obtain OCT point cloud data of healthy fundus, and obtain a hierarchical three-dimensional model of healthy fundus by processing it in the same way as the OCT point cloud data of diseased fundus. Select the lesion weight map corresponding to the target lesion from the lesion database, select the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical three-dimensional model of healthy fundus, and generate corresponding fundus tissue deformation in the hierarchical three-dimensional model of healthy fundus according to the lesion weight map corresponding to the target lesion to obtain OCT simulation images of fundus lesions.

[0056] In some embodiments, see Figure 2 Step S1 specifically includes:

[0057] Step S11: Use an OCT device to scan the fundus of the diseased eye and obtain OCT dense point cloud data of the fundus of the diseased eye.

[0058] Step S12: Generate a B-scan image sequence of the lesion from the OCT point cloud data of the lesion fundus, perform image segmentation on it, and obtain volumetric data of each physiological level of the lesion fundus. Specifically:

[0059] First, a B-scan image sequence was generated from the OCT point cloud data of the diseased fundus, see [link to documentation]. Figure 3 This is an OCT B-scan image of the fundus of a diseased eye. The image reveals the physiological structures of the fundus, including the various layers of the retina, the choroid, the sclera, and the lesion area. Significant gray-level differences exist between these physiological tissues. Based on these differences, B-scan image segmentation is performed. Possible image segmentation methods include graph-based boundary search methods and deep learning-based U-Net semantic segmentation methods. The B-scan image sequence is denoised and sharpened, and then segmented to obtain the coordinate set of the layer lines between each physiological structure. It is important to note that this embodiment does not segment the B-scan image according to the currently internationally accepted 10-layer retinal structure (as very thin structural layers are difficult to segment accurately). Instead, it incorporates pathological features, merging some layers without segmentation based on the significant gray-level differences between the physiological tissues. The majority of lesions fall within the actual segmentation layers, thus reducing the segmentation difficulty.

[0060] Next, image registration is performed: a relatively stable layer line of fundus physiological structure is selected as the reference marker for image sequence registration. In this embodiment, the registration reference layer line is the choroid-sclera interface (CSI). The first B-scan image in the B-scan image sequence is used as the reference image. The average CSI offset between the reference image and other B-scan images in the B-scan image sequence is calculated. The average offset is used to translate and adjust each B-scan image except the reference image, and the layer line coordinate set is updated to complete the image registration and improve the spatial consistency of the image sequence.

[0061] Finally, for each B-scan image in the B-scan image sequence, the points between two adjacent layer lines are stored in the data structure of the corresponding fundus physiological structure to obtain the volume data of each physiological layer of the diseased fundus.

[0062] Step S13: Reconstruct the physiological structures of the diseased fundus. The volumetric data of each physiological level of the diseased fundus obtained in Step S12 are reconstructed in three dimensions. Specifically, this embodiment uses volume rendering. Ray-casting is applied to the volumetric data of each physiological level. Transparency mapping parameters are set to make low-reflection signal areas, such as the transparent vitreous body and undetectable areas behind the sclera, completely transparent. Color mapping parameters are set to map physiological structures with grayscale differences to different colors. Finally, all volumetric data are added to the same renderer for rendering to obtain a hierarchical three-dimensional model of the diseased fundus.

[0063] Step S14: Identify the lesion level. Healthy fundus structures generally have uniform thickness, which can be used as a basis to determine whether a lesion has occurred layer by layer. The specific process is as follows: By controlling the rendering pipeline, the first layer of physiological structure is displayed separately. It is compared with prior knowledge to determine whether a lesion has occurred. Taking the lesion assessment of each layer in macular OCT as an example, the smoothness of the structure in each layer, any abnormal changes in the thickness of the physiological structure, and common lesion characteristics such as bulges caused by neovascularization are comprehensively considered. If no lesion has occurred, the next layer of physiological structure is displayed separately, and then it is determined whether a lesion has occurred in that next layer. If a lesion is determined, that layer is marked as the lesion level. If no lesion is determined, the above process is repeated until the lesion level is determined.

[0064] Step S15: Extraction of thickness variation distribution at the lesion level: Initialize a thickness variation distribution matrix, the size of which is consistent with the lateral plane resolution used by the OCT device for scanning. This matrix stores the thickness variation distribution of each point in the lesion level identified in step S14. In this embodiment, the thickness variation distribution matrix has a size of 512*512. For the physiological lesion level identified in step S14, extract the thickness variation distribution caused by the lesion. Extract the thickness of the lesion level between the upper and lower layer lines. Use the thickness of the unaffected area in the lesion level (which can also be understood as the original thickness of the lesion level before the lesion occurs) as the thickness threshold. When the thickness at a certain point is less than or equal to the thickness threshold, the value of the element corresponding to that point in the thickness variation distribution matrix is ​​0. When the thickness at a certain point is greater than the thickness threshold, the value of the element corresponding to that point in the thickness variation distribution matrix is ​​the difference between the thickness at that point and the thickness threshold, thus obtaining the thickness variation distribution matrix of the lesion level.

[0065] Step S16, Binarization and Connected Component Analysis: Treat the thickness variation distribution matrix of the lesion level as a grayscale image and record it as a thickness variation distribution map. Binarize it and examine all connected components. Healthy regions with thicknesses exceeding a threshold may exist due to factors such as blood vessels; these regions are reflected as smaller connected components in the binary image. Remove the connected components corresponding to healthy regions (this can be achieved by setting a connected component threshold; among all identified connected components, those with thicknesses less than the threshold are considered healthy regions, otherwise they are considered lesion regions), retaining the connected components corresponding to lesion regions. Then, use a minimum square bounding box to enclose the connected components corresponding to the lesion regions, setting the values ​​within the square region to 1 and the values ​​in the remaining regions to 0, thus generating a lesion region mask.

[0066] Step S17: The thickness variation distribution map is segmented using a lesion area mask to reduce the area of ​​invalid information and retain more lesion details. In this embodiment, the 512-pixel * 512-pixel thickness variation distribution map is segmented into a 267-pixel * 267-pixel lesion distribution map.

[0067] Step S18: To accelerate the generation of subsequent lesion images, the lesion distribution map needs to be downsampled to the target resolution and then grayscale normalized to convert it into a lesion weight map of a standardized size of 128 pixels * 128 pixels to describe the shape and extent of the lesion. The normalized grayscale value of each point in the lesion weight map is recorded as the weight of each point. The size of the lesion weight map is 2.61KB.

[0068] Step S19: For the remaining lesion types, refer to steps S11 to S18 to obtain lesion weight maps for different lesion types, and use them to construct a lesion database.

[0069] In some embodiments, see Figure 4 Step S2 specifically includes:

[0070] Step S21: Use an OCT device to scan the fundus of a healthy eye to obtain OCT dense point cloud data of the healthy fundus.

[0071] Step S22: Following the same method as step S12, generate a healthy B-scan image sequence from the OCT point cloud data of the healthy fundus, perform image segmentation on it, and obtain volume data of each physiological level of the healthy fundus.

[0072] Step S23: Following the same processing method as in step S13, reconstruct the physiological structure of the healthy fundus to obtain a hierarchical three-dimensional model of the healthy fundus.

[0073] Step S24: Select the lesion weight map corresponding to the target lesion from the lesion database. Taking into account the scanning parameters and the characteristics of the target lesion, rotate and scale the selected lesion weight map to ensure accurate simulation of the lesion area and orientation. Taking the simulation of central serous chorioretinopathy (CSCR) as an example, the scanning parameters are: a 6mm*6mm area in the macular region of the fundus, acquiring 512 B-scans of 512 pixels * 1044 pixels. For CCR, the lesion area is near the fovea. Considering the common area of ​​CCR and the resolution of this OCT scan, scaling the weight map to 256*256 is a reasonable choice. Rotate the weight map so that the position with the highest weight is aligned with the fovea, to match the lesion characteristics of CCR.

[0074] Step S25: Selecting the lesion generation location: From the hierarchical healthy fundus 3D model, select the center location of the lesion at the depth plane corresponding to the target lesion (i.e., in this application, it is assumed that there is a one-to-one correspondence between each lesion type and its occurrence level, i.e., the depth plane). Combine this with the parameters used in step S24 for rotating and scaling the selected lesion weight map to determine the influence range of the simulated lesion. In this way, the lesion is accurately embedded into the healthy fundus tissue, achieving precise spatial and directional control of the lesion area.

[0075] Step S26: Calculate the voxel displacement and deform the fundus tissue. Based on the scanning parameters and prior knowledge of the lesion's development stage, set the deformation intensity. Multiply this deformation intensity by the weight of each point in the rotated and scaled deformation weight map to obtain the displacement of each voxel within the simulated lesion's influence range. Perform displacement on each voxel within the simulated lesion's influence range to achieve fundus tissue deformation.

[0076] Step S27, Post-processing. Some optional post-processing operations are as follows: ① Adjust the pixel color of the displacement point to simulate the change in signal reflection intensity of the lesion area in the B-scan image. ② Fill the void area caused by displacement with voxels to simulate the fluid accumulation phenomenon that may be caused by the lesion. ③ Simulate the slight attenuation or displacement that may occur in healthy tissue when it is squeezed or deformed by the lesion. A Gaussian attenuation function can be used, multiplied by the deformation intensity set in step S26, to adjust the deformation intensity according to the distance from the center point of the lesion, so that the influence of the deformed area gradually weakens, achieves a natural transition between physiological levels, and ensures that the simulation results are more natural in space.

[0077] Step S28: Update the rendering pipeline. Add the deformed volume data back into the renderer and refresh the 3D model in the rendering window. The deformed fundus structure entity, i.e., the OCT simulation image of the fundus lesion, can be observed. Refer to Figure 5(a), which is the 3D model of the healthy fundus obtained in this embodiment. Figure 5(b) is the 3D model of the diseased fundus corresponding to Figure 5(a) generated in this embodiment, and it is stripped to the corresponding lesion occurrence level (i.e., the lesion level), where the generated lesions can be observed.

[0078] Understandably, this method, based on the physiological characteristics of fundus layering, segments OCT B-scan image sequences to extract physiological entities from point cloud data, constructing a hierarchical, materialized 3D fundus model. It then extracts fundus lesions and describes them using a small-scale data structure (i.e., a lesion weight map), which is beneficial for building a large lesion database and offers advantages in data volume. In the lesion simulation image generation stage, conventional 2D image processing methods can be used to scale and rotate the weight map to achieve flexible lesion simulation. Simulating lesions on the materialized 3D fundus model provides a realistic and intuitive visual effect. In teaching and research activities, this method can be used to simulate fundus lesions, solving the problem of difficult acquisition of fundus lesion images to some extent. In the field of deep learning, this method can generate training samples in batches, which helps train high-quality fundus lesion recognition models.

[0079] A second aspect of the present invention provides an apparatus for generating simulated OCT lesion images, comprising:

[0080] The lesion database construction module is used to acquire OCT point cloud data of the fundus of the diseased eye, generate a B-scan image sequence of the fundus of the diseased eye based on the OCT point cloud data, segment each B-scan image in the B-scan image sequence according to the gray-level differences of different physiological levels of the fundus of the diseased eye, obtain volume data of each physiological level of the fundus of the diseased eye, and perform three-dimensional reconstruction on the volume data of each physiological level to obtain a hierarchical three-dimensional model of the fundus of the diseased eye; extract the thickness change distribution map of the physiological level of the diseased eye from the three-dimensional model of the fundus of the diseased eye, segment it, extract the lesion location and downsample it to obtain a lesion weight map to describe the shape and extent of the lesion; and construct a lesion database using the lesion weight maps of different lesion types.

[0081] The OCT simulation image generation module for fundus lesions is used to acquire OCT point cloud data of healthy fundus, obtain hierarchical three-dimensional models of healthy fundus using the same processing method as OCT point cloud data of diseased fundus, select the lesion weight map corresponding to the target lesion from the lesion database, select the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical three-dimensional model of healthy fundus, and generate corresponding fundus tissue deformation in the hierarchical three-dimensional model of healthy fundus according to the lesion weight map corresponding to the target lesion, thereby obtaining OCT simulation images of fundus lesions.

[0082] It should be noted that the foregoing explanation of the embodiment of the method for generating OCT simulated images of fundus lesions also applies to the OCT simulated image generation device for fundus lesions in this embodiment, and will not be repeated here.

[0083] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing a computer program thereon, which is executed by a processor to perform the adaptive compensation method of the above embodiments.

[0084] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present invention. It should be noted that the electronic device in the embodiments of the present invention may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs, desktop computers, and servers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0085] like Figure 6As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 102 or a program loaded from a storage device 108 into a random access memory (RAM) 103. The RAM 103 also stores various programs and data required for the operation of the electronic device. The processing unit 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0086] Typically, the following devices can be connected to I / O interface 105: input devices 106 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, etc.; output devices 107 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 108 including, for example, magnetic tapes, hard disks, etc.; and communication devices 109. Communication device 109 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0087] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, this embodiment includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 109, or installed from a storage device 108, or installed from a ROM 102. When the computer program is executed by the processing device 101, it performs the functions defined in the methods of the embodiments of this disclosure.

[0088] It should be noted that the computer-readable medium described above in this invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0089] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0090] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the aforementioned method for generating simulated OCT images of fundus lesions.

[0091] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and Python, as well as conventional procedural programming languages ​​such as the "C-" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0093] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0094] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0095] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0096] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0097] Those skilled in the art will understand that implementing all or part of the steps of the methods in the above embodiments can be accomplished by instructing related hardware through a program. The developed program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0098] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0099] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. An OCT simulation image generation method for ocular fundus lesions, characterized by, The method comprises the following steps: Obtaining OCT point cloud data of a diseased fundus, generating a B-scan image sequence of the diseased fundus based on the OCT point cloud data, segmenting each B-scan image in the B-scan image sequence according to the gray difference of different physiological levels of the diseased fundus to obtain volume data of each physiological level of the diseased fundus, performing three-dimensional reconstruction on the volume data of each physiological level to obtain a three-dimensional model of the diseased fundus in different levels; extracting a thickness change distribution map of the physiological level where the disease occurs from the three-dimensional model of the diseased fundus in different levels, segmenting the thickness change distribution map, extracting a lesion position and performing downsampling to obtain a lesion weight map for describing the shape and range of the lesion; and constructing a lesion database by using the lesion weight maps of different lesion types; Obtaining OCT point cloud data of a healthy fundus, obtaining a three-dimensional model of the healthy fundus in different levels by the same processing method as the OCT point cloud data of the diseased fundus, selecting a lesion weight map corresponding to a target lesion from the lesion database, selecting a center position where the lesion occurs from a depth plane corresponding to the target lesion in the three-dimensional model of the healthy fundus in different levels, and generating a corresponding fundus tissue deformation in the three-dimensional model of the healthy fundus in different levels according to the lesion weight map corresponding to the target lesion to obtain an OCT simulation image of a fundus lesion.

2. The method of claim 1, wherein the method further comprises: The segmentation of each B-scan image in the B-scan image sequence according to the gray difference of different physiological levels of the diseased fundus to obtain volume data of each physiological level of the diseased fundus specifically comprises: Segmenting each B-scan image in the B-scan image sequence according to the gray difference of different physiological levels of the diseased fundus to obtain a set of delamination line coordinates between physiological structures of different layers; Selecting a delamination line of a stable fundus physiological structure as a reference mark, taking the first B-scan image in the B-scan image sequence as a reference image, calculating the average offset of the reference mark in the reference image and the remaining B-scan images, performing translation adjustment on each of the remaining B-scan images according to the average offset, and updating the set of delamination line coordinates; For each B-scan image in the B-scan image sequence, storing the points between two adjacent delamination lines into a data structure corresponding to the fundus physiological structure to obtain the volume data of each physiological level of the diseased fundus.

3. The method of claim 1, wherein the method further comprises: The three-dimensional reconstruction of the volume data of each physiological level to obtain a three-dimensional model of the diseased fundus in different levels specifically comprises: Applying a ray casting method to set transparency mapping parameters for the volume data of each physiological level; Setting color mapping parameters to map physiological structures with gray differences into different colors; Adding all the volume data to the same renderer for rendering to obtain the three-dimensional model of the diseased fundus in different levels.

4. The method of claim 1, wherein the method further comprises: The extraction of a thickness change distribution map of the physiological level where the disease occurs from the three-dimensional model of the diseased fundus in different levels, the segmentation of the thickness change distribution map, the extraction of a lesion position and the downsampling to obtain a lesion weight map for describing the shape and range of the lesion specifically comprise: The thickness of each hierarchical physiological structure is compared with prior knowledge to determine the lesion occurrence level; A thickness change distribution matrix is initialized, which has the same size as the lateral resolution of the OCT scan, and is used to store the thickness change distribution of each point in the lesion occurrence level; the thickness of the lesion occurrence level is extracted from the upper and lower layer lines of the lesion occurrence level, and the thickness of the area not affected by the lesion in the lesion occurrence level is taken as the thickness threshold; when the thickness of a certain point is less than or equal to the thickness threshold, the value of the element corresponding to the point in the thickness change distribution matrix is taken as 0, otherwise the value of the element corresponding to the point in the thickness change distribution matrix is taken as the difference between the thickness of the point and the thickness threshold, thereby obtaining the thickness change distribution matrix of the lesion occurrence level; The thickness change distribution matrix of the lesion occurrence level is regarded as a gray-scale image and is denoted as a thickness change distribution graph, which is binarized, all connected domains are checked, the connected domain corresponding to the healthy area is removed, the connected domain corresponding to the lesion area is retained, the smallest circumscribed rectangle surrounding the connected domain corresponding to the lesion area is determined, the value of the smallest circumscribed rectangle is set to 1, and the values of the remaining areas are set to 0, thereby generating a lesion area mask; The thickness change distribution graph of the lesion occurrence level is segmented using the lesion area mask to obtain a lesion distribution graph; The lesion distribution graph is down-sampled and gray-scale normalized to obtain a lesion weight graph, and the normalized gray value of each point in the lesion weight graph is denoted as the weight of each point.

5. The method of claim 4, wherein the method further comprises: The pixel size of the lesion weight graph is 128 pixels*128 pixels.

6. The method of claim 1, wherein the method further comprises: The center position of the lesion occurrence is selected from the depth plane corresponding to the target lesion in the hierarchical healthy fundus three-dimensional model, and the corresponding fundus tissue deformation is generated in the hierarchical healthy fundus three-dimensional model according to the lesion weight graph corresponding to the target lesion, which specifically includes: Rotating and scaling the selected lesion weight graph according to the scanning parameters and the characteristics of the target lesion; The center position of the lesion occurrence is selected from the depth plane corresponding to the target lesion in the hierarchical healthy fundus three-dimensional model, and the parameters of rotating and scaling the selected lesion weight graph are combined to determine the simulation lesion influence range; Setting the deformation intensity, multiplying the deformation intensity and the weight of each point in the rotated and scaled deformation weight graph to obtain the displacement amount of each voxel in the simulation lesion influence range, and traversing each voxel in the simulation lesion influence range to realize the deformation of the fundus tissue.

7. The method of claim 6, wherein the method further comprises: The realization of the deformation of the fundus tissue further includes any one or more of the following post-processing: ①Adjusting the pixel color of the displacement point to simulate the change of the signal reflection intensity of the lesion area in the B-scan image; ②Filling the hollow area generated due to displacement with voxels to simulate the effusion phenomenon that may be caused by the lesion; ③Multiplying the deformation intensity by a Gaussian decay function to adjust the deformation intensity according to the distance from the center position of the lesion, and simulating the attenuation or displacement of the healthy tissue when it is squeezed or deformed by the lesion.

8. An apparatus for generating an OCT simulation image of an ocular fundus lesion based on the method according to any one of claims 1 to 7, characterized in that The method comprises the following steps: A lesion database construction module is configured to acquire OCT point cloud data of a lesion fundus, generate a B-scan image sequence of the lesion fundus based on the OCT point cloud data, segment each B-scan image in the B-scan image sequence according to grayscale differences of different physiological levels of the lesion fundus to obtain volume data of each physiological level of the lesion fundus, perform three-dimensional reconstruction on the volume data of each physiological level respectively to obtain a three-dimensional model of the lesion fundus in hierarchical levels, extract a thickness change distribution map of a physiological level where a lesion occurs from the three-dimensional model of the lesion fundus in hierarchical levels, segment the thickness change distribution map, extract a lesion position and perform down-sampling to obtain a lesion weight map for describing a lesion shape and range, and construct a lesion database by using the lesion weight maps of different lesion types. An OCT simulation image generation module is configured to obtain a three-dimensional model of a healthy fundus in hierarchical levels in the same manner as the OCT point cloud data of the lesion fundus, select a lesion weight map corresponding to a target lesion from the lesion database, select a center position where a lesion occurs from a depth plane corresponding to the target lesion in the three-dimensional model of the healthy fundus in hierarchical levels, and generate a corresponding fundus tissue deformation in the three-dimensional model of the healthy fundus in hierarchical levels according to the lesion weight map corresponding to the target lesion to obtain an OCT simulation image of a fundus lesion.

9. An electronic device, comprising: Comprise: at least one processor, and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are configured to perform the fundus lesion OCT simulation image generation method in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the computer to perform the fundus lesion OCT simulation image generation method in any one of claims 1-7. The computer readable storage medium stores computer instructions for causing the computer to perform the fundus lesion OCT simulation image generation method in any one of claims 1-7.

Images