OCT lesion simulation image generation method and device, equipment and storage medium

Abstract

According to the OCT lesion simulation image generation method provided by the invention, each image in a B-scan image sequence generated by lesion fundus OCT point cloud data is segmented based on the gray difference of different physiological levels of the lesion fundus to obtain the volume data of each physiological level of the lesion fundus, and the volume data of each physiological level is subjected to three-dimensional reconstruction to obtain the OCT lesion simulation image. The method comprises the following steps: acquiring a hierarchical lesion fundus three-dimensional model, extracting a thickness change distribution diagram of a physiological hierarchy with lesion from the three-dimensional model, segmenting the thickness change distribution diagram, extracting a lesion position and performing down-sampling to obtain a lesion weight map for describing the shape and range of the lesion, and constructing a lesion database according to the lesion weight map; and applying the lesion weight map to the healthy fundus data to generate an OCT lesion simulation image. The lesion data stored in the method is small in size, and a large-scale local database is easy to construct; the generated lesion OCT image is visual and real, can be used for teaching and scientific research activities, and can be used as a training sample for deep learning.

Description

Technical Field

[0001] The present invention belongs to the field of medical imaging, and in particular provides a method, device, equipment and storage medium for generating fundus optical coherence tomography (OCT) lesion simulation images. Background Art

[0002] OCT is a technology that uses infrared light to scan the surface of human tissue, obtain depth information several millimeters below the tissue surface, and further realize three-dimensional imaging. As a non-contact, non-invasive optical detection method, OCT technology plays an important role in the diagnosis of ophthalmic diseases. In ophthalmic clinical practice, OCT technology is widely used to detect a variety of retinal diseases, such as macular holes, serous retinal detachment, age-related macular degeneration, etc. By imaging each layer of the retina, observing disease-specific imaging features, and evaluating biomarkers for diagnosis and prognosis of these diseases, doctors can detect early lesion areas and develop treatment plans. However, despite the continuous expansion of the clinical application of OCT, the scarcity of medical imaging data has become a bottleneck in education and research.

[0003] For medical education, rich case data is the key to mastering disease characteristics and diagnostic skills. However, some retinal diseases are relatively rare in clinical practice, which makes it difficult for medical students to come into contact with enough actual cases. In addition, the acquisition and sharing of medical images are restricted by ethical approval, privacy protection and acquisition costs, further exacerbating the scarcity of OCT images. For deep learning research, the scale and number of data sets are important influencing factors for model training. However, the scale of existing lesion OCT image data sets is generally small, and the types of diseases are relatively single. For example, the Retouch data set mainly focuses on the segmentation task of intraretinal fluid, the Retinal OCT-C8 data set covers a limited number of lesion types, and the sample size of some rare lesions is relatively scarce. The scarcity of this OCT image data directly limits the application of deep learning models in ophthalmic disease recognition. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] To this end, the purpose of the present invention is to provide a method, device, equipment and storage medium for generating OCT lesion simulation images in response to the scarcity of ophthalmic OCT image data. The present invention can extract lesion features from the OCT scanning data of the lesion fundus, and synthesize them with the healthy fundus OCT data to generate OCT lesion simulation images, so as to intuitively observe the physiological changes caused by the lesions. The present invention can be used for medical training and generating training data sets.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides a method for generating an OCT simulation image of fundus lesions, comprising the following steps:

[0008] Obtain OCT point cloud data of the diseased fundus, generate a B-scan image sequence of the diseased fundus based on the OCT point cloud data, segment each B-scan image in the B-scan image sequence according to the grayscale difference of different physiological levels of the diseased fundus, obtain volume data of each physiological level of the diseased fundus, perform three-dimensional reconstruction on the volume data of each physiological level, and obtain a hierarchical three-dimensional model of the diseased fundus; extract a thickness change distribution map of the physiological level where the lesion occurs from the hierarchical three-dimensional model of the diseased fundus, segment it, extract the lesion position and perform downsampling, and obtain a lesion weight map for describing the shape and range of the lesion; and construct a lesion database using the lesion weight maps of different lesion types;

[0009] Obtain OCT point cloud data of a healthy fundus, obtain a hierarchical healthy fundus three-dimensional model in the same processing manner as the OCT point cloud data of the diseased fundus, select a lesion weight map corresponding to a target lesion from the lesion database, select the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus three-dimensional model, generate corresponding fundus tissue deformation in the hierarchical healthy fundus three-dimensional model according to the lesion weight map corresponding to the target lesion, and obtain an OCT simulation image of the fundus lesion.

[0010] In some embodiments, segmenting each B-scan image in the B-scan image sequence according to the grayscale difference of different physiological levels of the diseased fundus to obtain volume data of each physiological level of the diseased fundus specifically includes:

[0011] Segmenting each B-scan image in the B-scan image sequence according to the grayscale difference of different physiological levels of the diseased fundus to obtain a hierarchical line coordinate set between each layer of physiological structure;

[0012] Selecting a layered line of a stable fundus physiological structure as a reference mark, taking the first B-scan image in the B-scan image sequence as a reference image, calculating an average offset between the reference image and the reference marks in the remaining B-scan images, performing translation adjustment on the remaining B-scan images according to the average offset, and updating the layered line coordinate set;

[0013] For each B-scan image in the B-scan image sequence, the points between two adjacent layering lines are stored in a data structure corresponding to the physiological structure of the fundus, so as to obtain the volume data of each physiological level of the diseased fundus.

[0014] In some embodiments, the three-dimensional reconstruction of the volume data of each physiological level is performed separately to obtain a hierarchical three-dimensional model of the diseased fundus, specifically including:

[0015] Apply ray casting method to set transparency mapping parameters for volume data at each physiological level;

[0016] Set color mapping parameters to map physiological structures with grayscale differences into different colors;

[0017] All volume data are added to the same renderer for rendering to obtain a hierarchical three-dimensional model of the diseased fundus.

[0018] In some embodiments, the step of extracting the thickness change distribution map of the physiological layer where the lesion occurs from the hierarchical lesion fundus three-dimensional model, segmenting it, extracting the lesion location and performing downsampling to obtain a lesion weight map for describing the shape and range of the lesion specifically includes:

[0019] The thickness of each level of physiological structure is compared with prior knowledge to determine the level of lesion occurrence;

[0020] Initialize a thickness change distribution matrix, whose size is consistent with the transverse plane resolution adopted by the OCT scan, for storing the thickness change distribution of each point in the lesion occurrence level; extract the thickness of the lesion occurrence level between the upper and lower stratification lines of the lesion occurrence level, and use the thickness of the area not affected by the lesion in the lesion occurrence level as the thickness threshold. When the thickness at a certain point extracted is less than or equal to the thickness threshold, the value of the element corresponding to the point in the thickness change distribution matrix is ​​taken as 0, otherwise the value of the element corresponding to the point in the thickness change distribution matrix is ​​taken as the difference between the thickness of the point and the thickness threshold, thereby obtaining the thickness change distribution matrix of the lesion occurrence level;

[0021] The thickness change distribution matrix of the lesion occurrence level is regarded as a grayscale image and recorded as a thickness change distribution map, which is binarized, and all connected domains therein are checked, the connected domains corresponding to the healthy area are removed, and the connected domains corresponding to the lesion area are retained, and the minimum circumscribed rectangle surrounding the connected domain corresponding to the lesion area is determined, and the value of the minimum circumscribed rectangle is set to 1, and the values ​​of the remaining areas are set to 0, so as to generate a lesion area mask;

[0022] Using the lesion area mask to segment the thickness change distribution map of the lesion occurrence level to obtain a lesion distribution map;

[0023] The lesion distribution map is downsampled and grayscale normalized to obtain the lesion weight map, and the normalized grayscale value of each point in the lesion weight map is recorded as the weight of each point.

[0024] In some embodiments, the pixel size of the lesion weight map is 128 pixels*128 pixels.

[0025] In some embodiments, selecting the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus 3D model, and generating corresponding fundus tissue deformation in the hierarchical healthy fundus 3D model according to the lesion weight map corresponding to the target lesion, specifically includes:

[0026] Rotate and scale the selected lesion weight map according to the scanning parameters and target lesion characteristics;

[0027] Selecting the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus three-dimensional model, and determining the influence range of the simulated lesion in combination with the parameters of the rotation and scaling operations performed on the selected lesion weight map;

[0028] Set the deformation strength, multiply the deformation strength by the weight of each point in the deformation weight map after rotation and scaling, and obtain the displacement of each voxel in the influence range of the simulated lesion. Iterate through each voxel in the influence range of the simulated lesion and displace it to achieve deformation of the fundus tissue.

[0029] In some embodiments, the step of achieving fundus tissue deformation further includes any one or more of the following post-processing:

[0030] ① Adjust the pixel color of the displacement point to simulate the change of signal reflection intensity of the lesion area in the B-scan image;

[0031] ② Fill the void area caused by displacement with voxels to simulate the effusion phenomenon that may be caused by the lesion;

[0032] ③ Use the Gaussian attenuation function to multiply the deformation intensity to adjust the deformation intensity according to the distance from the center of the lesion, so as to simulate the attenuation or displacement that may occur in healthy tissue when it is squeezed or deformed by the lesion.

[0033] The second aspect of the present invention provides a device based on the method for generating fundus lesion OCT simulation images according to any embodiment of the first aspect of the present invention, comprising:

[0034] A lesion database construction module is used to obtain OCT point cloud data of the lesion fundus, generate a B-scan image sequence of the lesion fundus based on the OCT point cloud data, segment each B-scan image in the B-scan image sequence according to the grayscale difference of different physiological levels of the lesion fundus, obtain volume data of each physiological level of the lesion fundus, perform three-dimensional reconstruction on the volume data of each physiological level, and obtain a hierarchical lesion fundus three-dimensional model; extract the thickness change distribution map of the physiological level where the lesion occurs from the hierarchical lesion fundus three-dimensional model, segment it, extract the lesion position and perform downsampling, and obtain a lesion weight map for describing the shape and range of the lesion; and construct a lesion database using the lesion weight maps of different lesion types;

[0035] The fundus lesion OCT simulation image generation module is used to obtain a hierarchical healthy fundus three-dimensional model in the same processing method as the OCT point cloud data of the diseased fundus, select a lesion weight map corresponding to the target lesion from the lesion database, select the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus three-dimensional model, and generate corresponding fundus tissue deformation in the hierarchical healthy fundus three-dimensional model according to the lesion weight map corresponding to the target lesion, so as to obtain an OCT simulation image of the fundus lesion.

[0036] A third aspect of the present invention provides an electronic device, comprising:

[0037] at least one processor, and a memory communicatively coupled to the at least one processor;

[0038] The memory stores instructions executable by the at least one processor, and the instructions are configured to execute the above-mentioned method for generating fundus lesion OCT simulation images.

[0039] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the above-mentioned method for generating fundus lesion OCT simulation images.

[0040] Features and beneficial effects of the present invention:

[0041] The present invention is based on high-definition OCT B-scan image segmentation, and converts the point cloud data collected by the OCT device into fundus physiological structures and lesion entities, on which the lesions are extracted and the lesions of healthy images are simulated. The lesion distribution data is saved in the form of a grayscale image by peeling off the retinal physiological layers layer by layer, exposing the lesions and sampling them; the lesion data can be applied to healthy fundus data through a program interface to generate simulated lesion OCT images. The saved lesion data is small in size, and it is easy to build a large-scale local database; the generated lesion OCT images are intuitive and realistic, and can be used for teaching and scientific research activities, and as training samples for deep learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is an overall flow chart of a method for generating an OCT lesion simulation image provided by an embodiment of the first aspect of the present invention;

[0043] Figure 2 It is a flow chart of constructing a lesion database in a method for generating an OCT lesion simulation image provided by an embodiment of the first aspect of the present invention;

[0044] Figure 3 An OCT B-scan image of a diseased fundus generated from point cloud data of the diseased fundus in the first aspect of the present invention;

[0045] Figure 4 It is a flowchart of generating an OCT lesion simulation image based on a constructed lesion database in an OCT lesion simulation image generation method provided by an embodiment of the first aspect of the present invention.

[0046] FIG5( a ) is a three-dimensional model of a healthy fundus obtained in the embodiment of the first aspect;

[0047] FIG5(b) is a three-dimensional model of the diseased fundus corresponding to FIG5(a) generated by the embodiment of the first aspect of the present invention and peeled off to the corresponding lesion occurrence level.

[0048] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the third aspect of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] On the contrary, the present application covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present application as defined by the claims. Further, in order to make the public have a better understanding of the present application, some specific details are described in detail in the detailed description of the present application below. Those skilled in the art can fully understand the present application without the description of these details.

[0051] The first embodiment of the present invention provides a method for generating an OCT image of fundus lesions, which is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] The overall process of an OCT lesion simulation image generation method provided by the first embodiment of the present invention is as follows: Figure 1 As shown in the figure. The lesions of the fundus can be regarded as the deformation of the healthy physiological structure, which can be described by the deformation weight distribution, reflecting the compression intensity of the lesion on different parts of the healthy tissue. The weight map is a data structure for storing the deformation weight distribution. This method extracts the weight map from the OCT scanning data of the lesion fundus, constructs the lesion database, and applies the weight map to the OCT scanning data of the healthy fundus to generate the OCT three-dimensional simulated lesion image.

[0053] See also Figure 1 The first aspect of the present invention provides a method for generating an OCT simulation image of fundus lesions, comprising the following steps:

[0054] Step S1, constructing a lesion database: obtaining OCT point cloud data of the lesion fundus, generating a B-scan image sequence of the lesion fundus based on the OCT point cloud data, wherein the grayscale layers in the B-scan image sequence correspond to different physiological layers of the retina of the lesion fundus, segmenting each B-scan image in the B-scan image sequence according to the grayscale difference of different physiological layers of the lesion fundus, obtaining volume data of each physiological layer of the lesion fundus, and performing three-dimensional reconstruction on the volume data of each physiological layer to obtain a hierarchical three-dimensional model of the lesion fundus; extracting a thickness change distribution map of the physiological layer where the lesion occurs from the three-dimensional model of the lesion fundus, segmenting it, extracting the lesion position and performing downsampling to obtain a lesion weight map for describing the shape and range of the lesion; constructing a lesion database using the lesion weight maps of different lesion types;

[0055] Step S2, generation of fundus lesion OCT simulation image: obtain OCT point cloud data of healthy fundus, obtain a hierarchical healthy fundus three-dimensional model in the same processing method as the OCT point cloud data of the diseased fundus, select a lesion weight map corresponding to the target lesion from the lesion database, select the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus three-dimensional model, and generate corresponding fundus tissue deformation in the hierarchical healthy fundus three-dimensional model according to the lesion weight map corresponding to the target lesion to obtain an OCT simulation image of the fundus lesion.

[0056] In some embodiments, see Figure 2 , step S1 specifically includes:

[0057] Step S11: Use an OCT device to scan the diseased fundus and obtain OCT dense point cloud data of the diseased fundus.

[0058] Step S12: Generate a B-scan image sequence of the lesion from the OCT point cloud data of the lesion fundus, perform image segmentation on it, and obtain volume data of each physiological level of the lesion fundus. Specifically:

[0059] First, generate a B-scan image sequence from the OCT point cloud data of the diseased fundus, see Figure 3 , is an OCT B-scan image of a diseased fundus, from which the physiological structure of the fundus can be observed, including the various layers of retinal tissue, choroid, sclera and lesion area. There are obvious grayscale differences between the physiological tissues. The B-scan image is segmented based on this grayscale difference. The optional image segmentation methods include a graph-based boundary search method, a U-Net semantic segmentation method based on deep learning, etc. The B-scan image sequence is denoised and sharpened, and a segmentation operation is performed to obtain a hierarchical line coordinate set between each layer of physiological structure. It should be particularly pointed out that when segmenting the B-scan image in this embodiment, it is not segmented according to the current internationally used 10-layer structure of the retina (for some structural layers with very thin layers, there is a problem of difficulty in accurate segmentation), but combined with pathology. In the segmentation process, according to the obvious grayscale differences between the physiological tissues, some levels are merged without segmentation. Most of the lesions are in the actual segmentation level, thereby reducing the difficulty of segmentation.

[0060] Afterwards, an image registration operation is performed: a relatively stable layer line of the fundus physiological structure is selected as a reference mark for image sequence registration. The registration reference layer line selected in this embodiment is the choroid-sclera interface (CSI); the first B-scan image in the B-scan image sequence is used as the reference image, and the CSI average offset between the reference image and other B-scan images in the B-scan image sequence is calculated. The B-scan images except the reference image are translated and adjusted by the average offset, and the layer line coordinate set is updated to complete the image registration, thereby improving the spatial consistency of the image sequence.

[0061] Finally, for each B-scan image in the B-scan image sequence, the points between two adjacent stratification lines are stored in the data structure corresponding to the physiological structure of the fundus, and the volume data of each physiological level of the diseased fundus are obtained.

[0062] Step S13, reconstruct the entity of the physiological structure of the diseased fundus. The volume data of each physiological level of the diseased fundus obtained in step S12 are reconstructed in three dimensions respectively. Specifically: this embodiment adopts volume rendering, applies the ray-casting algorithm to the volume data of each physiological level, sets the transparency mapping parameters, and sets the low-reflection signal areas, such as the transparent vitreous body and the undetectable area behind the sclera, to be completely transparent; sets the color mapping parameters, and maps the physiological structures with grayscale differences to different colors. Finally, all the volume data are added to the same renderer for rendering to obtain a hierarchical three-dimensional model of the diseased fundus.

[0063] Step S14, identify the level at which the lesion occurs. The thickness of most areas of the healthy fundus structure is uniform, and this can be used as a basis to determine whether a lesion occurs layer by layer. The specific process is: by controlling the rendering pipeline, the first-level physiological structure is displayed separately, and by comparing with prior knowledge, it is determined whether a lesion occurs. Taking the lesion judgment of each layer of the macular OCT as an example, it is considered whether the structure of each layer is roughly smooth, whether the thickness of each layer of the physiological structure changes abnormally, and whether there are common lesion characteristics such as bulges caused by new blood vessels; if no lesion occurs, the next level of physiological structure is displayed separately, and then it is determined whether the next level of physiological structure has a lesion; if it is determined that a lesion occurs, the level is marked as the lesion occurrence level. If it is determined that no lesion has occurred, the above process is repeated until the lesion occurrence level is determined.

[0064] Step S15, extraction of thickness change distribution of lesion occurrence level: Initialize a thickness change distribution matrix, whose size is consistent with the transverse plane resolution adopted by the OCT device scanning, for storing the thickness change distribution of each point in the lesion occurrence level identified in step S14. The size of the thickness change distribution matrix in this embodiment is 512*512. For the physiological level of lesion occurrence identified in step S14, extract the thickness change distribution caused by the lesion. Extract the thickness of the lesion occurrence level between the upper and lower stratification lines of the lesion occurrence level, and take the thickness of the area not affected by the lesion in the lesion occurrence level (which can also be understood as the original thickness of the lesion occurrence level when no lesion occurs) as the thickness threshold. When the thickness at a certain point extracted is less than or equal to the thickness threshold, the value of the element corresponding to the point in the thickness change distribution matrix is ​​taken as 0. When the thickness at a certain point extracted is greater than the thickness threshold, the value of the element corresponding to the point in the thickness change distribution matrix is ​​taken as the difference between the thickness of the point and the thickness threshold, thereby obtaining the thickness change distribution matrix of the lesion occurrence level.

[0065] Step S16, binarization and connected domain analysis: The thickness change distribution matrix of the lesion occurrence level is regarded as a grayscale image and recorded as a thickness change distribution map. It is binarized and all connected domains are checked. There may be healthy areas with thickness higher than the threshold due to factors such as blood vessels. These areas are reflected as smaller connected domains in the binary map. The connected domains corresponding to the healthy areas are eliminated (this can be achieved by setting the connected domain threshold. Among all the identified connected domains, the connected domains that are less than the connected threshold are regarded as healthy areas, otherwise they are regarded as lesion areas), and the connected domains corresponding to the lesion areas are retained. Then the connected domain corresponding to the lesion area is surrounded by the minimum square bounding box, the value in the square area is set to 1, and the value of the remaining area is set to 0, so as to generate a lesion area mask.

[0066] Step S17, segment the thickness variation distribution map using the lesion area mask to reduce the invalid information area and retain more lesion details. In this embodiment, the 512 pixel*512 pixel thickness variation distribution map is segmented into a lesion distribution map of 267 pixels*267 pixels.

[0067] Step S18: In order to speed up the generation of subsequent lesion images, the lesion distribution map must first be downsampled to the target resolution, and then grayscale normalized and converted into a lesion weight map of a standardized size of 128 pixels * 128 pixels for describing the shape and range of the lesion. The normalized grayscale value of each point in the lesion weight map is recorded as the weight of each point. The size of the lesion weight map is 2.61KB.

[0068] Step S19: For the remaining lesion types, refer to steps S11 to S18 to obtain lesion weight maps of different lesion types, and use them to construct a lesion database.

[0069] In some embodiments, see Figure 4 , step S2 specifically includes:

[0070] Step S21: Use an OCT device to scan a healthy fundus to obtain OCT dense point cloud data of the healthy fundus.

[0071] Step S22: in the same manner as step S12, a healthy B-scan image sequence is generated from the OCT point cloud data of the healthy fundus, and image segmentation is performed to obtain volume data of each physiological level of the healthy fundus.

[0072] Step S23: Reconstruct the physiological entity structure of the healthy fundus in the same processing manner as step S13 to obtain a hierarchical three-dimensional model of the healthy fundus.

[0073] Step S24, select the lesion weight map corresponding to the target lesion from the lesion database, comprehensively consider the scanning parameters and the characteristics of the target lesion, rotate and scale the selected lesion weight map to ensure accurate simulation of the lesion area and lesion direction. Taking the simulation of central serous chorioretinopathy (hereinafter referred to as central serous chorioretinopathy) as an example, the scanning parameters are: 6mm*6mm range of the macular area of ​​the fundus, and 512 B-scans of 512 pixels*1044 pixels are obtained. For central serous chorioretinopathy, the area of ​​​​the disease is near the fovea of ​​the macula. Combined with the common area of ​​central serous chorioretinopathy and the resolution of this OCT scan, it is a reasonable choice to enlarge the weight map to 256*256, and rotate the weight map so that the position with the largest weight is aligned with the fovea of ​​the macula to meet the lesion characteristics of central serous chorioretinopathy.

[0074] Step S25, select the location of lesion generation: select the center location of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus 3D model (that is, it is considered in this application that there is a one-to-one correspondence between each lesion type and the level where the lesion occurs, that is, the depth plane), and combine the parameters of the rotation and scaling operations on the selected lesion weight map in step S24 to determine the influence range of the simulated lesion. In this way, the lesion is accurately embedded in the healthy fundus tissue, achieving precise control of the space and direction of the lesion area.

[0075] Step S26, calculate the voxel displacement and deform the fundus tissue. Set the deformation strength according to the scanning parameters and the prior knowledge of the lesion at a certain development stage, multiply the deformation strength with the weight of each point in the deformation weight map after rotation and scaling, and obtain the displacement of each voxel in the influence range of the simulated lesion. Go through each voxel in the influence range of the simulated lesion and perform displacement to achieve fundus tissue deformation.

[0076] Step S27, post-processing. Some optional post-processing operations are as follows: ① Adjust the pixel color of the displacement point to simulate the change in the signal reflection intensity of the lesion area in the B-scan image. ② Fill the void area caused by the displacement with voxels to simulate the effusion phenomenon that may be caused by the lesion. ③ To simulate the slight attenuation or displacement that may occur in healthy tissue when it is squeezed or deformed by the lesion, you can consider using a Gaussian attenuation function and multiplying it with the deformation intensity set in step S26 to adjust the deformation intensity according to the distance from the center of the lesion, so that the influence of the deformed area is gradually weakened, achieving a natural transition between physiological levels, and ensuring that the simulation results are more natural in space.

[0077] Step S28, update the rendering pipeline. Add the deformed volume data back into the renderer, refresh the 3D model in the rendering window, and observe the deformed fundus structure entity, i.e., the fundus lesion OCT simulation image. See FIG5(a), which is the healthy fundus 3D model obtained in this embodiment, and FIG5(b) is the lesion fundus 3D model corresponding to FIG5(a) generated by this embodiment, and it is peeled off to the corresponding lesion occurrence level (i.e., lesion level), from which the generated lesion can be observed.

[0078] It can be understood that this method segments the OCT B-scan image sequence based on the physiological characteristics of fundus stratification, extracts the physiological entities of each fundus layer from the point cloud data, constructs a hierarchical and materialized three-dimensional fundus model, extracts fundus lesions, and describes the lesions with a small data structure (i.e., lesion weight map), which is conducive to building a large lesion database and has the advantage of data volume; in the lesion simulation image generation stage, the weight map can be scaled and rotated using conventional two-dimensional image processing methods to achieve flexible lesion simulation; lesion simulation of the materialized three-dimensional fundus model has a real and intuitive visual effect. In teaching and scientific research activities, this method can be used to simulate fundus lesions, which can solve the problem of difficulty in obtaining fundus lesion images to a certain extent. In the field of deep learning, this method can generate training samples in batches, which is helpful for training high-quality fundus lesion recognition models.

[0079] A method and device for generating an OCT lesion simulation image provided by an embodiment of the second aspect of the present invention includes:

[0080] A lesion database construction module is used to obtain OCT point cloud data of the lesion fundus, generate a B-scan image sequence of the lesion fundus based on the OCT point cloud data, segment each B-scan image in the B-scan image sequence according to the grayscale difference of different physiological levels of the lesion fundus, obtain volume data of each physiological level of the lesion fundus, perform three-dimensional reconstruction on the volume data of each physiological level, and obtain a hierarchical three-dimensional model of the lesion fundus; extract the thickness change distribution map of the physiological level where the lesion occurs from the three-dimensional model of the lesion fundus, segment it, extract the lesion position and perform downsampling to obtain a lesion weight map for describing the shape and range of the lesion; and construct a lesion database using the lesion weight maps of different lesion types;

[0081] The fundus lesion OCT simulation image generation module is used to obtain the OCT point cloud data of the healthy fundus, obtain a hierarchical healthy fundus three-dimensional model in the same processing method as the OCT point cloud data of the diseased fundus, select the lesion weight map corresponding to the target lesion from the lesion database, select the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus three-dimensional model, and generate corresponding fundus tissue deformation in the hierarchical healthy fundus three-dimensional model according to the lesion weight map corresponding to the target lesion to obtain the fundus lesion OCT simulation image.

[0082] It should be noted that the aforementioned explanation of the embodiment of the method for generating the fundus lesion OCT simulation image is also applicable to the fundus lesion OCT simulation image generating device of the present embodiment, and will not be repeated here.

[0083] In order to implement the above embodiment, the embodiment of the present disclosure further proposes a computer-readable storage medium, on which a computer program is stored. The program is executed by a processor to execute the adaptive compensation method of the above embodiment.

[0084] Reference below Figure 6 , which shows a schematic diagram of the structure of an electronic device suitable for implementing an embodiment of the present invention. It should be noted that the electronic device in the embodiment of the present invention may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, servers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0085] like Figure 6As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 102 or a program loaded from a storage device 108 to a random access memory (RAM) 103. In the RAM 103, various programs and data required for the operation of the electronic device are also stored. The processing device 101, the ROM 102, and the RAM 103 are connected to each other via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0086] Typically, the following devices may be connected to the I / O interface 105: an input device 106 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, etc.; an output device 107 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 108 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 109. The communication device 109 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 6 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0087] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present embodiment includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 109, or installed from the storage device 108, or installed from the ROM 102. When the computer program is executed by the processing device 101, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0088] It should be noted that the computer-readable medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0089] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0090] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method for generating fundus lesion OCT simulation images.

[0091] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, python, and conventional procedural programming languages, such as "C-" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0093] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0094] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0095] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0096] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0097] A person of ordinary skill in the art may understand that all or part of the steps carried out in the above-mentioned embodiment method may be completed by instructing the relevant hardware through a program, and the developed program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0098] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0099] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for generating an OCT simulation image of fundus lesions, characterized in that: The following steps are involved: Obtain OCT point cloud data of the diseased fundus, generate a B-scan image sequence of the diseased fundus based on the OCT point cloud data, segment each B-scan image in the B-scan image sequence according to the grayscale difference of different physiological levels of the diseased fundus, obtain volume data of each physiological level of the diseased fundus, perform three-dimensional reconstruction on the volume data of each physiological level, and obtain a hierarchical three-dimensional model of the diseased fundus; extract a thickness change distribution map of the physiological level where the lesion occurs from the hierarchical three-dimensional model of the diseased fundus, segment it, extract the lesion position and perform downsampling, and obtain a lesion weight map for describing the shape and range of the lesion; and construct a lesion database using the lesion weight maps of different lesion types; Obtain OCT point cloud data of a healthy fundus, obtain a hierarchical healthy fundus three-dimensional model in the same processing manner as the OCT point cloud data of the diseased fundus, select a lesion weight map corresponding to a target lesion from the lesion database, select the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus three-dimensional model, generate corresponding fundus tissue deformation in the hierarchical healthy fundus three-dimensional model according to the lesion weight map corresponding to the target lesion, and obtain an OCT simulation image of the fundus lesion.

2. The method for generating fundus lesion OCT simulation images according to claim 1, characterized in that: The segmentation of each B-scan image in the B-scan image sequence according to the grayscale difference of different physiological levels of the diseased fundus to obtain volume data of each physiological level of the diseased fundus specifically includes: Segmenting each B-scan image in the B-scan image sequence according to the grayscale difference of different physiological levels of the diseased fundus to obtain a hierarchical line coordinate set between each layer of physiological structure; Selecting a layered line of a stable fundus physiological structure as a reference mark, taking the first B-scan image in the B-scan image sequence as a reference image, calculating an average offset between the reference image and the reference marks in the remaining B-scan images, performing translation adjustment on the remaining B-scan images according to the average offset, and updating the layered line coordinate set; For each B-scan image in the B-scan image sequence, the points between two adjacent layering lines are stored in a data structure corresponding to the physiological structure of the fundus, so as to obtain the volume data of each physiological level of the diseased fundus.

3. The method for generating fundus lesion OCT simulation images according to claim 1, characterized in that: The three-dimensional reconstruction of the volume data of each physiological level is performed respectively to obtain a hierarchical three-dimensional model of the diseased fundus, which specifically includes: Apply ray casting method to set transparency mapping parameters for volume data at each physiological level; Set color mapping parameters to map physiological structures with grayscale differences into different colors; All volume data are added to the same renderer for rendering to obtain a hierarchical three-dimensional model of the diseased fundus.

4. The method for generating fundus lesion OCT simulation images according to claim 1, characterized in that: The step of extracting a thickness change distribution map of the physiological layer where the lesion occurs from the hierarchical lesion fundus three-dimensional model, segmenting the map, extracting the lesion location and performing downsampling to obtain a lesion weight map for describing the shape and range of the lesion specifically includes: The thickness of each level of physiological structure is compared with prior knowledge to determine the level of lesion occurrence; Initialize a thickness change distribution matrix, whose size is consistent with the transverse plane resolution adopted by the OCT scan, for storing the thickness change distribution of each point in the lesion occurrence level; extract the thickness of the lesion occurrence level between the upper and lower stratification lines of the lesion occurrence level, and use the thickness of the area not affected by the lesion in the lesion occurrence level as the thickness threshold. When the thickness at a certain point extracted is less than or equal to the thickness threshold, the value of the element corresponding to the point in the thickness change distribution matrix is ​​taken as 0, otherwise the value of the element corresponding to the point in the thickness change distribution matrix is ​​taken as the difference between the thickness of the point and the thickness threshold, thereby obtaining the thickness change distribution matrix of the lesion occurrence level; The thickness change distribution matrix of the lesion occurrence level is regarded as a grayscale image and recorded as a thickness change distribution map, which is binarized, and all connected domains therein are checked, the connected domains corresponding to the healthy area are removed, and the connected domains corresponding to the lesion area are retained, and the minimum circumscribed rectangle surrounding the connected domain corresponding to the lesion area is determined, and the value of the minimum circumscribed rectangle is set to 1, and the values ​​of the remaining areas are set to 0, so as to generate a lesion area mask; Using the lesion area mask to segment the thickness change distribution map of the lesion occurrence level to obtain a lesion distribution map; The lesion distribution map is downsampled and grayscale normalized to obtain the lesion weight map, and the normalized grayscale value of each point in the lesion weight map is recorded as the weight of each point.

5. The method for generating fundus lesion OCT simulation images according to claim 4, characterized in that: The pixel size of the lesion weight map is 128 pixels*128 pixels.

6. The method for generating fundus lesion OCT simulation images according to claim 1, characterized in that: The step of selecting the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus three-dimensional model, and generating corresponding fundus tissue deformation in the hierarchical healthy fundus three-dimensional model according to the lesion weight map corresponding to the target lesion, specifically includes: Rotate and scale the selected lesion weight map according to the scanning parameters and target lesion characteristics; Selecting the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus three-dimensional model, and determining the influence range of the simulated lesion in combination with the parameters of the rotation and scaling operations performed on the selected lesion weight map; Set the deformation strength, multiply the deformation strength by the weight of each point in the deformation weight map after rotation and scaling, and obtain the displacement of each voxel in the influence range of the simulated lesion. Iterate through each voxel in the influence range of the simulated lesion and displace it to achieve deformation of the fundus tissue.

7. The method for generating fundus lesion OCT simulation images according to claim 6, characterized in that: After the fundus tissue deformation is achieved, any one or more of the following post-processings may also be included: ① Adjust the pixel color of the displacement point to simulate the change of signal reflection intensity of the lesion area in the B-scan image; ② Fill the void area caused by displacement with voxels to simulate the effusion phenomenon that may be caused by the lesion; ③ Use the Gaussian attenuation function to multiply the deformation intensity to adjust the deformation intensity according to the distance from the center of the lesion, so as to simulate the attenuation or displacement that may occur in healthy tissue when it is squeezed or deformed by the lesion.

8. A device based on the method for generating fundus lesion OCT simulation images according to any one of claims 1 to 7, characterized in that: include: A lesion database construction module is used to obtain OCT point cloud data of the lesion fundus, generate a B-scan image sequence of the lesion fundus based on the OCT point cloud data, segment each B-scan image in the B-scan image sequence according to the grayscale difference of different physiological levels of the lesion fundus, obtain volume data of each physiological level of the lesion fundus, perform three-dimensional reconstruction on the volume data of each physiological level, and obtain a hierarchical lesion fundus three-dimensional model; extract the thickness change distribution map of the physiological level where the lesion occurs from the hierarchical lesion fundus three-dimensional model, segment it, extract the lesion position and perform downsampling, and obtain a lesion weight map for describing the shape and range of the lesion; and construct a lesion database using the lesion weight maps of different lesion types; The fundus lesion OCT simulation image generation module is used to obtain a hierarchical healthy fundus three-dimensional model in the same processing method as the OCT point cloud data of the diseased fundus, select a lesion weight map corresponding to the target lesion from the lesion database, select the center position of the lesion from the depth plane corresponding to the target lesion in the hierarchical healthy fundus three-dimensional model, and generate corresponding fundus tissue deformation in the hierarchical healthy fundus three-dimensional model according to the lesion weight map corresponding to the target lesion, so as to obtain an OCT simulation image of the fundus lesion.

9. An electronic device, characterized in that: include: at least one processor, and a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are configured to execute the method for generating fundus lesion OCT simulation images according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method for generating fundus lesion OCT simulation images according to any one of claims 1 to 7.

Images