A method, device and storage medium for OCT image stratification and lesion semantic segmentation

Abstract

The present invention discloses an OCT image stratification and lesion semantic segmentation method, device and storage medium. The method comprises: obtaining each BScan two-dimensional slice image data of OCT of one eye of a patient to be examined; inputting the slice image data one by one into a pre-trained first-stage convolutional neural network to obtain the first-stage stratification, optic disc area, and semantic segmentation prediction results of the macula fovea area; making a "region of interest" according to the obtained first-stage stratification, optic disc area, and semantic segmentation prediction results of the macula fovea area; obtaining a data map of the "region of interest"; inputting the data map of the "region of interest" into a second-stage convolutional neural network to obtain the second-stage stratification, drusen, pigment epithelial detachment, and semantic segmentation prediction results of choroidal neovascularization lesions. The present invention increases the proportion of lesion pixels to total pixels and reduces the learning difficulty of small-area positive lesions such as drusen.

Description

Technical Field

[0001] The present invention relates to an OCT image stratification and lesion semantic segmentation method, device and storage medium, belonging to the technical field of semantic segmentation. Background Art

[0002] Optical coherence tomography (OCT) is an imaging technology that forms fundus images. Because it can reflect the reflection and scattering characteristics of different physiological structures of the fundus to the incident weak coherent light, the three-dimensional image formed has depth information. Compared with images such as fundus color photography, it has unique advantages.

[0003] At present, the hierarchical analysis and lesion analysis of OCT images usually use deep convolutional neural networks to perform semantic segmentation of hierarchical and lesion segments on OCT slices or the whole, and then perform post-processing based on the segmentation results to obtain some physiological indicators of the fundus, such as the thickness of the pigment epithelium and the volume of drusen. Usually, deep learning models need to perform hierarchical / lesion segmentation on pixels in all areas of the collected OCT images during the training phase. The learned image information lacks specificity, and it is difficult to accurately identify small lesions (such as drusen) involved in age-related macular degeneration, which brings difficulties to the subsequent staging of age-related macular degeneration based on information such as the volume of the lesion. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide an OCT image stratification and lesion semantic segmentation method, device and storage medium, which increases the ratio of lesion pixels to total pixels and reduces the difficulty of learning small-area positive lesions such as drusen.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides an OCT image stratification and lesion semantic segmentation method, comprising:

[0007] Obtain each BScan two-dimensional slice image data of OCT of one eye of the patient to be examined, and perform normalization processing;

[0008] The normalized slice image data are input into the pre-trained first-stage convolutional neural network one by one to obtain the semantic segmentation prediction results of the first-stage layers, optic disc areas, and macular concave areas;

[0009] Create a "region of interest" based on the semantic segmentation prediction results of the first stage of stratification, optic disc area, and macular concave area;

[0010] Get the data graph of the "area of ​​interest" and normalize the data;

[0011] The normalized “region of interest” data graph is input into the second-stage convolutional neural network to obtain the semantic segmentation prediction results of the second-stage stratification, drusen, pigment epithelial detachment, and choroidal neovascularization lesions.

[0012] Furthermore, based on the semantic segmentation prediction results of the first stage of stratification, optic disc area, and submacular fovea area, a “region of interest” is created, including:

[0013] Select the slice with the thinnest nerve fiber layer in the column where the center point of the macular depression area in the OCT data label of the same eye is located, and the center point of the macular depression area is used as the center point of the "fovea centralis" of the OCT image of the eye;

[0014] The maximum value of the width of the optic disc area in all slice data labels of the same eye's OCT was selected as the "optic disc diameter" of the OCT image;

[0015] According to the values ​​of horizontal and vertical physical resolution and pixel resolution in the specific mode of the OCT model, the horizontal pixel numbers of the two optic disc diameters are converted into horizontal physical dimensions, and the "area of ​​interest circle" is determined with the fovea centralis as the center and the physical value of the optic disc diameter as the radius, and then converted into the pixel width of the "area of ​​interest" corresponding to the "area of ​​interest circle" in each slice;

[0016] On the slice, the column where the center point of the fovea is located is taken as the center point, the width is obtained by converting the pixel width, and the center of the circumscribed rectangle of the polygon of the non-background area of ​​the underlying eye structure label of these columns is taken as the center, and the specific number of pixels is used as the height to obtain the "area of ​​interest".

[0017] Furthermore, the training method of the convolutional neural network in the first stage includes:

[0018] Obtain OCT BScan two-dimensional slice image data and the corresponding semantic segmentation labels of layers, optic disc areas, and macular concave areas, perform data augmentation, normalization, and random shuffling to create the first-stage data set;

[0019] In each step of the first-stage network training, the processed image data in the first-stage data set is input into the first-stage convolutional neural network to obtain the semantic segmentation prediction results of the first-stage layers, optic disc areas, and submacular fovea areas, and the loss is calculated with the corresponding labels, and the gradient is back-propagated and the network parameters are updated;

[0020] Create "key areas" based on the BScan 2D slice image data of OCT and the corresponding semantic segmentation labels of the stratification, optic disc area, and macular concave area;

[0021] Obtain data maps of "key areas" and lesion labels of stratification, drusen, pigment epithelial detachment, and choroidal neovascularization, perform data augmentation, normalize the data, and randomly shuffle them to create the second-stage data set;

[0022] In each step of the second-stage network training, the processed image data in the second-stage data set is input into the convolutional neural network of the first stage to obtain the semantic segmentation prediction results of the second-stage stratification, drusen, pigment epithelial detachment, and choroidal neovascular lesions. The loss is calculated with the corresponding labels, and the gradient is back-propagated and the network parameters are updated.

[0023] Furthermore, the method of making the "key area" is the same as the method of making the "area of ​​interest".

[0024] Furthermore, it also includes: optimizing the network using Adam optimizer.

[0025] In a second aspect, the present invention provides an OCT image stratification and lesion semantic segmentation device, comprising:

[0026] A first acquisition unit is used to acquire each BScan two-dimensional slice image data of OCT of one eye of the patient to be examined, and perform normalization processing;

[0027] The first input unit is used to input the normalized slice image data one by one into the pre-trained first-stage convolutional neural network to obtain the semantic segmentation prediction results of the first-stage layers, optic disc areas, and macular concave areas;

[0028] A processing unit, used for making a "region of interest" according to the semantic segmentation prediction results of the first stage of stratification, optic disc area, and macular fovea area;

[0029] The second acquisition unit is used to acquire the data map of the "area of ​​interest" and perform normalization processing on the data;

[0030] The second input unit inputs the normalized “region of interest” data map into the second-stage convolutional neural network to obtain the second-stage semantic segmentation prediction results of stratification, drusen, pigment epithelial detachment, and choroidal neovascularization lesions.

[0031] In a third aspect, the present invention provides an OCT image layering and lesion semantic segmentation device, including a processor and a storage medium;

[0032] The storage medium is used to store instructions;

[0033] The processor is used to operate according to the instructions to execute the steps of any of the methods described above.

[0034] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention provides an OCT image stratification and lesion semantic segmentation method, device and storage medium. Since the training data of the second-stage network is concentrated on the "focus area", the learning of the model is more targeted at the main affected area of ​​the lesion of age-related macular degeneration; since the second-stage network reduces the sample input of irrelevant areas, the proportion of lesion pixels in the total pixels is increased in disguise, and the learning difficulty of small-area positive lesions such as drusen is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of an OCT image stratification and lesion semantic segmentation method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0039] Example 1

[0040] This embodiment introduces an OCT image layering and lesion semantic segmentation method, device and storage medium, including:

[0041] Obtain each BScan two-dimensional slice image data of OCT of one eye of the patient to be examined, and perform normalization processing;

[0042] The normalized slice image data are input into the pre-trained first-stage convolutional neural network one by one to obtain the semantic segmentation prediction results of the first-stage layers, optic disc areas, and macular concave areas;

[0043] Create a "region of interest" based on the semantic segmentation prediction results of the first stage of stratification, optic disc area, and macular concave area;

[0044] Get the data graph of the "area of ​​interest" and normalize the data;

[0045] The normalized “region of interest” data graph is input into the second-stage convolutional neural network to obtain the semantic segmentation prediction results of the second-stage stratification, drusen, pigment epithelial detachment, and choroidal neovascularization lesions.

[0046] The OCT image layering and lesion semantic segmentation method, device and storage medium provided in this embodiment specifically involve the following steps in their application process:

[0047] 1. Training and verification network stage

[0048] 1. First, the OCT BScan two-dimensional slice image data and the corresponding semantic segmentation labels of the layer, optic disc area, and macular concave area are augmented, normalized, and randomly shuffled to produce the first-stage data set;

[0049] 2. In each step of the first-stage network training, the processed image data in the first-stage data set is input into the first-stage convolutional neural network to obtain the semantic segmentation prediction results of the first-stage layer, optic disc area, and macular concave area, and the loss is calculated with the corresponding labels, and the gradient is back-propagated and the network parameters are updated. The network optimizer uses Adam;

[0050] 3. Create a "region of interest" based on the OCT BScan two-dimensional slice image data and the corresponding semantic segmentation labels of the layer, optic disc area, and macular concave area. The specific method is:

[0051] (1) Select the slice with the thinnest nerve fiber layer in the column where the center point of the macular concave area in the OCT data label of the same eye is located, and the center point of the macular concave area is used as the center point of the "fovea centralis" of the OCT image of the eye;

[0052] (2) The maximum value of the width of the optic disc area in the OCT data label of the same eye was selected as the “optic disc diameter” of the OCT image;

[0053] (3) According to the values ​​of horizontal and vertical physical resolution and pixel resolution in the specific mode of the OCT model, the horizontal pixel number of the two optic disc diameters is converted into the horizontal physical size, and the "area of ​​interest circle" is determined with the fovea centralis as the center and the physical value of the optic disc diameter as the radius, and then converted into the pixel width of the "area of ​​interest" corresponding to the "area of ​​interest circle" in each slice.

[0054] (4) On the slice, the column where the center point of the fovea is located is taken as the center point, the width is the pixel width converted in (3), and the center of the circumscribed rectangle of the polygon of the non-background area of ​​the underlying structure label of these columns is taken as the center, and the specific number of pixels is taken as the height (the height is an empirical value set according to the statistical value of the layer height of the OCT image), to obtain the "area of ​​interest";

[0055] 4. Take the data map of the "area of ​​interest" and the lesion label maps of stratification, drusen, pigment epithelial detachment, and choroidal neovascularization for data augmentation, normalize the data, and randomly shuffle them to make the second stage data set;

[0056] 5. In each step of the second-stage network training, the processed image data in the second-stage data set is input into the convolutional neural network of the first stage to obtain the semantic segmentation prediction results of the second-stage stratification, drusen, pigment epithelial detachment, and choroidal neovascularization lesions, and the loss is calculated with the corresponding labels, and the gradient is back-propagated and the network parameters are updated. The network optimizer uses Adam;

[0057] 2. Model Reasoning Stage

[0058] 1. Normalize each BScan 2D slice image data of OCT of one eye of a patient to be examined;

[0059] 2. Input the slice image data one by one into the trained first-stage convolutional neural network to obtain the semantic segmentation prediction results of the first-stage layers, optic disc areas, and macular concave areas;

[0060] 3. Create a "region of interest" based on the semantic segmentation prediction results of the stratification, optic disc area, and macular concave area of ​​the BScan 2D slice image data of the eye's OCT. The specific method is similar to the method of creating a "key area" in the training and verification stages:

[0061] (1) Select the slice with the thinnest nerve fiber layer in the column where the center point of the macular concave area in the first-stage network prediction result of the same eye's OCT is located, and the center point of the macular concave area is used as the center point of the "fovea centralis" of the OCT image of this eye; if no macular concave area is predicted on all slices, the solution in (3) is adopted;

[0062] (2) The maximum value of the width of the optic disc area in the prediction results of all slices of the first-stage network of the same eye OCT is selected as the "optic disc diameter" of the OCT image; if no optic disc area is predicted on all slices, the solution in (3) is adopted;

[0063] (3) When no macular depression area is predicted on any slice or no optic disc area is predicted on any slice, the “area of ​​interest” of each slice is the complete image of the entire slice;

[0064] (4) According to the values ​​of horizontal and vertical physical resolution and pixel resolution in the specific mode of the OCT model, the horizontal pixel numbers of the two optic disc diameters are converted into horizontal physical dimensions, and the "area of ​​interest circle" is determined with the fovea centralis as the center and the physical value of the optic disc diameter as the radius, and then converted into the pixel width of the "area of ​​interest" corresponding to the "area of ​​interest circle" in each slice;

[0065] (4) On the slice, the column where the center point of the fovea is located is taken as the center point, the width is the pixel width converted in (3), and the center of the circumscribed rectangle of the polygon of the non-background area in the prediction results of the underlying structure of these columns is taken as the center, and the specific number of pixels is taken as the height (the height is an empirical value set according to the statistical value of the layer height of the OCT image), to obtain the "area of ​​interest";

[0066] 4. Take the data map of the “area of ​​interest” and normalize the data;

[0067] 5. The normalized “region of interest” data map is input into the second-stage convolutional neural network to obtain the semantic segmentation prediction results of the second-stage stratification, drusen, pigment epithelial detachment, and choroidal neovascularization lesions.

[0068] Example 2

[0069] This embodiment provides an OCT image layering and lesion semantic segmentation device, including:

[0070] A first acquisition unit is used to acquire each BScan two-dimensional slice image data of OCT of one eye of the patient to be examined, and perform normalization processing;

[0071] The first input unit is used to input the normalized slice image data one by one into the pre-trained first-stage convolutional neural network to obtain the semantic segmentation prediction results of the first-stage layers, optic disc areas, and macular concave areas;

[0072] A processing unit, used for making a "region of interest" according to the semantic segmentation prediction results of the first stage of stratification, optic disc area, and macular fovea area;

[0073] The second acquisition unit is used to acquire the data map of the "area of ​​interest" and perform normalization processing on the data;

[0074] The second input unit inputs the normalized “region of interest” data map into the second-stage convolutional neural network to obtain the second-stage semantic segmentation prediction results of stratification, drusen, pigment epithelial detachment, and choroidal neovascularization lesions.

[0075] Example 3

[0076] This embodiment provides an OCT image layering and lesion semantic segmentation device, including a processor and a storage medium;

[0077] The storage medium is used to store instructions;

[0078] The processor is used to operate according to the instructions to execute the steps of the method according to any one of Embodiment 1.

[0079] Example 4

[0080] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of any method described in Embodiment 1 are implemented.

[0081] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for OCT image stratification and lesion semantic segmentation, characterized in that: include: Obtain each BScan two-dimensional slice image data of OCT of one eye of the patient to be examined, and perform normalization processing; The normalized slice image data are input into the pre-trained first-stage convolutional neural network one by one to obtain the semantic segmentation prediction results of the first-stage layers, optic disc areas, and macular concave areas; Create a "region of interest" based on the semantic segmentation prediction results of the first stage of stratification, optic disc area, and macular concave area; Get the data graph of the "area of ​​interest" and normalize the data; The normalized "area of ​​interest" data graph is input into the second-stage convolutional neural network to obtain the semantic segmentation prediction results of the second-stage stratification, drusen, pigment epithelial detachment, and choroidal neovascularization lesions; According to the semantic segmentation prediction results of the first stage of stratification, optic disc area, and macular concave area, the "region of interest" is created, including: Select the thinnest slice of the nerve fiber layer in the column where the center point of the macular depression area in the OCT data label of the same eye is located, and the center point of the macular depression area is used as the center point of the "fovea centralis" of the OCT image of the eye; The maximum value of the width of the optic disc area in all slice data labels of the same eye's OCT was selected as the "optic disc diameter" of the OCT image; According to the values ​​of horizontal and vertical physical resolution and pixel resolution in the specific mode of the OCT model, the horizontal pixel numbers of the two optic disc diameters are converted into horizontal physical dimensions, and the "area of ​​interest circle" is determined with the fovea centralis as the center and the physical value of the optic disc diameter as the radius, and then converted into the pixel width of the "area of ​​interest" corresponding to the "area of ​​interest circle" in each slice; On the slice, the column where the center point of the fovea is located is taken as the center point, the width is the pixel width obtained by conversion, and the center of the circumscribed rectangle of the polygon of the non-background area of ​​the underlying eye structure label of these columns is taken as the center, and the specific number of pixels is taken as the height to obtain the "area of ​​interest"; The training method of the convolutional neural network in the first stage includes: Obtain OCT BScan two-dimensional slice image data and the corresponding semantic segmentation labels of layers, optic disc areas, and macular concave areas, perform data augmentation, normalization, and random shuffling to create the first-stage data set; In each step of the first-stage network training, the processed image data in the first-stage data set is input into the first-stage convolutional neural network to obtain the semantic segmentation prediction results of the first-stage layers, optic disc areas, and submacular fovea areas, and the loss is calculated with the corresponding labels, and the gradient is back-propagated and the network parameters are updated; Create "key areas" based on the BScan 2D slice image data of OCT and the corresponding semantic segmentation labels of the stratification, optic disc area, and macular concave area; Obtain the data maps of "key areas" and the lesion labels of stratification, drusen, pigment epithelial detachment, and choroidal neovascularization, perform data augmentation, normalize the data, and randomly shuffle them to create the second-stage data set; In each step of the second-stage network training, the processed image data in the second-stage data set is input into the convolutional neural network of the first stage to obtain the semantic segmentation prediction results of the second-stage stratification, drusen, pigment epithelial detachment, and choroidal neovascular lesions. The loss is calculated with the corresponding labels, and the gradient is back-propagated and the network parameters are updated.

2. The OCT image layering and lesion semantic segmentation method according to claim 1, characterized in that: The method of making the "key area" is the same as the method of making the "area of ​​interest".

3. The OCT image layering and lesion semantic segmentation method according to claim 2, characterized in that: Also includes: The Adam optimizer is used to optimize the network.

4. An OCT image stratification and lesion semantic segmentation device, characterized in that: include: A first acquisition unit is used to acquire each BScan two-dimensional slice image data of OCT of one eye of the patient to be examined, and perform normalization processing; The first input unit is used to input the normalized slice image data one by one into the pre-trained first-stage convolutional neural network to obtain the semantic segmentation prediction results of the first-stage layers, optic disc areas, and macular concave areas; A processing unit, used for generating a "region of interest" according to the semantic segmentation prediction results of the first stage of stratification, optic disc area, and macular concave area; The second acquisition unit is used to acquire the data map of the "area of ​​interest" and normalize the data; The second input unit inputs the normalized "area of ​​interest" data graph into the second-stage convolutional neural network to obtain the semantic segmentation prediction results of the second-stage stratification, drusen, pigment epithelial detachment, and choroidal neovascularization lesions; Wherein, in the processing unit, according to the semantic segmentation prediction results of the first stage of stratification, optic disc area, and macular concave area, a "region of interest" is prepared, including: Select the thinnest slice of the nerve fiber layer in the column where the center point of the macular depression area in the OCT data label of the same eye is located, and the center point of the macular depression area is used as the center point of the "fovea centralis" of the OCT image of the eye; The maximum value of the width of the optic disc area in all slice data labels of the same eye's OCT was selected as the "optic disc diameter" of the OCT image; According to the values ​​of horizontal and vertical physical resolution and pixel resolution in the specific mode of the OCT model, the horizontal pixel numbers of the two optic disc diameters are converted into horizontal physical dimensions, and the "area of ​​interest circle" is determined with the fovea centralis as the center and the physical value of the optic disc diameter as the radius, and then converted into the pixel width of the "area of ​​interest" corresponding to the "area of ​​interest circle" in each slice; On the slice, the column where the center point of the fovea is located is taken as the center point, the width is the pixel width obtained by conversion, and the center of the circumscribed rectangle of the polygon of the non-background area of ​​the underlying eye structure label of these columns is taken as the center, and the specific number of pixels is taken as the height to obtain the "area of ​​interest"; In the first input unit, the training method of the convolutional neural network in the first stage includes: Obtain OCT BScan two-dimensional slice image data and the corresponding semantic segmentation labels of layers, optic disc areas, and macular concave areas, perform data augmentation, normalization, and random shuffling to create the first-stage data set; In each step of the first-stage network training, the processed image data in the first-stage data set is input into the first-stage convolutional neural network to obtain the semantic segmentation prediction results of the first-stage layers, optic disc areas, and submacular fovea areas, and the loss is calculated with the corresponding labels, and the gradient is back-propagated and the network parameters are updated; Create "key areas" based on the BScan 2D slice image data of OCT and the corresponding semantic segmentation labels of the stratification, optic disc area, and macular concave area; Obtain the data maps of "key areas" and the lesion labels of stratification, drusen, pigment epithelial detachment, and choroidal neovascularization, perform data augmentation, normalize the data, and randomly shuffle them to create the second-stage data set; In each step of the second-stage network training, the processed image data in the second-stage data set is input into the convolutional neural network of the first stage to obtain the semantic segmentation prediction results of the second-stage stratification, drusen, pigment epithelial detachment, and choroidal neovascular lesions. The loss is calculated with the corresponding labels, and the gradient is back-propagated and the network parameters are updated.

5. An OCT image layering and lesion semantic segmentation device, characterized by: including processor and storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images