Three-dimensional optical coherence tomography image segmentation method, device and equipment based on Unet network and medium

Abstract

The application discloses a three-dimensional optical coherence tomography image segmentation method and device based on a Unet network, equipment and a medium, and relates to the field of image processing. The method comprises the following steps: determining affine transformation parameters based on adjacent two-dimensional tomographic images in an original three-dimensional OCT image; the original three-dimensional OCT image is an image obtained after optical coherence tomography is performed on a target eye region; using a first Unet network to identify layering lines from each two-dimensional tomographic image of the original three-dimensional OCT image; transforming the original three-dimensional OCT image based on the affine transformation parameters, and processing corresponding two-dimensional tomographic images in the transformed three-dimensional OCT image using the layering lines to obtain a processed three-dimensional OCT image; and using a second Unet network to segment the retinal layered structure of the processed three-dimensional OCT image to obtain a segmentation result. The application can improve the segmentation continuity and accuracy of the three-dimensional OCT image.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method, apparatus, device, and medium for three-dimensional optical coherence tomography image segmentation based on Unet networks. Background Technology

[0002] Optical coherence tomography (OCT) is a high-resolution, non-invasive imaging technique widely used in ophthalmology, dermatology, cardiology, and many other fields. Three-dimensional OCT images provide three-dimensional structural information of biological tissues; however, extracting useful information from complex three-dimensional OCT images is a challenging task, especially when precise segmentation of specific or diseased tissues is required.

[0003] Currently, most common methods are based on segmentation and recognition of two-dimensional OCT tomographic scans. However, these methods are difficult to accurately identify areas where large blood vessels intersect. Furthermore, for patients with fundus diseases, the tissue structure may have abnormalities or weak signals, which can lead to inaccurate recognition of a single two-dimensional OCT tomographic scan. As a result, the segmentation results projected onto a three-dimensional OCT image have poor continuity, making it difficult to achieve accurate segmentation of the three-dimensional OCT image. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for segmenting three-dimensional optical coherence tomography (OCT) images based on a Unet network, which can ensure the segmentation continuity of three-dimensional OCT images and improve the segmentation accuracy of three-dimensional OCT images. The specific solution is as follows:

[0005] In a first aspect, this application provides a three-dimensional optical coherence tomography image segmentation method based on the Unet network, including:

[0006] The original three-dimensional optical coherence tomography (OCT) image is acquired, and the affine transformation parameters are determined based on the adjacent two-dimensional tomographic images in the original three-dimensional OCT image; the original three-dimensional OCT image is the image obtained after performing optical coherence tomography on the target eye region;

[0007] The first preset Unet network is used to identify each two-dimensional tomographic scan in the original three-dimensional optical coherence tomographic image, so as to identify the layer lines from each two-dimensional tomographic scan.

[0008] The original three-dimensional optical coherence tomography image is transformed based on the affine transformation parameters to obtain the transformed three-dimensional optical coherence tomography image.

[0009] The corresponding two-dimensional tomographic images in the transformed three-dimensional optical coherence tomography image are processed using the layered lines to flatten the corresponding two-dimensional tomographic images, thereby obtaining the processed three-dimensional optical coherence tomography image.

[0010] The processed three-dimensional optical coherence tomography image is cut into several overlapping sub-blocks, and the retinal layer structure is segmented in parallel using a second preset Unet network. The layer structure segmentation result of the processed three-dimensional optical coherence tomography image is determined by stitching together the segmentation results of each sub-block.

[0011] Optionally, determining the affine transformation parameters based on adjacent two-dimensional tomographic images in the original three-dimensional optical coherence tomography image includes:

[0012] The adjacent two-dimensional tomographic scan images are cropped to obtain a reference image and an image to be registered. Translation parameters are determined based on the reference image and the image to be registered. The translation parameters are then used to perform preliminary registration on the image to be registered to obtain a preliminary registered image.

[0013] The overlapping region of the preliminary registration map and the reference map is determined, and the similarity between the preliminary registration map and the reference map in the overlapping region is determined based on the gray values ​​of each pixel in the overlapping region.

[0014] If the similarity is not less than a preset threshold, then the translation parameter is determined as an affine transformation parameter;

[0015] If the similarity is less than a preset threshold, then scaling and rotation parameters are determined based on the reference image and the preliminary registration image, and affine transformation parameters are determined based on the scaling and rotation parameters and the translation parameters.

[0016] Optionally, the step of cropping the adjacent two-dimensional tomographic scans to obtain a reference image and a registration image, and determining translation parameters based on the reference image and the registration image, includes:

[0017] Determine the average gray value of each row of pixels in any image; the any image is any one of the adjacent two-dimensional tomographic scan images;

[0018] Based on the maximum value in the average grayscale value, the center line of the region is determined from any image, and a rectangular region of a first preset size is cropped from any image according to the center line of the region to obtain the cropped image.

[0019] The cropped images corresponding to the adjacent two-dimensional tomographic scan images are determined as the reference image and the image to be registered. Fourier transform is performed on the reference image and the image to be registered, and a cross power spectrum matrix is ​​constructed based on the first image obtained after the Fourier transform.

[0020] An inverse Fourier transform is performed on the cross-power spectrum matrix to obtain the impulse response function, and a translation parameter is determined based on the position corresponding to the pulse peak of the impulse response function.

[0021] Optionally, determining the scaling and rotation parameters based on the reference map and the preliminary registration map includes:

[0022] Perform Fourier transform on the reference image and the preliminary registration image, and determine the amplitude spectrum of the second image obtained after Fourier transform in the Cartesian coordinate system;

[0023] The amplitude spectrum is converted from Cartesian coordinates to polar coordinates, and scaling and rotation parameters are determined based on the amplitude spectrum in polar coordinates.

[0024] Optionally, the step of using a first preset Unet network to identify each two-dimensional tomographic image in the original three-dimensional optical coherence tomography image, so as to identify the layer lines from each two-dimensional tomographic image, includes:

[0025] The first preset Unet network is used to identify each two-dimensional tomographic image in the original three-dimensional optical coherence tomography image, so as to generate a probability value for each pixel in each two-dimensional tomographic image.

[0026] Target pixels are selected from each column of pixels in each of the two-dimensional tomographic scan images; the probability value of the target pixel is the maximum value among the probability values ​​of the column in which the target pixel is located.

[0027] The layer lines in each two-dimensional tomographic scan are determined based on the target pixels in each two-dimensional tomographic scan.

[0028] Mean filtering is used to smooth the layer lines in each two-dimensional tomographic scan to optimize the layer lines in each two-dimensional tomographic scan.

[0029] Optionally, the step of processing the corresponding two-dimensional tomographic images in the transformed three-dimensional optical coherence tomography image using the layered lines to flatten the corresponding two-dimensional tomographic images and obtain the processed three-dimensional optical coherence tomography image includes:

[0030] Based on the ordinate of each pixel on the layer line, determine the average ordinate corresponding to the layer line;

[0031] The corresponding two-dimensional tomographic scan in the transformed three-dimensional optical coherence tomography image is processed using the layered lines to determine the target ordinate corresponding to the abscissa of each pixel in the corresponding two-dimensional tomographic scan from the layered lines. The ordinate of each pixel is then transformed using the difference between the target ordinate and the average ordinate to flatten the corresponding two-dimensional tomographic scan, resulting in a flattened two-dimensional tomographic scan.

[0032] A rectangular area of ​​a second preset size is extracted from the flattened two-dimensional tomographic image based on the flattened layer lines to obtain the extracted two-dimensional tomographic image.

[0033] Based on the extracted two-dimensional tomographic images, the processed three-dimensional optical coherence tomographic image is determined.

[0034] Optionally, the retinal layered structure includes a first layered structure and a second layered structure, wherein the first layered structure includes a retinal nerve fiber layer, and the second layered structure includes a ganglion cell complex, an outer plexiform layer, and a pigment epithelium layer.

[0035] Accordingly, after determining the hierarchical structure segmentation result of the processed three-dimensional optical coherence tomography image by stitching together the segmentation results of each sub-block, the method further includes:

[0036] The layered structure segmentation result of the processed three-dimensional optical coherence tomography image is unfolded into several two-dimensional images along the axial direction;

[0037] Based on the grayscale value of each pixel and the previous pixel in the two-dimensional image, the first gradient value of each pixel is determined to construct the upward gradient field of the two-dimensional image.

[0038] Based on the grayscale values ​​of each pixel and the next pixel in the two-dimensional image, a second gradient value for each pixel is determined to construct the down-direction gradient field of the two-dimensional image.

[0039] The first and second layered structures in the two-dimensional image are bidirectionally searched using the upward gradient field and the downward gradient field, respectively, to search for gradient extrema from both sides of each layered structure. The gradient extrema are then used to perform gradient correction on the corresponding layered structures in the two-dimensional image to obtain the corrected image.

[0040] The corrected hierarchical structure segmentation result of the processed three-dimensional optical coherence tomography image is determined based on each of the corrected images;

[0041] In this configuration, the previous pixel and the next pixel are both located in the same column as each pixel, with the previous pixel located in the row above each pixel and the next pixel located in the row below each pixel.

[0042] Secondly, this application provides a three-dimensional optical coherence tomography image segmentation device based on a Unet network, comprising:

[0043] The transformation parameter determination module is used to acquire the original three-dimensional optical coherence tomography image and determine the affine transformation parameters based on the adjacent two-dimensional tomographic images in the original three-dimensional optical coherence tomography image; the original three-dimensional optical coherence tomography image is the image obtained after optical coherence tomography imaging of the target eye region;

[0044] The layer line recognition module is used to identify each two-dimensional tomographic scan image in the original three-dimensional optical coherence tomographic scan image using a first preset Unet network, so as to identify the layer lines from each two-dimensional tomographic scan image respectively.

[0045] The image transformation module is used to transform the original three-dimensional optical coherence tomography image based on the affine transformation parameters to obtain the transformed three-dimensional optical coherence tomography image.

[0046] The image processing module is used to process the corresponding two-dimensional tomographic images in the transformed three-dimensional optical coherence tomography image using the layered lines, so as to flatten the corresponding two-dimensional tomographic images and obtain the processed three-dimensional optical coherence tomography image.

[0047] The image segmentation module is used to cut the processed three-dimensional optical coherence tomography image into several overlapping sub-blocks, and to use a second preset Unet network to segment the retinal layer structure of each sub-block in parallel, so as to determine the layer structure segmentation result of the processed three-dimensional optical coherence tomography image by stitching together the segmentation results of each sub-block.

[0048] Thirdly, this application provides an electronic device, comprising:

[0049] Memory, used to store computer programs;

[0050] A processor is used to execute the computer program to implement the aforementioned three-dimensional optical coherence tomography image segmentation method based on Unet network.

[0051] Fourthly, this application provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned three-dimensional optical coherence tomography image segmentation method based on the Unet network.

[0052] In this application, an original three-dimensional optical coherence tomography (OCT) image is acquired, and affine transformation parameters are determined based on adjacent two-dimensional tomographic images in the original three-dimensional OCT image. The original three-dimensional OCT image is an image obtained after performing optical coherence tomography on the target eye region. A first preset Unet network is used to identify each two-dimensional tomographic image in the original three-dimensional OCT image to identify the layer lines from each two-dimensional tomographic image. The original three-dimensional OCT image is then transformed based on the affine transformation parameters. A transformed three-dimensional optical coherence tomography (OCT) image is obtained. The corresponding two-dimensional tomographic images in the transformed three-dimensional OCT image are processed using the layer lines to flatten the corresponding two-dimensional tomographic images, resulting in a processed three-dimensional OCT image. The processed three-dimensional OCT image is then divided into several overlapping sub-blocks, and each sub-block is segmented into retinal layer structures in parallel using a second preset Unet network. The layer structure segmentation result of the processed three-dimensional OCT image is determined by stitching together the segmentation results of each sub-block.

[0053] Therefore, this application segments a three-dimensional optical coherence tomography (OCT) image into several overlapping sub-blocks and uses a second preset Unet network to segment the retinal layer structure of each sub-block in parallel. The segmentation results of each sub-block are then stitched together to determine the layer structure segmentation result of the three-dimensional OCT image. By segmenting the three-dimensional OCT image into several overlapping sub-blocks, the second preset Unet network has more contextual information when segmenting the three-dimensional OCT image, improving the segmentation effect. Furthermore, by segmenting the retinal layer structure of each sub-block in parallel, this application can significantly improve the segmentation efficiency of the three-dimensional OCT image. Moreover, by combining a first preset Unet network for recognizing two-dimensional tomographic images and a second preset Unet network for segmenting three-dimensional OCT images, this application not only ensures the continuity of the three-dimensional OCT image segmentation but also improves the segmentation accuracy. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0055] Figure 1 This application discloses a flowchart of a three-dimensional optical coherence tomography image segmentation method based on a Unet network.

[0056] Figure 2 This is a schematic diagram of the layering lines of a pigmented epithelial layer disclosed in this application;

[0057] Figure 3 This is a schematic diagram of a truncated two-dimensional tomographic image disclosed in this application;

[0058] Figure 4 This is a schematic diagram of an enhanced two-dimensional tomographic image disclosed in this application;

[0059] Figure 5 This is a schematic diagram of a modified hierarchical structure segmentation result disclosed in this application;

[0060] Figure 6 This is a flowchart illustrating the determination of affine transformation parameters as disclosed in this application;

[0061] Figure 7 This is a schematic diagram of a three-dimensional optical coherence tomography image segmentation device based on a Unet network disclosed in this application;

[0062] Figure 8 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Segmentation based on two-dimensional OCT tomography images is difficult to accurately identify areas where large blood vessels intersect. Furthermore, for patients with fundus diseases, structural mutations and weak signals in certain areas can lead to inaccurate segmentation of a single two-dimensional OCT tomography image, resulting in poor continuity in the segmentation results projected onto a three-dimensional OCT image, hindering accurate segmentation of the 3D OCT image. Therefore, this application provides a three-dimensional optical coherence tomography image segmentation method based on a Unet network, which can ensure the continuity of 3D optical coherence tomography image segmentation and improve the segmentation accuracy.

[0065] See Figure 1As shown, this embodiment of the invention discloses a three-dimensional optical coherence tomography image segmentation method based on Unet networks, including:

[0066] Step S11: Obtain the original three-dimensional optical coherence tomography image, and determine the affine transformation parameters based on the adjacent two-dimensional tomographic images in the original three-dimensional optical coherence tomography image; the original three-dimensional optical coherence tomography image is the image obtained after optical coherence tomography imaging of the target eye region.

[0067] In this embodiment, optical coherence tomography (OCT) is performed on the target eye region to obtain a raw three-dimensional OCT image. It should be noted that the three-dimensional OCT image is a three-dimensional image constructed by superimposing multiple two-dimensional tomographic images.

[0068] Considering that during the process of optical coherence tomography of the target eye region, eye movement or camera focus shift can easily cause rigid transformations such as translation, rotation and scaling between adjacent two-dimensional tomographic images in the original three-dimensional optical coherence tomography image, ultimately resulting in spatial mismatch in the original three-dimensional optical coherence tomography image.

[0069] Based on this, this application adopts a two-stage registration strategy to register adjacent two-dimensional tomographic images in the original three-dimensional optical coherence tomography image, so as to determine the affine transformation parameters between adjacent two-dimensional tomographic images, thereby performing an affine transformation on the original three-dimensional optical coherence tomography image and solving the problem of spatial mismatch in the original three-dimensional optical coherence tomography image.

[0070] Specifically, adjacent two-dimensional tomographic images in the original three-dimensional optical coherence tomography (OCT) image are cropped to obtain a reference image and an image to be registered. Then, Fourier transform is used to perform preliminary registration of the reference image and the image to be registered to obtain translation parameters. The image to be registered is translated using the translation parameters to obtain a preliminary registration image. It is determined whether the preliminary registration image meets the preset registration accuracy requirements. If it does, the translation parameters are determined as affine transformation parameters. If it does not, Fourier-Merlin transform is used to perform high-precision registration of the reference image and the preliminary registration image to obtain scaling and rotation parameters. Then, the affine transformation parameters are determined based on the scaling and rotation parameters and the translation parameters.

[0071] Step S12: Use the first preset Unet network to identify each two-dimensional tomographic scan in the original three-dimensional optical coherence tomographic image, so as to identify the layer lines from each two-dimensional tomographic scan.

[0072] The first preset Unet network in this application embodiment can be a 2D-Unet network, used to identify the layer lines of the preset retinal layer from the two-dimensional tomographic scan image. Furthermore, the core features of the Unet network are U-shaped symmetric structure, encoder-decoder design, and skip connections, which can effectively fuse multi-scale features and solve the problem of detail loss under small sample data.

[0073] It should be noted that the first preset Unet network in this application specifically includes an encoder, a decoder, and an output layer; wherein, the encoder adopts a 4-level downsampling module, each downsampling module contains two 3×3 convolutions + ReLU (Rectified Linear Unit) + 2×2 max pooling, and the number of channels increases from 64 to 512; the decoder adopts a 4-level upsampling module, which restores the resolution through transposed convolution and fuses multi-scale features with the encoder through skip connections; the output layer adopts a 1×1 convolution + Sigmoid activation function, which is used to generate a probability value for each pixel in the two-dimensional tomographic scan image, and the probability value ranges between [0,1].

[0074] Specifically, the first preset Unet network is used to identify each two-dimensional tomographic image in the original three-dimensional optical coherence tomography (OCT) image, generating a probability value for each pixel in each two-dimensional tomographic image. Target pixels are selected from each column of pixels in each two-dimensional tomographic image, where the probability value of the target pixel is the maximum value among the probability values ​​of the column containing the target pixel. Layer lines in each two-dimensional tomographic image are determined based on the target pixels. Mean filtering is used to smooth the layer lines in each two-dimensional tomographic image to optimize them. For example, the layer lines are taken as the layer lines of the pigment epithelium (RPE). Figure 2 As shown, the red lines are the layering lines of the pigment epithelium. In addition to the pigment epithelium, other retinal layers may also be used in this application, and no specific limitation is made here.

[0075] Taking any two-dimensional tomographic scan image as an example, the first preset Unet network is used to identify the image, generating a probability value P(x,y) for each pixel (x,y) in the image; where x represents the x-coordinate and y represents the y-coordinate. For each column of pixels in the image, a target pixel is selected, where the probability value of the target pixel is the maximum of the probability values ​​of the pixels in the same column. The target pixels in the image are then smoothly connected to obtain the layer lines. Mean filtering is then used to smooth these layer lines, optimizing them.

[0076] In the process of smoothing the layer lines in any two-dimensional tomographic scan using mean filtering, for any pixel on the layer line, the neighborhood pixels of any pixel are determined from any two-dimensional tomographic scan. Based on the neighborhood pixels and the average values ​​of the horizontal and vertical coordinates of any pixel, an optimized pixel is determined. This optimized pixel is then used to replace any pixel, thereby optimizing the layer line, suppressing local jitter, and improving the continuity and smoothness of the layer line.

[0077] Step S13: Transform the original three-dimensional optical coherence tomography image based on the affine transformation parameters to obtain the transformed three-dimensional optical coherence tomography image.

[0078] In this embodiment, each two-dimensional tomographic image in the original three-dimensional optical coherence tomography image is transformed based on affine transformation parameters to obtain the transformed three-dimensional optical coherence tomography image.

[0079] When the affine transformation parameters are translation parameters At that time, based on the affine transformation parameters, each two-dimensional tomographic image in the original three-dimensional optical coherence tomography (OCT) image is translated to obtain the transformed three-dimensional OCT image. Among them, This represents the change in the horizontal axis. Let represent the change in the ordinate. In this case, the translation transformation of any pixel (x, y) in a two-dimensional tomographic image can be expressed as:

[0080] .

[0081] The affine transformation parameters are based on scaling and rotation parameters. Translation parameters Determined parameters At that time, based on the affine transformation parameters, each two-dimensional tomographic image in the original three-dimensional optical coherence tomography (OCT) image is transformed by translation, scaling, and rotation to obtain the transformed three-dimensional OCT image. Here, s represents the scaling factor. Let represent the rotation angle. Then, for any pixel (x, y) in a two-dimensional tomographic image, the translation, scaling, and rotation transformations can be expressed as:

[0082] .

[0083] In this way, by using affine transformation parameters to transform the original three-dimensional optical coherence tomography image, this application can eliminate the misalignment of the original three-dimensional optical coherence tomography image caused by eye movement or camera focus shift.

[0084] Step S14: Use the layered lines to process the corresponding two-dimensional tomographic images in the transformed three-dimensional optical coherence tomography image to flatten the corresponding two-dimensional tomographic images and obtain the processed three-dimensional optical coherence tomography image.

[0085] In this embodiment of the application, for the layer lines in each two-dimensional tomographic scan image, the corresponding two-dimensional tomographic scan image in the transformed three-dimensional optical coherence tomographic scan image is processed using the layer lines to flatten the corresponding two-dimensional tomographic scan image, thereby obtaining the processed three-dimensional optical coherence tomographic scan image.

[0086] Specifically, based on the ordinate of each pixel on the layer line, the average ordinate corresponding to the layer line is determined; the corresponding two-dimensional tomographic scan in the transformed three-dimensional optical coherence tomography image is processed using the layer line to determine the target ordinate corresponding to the abscissa of each pixel in the corresponding two-dimensional tomographic scan from the layer line, and the ordinate of each pixel is transformed using the difference between the target ordinate and the average ordinate to flatten the corresponding two-dimensional tomographic scan, resulting in a flattened two-dimensional tomographic scan; a rectangular area of ​​a second preset size is extracted from the flattened two-dimensional tomographic scan based on the flattened layer line to obtain a truncated two-dimensional tomographic scan; based on each truncated two-dimensional tomographic scan, the processed three-dimensional optical coherence tomographic image is determined.

[0087] Taking the layer line as y=f(x), x=1,2,…,W as an example, where x represents the x-coordinate of a pixel and y represents the y-coordinate of a pixel, the average y-coordinate corresponding to the layer line is then... Furthermore, for each pixel (x, y) in the corresponding two-dimensional tomographic image, the transformation of its ordinate can be expressed as: Where I(x,y) represents the grayscale value of a pixel.

[0088] Furthermore, after flattening the corresponding two-dimensional tomographic scan based on the stratification lines to obtain a flattened two-dimensional tomographic scan, 256 pixels (from stratification line -192 to stratification line +64) are cropped from the flattened two-dimensional tomographic scan along the stratification lines to obtain a cropped two-dimensional tomographic scan. Taking the stratification lines as the stratification lines of the pigment epithelium as an example... Figure 3 That is Figure 2 The corresponding cropped two-dimensional tomographic image can cover the inner and outer nuclear layers of the retina to the choroid. This operation can focus the image on the OCT physiological tissue area and compress the vertical resolution to 25% of the original size. For example, the size of a single frame image can be compressed from 1024×1024 pixels to 256×1024 pixels, and the computational complexity can be reduced by 75% accordingly.

[0089] To improve the clarity of the truncated 2D tomographic image, this application can first apply a median filtering algorithm to reduce speckle noise, then use the CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm to enhance the brightness and contrast of the image, thereby enhancing the image signal, and finally use Gaussian filtering to further enhance the image. Figure 4 That is Figure 3 The corresponding enhanced two-dimensional tomographic scan can not only improve the continuity of OCT tissue, but also ensure the integrity of the OCT image.

[0090] Step S15: Cut the processed three-dimensional optical coherence tomography image into several overlapping sub-blocks, and use a second preset Unet network to perform retinal layering structure segmentation on each sub-block in parallel, so as to determine the layering structure segmentation result of the processed three-dimensional optical coherence tomography image by stitching together the segmentation results of each sub-block.

[0091] The second preset Unet network in this embodiment can be a 3D-Unet network, used to segment the retinal layer structure of the three-dimensional optical coherence tomography image to obtain the layer structure segmentation result of the three-dimensional optical coherence tomography image.

[0092] It should be noted that the second pre-defined Unet network in this application specifically includes an encoder, a decoder, and an output layer; wherein, the encoder adopts a 4-level downsampling module, and the number of channels in each downsampling module increases progressively (64→128→256→512); the decoder adopts a 4-level upsampling module, and each upsampling module contains a transposed convolution with a stride of 2, which is used to restore resolution and fuse features of the same scale with the encoder via skip connections; the number of output channels of the output layer corresponds to the four anatomical structures of the retina (RNFL (Retinal Nerve Fibers Layer), GCC (Ganglion Cell Complex), OPL (Outer Plexiform Layer), RPE (Retinal Pigment Epithelium)), and the output layer adopts the Softmax activation function to normalize the interlayer probabilities.

[0093] Specifically, using a preset sliding window and a preset sliding step size, the processed 3D optical coherence tomography (OCT) image is segmented into several overlapping sub-blocks. A GPU (Graphics Processing Unit) is used to perform retinal layering on each sub-block in parallel based on a second preset Unet network. The layering structure segmentation result of the processed 3D OCT image is determined by stitching together the segmentation results of each sub-block. For example, this application can segment the processed 3D OCT image along the depth axis into 16 sub-blocks, with 8 layers overlapping between adjacent sub-blocks to ensure inter-layer continuity and provide the model with more comprehensive contextual information.

[0094] Taking a processed 3D optical coherence tomography (OCT) image of size 3×3×3 as an example, with a preset sliding window size of 2×2×2 and a preset sliding step size of 1, the processed 3D OCT image can be segmented into eight overlapping sub-blocks of size 2×2×2. This allows the second preset Unet network to have more contextual information when segmenting the 3D OCT image, improving the segmentation effect.

[0095] Considering that the retinal layer structure includes a first layer structure and a second layer structure, the first layer structure includes the retinal nerve fiber layer, and the second layer structure includes the ganglion cell complex, the outer plexiform layer, and the pigment epithelium layer, in order to improve the segmentation rationality and visual smoothness of the layer structure segmentation results of the processed three-dimensional optical coherence tomography image, the layer structure segmentation results of the processed three-dimensional optical coherence tomography image are unfolded into several two-dimensional images along the axis, and gradient correction optimization processing is performed on several two-dimensional images to obtain the corrected layer structure segmentation results.

[0096] Specifically, based on the grayscale values ​​of each pixel and the previous pixel in the 2D image, the first gradient value of each pixel is determined to construct the upward gradient field of the 2D image; based on the grayscale values ​​of each pixel and the next pixel in the 2D image, the second gradient value of each pixel is determined to construct the downward gradient field of the 2D image; the upward and downward gradient fields are used to perform a bidirectional search on the first and second layered structures in the 2D image, respectively, to search for gradient extrema from both sides of each layered structure, and the gradient extrema are used to perform gradient correction on the corresponding layered structures in the 2D image to obtain the corrected image; based on each corrected image, the corrected layered structure segmentation result of the processed 3D optical coherence tomography image is determined, such as... Figure 5 The image shows the corrected hierarchical segmentation result. It is evident that the corrected hierarchical segmentation result is clearer and more distinct. Specifically, the previous pixel and the next pixel are located in the same column as each pixel, with the previous pixel in the row above each pixel and the next pixel in the row below each pixel.

[0097] Assume that the coordinates of each pixel in a 2D image are (x, y), the coordinates of the previous pixel are (x, y+1), and the coordinates of the next pixel are (x, y-1). Furthermore, the first gradient value of each pixel is... I(x,y) represents the grayscale value of each pixel, and the second gradient value of each pixel is... It should be noted that the upward gradient field and the downward gradient field characterize the boundary features from bright to dark and from dark to bright, respectively.

[0098] By using the upward gradient field to perform a bidirectional search on the retinal nerve fiber layer in a two-dimensional image, gradient extrema are searched from both sides of the retinal nerve fiber layer. The gradient extrema are then used to correct the gradient of the retinal nerve fiber layer in the two-dimensional image, which can capture the transition of light and dark from the retinal nerve fiber layer to the vitreous body.

[0099] By using the downward gradient field to perform a bidirectional search on the ganglion cell complex / outer plexus layer / pigmented epithelium in a two-dimensional image, gradient extrema are searched from both sides of the ganglion cell complex / outer plexus layer / pigmented epithelium, and gradient correction is performed on the ganglion cell complex / outer plexus layer / pigmented epithelium in the two-dimensional image using the gradient extrema, which can better adapt to the lower boundary reflection enhancement characteristics.

[0100] In this way, considering that gradient correction is a post-processing technique based on the matching mechanism between the optical properties of biological tissues and gradient directions, this technique improves the segmentation accuracy of three-dimensional optical coherence tomography images by analyzing the brightness gradient characteristics of the retinal layered structure to refine the initial layered structure segmentation results.

[0101] In addition, the training process of the first preset Unet network in this application first acquires several two-dimensional tomographic scans and marks the layer lines of the preset retinal layers on the two-dimensional tomographic scans to construct an initial dataset. Then, data augmentation is performed on the initial dataset to obtain the target dataset. The data augmentation includes geometric deformation: horizontal / vertical flipping (probability 50%), random rotation (-30°~30°), elastic deformation (simulating tissue stretching caused by eye movement); noise injection: adding Gaussian noise (simulating device noise), brightness fluctuation (±20%), etc. Furthermore, a loss function is constructed by combining the Dice coefficient and edge-sensitive loss, and the initial learning rate is set to 1e-4, and the minimum learning rate is set to 1e-6. Then, the 2D-Unet network is trained using the loss function and cosine annealing strategy based on the target dataset to obtain the first preset Unet network.

[0102] The training process for the second preset Unet network begins by acquiring several 3D optical coherence tomography (OCT) images and corresponding retinal layer segmentation results to construct an initial dataset. This initial dataset is then augmented to obtain the target dataset. The augmentation includes spatial deformation: 3D elastic deformation (simulating retinal surface morphology), random rotation (simulating scanning angle deviation), and scaling (0.9–1.1 times, adapting to different imaging resolutions); and noise injection: adding speckle noise (simulating low signal-to-noise ratio regions), motion artifacts (simulating slight eye movements in patients), and device-specific noise templates. A hybrid loss function is then used to train the 3D-Unet network based on the target dataset to obtain the second preset Unet network. The hybrid loss function primarily uses the Dice coefficient as the loss, supplemented by topological continuity constraints to avoid interlayer breaks. Furthermore, during training, this application can start with low-resolution 3D OCT images and gradually transition to high-resolution 3D OCT images to enhance the second preset Unet network's segmentation effect on the retinal layer structure of the 3D OCT images.

[0103] Therefore, this application segments a three-dimensional optical coherence tomography (OCT) image into several overlapping sub-blocks and uses a second preset Unet network to segment the retinal layer structure of each sub-block in parallel. The segmentation results of each sub-block are then stitched together to determine the layer structure segmentation result of the three-dimensional OCT image. By segmenting the three-dimensional OCT image into several overlapping sub-blocks, the second preset Unet network has more contextual information when segmenting the three-dimensional OCT image, improving the segmentation effect. Furthermore, by segmenting the retinal layer structure of each sub-block in parallel, this application can significantly improve the segmentation efficiency of the three-dimensional OCT image. Moreover, by combining a first preset Unet network for recognizing two-dimensional tomographic images and a second preset Unet network for segmenting three-dimensional OCT images, this application not only ensures the continuity of the three-dimensional OCT image segmentation but also improves the segmentation accuracy.

[0104] As described in the previous embodiment, this application describes a complete segmentation process for three-dimensional optical coherence tomography images based on the Unet network. Next, this application will elaborate on how to determine the affine transformation parameters. See [link to previous document]. Figure 6 As shown, this embodiment of the invention discloses a process for determining affine transformation parameters, including:

[0105] Step S21: Crop adjacent two-dimensional tomographic images in the original three-dimensional optical coherence tomography image to obtain a reference image and a registration image, and determine translation parameters based on the reference image and the registration image, and use the translation parameters to perform preliminary registration on the registration image to obtain a preliminary registration image.

[0106] This application takes into account that the original three-dimensional optical coherence tomography image has a high resolution (usually 1024×1024 pixels), but the signal of effective biological tissue only occupies a local area. In order to reduce computational redundancy, this application proposes a signal localization algorithm based on the statistics of lateral gray-level extreme values.

[0107] Specifically, for any image in adjacent two-dimensional tomographic scans, the average gray value of each row of pixels in any image is determined; based on the maximum value in the average gray value, the center line of the region is determined from any image, and a rectangular region of a first preset size is cropped from any image according to the center line to obtain the cropped image; the cropped images corresponding to adjacent two-dimensional tomographic scans are determined as the reference image and the image to be registered, and Fourier transform is performed on the reference image and the image to be registered, and a cross-power spectrum matrix is ​​constructed based on the first image obtained after the Fourier transform; then, an inverse Fourier transform is performed on the cross-power spectrum matrix to obtain the impulse response function, and the translation parameter is determined based on the position corresponding to the pulse peak of the impulse response function.

[0108] For example, for any image in adjacent two-dimensional tomographic scans, the average gray value of each row of pixels in any image is calculated to generate a gray-level distribution curve. Where W represents the width of any image, (x,y) represents the coordinates of a pixel, and I(x,y) represents the gray value of pixel (x,y). This is based on the maximum value of the average gray value, which is also the maximum value of the gray-level distribution curve. Determine the center line of the region from any image, in order to Using the center line of the region as a reference, a rectangular area of ​​200 pixels above and below it (total height 401 pixels) is cropped to obtain the cropped image. In this way, the computational cost can be reduced by about 60% while preserving the complete biological tissue structure.

[0109] Furthermore, this application determines the cropped images corresponding to adjacent two-dimensional tomographic scans as the reference image and the image to be registered, respectively, and performs frequency domain analysis on the reference image and the image to be registered based on the translation invariance principle of Fourier transform, including: for the reference image... and the image to be registered Perform a Fourier transform and construct a cross-power spectrum matrix based on the first image obtained after the Fourier transform, specifically as follows:

[0110] ;

[0111] ;

[0112] ;

[0113] Where C(u,v) represents the cross-power spectrum matrix, and (u,v) represents the frequency domain coordinates, i.e., the frequencies in the horizontal and vertical directions. denoted as complex conjugate, and F denotes Fourier transform.

[0114] Then, an inverse Fourier transform is performed on the cross-power spectrum matrix to obtain the impulse response function, i.e. ,in, This represents the inverse Fourier transform. Then, based on the position corresponding to the peak value of the impulse response function, the translation parameters are determined. These translation parameters are then used to perform preliminary registration of the images to be registered, resulting in a preliminary registered image. .

[0115] Step S22: Determine the overlapping area of ​​the preliminary registration map and the reference map, and determine the similarity of the preliminary registration map and the reference map in the overlapping area based on the gray values ​​of each pixel in the overlapping area.

[0116] In this embodiment of the application, after the initial registration, it is necessary to quantify the similarity of the overlapping areas between the reference map and the initial registration map. Specifically, the normalized cross-correlation (NCC) is used as the evaluation index:

[0117] ;

[0118] NCC represents the preliminary registration map. and baseline diagram In overlapping areas similarity, This represents the average grayscale value of each pixel in the overlapping region of the initial registration map. This represents the average grayscale value of each pixel in the overlapping region of the reference image. This represents the grayscale value of pixel (x, y) in the initial registration image. This represents the grayscale value of a pixel (x, y) in the baseline image.

[0119] It should be noted that the value of NCC ranges from [-1, 1]. The closer the NCC is to 1, the higher the similarity between the preliminary registration map and the reference map in the overlapping area. The closer the NCC is to -1, the lower the similarity between the preliminary registration map and the reference map in the overlapping area.

[0120] Step S23: If the similarity is not less than a preset threshold, then the translation parameter is determined as an affine transformation parameter.

[0121] In this application, if the similarity between the preliminary registration map and the reference map in the overlapping area is not less than a preset threshold, and the preset threshold is 0.85 according to experimental statistics, then the preliminary registration is reliable, and the translation parameter can be directly determined as the affine transformation parameter.

[0122] Step S24: If the similarity is less than a preset threshold, then the scaling and rotation parameters are determined based on the reference image and the preliminary registration image, and the affine transformation parameters are determined based on the scaling and rotation parameters and the translation parameters.

[0123] In this application, if the similarity between the preliminary registration map and the reference map in the overlapping area is less than a preset threshold, it indicates that there is still rotation or scaling deformation. It is necessary to further determine the scaling and rotation parameters based on the reference map and the preliminary registration map, and determine the affine transformation parameters based on the scaling and rotation parameters and the translation parameters.

[0124] Specifically, regarding the baseline diagram and preliminary registration map Perform a Fourier transform and determine the second image obtained after the Fourier transform. and Amplitude spectrum in Cartesian coordinate system and To eliminate the effects of translation, the amplitude spectrum is then converted from Cartesian coordinates to polar coordinates, and the scaling and rotation parameters are determined based on the amplitude spectrum in polar coordinates, as shown in the following formula:

[0125] ;

[0126] Where r represents the polar radius. Indicates the polar angle, and s represents the scaling factor. This indicates the rotation angle, which is used to further improve registration accuracy.

[0127] Therefore, this application considers that the original three-dimensional optical coherence tomography (OCT) image has a high resolution, but the signal of effective biological tissue only occupies a local area. In order to reduce computational redundancy, this application proposes a signal localization algorithm based on lateral gray-level extreme value statistics. By cropping adjacent two-dimensional tomographic images in the original three-dimensional OCT image, and registering based on the cropped reference image and the image to be registered, the computational load can be significantly reduced while preserving the complete biological tissue structure. Furthermore, this application adopts a two-stage registration to determine the translation parameters and scaling and rotation parameters respectively, so as to improve the image registration accuracy. Then, by transforming the original three-dimensional OCT image, the spatial misalignment caused by eye movement or camera focus shift in the original three-dimensional OCT image can be eliminated.

[0128] See Figure 7 As shown, this embodiment of the invention discloses a three-dimensional optical coherence tomography image segmentation device based on a Unet network, comprising:

[0129] The transformation parameter determination module 11 is used to acquire the original three-dimensional optical coherence tomography image and determine the affine transformation parameters based on the adjacent two-dimensional tomography images in the original three-dimensional optical coherence tomography image; the original three-dimensional optical coherence tomography image is the image obtained after optical coherence tomography imaging of the target eye region;

[0130] The layer line recognition module 12 is used to identify each two-dimensional tomographic scan in the original three-dimensional optical coherence tomographic scan image using a first preset Unet network, so as to identify the layer lines from each two-dimensional tomographic scan.

[0131] Image transformation module 13 is used to transform the original three-dimensional optical coherence tomography image based on the affine transformation parameters to obtain the transformed three-dimensional optical coherence tomography image.

[0132] Image processing module 14 is used to process the corresponding two-dimensional tomographic images in the transformed three-dimensional optical coherence tomography image using the layered lines, so as to flatten the corresponding two-dimensional tomographic images and obtain the processed three-dimensional optical coherence tomography image.

[0133] The image segmentation module 15 is used to cut the processed three-dimensional optical coherence tomography image into several overlapping sub-blocks, and to use a second preset Unet network to segment the retinal layer structure of each sub-block in parallel, so as to determine the layer structure segmentation result of the processed three-dimensional optical coherence tomography image by stitching together the segmentation results of each sub-block.

[0134] Therefore, this application segments a three-dimensional optical coherence tomography (OCT) image into several overlapping sub-blocks and uses a second preset Unet network to segment the retinal layer structure of each sub-block in parallel. The segmentation results of each sub-block are then stitched together to determine the layer structure segmentation result of the three-dimensional OCT image. By segmenting the three-dimensional OCT image into several overlapping sub-blocks, the second preset Unet network has more contextual information when segmenting the three-dimensional OCT image, improving the segmentation effect. Furthermore, by segmenting the retinal layer structure of each sub-block in parallel, this application can significantly improve the segmentation efficiency of the three-dimensional OCT image. Moreover, by combining a first preset Unet network for recognizing two-dimensional tomographic images and a second preset Unet network for segmenting three-dimensional OCT images, this application not only ensures the continuity of the three-dimensional OCT image segmentation but also improves the segmentation accuracy.

[0135] In some specific embodiments, the transformation parameter determination module 11 includes:

[0136] The preliminary registration submodule is used to crop the adjacent two-dimensional tomographic scan images to obtain a reference image and an image to be registered, and to determine translation parameters based on the reference image and the image to be registered, and to perform preliminary registration on the image to be registered using the translation parameters to obtain a preliminary registration image;

[0137] A similarity determination unit is used to determine the overlapping region of the preliminary registration map and the reference map, and to determine the similarity between the preliminary registration map and the reference map in the overlapping region based on the gray values ​​of each pixel in the overlapping region.

[0138] The first parameter determination unit is used to determine the translation parameter as an affine transformation parameter if the similarity is not less than a preset threshold.

[0139] The second parameter determination unit is used to determine scaling and rotation parameters based on the reference image and the preliminary registration image if the similarity is less than a preset threshold, and to determine affine transformation parameters based on the scaling and rotation parameters and the translation parameters.

[0140] In some specific embodiments, the preliminary registration submodule includes:

[0141] An average grayscale value determination unit is used to determine the average grayscale value of each row of pixels in any image; wherein any image is any one of the adjacent two-dimensional tomographic scan images;

[0142] The first cropping unit is used to determine the region center line from any image based on the maximum value in the average gray value, and to crop a rectangular region of a first preset size from any image according to the region center line to obtain the cropped image.

[0143] The first Fourier transform unit is used to determine the cropped images corresponding to the adjacent two-dimensional tomographic scan images as the reference image and the image to be registered, respectively, to perform Fourier transform on the reference image and the image to be registered, and to construct a cross power spectrum matrix based on the first image obtained after the Fourier transform.

[0144] The translation parameter determination unit is used to perform an inverse Fourier transform on the cross-power spectrum matrix to obtain the impulse response function, and to determine the translation parameter based on the position corresponding to the pulse peak of the impulse response function.

[0145] In some specific embodiments, the second parameter determining unit includes:

[0146] The second Fourier transform unit is used to perform Fourier transform on the reference image and the preliminary registration image, and to determine the amplitude spectrum of the second image obtained after Fourier transform in the Cartesian coordinate system.

[0147] The scaling and rotation parameter determination unit is used to convert the amplitude spectrum from the Cartesian coordinate system to the polar coordinate system, and determine the scaling and rotation parameters based on the amplitude spectrum in the polar coordinate system.

[0148] In some specific embodiments, the layer line recognition module 12 includes:

[0149] The probability value generation unit is used to identify each two-dimensional tomographic image in the original three-dimensional optical coherence tomographic image using a first preset Unet network, so as to generate a probability value for each pixel in each two-dimensional tomographic image.

[0150] The target pixel selection unit is used to select target pixels from each column of pixels in each two-dimensional tomographic scan image; the probability value of the target pixel is the maximum value among the probability values ​​of the column in which the target pixel is located.

[0151] A layer line determination unit is used to determine the layer lines in each two-dimensional tomographic scan image based on the target pixel points in each two-dimensional tomographic scan image.

[0152] The layer line optimization unit is used to smooth the layer lines in each two-dimensional tomographic scan using mean filtering, so as to optimize the layer lines in each two-dimensional tomographic scan.

[0153] In some specific embodiments, the image processing module 14 includes:

[0154] The ordinate averaging unit is used to determine the average ordinate corresponding to the layer line based on the ordinate of each pixel point on the layer line.

[0155] The ordinate transformation unit is used to process the corresponding two-dimensional tomographic scan in the transformed three-dimensional optical coherence tomography image using the layering line, so as to determine the target ordinate corresponding to the horizontal coordinate of each pixel in the corresponding two-dimensional tomographic scan from the layering line, and transform the ordinate of each pixel using the difference between the target ordinate and the average ordinate, so as to flatten the corresponding two-dimensional tomographic scan and obtain the flattened two-dimensional tomographic scan.

[0156] The second interception unit is used to intercept a rectangular region of a second preset size from the flattened two-dimensional tomographic image based on the flattened layer lines, so as to obtain the intercepted two-dimensional tomographic image.

[0157] The processed image determination unit is used to determine the processed three-dimensional optical coherence tomography image based on each of the extracted two-dimensional tomographic images.

[0158] In some specific embodiments, the retinal layered structure includes a first layered structure and a second layered structure, wherein the first layered structure includes a retinal nerve fiber layer, and the second layered structure includes a ganglion cell complex, an outer plexiform layer, and a pigment epithelium layer;

[0159] Correspondingly, the three-dimensional optical coherence tomography image segmentation device based on the Unet network further includes:

[0160] The image unfolding unit is used to unfold the layered structure segmentation result of the processed three-dimensional optical coherence tomography image into several two-dimensional images along the axis.

[0161] The upward gradient field construction unit is used to determine the first gradient value of each pixel based on the gray value of each pixel and the previous pixel in the two-dimensional image, so as to construct the upward gradient field of the two-dimensional image.

[0162] The down-direction gradient field construction unit is used to determine the second gradient value of each pixel based on the gray value of each pixel and the next pixel in the two-dimensional image, so as to construct the down-direction gradient field of the two-dimensional image.

[0163] The gradient correction unit is used to perform bidirectional search on the first layer structure and the second layer structure in the two-dimensional image using the upward gradient field and the downward gradient field, respectively, to search for gradient extreme points from both sides of each layer structure, and to perform gradient correction on the corresponding layer structure in the two-dimensional image using the gradient extreme points, so as to obtain the corrected image.

[0164] The correction result determination unit is used to determine the corrected layer structure segmentation result of the processed three-dimensional optical coherence tomography image based on each of the corrected images;

[0165] In this configuration, the previous pixel and the next pixel are both located in the same column as each pixel, with the previous pixel located in the row above each pixel and the next pixel located in the row below each pixel.

[0166] Furthermore, embodiments of this application also disclose an electronic device, Figure 8 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0167] Figure 8 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the three-dimensional optical coherence tomography image segmentation method based on the Unet network disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0168] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0169] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0170] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the Unet network-based three-dimensional optical coherence tomography image segmentation method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0171] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed three-dimensional optical coherence tomography image segmentation method based on the Unet network. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0172] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0173] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0174] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0175] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0176] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A three-dimensional optical coherence tomography image segmentation method based on Unet network, characterized in that, include: Acquire the original three-dimensional optical coherence tomography image, and determine the affine transformation parameters based on the adjacent two-dimensional tomography images in the original three-dimensional optical coherence tomography image; The original three-dimensional optical coherence tomography image is the image obtained after optical coherence tomography of the target eye region; The first preset Unet network is used to identify each two-dimensional tomographic scan in the original three-dimensional optical coherence tomographic image, so as to identify the layer lines from each two-dimensional tomographic scan. The original three-dimensional optical coherence tomography image is transformed based on the affine transformation parameters to obtain the transformed three-dimensional optical coherence tomography image. The corresponding two-dimensional tomographic images in the transformed three-dimensional optical coherence tomography image are processed using the layered lines to flatten the corresponding two-dimensional tomographic images, thereby obtaining the processed three-dimensional optical coherence tomography image. The processed three-dimensional optical coherence tomography image is cut into several overlapping sub-blocks, and the retinal layer structure is segmented in parallel using a second preset Unet network. The layer structure segmentation result of the processed three-dimensional optical coherence tomography image is determined by stitching together the segmentation results of each sub-block.

2. The three-dimensional optical coherence tomography image segmentation method based on Unet network according to claim 1, characterized in that, The determination of affine transformation parameters based on adjacent two-dimensional tomographic images in the original three-dimensional optical coherence tomographic image includes: The adjacent two-dimensional tomographic scan images are cropped to obtain a reference image and an image to be registered. Translation parameters are determined based on the reference image and the image to be registered. The translation parameters are then used to perform preliminary registration on the image to be registered to obtain a preliminary registered image. The overlapping region of the preliminary registration map and the reference map is determined, and the similarity between the preliminary registration map and the reference map in the overlapping region is determined based on the gray values ​​of each pixel in the overlapping region. If the similarity is not less than a preset threshold, then the translation parameter is determined as an affine transformation parameter; If the similarity is less than a preset threshold, then scaling and rotation parameters are determined based on the reference image and the preliminary registration image, and affine transformation parameters are determined based on the scaling and rotation parameters and the translation parameters.

3. The three-dimensional optical coherence tomography image segmentation method based on Unet network according to claim 2, characterized in that, The step of cropping the adjacent two-dimensional tomographic scans to obtain a reference image and a registration image, and determining translation parameters based on the reference image and the registration image, includes: Determine the average gray value of each row of pixels in any image; the any image is any one of the adjacent two-dimensional tomographic scan images; Based on the maximum value in the average grayscale value, the center line of the region is determined from any image, and a rectangular region of a first preset size is cropped from any image according to the center line of the region to obtain the cropped image. The cropped images corresponding to the adjacent two-dimensional tomographic scan images are determined as the reference image and the image to be registered. Fourier transform is performed on the reference image and the image to be registered, and a cross power spectrum matrix is ​​constructed based on the first image obtained after the Fourier transform. An inverse Fourier transform is performed on the cross-power spectrum matrix to obtain the impulse response function, and a translation parameter is determined based on the position corresponding to the pulse peak of the impulse response function.

4. The three-dimensional optical coherence tomography image segmentation method based on Unet network according to claim 2, characterized in that, Determining scaling and rotation parameters based on the reference map and the preliminary registration map includes: Perform Fourier transform on the reference image and the preliminary registration image, and determine the amplitude spectrum of the second image obtained after Fourier transform in the Cartesian coordinate system; The amplitude spectrum is converted from Cartesian coordinates to polar coordinates, and scaling and rotation parameters are determined based on the amplitude spectrum in polar coordinates.

5. The three-dimensional optical coherence tomography image segmentation method based on Unet network according to claim 1, characterized in that, The step of using a first preset Unet network to identify each two-dimensional tomographic image in the original three-dimensional optical coherence tomography image, so as to identify the layer lines from each two-dimensional tomographic image, includes: The first preset Unet network is used to identify each two-dimensional tomographic image in the original three-dimensional optical coherence tomography image, so as to generate a probability value for each pixel in each two-dimensional tomographic image. Target pixels are selected from each column of pixels in each of the two-dimensional tomographic scan images; the probability value of the target pixel is the maximum value among the probability values ​​of the column in which the target pixel is located. The layer lines in each two-dimensional tomographic scan are determined based on the target pixels in each two-dimensional tomographic scan. Mean filtering is used to smooth the layer lines in each two-dimensional tomographic scan to optimize the layer lines in each two-dimensional tomographic scan.

6. The three-dimensional optical coherence tomography image segmentation method based on Unet network according to claim 1, characterized in that, The process of using the layered lines to process the corresponding two-dimensional tomographic images in the transformed three-dimensional optical coherence tomography image to flatten the corresponding two-dimensional tomographic images, thereby obtaining the processed three-dimensional optical coherence tomography image, includes: Based on the ordinate of each pixel on the layer line, determine the average ordinate corresponding to the layer line; The corresponding two-dimensional tomographic scan image in the transformed three-dimensional optical coherence tomography image is processed using the layered lines to determine the target ordinate corresponding to the abscissa of each pixel in the corresponding two-dimensional tomographic scan image from the layered lines. The ordinate of each pixel is then transformed using the difference between the target ordinate and the average ordinate to flatten the corresponding two-dimensional tomographic scan image, resulting in a flattened two-dimensional tomographic scan image. A rectangular area of ​​a second preset size is extracted from the flattened two-dimensional tomographic image based on the flattened layer lines to obtain the extracted two-dimensional tomographic image. Based on the extracted two-dimensional tomographic images, the processed three-dimensional optical coherence tomographic image is determined.

7. The three-dimensional optical coherence tomography image segmentation method based on Unet network according to any one of claims 1 to 6, characterized in that, The retinal layered structure includes a first layered structure and a second layered structure. The first layered structure includes a retinal nerve fiber layer, and the second layered structure includes a ganglion cell complex, an outer plexiform layer, and a pigment epithelium layer. Accordingly, after determining the hierarchical structure segmentation result of the processed three-dimensional optical coherence tomography image by stitching together the segmentation results of each sub-block, the method further includes: The layered structure segmentation result of the processed three-dimensional optical coherence tomography image is unfolded into several two-dimensional images along the axial direction; Based on the grayscale value of each pixel and the previous pixel in the two-dimensional image, the first gradient value of each pixel is determined to construct the upward gradient field of the two-dimensional image. Based on the grayscale values ​​of each pixel and the next pixel in the two-dimensional image, a second gradient value for each pixel is determined to construct the down-direction gradient field of the two-dimensional image. The first and second layered structures in the two-dimensional image are bidirectionally searched using the upward gradient field and the downward gradient field, respectively, to search for gradient extrema from both sides of each layered structure. The gradient extrema are then used to perform gradient correction on the corresponding layered structures in the two-dimensional image to obtain the corrected image. The corrected hierarchical structure segmentation result of the processed three-dimensional optical coherence tomography image is determined based on each of the corrected images; In this configuration, the previous pixel and the next pixel are both located in the same column as each pixel, with the previous pixel located in the row above each pixel and the next pixel located in the row below each pixel.

8. A three-dimensional optical coherence tomography image segmentation device based on a Unet network, characterized in that, include: The transformation parameter determination module is used to acquire the original three-dimensional optical coherence tomography image and determine the affine transformation parameters based on the adjacent two-dimensional tomographic images in the original three-dimensional optical coherence tomography image; the original three-dimensional optical coherence tomography image is the image obtained after optical coherence tomography imaging of the target eye region; The layer line recognition module is used to identify each two-dimensional tomographic scan image in the original three-dimensional optical coherence tomographic scan image using a first preset Unet network, so as to identify the layer lines from each two-dimensional tomographic scan image respectively. The image transformation module is used to transform the original three-dimensional optical coherence tomography image based on the affine transformation parameters to obtain the transformed three-dimensional optical coherence tomography image. The image processing module is used to process the corresponding two-dimensional tomographic images in the transformed three-dimensional optical coherence tomography image using the layered lines, so as to flatten the corresponding two-dimensional tomographic images and obtain the processed three-dimensional optical coherence tomography image. The image segmentation module is used to cut the processed three-dimensional optical coherence tomography image into several overlapping sub-blocks, and to use a second preset Unet network to segment the retinal layer structure of each sub-block in parallel, so as to determine the layer structure segmentation result of the processed three-dimensional optical coherence tomography image by stitching together the segmentation results of each sub-block.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the three-dimensional optical coherence tomography image segmentation method based on the Unet network as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the three-dimensional optical coherence tomography image segmentation method based on the Unet network as described in any one of claims 1 to 7.

Images