OCT image segmentation method and device, electronic equipment and storage medium

Abstract

The present application relates to artificial intelligence and digital medical treatment, and provides an OCT image segmentation method and device, electronic equipment and storage medium, edge lines are determined through a plurality of layered masks of an original OCT image, a first gradient weight is calculated according to the edge lines, and a second gradient weight is calculated according to the original OCT image, the first gradient weight and the second gradient weight join the attention mechanism of edge perception, the edges between layers are distinguished, greater weight is given to the edges, so that the network pays more attention to the punishment at the edges when learning, more accurate edge features are obtained, the segmentation performance of the model is improved, and the accuracy of OCT image segmentation is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an OCT image segmentation method, apparatus, electronic device, and storage medium. Background Technology

[0002] Optical coherence tomography (OCT), due to its non-contact and non-invasive nature, has become a routine examination in the diagnosis and treatment of eye diseases, providing doctors with images of retinal structures. Segmenting lesion areas in OCT images, such as subretinal fluid accumulation, intraretinal fluid accumulation, subretinal hyperreflective material, and pigment epithelium detachment, is fundamental for reliable diagnosis of fundus diseases.

[0003] Currently, many scholars have studied the segmentation of fundus structures in macular OCT scans, including the retinal nerve fiber layer (RNFL), the ganglion cell-inner reticular layer (GCIPL), and the choroid. With the innovation of computational methods, a large number of OCT image segmentation methods based on deep learning have also emerged.

[0004] In existing technologies, due to the blurring of edges between OCT layers, the accuracy of inter-layer segmentation is low when training OCT image segmentation models based on deep learning. Summary of the Invention

[0005] In view of the above, it is necessary to propose an OCT image segmentation method, apparatus, electronic device and storage medium that can improve the accuracy of OCT image segmentation.

[0006] A first aspect of the present invention provides an OCT image segmentation method, the method comprising:

[0007] Obtain multiple layered masks from the original OCT image;

[0008] The edge lines are determined based on the multiple layered masks;

[0009] The first gradient weight is calculated based on the edge line;

[0010] The second gradient weight is calculated based on the original OCT image;

[0011] An OCT image segmentation model is obtained by training the original OCT image and the corresponding first gradient weight and second gradient weight;

[0012] The OCT image segmentation model is invoked to segment the OCT image to be segmented.

[0013] In an optional implementation, training the OCT image segmentation model using the original OCT image and the corresponding first gradient weights and second gradient weights includes:

[0014] The pixel-level weights of the original OCT image are calculated based on the first gradient weight and the second gradient weight.

[0015] An OCT image segmentation model is obtained by training the original OCT image and the corresponding pixel-level weights.

[0016] In an optional implementation, calculating the pixel-level weights of the original OCT image based on the first gradient weights and the second gradient weights includes:

[0017] Determine whether the second gradient weight corresponding to each pixel in the original OCT image is greater than a preset gradient weight threshold;

[0018] When the second gradient weight corresponding to the first pixel in the original OCT image is greater than the preset gradient weight threshold, the second gradient weight corresponding to the first pixel is determined as the pixel-level weight of the first pixel.

[0019] When the second gradient weight corresponding to the second pixel in the original OCT image is less than the preset gradient weight threshold, the product of the second gradient weight and the first gradient weight corresponding to the second pixel is determined as the pixel-level weight of the second pixel.

[0020] In an optional implementation, training the OCT image segmentation model using the original OCT image and the corresponding pixel-level weights includes:

[0021] The original OCT image and the corresponding pixel-level weights are input into a preset segmentation network for segmentation processing to obtain a segmented image and segmentation labels.

[0022] The first loss function value is calculated based on the segmented image and the corresponding gold standard image;

[0023] The second loss function value is calculated based on the segmentation label and the corresponding hierarchical label;

[0024] The total loss function value is calculated based on the first loss function value and the second loss function value.

[0025] The OCT image segmentation model is obtained by iteratively training the preset segmentation network based on the total loss function value.

[0026] In an optional implementation, before acquiring multiple layered masks of the original OCT image, the method further includes:

[0027] Perform an affine transformation on the original OCT image to obtain the transformed image;

[0028] For each pixel in the transformed image, a displacement in the x and y directions is generated to obtain a random displacement field;

[0029] The random displacement field is filtered;

[0030] Deformation control is applied to the filtered random displacement field based on a preset deformation control factor.

[0031] The original OCT image is processed based on the random displacement field after deformation to obtain an enhanced OCT image;

[0032] The process of obtaining multiple layered masks for the original OCT image includes obtaining multiple layered masks for the enhanced OCT image.

[0033] In an optional implementation, generating a displacement in the x and y directions for each pixel in the transformed image to obtain a random displacement field includes:

[0034] Obtain the coordinates of each pixel in the transformed image;

[0035] The range of the stated coordinate values ​​and the preset coordinate values ​​are compared to obtain the comparison result.

[0036] Based on the comparison results, the pixels are divided into middle pixels and edge pixels;

[0037] Generate a first displacement in the x and y directions for the intermediate pixel;

[0038] A second displacement in the x and y directions is generated for the edge pixel;

[0039] The random displacement field is obtained based on the first displacement and the second displacement;

[0040] Wherein, the first displacement is greater than the second displacement.

[0041] In an optional implementation, determining the edge line based on the plurality of layered masks includes:

[0042] The multiple layered masks are superimposed onto an image to obtain a superimposed image; the gradient value of the superimposed image is calculated; the edge line is determined based on the gradient value; or

[0043] Morphological dilation is performed on each of the layered masks to obtain a dilated image; morphological erosion is performed on each of the layered masks to obtain an eroded image; a difference image is calculated based on each dilated image and the corresponding eroded image; and the edge line is obtained based on the difference image.

[0044] A second aspect of the present invention provides an OCT image segmentation apparatus, the apparatus comprising:

[0045] The acquisition module is used to acquire multiple layered masks of the original OCT image;

[0046] The determination module is used to determine the edge lines based on the multiple layered masks;

[0047] The first calculation module is used to calculate the first gradient weight based on the edge line;

[0048] The second calculation module is used to calculate the second gradient weight based on the original OCT image;

[0049] The training module is used to train an OCT image segmentation model using the original OCT image and the corresponding first gradient weight and second gradient weight;

[0050] The segmentation module is used to call the OCT image segmentation model to segment the OCT image to be segmented.

[0051] A third aspect of the present invention provides an electronic device comprising a processor and a memory, wherein the processor is configured to implement the OCT image segmentation method when executing a computer program stored in the memory.

[0052] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the OCT image segmentation method.

[0053] The OCT image segmentation method, apparatus, electronic device, and storage medium provided by this invention determine edge lines through multiple layered masks of the original OCT image, calculate a first gradient weight based on the edge lines, and calculate a second gradient weight based on the original OCT image. The first and second gradient weights incorporate an edge-aware attention mechanism to distinguish between interlayer edges and assign greater weights to edges. This allows the network to focus more on penalizing edges during learning, obtaining more accurate edge features, improving the model's segmentation performance, and increasing the accuracy of OCT image segmentation. Attached Figure Description

[0054] Figure 1 This is a flowchart of the OCT image segmentation method provided in Embodiment 1 of the present invention.

[0055] Figure 2 This is a schematic diagram of the layered mask of the original OCT image provided in an embodiment of the present invention.

[0056] Figure 3 This is a structural diagram of the OCT image segmentation device provided in Embodiment 2 of the present invention.

[0057] Figure 4 This is a schematic diagram of the structure of the electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0058] To better understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing an embodiment in one alternative implementation and is not intended to be limiting of the invention.

[0060] The OCT image segmentation method provided in this embodiment of the invention is executed by an electronic device, and correspondingly, the OCT image segmentation device runs in the electronic device.

[0061] The embodiments of this invention can standardize data processing based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0062] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0063] Example 1

[0064] Figure 1 This is a flowchart of the OCT image segmentation method provided in an embodiment of the present invention. The OCT image segmentation method specifically includes the following steps. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0065] S11, obtain multiple layered masks of the original OCT image.

[0066] The original OCT image can be an ocular OCT image, which refers to an image obtained by scanning the eyes of a patient with an eye disease using Optical Coherence Tomography (OCT) technology. When the electronic device is an OCT device, it scans the patient's eyes in real time to obtain the original ocular OCT image and stores it in a database. When the electronic device is not an OCT device, the OCT device acquires an OCT image of the patient's eyes and sends it to the electronic device. The electronic device first stores the original OCT image in its database and then retrieves it from the database. In other words, the original OCT image can be an ocular OCT image obtained by the electronic device in real time from the OCT device, or it can be a pre-stored ocular OCT image retrieved from the internal or external memory of the electronic device.

[0067] An electronic device acquires multiple raw OCT images and a corresponding gold standard image for each raw OCT image as a training sample image set to train an OCT image segmentation model. Here, raw OCT images refer to unprocessed OCT images. Gold standard images refer to pre-segmented lesion images; for example, experts, based on their professional medical knowledge, delineate the locations of the lesions to be segmented from the unprocessed raw OCT images and perform structural layering, meaning the gold standard image has pre-annotated layered masks. The gold standard image can be obtained by experts annotating the structure of each lesion location in the unprocessed raw OCT image.

[0068] It should be noted that the gold standard image is the same size as the original OCT image. Multiple layered masks annotated by experts from the unprocessed original OCT image, based on their professional medical knowledge, are the same size as the original OCT image. These multiple layered masks from the original OCT image include: the retinal nerve fiber layer (RNFL), the ganglion cell and inner reticular layer (GCIPL), and the choroid layer.

[0069] In an optional implementation, before acquiring multiple layered masks of the original OCT image, the method further includes:

[0070] Perform an affine transformation on the original OCT image to obtain the transformed image;

[0071] For each pixel in the transformed image, a displacement in the x and y directions is generated to obtain a random displacement field;

[0072] The random displacement field is filtered;

[0073] Deformation control is applied to the filtered random displacement field based on a preset deformation control factor.

[0074] The original OCT image is processed based on the random displacement field after deformation to obtain an enhanced OCT image.

[0075] Affine transformations in spatial transformations correspond to five types of transformations: translation, scaling, rotation, flipping, and shearing. The process of transforming the original image into the transformed image using these five transformations can be described using an affine transformation matrix. This transformation process can be obtained by multiplying the original image by a 2x3 matrix. Affine transformations are existing technology and will not be elaborated upon in detail here.

[0076] The electronic device can generate a displacement in the x and y directions for each pixel in the transformed image in a random manner, resulting in a random displacement field. The displacement in the x direction is denoted as Δx, and the displacement in the y direction is denoted as Δy, both randomly generated for each pixel in the transformed image. The ranges of displacements Δx and Δy are between (-1, 1). Generating displacements randomly enhances the diversity of displacements, thereby enhancing the diversity of the random displacement field, and consequently, the enhanced OCT image obtained by processing the original OCT image based on the random displacement field possesses greater diversity.

[0077] Electronic devices can use a Gaussian distribution to convolve the generated random displacement field, thereby smoothing the random displacement field, i.e., filtering the random displacement field. After filtering the random displacement field, the electronic device obtains a pre-set deformation control factor α, and multiplies the deformation control factor α with each displacement in the filtered random displacement field to control the deformation intensity of the random displacement field.

[0078] Finally, the original OCT image is processed based on the deformed random displacement field to obtain an enhanced OCT image. Specifically, each pixel in the original OCT image is acquired, and the deformation displacement corresponding to each pixel is obtained from the deformed random displacement field. This deformation displacement is then used to perform a translation transformation on the corresponding pixel. Once all pixels in the original OCT image have undergone this translation transformation, the enhanced OCT image is obtained.

[0079] It should be understood that obtaining multiple layered masks from the original OCT image includes obtaining multiple layered masks from the enhanced OCT image. That is, experts annotate multiple layered masks from the enhanced OCT image based on their professional medical knowledge.

[0080] In an optional implementation, generating a displacement in the x and y directions for each pixel in the transformed image to obtain a random displacement field includes:

[0081] Obtain the coordinates of each pixel in the transformed image;

[0082] The range of the stated coordinate values ​​and the preset coordinate values ​​are compared to obtain the comparison result.

[0083] Based on the comparison results, the pixels are divided into middle pixels and edge pixels;

[0084] Generate a first displacement in the x and y directions for the intermediate pixel;

[0085] A second displacement in the x and y directions is generated for the edge pixel;

[0086] The random displacement field is obtained based on the first displacement and the second displacement;

[0087] Wherein, the first displacement is greater than the second displacement.

[0088] A planar coordinate system can be pre-constructed, with the top-left corner of the transformed image as the origin, the horizontal line containing the width of the transformed image as the x-axis, and the vertical line containing the height of the transformed image as the y-axis. After establishing the planar coordinate system, the electronic device acquires the coordinate values ​​of each pixel in the transformed image within the planar coordinate system. The coordinate value of each pixel is represented as (xb, yb), where xb is the x-coordinate and yb is the y-coordinate.

[0089] The range of preset coordinate values ​​includes the range of preset horizontal coordinate values ​​and the range of preset vertical coordinate values. The range of preset horizontal coordinate values ​​is [xf1, xf2], and the range of preset vertical coordinate values ​​is [yf1, yf2].

[0090] The comparison of the range of the coordinate values ​​and the preset coordinate values ​​includes: comparing the range of the horizontal coordinate values ​​in the coordinate values ​​with the range of the preset horizontal coordinate values, and comparing the range of the vertical coordinate values ​​in the coordinate values ​​with the range of the preset vertical coordinate values.

[0091] When the x-coordinate value of the coordinates is within the range of preset x-coordinate values ​​and the y-coordinate value of the coordinates is within the range of preset y-coordinate values, the comparison result is: the coordinates are within the range of preset coordinate values.

[0092] When the x-coordinate value in the coordinate values ​​is not within the range of the preset x-coordinate values, and / or the y-coordinate value in the coordinate values ​​is within the range of the preset y-coordinate values, the comparison result is: the coordinate values ​​are not within the range of the preset coordinate values.

[0093] For a given pixel, if the comparison result indicates that the coordinate value is within the range of preset coordinate values, the pixel is classified as an intermediate pixel. For a given pixel, if the comparison result indicates that the coordinate value is not within the range of preset coordinate values, the pixel is classified as an edge pixel.

[0094] Since the first displacement is greater than the second displacement, a larger displacement in the x and y directions is generated for the middle pixels, and a smaller displacement in the x and y directions is generated for the edge pixels. Generally, the lesion area is mostly located in the middle part of the image, while the edge parts are relatively blurry. Therefore, a larger displacement is used for the middle pixels, resulting in greater deformation of the middle pixels, while a smaller displacement is used for the edge pixels, resulting in less deformation of the edge pixels. Otherwise, if a larger displacement is used for the edge pixels, the deformation of the edge pixels will be greater, making the edge parts increasingly blurry.

[0095] After obtaining multiple raw OCT images, electronic devices need to perform deformation processing on each raw OCT image to enhance the raw OCT image.

[0096] Before deforming the original OCT image, the electronic device can also perform grayscale transformation and / or angle transformation on the original OCT image. The electronic device can perform grayscale transformation on the original OCT image by applying histogram processing. A histogram is a function of grayscale levels, describing the number of pixels at a given grayscale level in the image. During grayscale equalization, the range of grayscale values ​​for the image pixels is first determined, and the grayscale values ​​are divided into several levels using appropriate grayscale intervals. Grayscale levels with a large number of pixels in the image are broadened, while grayscale levels with a small number of pixels are compressed, thereby expanding the dynamic range of pixel values, improving contrast and grayscale tone variation, and making the image clearer. The electronic device can preset an angle range, randomly select an angle within the preset angle range, and rotate the original OCT image counterclockwise or clockwise by that angle. The electronic device then deforms the original OCT image after grayscale transformation and / or angle transformation to obtain an enhanced OCT image.

[0097] S12, determine the edge line based on the multiple layered masks.

[0098] Since a pixel belongs to one and only one of multiple layers, an electronic device can calculate the interlayer edge lines based on multiple layer masks, and the interlayer edge lines are also the layer edge lines.

[0099] In an optional implementation, determining the edge line based on the plurality of layered masks includes:

[0100] The multiple layered masks are superimposed onto an image to obtain a superimposed image;

[0101] Calculate the gradient value of the overlaid image;

[0102] The edge line is determined based on the gradient value.

[0103] like Figure 2 The above, Figure 2 The leftmost image shows multiple layered masks, such as the retinal nerve fiber layer (RNFL), the ganglion cell and inner plexiform layer (GCIPL), and the choroid layer. Figure 2 In the middle part of the image, each layered mask is treated as a channel. Three layered masks correspond to exactly three channels. Superimposing these three channels forms a shape like... Figure 2 The image on the right (overlay image) allows us to calculate the gradient of the resulting image.

[0104] An image edge is the set of pixels that connect two regions of the image; pixels on an edge are called edge points. Image gradients calculate the rate of change in an image. Edge regions of an image exhibit larger changes in grayscale values, resulting in larger gradient values; conversely, smoother regions of the image show smaller changes in grayscale values, leading to smaller gradient values.

[0105] In the above embodiments, after the electronic device calculates the gradient value of each pixel in the overlay image, it compares the gradient value of each pixel with a preset gradient threshold. If the gradient value of a pixel is greater than the preset gradient threshold, the pixel is considered an edge pixel; if the gradient value of a pixel is less than the preset gradient threshold, the pixel is considered a non-edge pixel. The edge line can then be obtained based on the edge pixels.

[0106] In another alternative implementation, determining the edge line based on the plurality of layered masks includes:

[0107] Each of the layered masks is subjected to morphological dilation to obtain a dilated image;

[0108] Each of the layered masks is subjected to morphological erosion processing to obtain an eroded image;

[0109] A difference image is calculated based on each of the dilation images and the corresponding erosion images;

[0110] The edge line is obtained from the difference image.

[0111] Dilation expands the edges of an image, filling in the edges or internal pits of a target. Erosion removes the edges of an image, eliminating the "burrs" at the edges of a target. The electronic device first performs morphological dilation on each layered mask, then performs morphological erosion on each layered mask. This enhances important object features within the layered masks. Therefore, after subtracting each dilated image from its corresponding eroded image, the edge information in the resulting difference image will be more pronounced.

[0112] In the above optional implementation, the electronic device obtains the edge line through multiple layered masks instead of calculating the gradient through the original OCT image to obtain the edge characteristics. This is because the original OCT image is relatively blurry, while the multiple layered masks are pre-annotated. Whether each layered mask is used as a channel in a superimposed image or morphological dilation and erosion processing is used, the edges can be strengthened, thereby obtaining clear edge lines.

[0113] S13, calculate the first gradient weight based on the edge line.

[0114] After determining the edge line, the electronic device performs Gaussian filtering on the edge line to obtain the first gradient weight, which makes the model pay more attention to the penalty at the edge during learning and obtain more accurate edge features.

[0115] S14, calculate the second gradient weight based on the original OCT image.

[0116] The electronic device can use a preset filtering kernel to calculate the second gradient weights of the original OCT image. The preset filtering kernel includes a preset first filtering kernel and a preset second filtering kernel. The preset first filtering kernel and the preset second filtering kernel can be preset in the electronic device or adjusted by the user as needed. This embodiment of the invention does not impose specific limitations on the preset first filtering kernel and the preset second filtering kernel.

[0117] The first and second preset filter kernels are either the Sobel kernel, the Privette kernel, or the Roberts kernel, etc.

[0118] For example, suppose the first filter kernel is a transverse Sobel kernel:

[0119]

[0120] For example, suppose the second filter kernel is preset to be a longitudinal Sobel kernel:

[0121]

[0122] The original OCT image is represented by FA, ​​where Gx and Gy represent the grayscale values ​​of the image after horizontal and vertical edge detection, respectively. In other words, Gx represents the horizontal gradient and Gy represents the vertical gradient. The calculation formula is as follows:

[0123]

[0124] It should be noted that the horizontal and vertical gradients are calculated here for each pixel (x, y) in the original OCT image.

[0125] The second gradient weight of the original OCT image is determined by calculating the horizontal and vertical gradients. Specifically, a gradient vector is obtained based on the horizontal and vertical gradients, and the gradient vector is (horizontal gradient, vertical gradient). The gradient magnitude is calculated based on the gradient vector and used as the second gradient weight of the corresponding pixel.

[0126] In the above optional implementation, the preset amplitude threshold is used to measure the edge sharpness of the original OCT image. When the gradient amplitude of a certain pixel is greater than the preset amplitude threshold, it indicates that the edge image corresponding to the pixel is clear. When the gradient amplitude of a certain pixel is greater than the preset amplitude threshold, it indicates that the edge image corresponding to the pixel is blurry.

[0127] S15, an OCT image segmentation model is trained using the original OCT image and the corresponding first gradient weight and second gradient weight.

[0128] Electronic devices can initialize the network architecture of an OCT image segmentation model. For example, a convolutional neural network model, such as a U-shaped convolutional neural network (U-Net), can be used as the network architecture of the OCT image segmentation model.

[0129] Since the first gradient weight is calculated based on the edge line, which is determined by the layered mask, and the second gradient weight is calculated based on the original OCT image, and the size of the layered mask is the same as the size of the original OCT image, the first gradient weight and the second gradient weight have the same magnitude and the same size as the original OCT image. Therefore, the electronic device can train the initialized network architecture based on the original OCT image and the corresponding first gradient weight and second gradient weight to obtain the OCT image segmentation model.

[0130] In an optional implementation, training the OCT image segmentation model using the original OCT image and the corresponding first gradient weights and second gradient weights includes:

[0131] The pixel-level weights of the original OCT image are calculated based on the first gradient weight and the second gradient weight.

[0132] An OCT image segmentation model is obtained by training the original OCT image and the corresponding pixel-level weights.

[0133] In an optional implementation, calculating the pixel-level weights of the original OCT image based on the first gradient weights and the second gradient weights includes:

[0134] Determine whether the second gradient weight corresponding to each pixel in the original OCT image is greater than a preset gradient weight threshold;

[0135] When the second gradient weight corresponding to the first pixel in the original OCT image is greater than the preset gradient weight threshold, the second gradient weight corresponding to the first pixel is determined as the pixel-level weight of the first pixel.

[0136] When the second gradient weight corresponding to the second pixel in the original OCT image is less than the preset gradient weight threshold, the product of the second gradient weight and the first gradient weight corresponding to the second pixel is determined as the pixel-level weight of the second pixel.

[0137] A larger weight in the second gradient indicates a clear edge image, while a smaller weight indicates a blurry edge image. Blurry pixels are considered hard samples in the data. For hard samples, the edge gradient calculation method can automatically determine the degree of blurriness, assigning greater weights to more blurry pixels, thus acting as a dual attention mechanism. This allows the network to focus more on edge learning efficiency during training, improving the accuracy of the OCT image segmentation model.

[0138] In an optional implementation, training the OCT image segmentation model using the original OCT image and the corresponding pixel-level weights includes:

[0139] The original OCT image and the corresponding pixel-level weights are input into a preset segmentation network for segmentation processing to obtain a segmented image and segmentation labels.

[0140] The first loss function value is calculated based on the segmented image and the corresponding gold standard image;

[0141] The second loss function value is calculated based on the segmentation label and the corresponding hierarchical label;

[0142] The total loss function value is calculated based on the first loss function value and the second loss function value.

[0143] The OCT image segmentation model is obtained by iteratively training the preset segmentation network based on the total loss function value.

[0144] The preset segmentation network can include a downsampling stage and an upsampling stage. The downsampling stage includes multiple convolutional layers and pooling layers. The downsampling stage is used to extract features from the input original OCT image. The upsampling stage includes multiple deconvolutional layers. The upsampling stage is used to gradually restore the details of the original OCT image. At the same time, skip connections are added between feature layers of the same resolution to achieve segmentation of the target object in the original OCT image.

[0145] Specifically, the original OCT image is input into the preset segmentation network. Images of different scales are generated through sampling and then fed into convolutional layers of corresponding scales during the downsampling stage. The output of the preset segmentation network is then upsampled and concatenated to obtain the segmented image. Understandably, because the preset segmentation network fully considers the impact of images of different scales on image segmentation, it improves the performance of the convolutional neural network model, thereby increasing the accuracy of segmenting the original OCT image.

[0146] The preset segmentation network also includes a first convolutional layer, a first ReLU activation layer and a first batch normalization layer, a second convolutional layer, a second ReLU activation layer and a second batch normalization layer.

[0147] In each first convolutional layer, a non-linear activation function is used to classify the output, achieving the first comparison of the images. The result of the first comparison reflects how close the segmented image is to the gold standard image. The larger the result of the first comparison, the closer the segmented image is to the gold standard image; the smaller the result of the second comparison, the more different the segmented image is from the gold standard image.

[0148] In each second convolutional layer, a non-linear activation function is used to classify the output, achieving a second alignment of the hierarchical labels. The result of the second alignment reflects the closeness between the segmentation label and the hierarchical label. A larger second alignment result indicates a more consistent segmentation label and hierarchical label, while a smaller result indicates a more different segmentation label and hierarchical label.

[0149] In this embodiment, the loss function value is used to estimate the degree of inconsistency between the model's predicted values ​​and the true values. It is a non-negative real-valued function; the smaller the loss function value, the higher the accuracy of the model's predictions. The first loss function value L1 can be Euclidean distance or cosine angle. The second loss function value L2 can be calculated using the cross-entropy loss function, L2 = -SUM(y*logy'), where y represents the true segmentation label and y' represents the predicted segmentation label.

[0150] The total loss function value is obtained by summing the first and second loss function values. The preset segmentation network is then iteratively trained based on this total loss function value until it converges. At this point, the iterative training stops, and the network model after iterative training stops is identified as the OCT image segmentation model. Iterative training is a model training method in deep learning used to optimize the model. The iterative training process in this step is as follows: In each training iteration, all training samples are sequentially read in, and the current total loss function value is calculated. A stochastic gradient descent algorithm is used to determine the gradient descent direction, causing the total loss function value to gradually decrease and reach a stable state, thereby optimizing the parameters of the constructed network model.

[0151] Convergence of the total loss function value means that the total loss function value is close to 0, for example, less than 0.1.

[0152] S16, The OCT image segmentation model is invoked to segment the OCT image to be segmented.

[0153] The OCT image to be segmented is the object for which lesion regions need to be segmented. The OCT image to be segmented is input into the OCT image segmentation model, and the OCT image segmentation model outputs the lesion regions in the OCT image to be segmented.

[0154] In existing technologies, ocular OCT images are generally captured as tomographic slices, resulting in strong homogeneity between the layers of the eye and thus fewer lesion patterns. This limits the number of samples that can be used as training data, leading to poor generalization ability of the OCT image segmentation models trained from these samples. This invention uses a deformation method to break down the homogeneity between layers in OCT tomographic scans from the same patient, simulating uneven layer thickness and lesions, thereby expanding the diversity of layer morphology and improving the model's generalization ability.

[0155] Furthermore, OCT images are often unclear, especially the boundaries between different layers are very blurred, making it difficult for the network to accurately distinguish them. In existing technologies, deep learning methods often directly use undifferentiated cross-entropy loss calculations, backpropagating each pixel indiscriminately. This results in poor edge segmentation at each OCT layer, with jagged edges or even large-area misclassification, leading to low overall accuracy. This invention first incorporates an edge-aware attention mechanism, distinguishing edges between layers and assigning greater weight to them. This allows the network to focus more on edge penalties during learning, obtaining more accurate edge features and improving the model's segmentation performance. Second, in calculating the loss function, it not only calculates the overall loss between the segmented image and the gold standard image but also the detailed loss between the segmentation label and the layer label. Therefore, it constrains the model's training from both overall and detailed perspectives, further improving the model's segmentation performance.

[0156] The method described in this invention can be applied to the field of digital healthcare. OCT images can provide cross-sectional information of the retina, thus allowing for more accurate analysis of retinal structure. Layer segmentation is helpful in diagnosing many retinal and optic nerve diseases, such as glaucoma, macular degeneration, or diabetic retinopathy. In the diagnosis of glaucoma, OCT is more effective than fundus color images in detecting early cases.

[0157] Example 2

[0158] Figure 3 This is a structural diagram of the OCT image segmentation device provided in an embodiment of the present invention.

[0159] In some embodiments, the OCT image segmentation apparatus 30 may include multiple functional modules composed of computer program segments. The computer programs for each program segment in the OCT image segmentation apparatus 30 may be stored in the memory of an electronic device and executed by at least one processor to perform (see details). Figure 1 (Description) The function of OCT image segmentation.

[0160] In this embodiment, the OCT image segmentation device 30 can be divided into multiple functional modules according to its functions. These functional modules may include: an acquisition module 301, a determination module 302, a first calculation module 303, a second calculation module 304, a training module 305, and a segmentation module 306. The term "module" in this invention refers to a series of computer program segments that can be executed by at least one processor and perform a fixed function, and which are stored in memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0161] The acquisition module 301 is used to acquire multiple layered masks of the original OCT image.

[0162] The original OCT image can be an ocular OCT image, which refers to an image obtained by scanning the eyes of a patient with an eye disease using Optical Coherence Tomography (OCT) technology. When the electronic device is an OCT device, it scans the patient's eyes in real time to obtain the original ocular OCT image and stores it in a database. When the electronic device is not an OCT device, the OCT device acquires an OCT image of the patient's eyes and sends it to the electronic device. The electronic device first stores the original OCT image in its database and then retrieves it from the database. In other words, the original OCT image can be an ocular OCT image obtained by the electronic device in real time from the OCT device, or it can be a pre-stored ocular OCT image retrieved from the internal or external memory of the electronic device.

[0163] An electronic device acquires multiple raw OCT images and a corresponding gold standard image for each raw OCT image as a training sample image set to train an OCT image segmentation model. Here, raw OCT images refer to unprocessed OCT images. Gold standard images refer to pre-segmented lesion images; for example, experts, based on their professional medical knowledge, delineate the locations of the lesions to be segmented from the unprocessed raw OCT images and perform structural layering, meaning the gold standard image has pre-annotated layered masks. The gold standard image can be obtained by experts annotating the structure of each lesion location in the unprocessed raw OCT image.

[0164] It should be noted that the gold standard image is the same size as the original OCT image. Multiple layered masks annotated by experts from the unprocessed original OCT image, based on their professional medical knowledge, are the same size as the original OCT image. These multiple layered masks from the original OCT image include: the retinal nerve fiber layer (RNFL), the ganglion cell and inner reticular layer (GCIPL), and the choroid layer.

[0165] In an optional implementation, before acquiring multiple layered masks of the original OCT image, the method further includes:

[0166] Perform an affine transformation on the original OCT image to obtain the transformed image;

[0167] For each pixel in the transformed image, a displacement in the x and y directions is generated to obtain a random displacement field;

[0168] The random displacement field is filtered;

[0169] Deformation control is applied to the filtered random displacement field based on a preset deformation control factor.

[0170] The original OCT image is processed based on the random displacement field after deformation to obtain an enhanced OCT image.

[0171] Affine transformations in spatial transformations correspond to five types of transformations: translation, scaling, rotation, flipping, and shearing. The process of transforming the original image into the transformed image using these five transformations can be described using an affine transformation matrix. This transformation process can be obtained by multiplying the original image by a 2x3 matrix. Affine transformations are existing technology and will not be elaborated upon in detail here.

[0172] The electronic device can generate a displacement in the x and y directions for each pixel in the transformed image in a random manner, resulting in a random displacement field. The displacement in the x direction is denoted as Δx, and the displacement in the y direction is denoted as Δy, both randomly generated for each pixel in the transformed image. The ranges of displacements Δx and Δy are between (-1, 1). Generating displacements randomly enhances the diversity of displacements, thereby enhancing the diversity of the random displacement field, and consequently, the enhanced OCT image obtained by processing the original OCT image based on the random displacement field possesses greater diversity.

[0173] Electronic devices can use a Gaussian distribution to convolve the generated random displacement field, thereby smoothing the random displacement field, i.e., filtering the random displacement field. After filtering the random displacement field, the electronic device obtains a pre-set deformation control factor α, and multiplies the deformation control factor α with each displacement in the filtered random displacement field to control the deformation intensity of the random displacement field.

[0174] Finally, the original OCT image is processed based on the deformed random displacement field to obtain an enhanced OCT image. Specifically, each pixel in the original OCT image is acquired, and the deformation displacement corresponding to each pixel is obtained from the deformed random displacement field. This deformation displacement is then used to perform a translation transformation on the corresponding pixel. Once all pixels in the original OCT image have undergone this translation transformation, the enhanced OCT image is obtained.

[0175] It should be understood that obtaining multiple layered masks from the original OCT image includes obtaining multiple layered masks from the enhanced OCT image. That is, experts annotate multiple layered masks from the enhanced OCT image based on their professional medical knowledge.

[0176] In an optional implementation, generating a displacement in the x and y directions for each pixel in the transformed image to obtain a random displacement field includes:

[0177] Obtain the coordinates of each pixel in the transformed image;

[0178] The range of the stated coordinate values ​​and the preset coordinate values ​​are compared to obtain the comparison result.

[0179] Based on the comparison results, the pixels are divided into middle pixels and edge pixels;

[0180] Generate a first displacement in the x and y directions for the intermediate pixel;

[0181] A second displacement in the x and y directions is generated for the edge pixel;

[0182] The random displacement field is obtained based on the first displacement and the second displacement;

[0183] Wherein, the first displacement is greater than the second displacement.

[0184] A planar coordinate system can be pre-constructed, with the top-left corner of the transformed image as the origin, the horizontal line containing the width of the transformed image as the x-axis, and the vertical line containing the height of the transformed image as the y-axis. After establishing the planar coordinate system, the electronic device acquires the coordinate values ​​of each pixel in the transformed image within the planar coordinate system. The coordinate value of each pixel is represented as (xb, yb), where xb is the x-coordinate and yb is the y-coordinate.

[0185] The range of preset coordinate values ​​includes the range of preset horizontal coordinate values ​​and the range of preset vertical coordinate values. The range of preset horizontal coordinate values ​​is [xf1, xf2], and the range of preset vertical coordinate values ​​is [yf1, yf2].

[0186] The comparison of the range of the coordinate values ​​and the preset coordinate values ​​includes: comparing the range of the horizontal coordinate values ​​in the coordinate values ​​with the range of the preset horizontal coordinate values, and comparing the range of the vertical coordinate values ​​in the coordinate values ​​with the range of the preset vertical coordinate values.

[0187] When the x-coordinate value of the coordinates is within the range of preset x-coordinate values ​​and the y-coordinate value of the coordinates is within the range of preset y-coordinate values, the comparison result is: the coordinates are within the range of preset coordinate values.

[0188] When the x-coordinate value in the coordinate values ​​is not within the range of the preset x-coordinate values, and / or the y-coordinate value in the coordinate values ​​is within the range of the preset y-coordinate values, the comparison result is: the coordinate values ​​are not within the range of the preset coordinate values.

[0189] For a given pixel, if the comparison result indicates that the coordinate value is within the range of preset coordinate values, the pixel is classified as an intermediate pixel. For a given pixel, if the comparison result indicates that the coordinate value is not within the range of preset coordinate values, the pixel is classified as an edge pixel.

[0190] Since the first displacement is greater than the second displacement, a larger displacement in the x and y directions is generated for the middle pixels, and a smaller displacement in the x and y directions is generated for the edge pixels. Generally, the lesion area is mostly located in the middle part of the image, while the edge parts are relatively blurry. Therefore, a larger displacement is used for the middle pixels, resulting in greater deformation of the middle pixels, while a smaller displacement is used for the edge pixels, resulting in less deformation of the edge pixels. Otherwise, if a larger displacement is used for the edge pixels, the deformation of the edge pixels will be greater, making the edge parts increasingly blurry.

[0191] After obtaining multiple raw OCT images, electronic devices need to perform deformation processing on each raw OCT image to enhance the raw OCT image.

[0192] Before deforming the original OCT image, the electronic device can also perform grayscale transformation and / or angle transformation on the original OCT image. The electronic device can perform grayscale transformation on the original OCT image by applying histogram processing. A histogram is a function of grayscale levels, describing the number of pixels at a given grayscale level in the image. During grayscale equalization, the range of grayscale values ​​for the image pixels is first determined, and the grayscale values ​​are divided into several levels using appropriate grayscale intervals. Grayscale levels with a large number of pixels in the image are broadened, while grayscale levels with a small number of pixels are compressed, thereby expanding the dynamic range of pixel values, improving contrast and grayscale tone variation, and making the image clearer. The electronic device can preset an angle range, randomly select an angle within the preset angle range, and rotate the original OCT image counterclockwise or clockwise by that angle. The electronic device then deforms the original OCT image after grayscale transformation and / or angle transformation to obtain an enhanced OCT image.

[0193] The determining module 302 is used to determine the edge line based on the plurality of layered masks.

[0194] Since a pixel belongs to one and only one of multiple layers, an electronic device can calculate the interlayer edge lines based on multiple layer masks, and the interlayer edge lines are also the layer edge lines.

[0195] In an optional implementation, determining the edge line based on the plurality of layered masks includes:

[0196] The multiple layered masks are superimposed onto an image to obtain a superimposed image;

[0197] Calculate the gradient value of the overlaid image;

[0198] The edge line is determined based on the gradient value.

[0199] like Figure 2 The above, Figure 2 The leftmost image shows multiple layered masks, such as the retinal nerve fiber layer (RNFL), the ganglion cell and inner plexiform layer (GCIPL), and the choroid layer. Figure 2 In the middle part of the image, each layered mask is treated as a channel. Three layered masks correspond to exactly three channels. Superimposing these three channels forms a shape like... Figure 2 The image on the right (overlay image) allows us to calculate the gradient of the resulting image.

[0200] An image edge is the set of pixels that connect two regions of the image; pixels on an edge are called edge points. Image gradients calculate the rate of change in an image. Edge regions of an image exhibit larger changes in grayscale values, resulting in larger gradient values; conversely, smoother regions of the image show smaller changes in grayscale values, leading to smaller gradient values.

[0201] In the above embodiments, after the electronic device calculates the gradient value of each pixel in the overlay image, it compares the gradient value of each pixel with a preset gradient threshold. If the gradient value of a pixel is greater than the preset gradient threshold, the pixel is considered an edge pixel; if the gradient value of a pixel is less than the preset gradient threshold, the pixel is considered a non-edge pixel. The edge line can then be obtained based on the edge pixels.

[0202] In another alternative implementation, determining the edge line based on the plurality of layered masks includes:

[0203] Each of the layered masks is subjected to morphological dilation to obtain a dilated image;

[0204] Each of the layered masks is subjected to morphological erosion processing to obtain an eroded image;

[0205] A difference image is calculated based on each of the dilation images and the corresponding erosion images;

[0206] The edge line is obtained from the difference image.

[0207] Dilation expands the edges of an image, filling in the edges or internal pits of a target. Erosion removes the edges of an image, eliminating the "burrs" at the edges of a target. The electronic device first performs morphological dilation on each layered mask, then performs morphological erosion on each layered mask. This enhances important object features within the layered masks. Therefore, after subtracting each dilated image from its corresponding eroded image, the edge information in the resulting difference image will be more pronounced.

[0208] In the above optional implementation, the electronic device obtains the edge line through multiple layered masks instead of calculating the gradient through the original OCT image to obtain the edge characteristics. This is because the original OCT image is relatively blurry, while the multiple layered masks are pre-annotated. Whether each layered mask is used as a channel in a superimposed image or morphological dilation and erosion processing is used, the edges can be strengthened, thereby obtaining clear edge lines.

[0209] The first calculation module 303 is used to calculate the first gradient weight based on the edge line.

[0210] After determining the edge line, the electronic device performs Gaussian filtering on the edge line to obtain the first gradient weight, which makes the model pay more attention to the penalty at the edge during learning and obtain more accurate edge features.

[0211] The second calculation module 304 is used to calculate the second gradient weight based on the original OCT image.

[0212] The electronic device can use a preset filtering kernel to calculate the second gradient weights of the original OCT image. The preset filtering kernel includes a preset first filtering kernel and a preset second filtering kernel. The preset first filtering kernel and the preset second filtering kernel can be preset in the electronic device or adjusted by the user as needed. This embodiment of the invention does not impose specific limitations on the preset first filtering kernel and the preset second filtering kernel.

[0213] The first and second preset filter kernels are either the Sobel kernel, the Privette kernel, or the Roberts kernel, etc.

[0214] For example, suppose the first filter kernel is a transverse Sobel kernel:

[0215]

[0216] For example, suppose the second filter kernel is preset to be a longitudinal Sobel kernel:

[0217]

[0218] The original OCT image is represented by FA, ​​where Gx and Gy represent the grayscale values ​​of the image after horizontal and vertical edge detection, respectively. In other words, Gx represents the horizontal gradient and Gy represents the vertical gradient. The calculation formula is as follows:

[0219]

[0220] It should be noted that the horizontal and vertical gradients are calculated here for each pixel (x, y) in the original OCT image.

[0221] The second gradient weight of the original OCT image is determined by calculating the horizontal and vertical gradients. Specifically, a gradient vector is obtained based on the horizontal and vertical gradients, and the gradient vector is (horizontal gradient, vertical gradient). The gradient magnitude is calculated based on the gradient vector and used as the second gradient weight of the corresponding pixel.

[0222] In the above optional implementation, the preset amplitude threshold is used to measure the edge sharpness of the original OCT image. When the gradient amplitude of a certain pixel is greater than the preset amplitude threshold, it indicates that the edge image corresponding to the pixel is clear. When the gradient amplitude of a certain pixel is greater than the preset amplitude threshold, it indicates that the edge image corresponding to the pixel is blurry.

[0223] The training module 305 is used to train an OCT image segmentation model using the original OCT image and the corresponding first gradient weight and second gradient weight.

[0224] Electronic devices can initialize the network architecture of an OCT image segmentation model. For example, a convolutional neural network model, such as a U-shaped convolutional neural network (U-Net), can be used as the network architecture of the OCT image segmentation model.

[0225] Since the first gradient weight is calculated based on the edge line, which is determined by the layered mask, and the second gradient weight is calculated based on the original OCT image, and the size of the layered mask is the same as the size of the original OCT image, the first gradient weight and the second gradient weight have the same magnitude and the same size as the original OCT image. Therefore, the electronic device can train the initialized network architecture based on the original OCT image and the corresponding first gradient weight and second gradient weight to obtain the OCT image segmentation model.

[0226] In an optional implementation, training the OCT image segmentation model using the original OCT image and the corresponding first gradient weights and second gradient weights includes:

[0227] The pixel-level weights of the original OCT image are calculated based on the first gradient weight and the second gradient weight.

[0228] An OCT image segmentation model is obtained by training the original OCT image and the corresponding pixel-level weights.

[0229] In an optional implementation, calculating the pixel-level weights of the original OCT image based on the first gradient weights and the second gradient weights includes:

[0230] Determine whether the second gradient weight corresponding to each pixel in the original OCT image is greater than a preset gradient weight threshold;

[0231] When the second gradient weight corresponding to the first pixel in the original OCT image is greater than the preset gradient weight threshold, the second gradient weight corresponding to the first pixel is determined as the pixel-level weight of the first pixel.

[0232] When the second gradient weight corresponding to the second pixel in the original OCT image is less than the preset gradient weight threshold, the product of the second gradient weight and the first gradient weight corresponding to the second pixel is determined as the pixel-level weight of the second pixel.

[0233] A larger weight in the second gradient indicates a clear edge image, while a smaller weight indicates a blurry edge image. Blurry pixels are considered hard samples in the data. For hard samples, the edge gradient calculation method can automatically determine the degree of blurriness, assigning greater weights to more blurry pixels, thus acting as a dual attention mechanism. This allows the network to focus more on edge learning efficiency during training, improving the accuracy of the OCT image segmentation model.

[0234] In an optional implementation, training the OCT image segmentation model using the original OCT image and the corresponding pixel-level weights includes:

[0235] The original OCT image and the corresponding pixel-level weights are input into a preset segmentation network for segmentation processing to obtain a segmented image and segmentation labels.

[0236] The first loss function value is calculated based on the segmented image and the corresponding gold standard image;

[0237] The second loss function value is calculated based on the segmentation label and the corresponding hierarchical label;

[0238] The total loss function value is calculated based on the first loss function value and the second loss function value.

[0239] The OCT image segmentation model is obtained by iteratively training the preset segmentation network based on the total loss function value.

[0240] The preset segmentation network can include a downsampling stage and an upsampling stage. The downsampling stage includes multiple convolutional layers and pooling layers. The downsampling stage is used to extract features from the input original OCT image. The upsampling stage includes multiple deconvolutional layers. The upsampling stage is used to gradually restore the details of the original OCT image. At the same time, skip connections are added between feature layers of the same resolution to achieve segmentation of the target object in the original OCT image.

[0241] Specifically, the original OCT image is input into the preset segmentation network. Images of different scales are generated through sampling and then fed into convolutional layers of corresponding scales during the downsampling stage. The output of the preset segmentation network is then upsampled and concatenated to obtain the segmented image. Understandably, because the preset segmentation network fully considers the impact of images of different scales on image segmentation, it improves the performance of the convolutional neural network model, thereby increasing the accuracy of segmenting the original OCT image.

[0242] The preset segmentation network also includes a first convolutional layer, a first ReLU activation layer and a first batch normalization layer, a second convolutional layer, a second ReLU activation layer and a second batch normalization layer.

[0243] In each first convolutional layer, a non-linear activation function is used to classify the output, achieving the first comparison of the images. The result of the first comparison reflects how close the segmented image is to the gold standard image. The larger the result of the first comparison, the closer the segmented image is to the gold standard image; the smaller the result of the second comparison, the more different the segmented image is from the gold standard image.

[0244] In each second convolutional layer, a non-linear activation function is used to classify the output, achieving a second alignment of the hierarchical labels. The result of the second alignment reflects the closeness between the segmentation label and the hierarchical label. A larger second alignment result indicates a more consistent segmentation label and hierarchical label, while a smaller result indicates a more different segmentation label and hierarchical label.

[0245] In this embodiment, the loss function value is used to estimate the degree of inconsistency between the model's predicted values ​​and the true values. It is a non-negative real-valued function; the smaller the loss function value, the higher the accuracy of the model's predictions. The first loss function value L1 can be Euclidean distance or cosine angle. The second loss function value L2 can be calculated using the cross-entropy loss function, L2 = -SUM(y*logy'), where y represents the true segmentation label and y' represents the predicted segmentation label.

[0246] The total loss function value is obtained by summing the first and second loss function values. The preset segmentation network is then iteratively trained based on this total loss function value until it converges. At this point, the iterative training stops, and the network model after iterative training stops is identified as the OCT image segmentation model. Iterative training is a model training method in deep learning used to optimize the model. The iterative training process in this step is as follows: In each training iteration, all training samples are sequentially read in, and the current total loss function value is calculated. A stochastic gradient descent algorithm is used to determine the gradient descent direction, causing the total loss function value to gradually decrease and reach a stable state, thereby optimizing the parameters of the constructed network model.

[0247] Convergence of the total loss function value means that the total loss function value is close to 0, for example, less than 0.1.

[0248] The segmentation module 306 is used to call the OCT image segmentation model to segment the OCT image to be segmented.

[0249] The OCT image to be segmented is the object for which lesion regions need to be segmented. The OCT image to be segmented is input into the OCT image segmentation model, and the OCT image segmentation model outputs the lesion regions in the OCT image to be segmented.

[0250] In existing technologies, ocular OCT images are generally captured as tomographic slices, resulting in strong homogeneity between the layers of the eye and thus fewer lesion patterns. This limits the number of samples that can be used as training data, leading to poor generalization ability of the OCT image segmentation models trained from these samples. This invention uses a deformation method to break down the homogeneity between layers in OCT tomographic scans from the same patient, simulating uneven layer thickness and lesions, thereby expanding the diversity of layer morphology and improving the model's generalization ability.

[0251] Furthermore, OCT images are often unclear, especially the boundaries between different layers are very blurred, making it difficult for the network to accurately distinguish them. In existing technologies, deep learning methods often directly use undifferentiated cross-entropy loss calculations, backpropagating each pixel indiscriminately. This results in poor edge segmentation at each OCT layer, with jagged edges or even large-area misclassification, leading to low overall accuracy. This invention first incorporates an edge-aware attention mechanism, distinguishing edges between layers and assigning greater weight to them. This allows the network to focus more on edge penalties during learning, obtaining more accurate edge features and improving the model's segmentation performance. Second, in calculating the loss function, it not only calculates the overall loss between the segmented image and the gold standard image but also the detailed loss between the segmentation label and the layer label. Therefore, it constrains the model's training from both overall and detailed perspectives, further improving the model's segmentation performance.

[0252] The device described in this invention can be applied to the field of digital healthcare. OCT images can provide cross-sectional information of the retina, thus allowing for more accurate analysis of retinal structure. Layer segmentation is helpful in diagnosing many retinal and optic nerve diseases, such as glaucoma, macular degeneration, or diabetic retinopathy. In the diagnosis of glaucoma, OCT is more effective than fundus color images in detecting early cases.

[0253] Example 3

[0254] This embodiment provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps described in the OCT image segmentation method embodiment above, for example... Figure 1 S11-S16 as shown:

[0255] S11, obtain multiple layered masks of the original OCT image;

[0256] S12, determine the edge line based on the multiple layered masks;

[0257] S13, calculate the first gradient weight based on the edge line;

[0258] S14, calculate the second gradient weight based on the original OCT image;

[0259] S15, an OCT image segmentation model is trained using the original OCT image and the corresponding first gradient weight and second gradient weight;

[0260] S16, The OCT image segmentation model is invoked to segment the OCT image to be segmented.

[0261] Alternatively, when the computer program is executed by the processor, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 3 Modules 301-306 in the document:

[0262] The acquisition module 301 is used to acquire multiple layered masks of the original OCT image;

[0263] The determining module 302 is used to determine the edge line based on the plurality of layered masks;

[0264] The first calculation module 303 is used to calculate the first gradient weight based on the edge line;

[0265] The second calculation module 304 is used to calculate the second gradient weight based on the original OCT image;

[0266] The training module 305 is used to train an OCT image segmentation model using the original OCT image and the corresponding first gradient weight and second gradient weight;

[0267] The segmentation module 306 is used to call the OCT image segmentation model to segment the OCT image to be segmented.

[0268] Example 4

[0269] See Figure 4 The diagram shown is a structural schematic of an electronic device provided in an embodiment of the present invention. In a preferred embodiment of the present invention, the electronic device 4 includes a memory 41, at least one processor 42, at least one communication bus 43, and a transceiver 44.

[0270] Those skilled in the art should understand that Figure 4 The structure of the electronic device shown does not constitute a limitation of the embodiments of the present invention. It can be a bus structure or a star structure. The electronic device 4 may also include more or fewer other hardware or software than shown, or different component arrangements.

[0271] In some embodiments, the electronic device 4 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital processors, and embedded devices. The electronic device 4 may also include client devices, including, but not limited to, any electronic product capable of human-computer interaction with a client via a keyboard, mouse, remote control, touchpad, or voice control device, such as personal computers, tablet computers, smartphones, and digital cameras.

[0272] The electronic device 4 described herein is merely an example. Other existing or future electronic products that are adaptable to this invention should also be included within the scope of protection of this invention and are incorporated herein by reference.

[0273] In some embodiments, the memory 41 stores a computer program that, when executed by the at least one processor 42, implements all or part of the steps in the OCT image segmentation method described above. The memory 41 includes a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0274] Furthermore, the computer-readable storage medium may primarily include a program storage area and a data storage area, wherein the program storage area may store the operating system, at least one application required for a function, etc.; and the data storage area may store data created based on the use of blockchain nodes, etc.

[0275] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0276] In some embodiments, the at least one processor 42 is the control unit of the electronic device 4, connecting various components of the electronic device 4 via various interfaces and lines. It executes programs or modules stored in the memory 41 and calls data stored in the memory 41 to perform various functions and process data of the electronic device 4. For example, when the at least one processor 42 executes a computer program stored in the memory, it implements all or part of the steps of the OCT image segmentation method described in this embodiment of the invention; or it implements all or part of the functions of the OCT image segmentation device. The at least one processor 42 may be composed of integrated circuits, such as a single-packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.

[0277] In some embodiments, the at least one communication bus 43 is configured to enable communication between the memory 41 and the at least one processor 42, etc.

[0278] Although not shown, the electronic device 4 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 42 via a power management device, thereby enabling functions such as charging, discharging, and power consumption management. The power supply may also include one or more DC or AC power sources, a recharging device, a power fault detection circuit, a power converter or inverter, a power status indicator, or any other components. The electronic device 4 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0279] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) or processor to execute portions of the methods described in the various embodiments of the present invention.

[0280] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0281] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0282] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0283] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other elements, and the singular does not exclude the plural. Multiple elements or devices recited in the specification may also be implemented by a single element or device in software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

[0284] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An OCT image segmentation method, characterized by, The method comprises: obtaining a plurality of layered masks of an original OCT image; determining an edge line based on the plurality of layered masks; calculating a first gradient weight according to the edge line; calculating a second gradient weight according to the original OCT image; training an OCT image segmentation model using the original OCT image and the corresponding first gradient weight and second gradient weight, comprising: calculating a pixel-level weight of the original OCT image according to the first gradient weight and the second gradient weight; training an OCT image segmentation model using the original OCT image and the corresponding pixel-level weight; calling the OCT image segmentation model to segment a to-be-segmented OCT image.

2. The OCT image segmentation method of claim 1, wherein, The calculation of the pixel-level weight of the original OCT image according to the first gradient weight and the second gradient weight comprises: determining whether the second gradient weight corresponding to each pixel point in the original OCT image is greater than a preset gradient weight threshold; when the second gradient weight corresponding to a first pixel point in the original OCT image is greater than the preset gradient weight threshold, determining the second gradient weight corresponding to the first pixel point as the pixel-level weight of the first pixel point; when the second gradient weight corresponding to a second pixel point in the original OCT image is less than the preset gradient weight threshold, determining the product of the second gradient weight corresponding to the second pixel point and the first gradient weight as the pixel-level weight of the second pixel point.

3. The OCT image segmentation method of claim 2, wherein, The training of the OCT image segmentation model using the original OCT image and the corresponding pixel-level weight comprises: inputting the original OCT image and the corresponding pixel-level weight into a preset segmentation network for segmentation processing to obtain a segmentation image and a segmentation label; calculating a first loss function value according to the segmentation image and the corresponding gold standard image; calculating a second loss function value according to the segmentation label and the corresponding layered label; calculating a total loss function value based on the first loss function value and the second loss function value; iteratively training the preset segmentation network based on the total loss function value to obtain an OCT image segmentation model.

4. The OCT image segmentation method according to any one of claims 1 to 3, wherein, Before the obtaining of the plurality of layered masks of the original OCT image, the method further comprises: performing affine transformation on the original OCT image to obtain a transformed image; generating a displacement in x and y directions for each pixel point in the transformed image to obtain a random displacement field; filtering the random displacement field; controlling the deformation of the filtered random displacement field according to a preset deformation control factor; processing the original OCT image according to the deformed random displacement field to obtain an enhanced OCT image; the obtaining of the plurality of layered masks of the original OCT image comprises the obtaining of the plurality of layered masks of the enhanced OCT image.

5. The OCT image segmentation method of claim 4, wherein, The generation of a displacement in x and y directions for each pixel point in the transformed image to obtain a random displacement field comprises: obtaining coordinate values of each pixel point in the transformed image; comparing the coordinate values with a range of preset coordinate values to obtain a comparison result; According to the comparison result, the pixel points are divided into middle pixel points and edge pixel points; A first displacement in x and y directions is generated for the middle pixel points; A second displacement in x and y directions is generated for the edge pixel points; The random displacement field is obtained according to the first displacement and the second displacement; The first displacement is greater than the second displacement.

6. The OCT image segmentation method of claim 5, wherein, The determination of the edge line based on the plurality of layered masks comprises: The plurality of layered masks are superimposed into an image to obtain a superimposed image; the gradient value of the superimposed image is calculated; and the edge line is determined according to the gradient value; or Each of the layered masks is subjected to morphological dilation processing to obtain a dilated image; each of the layered masks is subjected to morphological erosion processing to obtain an eroded image; a difference image is calculated according to each of the dilated images and the corresponding eroded images; and the edge line is obtained according to the difference image.

7. An OCT image segmentation apparatus characterized by comprising: The device is used to implement the OCT image segmentation method according to any one of claims 1 to 6, and the device comprises: An acquisition module is configured to acquire a plurality of layered masks of an original OCT image; A determination module is configured to determine an edge line based on the plurality of layered masks; A first calculation module is configured to calculate a first gradient weight according to the edge line; A second calculation module is configured to calculate a second gradient weight according to the original OCT image; A training module is configured to train an OCT image segmentation model using the original OCT image and the corresponding first gradient weight and second gradient weight; A segmentation module is configured to call the OCT image segmentation model to segment a to-be-segmented OCT image.

8. An electronic device, comprising: The electronic device comprises a processor and a memory, and the processor is configured to implement the OCT image segmentation method according to any one of claims 1 to 6 when executing a computer program stored in the memory.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the OCT image segmentation method according to any one of claims 1 to 6.

Images