A segmentation method and scoring method and processing device for OCT images

Abstract

The present invention relates to a segmentation method, scoring method and processing device of an OCT image, and belongs to the field of image processing technology related to the eye. The segmentation method includes: reading an OCT image, which is a horizontal cross-sectional view generated with the fovea maculae on the retina as the center point; determining a background image, a retinal nerve fiber layer, an inner retina, an outer retina and a high reflection point by annotating the OCT image at the pixel level; training the original OCT image and the annotated OCT image using an artificial intelligence model to obtain a trained automatic recognition model; inputting a number of OCT images into the automatic recognition model, and the automatic recognition model automatically determines the background image, the retinal nerve fiber layer, the inner retina, the outer retina and the high reflection point. The present invention can automatically detect high reflection points in OCT images, improve recognition accuracy and consistency, help improve the work efficiency of medical staff, and provide more accurate treatment suggestions for patients.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an OCT image segmentation method, a scoring method and a processing device. Background Art

[0002] Highly reflective points refer to a type of points distributed on the retina in OCT images with a diameter of ≤30μm, no artifacts, and reflectivity similar to that of the retinal nerve fiber layer. They have been shown not to correspond to any specific lesions (such as blood vessels and exudates) in color fundus images and enface OCT (structural OCT) scans. Current studies believe that high-reflective points are mainly derived from activated microglia in the retina or aggregates of monocytes and macrophages in the blood. Their number and distribution change significantly in a variety of inflammatory-related fundus diseases, such as diabetic retinopathy, diabetic macular edema, and age-related macular degeneration. They are considered to be biomarkers of retinal neuroinflammatory response in OCT imaging. Therefore, automatic detection and segmentation of high-reflective points in OCT images, and identification of changes in the number and distribution area of ​​high-reflective points are of great guiding significance for the early diagnosis of retinal inflammatory diseases, assessment of disease progression and prognosis, and establishment of precise treatment plans.

[0003] In the past, high reflection points mainly relied on manual identification and counting by professionals, which consumed a lot of time and energy and had certain inaccuracies. With the advancement of science and technology, some researchers have also segmented high reflection points based on some traditional algorithms such as morphological component analysis, region growing algorithm, morphological reconstruction and component tree and traditional image recognition methods. However, most of the high reflection point recognition algorithms based on these traditional algorithms have complicated steps and require researchers to perform complex feature extraction, resulting in poor accuracy and precision of the algorithm. Deep learning algorithms can avoid the limitations of traditional algorithms and can automatically learn data through training. Therefore, establishing a more accurate, automated and scientific image processing and segmentation method based on deep learning algorithms to improve the accuracy and scientificity of high reflection point recognition is of great significance to improving the work efficiency of medical staff and providing accurate treatment recommendations for patients.

[0004] For example, a patent application with application publication number CN116386820A discloses a method for recommending similar case information based on deep learning, including: controlling the OCT scanning device to perform fundus scanning according to a preset scanning mode to collect fundus OCT scanning images of the macular area of ​​the fundus; inputting the fundus OCT scanning image into a trained OCT multi-lesion classification model to obtain the lesion features of the fundus OCT scanning image, and inputting the lesion features of the fundus OCT scanning image into a trained feature generation model to generate lesion features in the eye dimension; comparing the lesion features in the eye dimension with multiple annotated lesion features, obtaining target lesion features that match the lesion features in the eye dimension from multiple annotated lesion features, and using the OCT case information corresponding to the target lesion features as similar case information of the fundus OCT scanning image. The technical solution of this patent is to identify high reflective points in lesions by extracting complex features. The defects of this method include: requiring medical personnel with rich work experience to confirm the identified target lesion features.

[0005] Therefore, how to improve the recognition accuracy of high-reflection points in OCT images is a technical problem that needs to be solved by the current existing technology.

[0006] In addition, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the applicant studied a large number of documents and patents when making the present invention, but due to space limitations, not all details and contents are listed in detail. However, this does not mean that the present invention does not have the characteristics of these prior arts. On the contrary, the present invention already has all the characteristics of the prior art, and the applicant reserves the right to add relevant prior art to the background technology. Summary of the invention

[0007] At present, the identification of high-reflection points in OCT (optical coherence tomography) images mainly relies on manual identification and counting by professionals, which consumes a lot of time and energy and has certain inaccuracies. For example, the current clinical identification method for high-reflection points in OCT images is simply for clinicians to simply look at the number, and rarely directly quantify; accurate identification and quantification usually requires professional counting personnel to manually partition, identify, and quantify their number. Manual stratification and quantification of OCT images is a repetitive and complicated task that requires a lot of manpower and time, and the accuracy is poor. The main technical problem solved by the present invention is to overcome the shortcomings and deficiencies of the prior art and improve the accuracy and efficiency of high-reflection point identification.

[0008] The prior art has already presented a technical solution for collecting images of target lesions through OCT technology and analyzing the collected OCT images to obtain the severity of the lesions. For example, the patent document with publication number CN115345811A discloses a guided calcified plaque detection and scoring method, which includes: obtaining an OCT image sequence of the target lumen, inputting the OCT image sequence into a preset segmentation model for processing, outputting a mask image sequence of the OCT image sequence, the mask image sequence is used to identify the calcified plaque area in the OCT image sequence, calculating the size parameters of the calcified plaque in the OCT image sequence according to the mask image sequence, and determining the severity score of the calcified plaque in the target lumen according to the size parameters. However, this technical solution is mainly aimed at area-type calcified lesions extending and distributed along the surface of the vascular wall, in order to more accurately obtain the distribution status of these lesions on the inner wall of the blood vessel, and at the same time, it is necessary to collect OCT images that are closer to the surface structure of the blood vessel as much as possible to accurately obtain the spread distribution status of the calcified lesions on the surface of the blood vessel, which is different from the retinal inflammatory disease detection and evaluation indicators targeted by the present invention. The present invention is mainly aimed at lesions distributed at different levels along the depth of the retina. These lesions are distributed in a dotted manner in different retinal layers and regions. If only the processing method for area-type lesions is used, the distribution state of lesions at different layer thicknesses on the retinal layer cannot be obtained. Therefore, if the area-type lesion OCT image analysis method of the prior art is relied upon, the specific distribution state of retinal inflammatory disease biomarkers cannot be determined, and the severity of retinal inflammatory diseases cannot be accurately assessed based on the specific distribution state. Therefore, the present invention avoids the use of the above-mentioned prior art OCT image analysis processing method.

[0009] In view of the shortcomings of the prior art, the present invention provides a method for segmenting an OCT image from a first aspect, the method comprising: reading an OCT image, the OCT image being a horizontal cross-sectional view generated with the fovea maculae on the retina as the center point; determining the background image, the retinal nerve fiber layer, the inner retina, the outer retina and the high reflection point by pixel-level annotation of the OCT image; using the read original OCT image and the annotated OCT image to train under the condition of using an artificial intelligence model to obtain a trained automatic recognition model; inputting a number of OCT images into the automatic recognition model, and the automatic recognition model automatically determines the background image, the retinal nerve fiber layer, the inner retina, the outer retina and the high reflection point. In the process of automatic determination using the automatic recognition model, the fovea maculae is identified according to the following steps: for the inner retina automatically segmented by the automatic recognition model, the inner retina is processed by an image processing algorithm related to the graphic morphology to determine the narrowest position of the inner retina; the determined narrowest position is identified as the fovea maculae of the OCT image; with the fovea maculae as the center point, the positions of the inner retina and the outer retina and the number of high reflection points are determined by its preset diameter.

[0010] Different from the prior art, the present invention chooses to use the original OCT image of the horizontal cross-sectional angle of the retinal fovea. Then, the original OCT image is combined with the OCT image annotated at the pixel level, and trained through an artificial intelligence model to form an automatic recognition model. Based on the above-mentioned distinguishing technical features, the problems to be solved by the present invention may include: how to accurately identify high-reflection points at different depths of the retinal layer to clarify the inflammatory nature and degree of retinal diseases. Specifically, the high-reflection point information obtained by performing OCT detection on the retina is the point distributed on the retina in the OCT image. The high-reflection points mainly come from activated microglia in the retina or aggregates of monocytes and macrophages in the blood. Information such as their number and distribution status can be used as a biomarker for a variety of inflammatory-related fundus diseases. The high-reflection point information on the retina is mainly distributed in different spatial positions of the retinal layer, usually with different depths and numbers. Therefore, in order to obtain more accurate information about high-reflection points in OCT images, it is necessary to use the original OCT image of the horizontal cross-section angle through the retinal fovea as the basis for analysis, and it is necessary to avoid using a technical solution similar to the above-mentioned prior art that only performs OCT image analysis on the surface lesions of the object being measured. Otherwise, it will not be possible to accurately obtain the information about high-reflection points in different retinal layers, and thus it will not be possible to accurately identify biomarkers of inflammatory fundus diseases.

[0011] The present invention uses an artificial intelligence model to identify and segment high-reflection points in the OCT image to be evaluated, and can automatically detect high-reflection points in the OCT image to be evaluated, which can save the time and manpower costs of counting personnel and clinicians to a great extent, improve accuracy, and thus help provide accurate treatment recommendations for patients.

[0012] According to a preferred embodiment, the method also includes: processing the automatic segmentation results of the automatic recognition model, the processing method including: applying morphological operations to smooth the segmentation boundaries; using the U-Net deep learning network structure with an added attention mechanism to fine-tune the segmentation results; generating a visual image of the segmentation results, including marking each segmentation area with different colors.

[0013] Fine-tuning the segmentation results can effectively reduce the jagged effect and noise artifacts generated during the segmentation process, making the segmentation boundaries smoother and more continuous, thereby improving the precision and accuracy of the segmentation. Fine-tuning the segmentation results can correct the errors of the automatic segmentation model in specific areas, further improving the precision of the segmentation results. Marking the segmented areas makes the segmentation results more intuitive and easy to understand. The retinal structures of each layer are clearly defined, and the markings of different colors help clinicians quickly identify and evaluate different lesion areas of the retina, improving diagnostic efficiency.

[0014] According to a preferred embodiment, the pixel-level annotation method includes: annotating the background image, retinal nerve fiber layer, inner retina, outer retina and high reflection points according to the pixel features corresponding to each characteristic area in the OCT image to be evaluated. Among them, the high reflection point is displayed with a brightness similar to that of the retinal nerve fiber layer, and the brightness of the high reflection point is different from that of the inner retina and the outer retina. The pixel resolution of each part in the OCT image to be evaluated is different. Annotation by pixel features helps to effectively divide and accurately identify the various areas of the OCT image to be evaluated, avoiding incorrect segmentation of the artificial intelligence model.

[0015] According to a preferred embodiment, when the high reflection point is located in the edema area, the high reflection point is classified into one of the inner retina or the outer retina corresponding to the location of the edema area. Identifying the retinal location of the high reflection point helps to determine the stage of the disease. Generally, in the early stage of the disease, the high reflection point is mainly located in the inner retina, and in the middle and late stages, it is mainly located in the outer retina.

[0016] According to a preferred embodiment, the method further includes: pixel-level annotation of high reflection points includes:

[0017] A pixel brightness threshold is set, and pixels exceeding the pixel brightness threshold are preliminarily marked as candidate areas of high reflection points; connected domain analysis is performed on the candidate areas, and areas with an area smaller than the preset threshold are eliminated to reduce the impact of noise; morphological processing is performed on the remaining areas to optimize the shape and size of the high reflection points.

[0018] By setting the pixel brightness threshold, the pixels exceeding the threshold are preliminarily marked as high reflection point candidate areas, thus achieving preliminary identification of high reflection points. This method can effectively distinguish high reflection points from background noise, thereby improving the marking accuracy of candidate areas. The connected domain analysis technology is used to process the preliminarily marked candidate areas, and areas with an area smaller than the preset threshold are eliminated. This step effectively reduces noise points and small area artifacts in the image, making the final identification of high reflection points more accurate and reliable.

[0019] According to a preferred embodiment, the artificial intelligence model is a U-Net deep learning network structure, and the recognition method of the U-Net deep learning network structure for high reflection points includes: adding an attention mechanism to the encoder part of the U-Net deep learning network structure to improve the sensitivity to small-sized high reflection points; using multi-scale feature fusion in the decoder part to better capture high reflection points of different sizes.

[0020] The attention mechanism can give the model better focusing ability, so that it can more accurately identify and separate small-sized high-reflection points when processing OCT images with complex backgrounds, effectively improving the recognition rate of small-sized high-reflection points. Multi-scale feature fusion integrates information from different scales to enhance the model's comprehensive understanding of the shape and size of high-reflection points, thereby improving the overall recognition accuracy and robustness.

[0021] The present invention provides a scoring method for OCT images from a second aspect, the method comprising: inputting an OCT image to be evaluated into a trained automatic recognition model; automatically determining the background image, retinal nerve fiber layer, inner retina, outer retina and high reflection points of the input OCT image by the automatic recognition model; wherein, in the process of automatic determination using the automatic recognition model, the fovea is identified according to the following steps: for the inner retina automatically segmented by the automatic recognition model, the inner retina is processed by an image processing algorithm related to graphic morphology to determine the narrowest position of the inner retina; the determined narrowest position is identified as the fovea of ​​the OCT image; with the fovea as the center point, the position of the inner and outer retina and the number of high reflection points are determined by its preset diameter; the inflammation score of the inner and outer retina of the input OCT image is determined according to a preset high reflection point number threshold; the thickness change of each retinal layer is calculated; and a comprehensive score is generated by combining the number of high reflection points and the retinal thickness change using a machine learning algorithm.

[0022] The present invention determines the inflammation situation based on the number of high-reflection points, providing reliable reference information for medical personnel. In addition, some studies have shown that during the inflammatory activity, the high-reflection points of the image related to inflammation develop from the inner layer to the outer layer. The method of the present invention can continuously track the related hypothesis of inflammatory changes, and from the perspective of scientific statistics, it has created a new working path for conducting more sophisticated research on fundus lesions and provided an accurate basis for the selection of clinical treatment plans.

[0023] According to a preferred embodiment, in the scoring method, the statistical method of high reflection points includes: binarizing the high reflection point probability map output by the automatic recognition model; using a connected domain labeling algorithm to classify adjacent high reflection point pixels as one high reflection point; and calculating the number of high reflection points in the inner retina and the outer retina respectively.

[0024] Binarization ensures clear positioning of high-reflection points and effectively improves the accuracy and consistency of preliminary marking. Adjacent high-reflection point pixels are classified as one high-reflection point using the connected domain labeling algorithm. This processing method can efficiently integrate connected high-reflection point pixel groups, avoid repeated counting and misjudgment, and help to accurately count the number of high-reflection points that actually exist. The number of high-reflection points in the inner and outer retina is calculated separately, and independent statistics of high-reflection points in different retinal layers are achieved. This hierarchical statistical method can provide more detailed and accurate statistical results.

[0025] According to a preferred embodiment, in the scoring method, the method further includes: calculating the morphological characteristics of the high reflection points, the morphological characteristics including area, perimeter and circularity; and performing a more detailed grading of the severity of inflammation according to the number and morphological characteristics of the high reflection points. The morphological characteristics can provide detailed information about the size, shape and structure of the high reflection points, which helps to more fully understand their morphological characteristics. The present invention not only considers the number of high reflection points, but also comprehensively considers their morphological characteristics, so that the inflammation grading is more detailed and accurate, which helps to improve the accuracy of diagnosis.

[0026] The present invention provides an OCT image processing device from a third aspect, including a processor and a display terminal, the processor being communicatively connected to the display terminal, and an automatic recognition model being arranged in the processor; a plurality of OCT images are input into the automatic recognition model, and the automatic recognition model is used to automatically determine a background image, a retinal nerve fiber layer, an inner retina, an outer retina, and high reflection points; wherein, in the process of automatically determining using the automatic recognition model, the fovea is identified according to the following steps: for the inner retina automatically segmented by the automatic recognition model, the inner retina is processed by an image processing algorithm related to the graphic morphology to determine the narrowest position of the inner retina; the determined narrowest position is identified as the fovea of ​​the OCT image; with the fovea as the center point, the positions of the inner retina and the outer retina and the number of high reflection points are determined by their preset diameters.

[0027] The device of the present invention uses an automatic recognition model in the processor, which can efficiently process several input OCT images. By automatically determining the background image, retinal nerve fiber layer, inner retina, outer retina and high reflection points, manual intervention and processing time are greatly reduced, and the speed and efficiency of data processing are improved. The device can accurately identify the narrowest position of the inner retina, and then determine the fovea. This process ensures the accurate identification of the position of the fovea, provides precise reference points, and makes the subsequent determination of the position of the inner and outer retina and the number of high reflection points more accurate; not only that, it also makes the positioning of other structures based on the fovea more reliable, thereby improving the overall imaging quality. Setting a preset diameter to determine the position of the inner and outer retina and the number of high reflection points provides a complete retinal imaging solution. Such an imaging method ensures the comprehensive visualization of the retinal layers and lesions, allowing doctors to more comprehensively evaluate the health of the retina.

[0028] According to a preferred embodiment, the automatic recognition model annotates the background image, retinal nerve fiber layer, inner retina, outer retina and high reflection points according to the pixel features corresponding to each characteristic area in the OCT image to be evaluated, wherein the high reflection points are displayed with a brightness similar to that of the retinal nerve fiber layer, and the brightness of the high reflection points is different from that of the inner retina and the outer retina. The processing device of the present invention displays the layer distribution of each part through the brightness difference in the image, and clearly displays the distribution position of the high reflection points, so that medical personnel can quickly determine the distribution and number of high reflection points, and then judge the inflammation situation.

[0029] According to a preferred embodiment, the automatic recognition model in the processor scores the inflammation of the inner retina and the outer retina of the OCT image, and the display terminal displays the processed OCT image and its score; wherein the scoring method includes: taking the fovea centralis as the center point and a preset diameter of the fovea centralis to determine the position of the inner retina and the outer retina and the number of high reflection points; counting the number of high reflection points within the inner retina and the outer retina; and determining the inflammation score of the inner retina and the outer retina of the input OCT image to be evaluated according to a preset high reflection point number threshold.

[0030] The processing device of the present invention can count the location and number of high reflection points and determine the inflammation score, and display and provide reference information of eye inflammation conditions to medical personnel through a display terminal, thereby reducing the dependence of high reflection point identification on experienced medical personnel, so that medical personnel can make accurate judgments based on the information provided by the processing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of one of the original OCT images and the OCT image after annotation according to a preferred embodiment of the present invention; the schematic diagram shows the normal structure of the macula;

[0032] Figure 2 It is a schematic diagram of another original OCT image and the OCT image after annotation according to a preferred embodiment of the present invention; the schematic diagram includes a macular edema area;

[0033] Figure 3 It is a schematic diagram of the steps of the OCT image segmentation method of the present invention;

[0034] Figure 4 It is a connection diagram of the OCT image processing device of the present invention.

[0035] Reference numerals list

[0036] 100: OCT image; 110: background image; 120: retinal nerve fiber layer; 130: inner retina; 140: outer retina; 150: high reflection point; 200: processor; 210: automatic recognition model; 300: display terminal. DETAILED DESCRIPTION

[0037] The following is a detailed description with reference to the accompanying drawings.

[0038] The present invention explains some of the noun terms.

[0039] High reflective points 150 (HRF): In OCT images, they are distributed in the inner retina 130 and the outer retina 140, with clear boundaries, a size of ≤30 μm, moderate reflectivity (similar to the retinal nerve fiber layer 120), no artifacts, and no visible lesions on the corresponding color fundus images.

[0040] Macular area: The macula is the thinnest and sharpest area on the retina, located 3-4mm temporal to the optic nerve head at the posterior pole of the eyeball. Its central depression is called the central fovea (fovea centralis), where only pigment epithelium and cone cells are located, with a thickness of only 0.1mm, so vision is the sharpest. The center of the macula is mostly composed of cone cells, which are responsible for light vision and color vision. The surrounding area of ​​the macula is composed of rod cells, which are responsible for dark vision.

[0041] Fovea: There is a small depression in the center of the macula called the fovea. Reflective spots can be seen in the fovea, called foveal reflections, which is the most sensitive part of the retina.

[0042] Retinal nerve fiber layer 120: It is composed of ganglion cell axons, ie, nerve fibers.

[0043] Inner retina 130: from the ganglion cell layer to the outer plexiform layer.

[0044] Outer retina 140: from outer nuclear layer to external limiting membrane.

[0045] Areas of macular edema: Focal or diffuse thickening of the retina in the macula, with accumulation of intraretinal fluid.

[0046] Example 1

[0047] The identification and counting of the high reflection points 150 in the current OCT image 100 needs to be done by professionals with rich work experience, which takes a lot of time and energy and has certain inaccuracies. With the advancement of science and technology, some researchers have segmented the high reflection points 150 based on some traditional algorithms such as morphological component analysis, region growing algorithm, morphological reconstruction and component tree and traditional image recognition methods, but the high reflection point 150 recognition algorithms based on these traditional algorithms are mostly complicated and require designers to perform complex feature extraction, resulting in poor accuracy and precision of the algorithm. Therefore, the present invention improves the accuracy and scientificity of the recognition of high reflection points 150 by improving the intelligent model, improves the work efficiency of medical staff, and provides accurate treatment suggestions for patients.

[0048] In view of the shortcomings of the prior art, the present invention provides an OCT image segmentation method, such as Figure 3 The segmentation method of the OCT image 100 includes:

[0049] S1: Read an OCT image 100. The OCT image 100 is a horizontal cross-sectional view generated with the fovea maculae on the retina as the center point.

[0050] S2: Determine the background image 110 , the retinal nerve fiber layer 120 , the inner retina 130 , the outer retina 140 and the high reflection point 150 by performing pixel-level annotation on the OCT image 100 .

[0051] S3: The read original OCT image and the annotated OCT image are used to perform training using an artificial intelligence model to obtain a trained automatic recognition model 210 .

[0052] S4: The OCT image 100 is input to the trained automatic recognition model 210. The automatic recognition model 210 automatically determines the background image 110, the retinal nerve fiber layer 120, the inner retina 130, the outer retina 140 and the high reflection point 150.

[0053] In the process of performing automatic determination using the automatic identification model 210, the fovea is identified according to the following steps:

[0054] S41: For the inner retina obtained by automatic segmentation of the model, the inner retina is processed by an image processing algorithm related to the graphic morphology so as to determine the narrowest position of the inner retina.

[0055] S42 : Identify the determined narrowest position as the fovea of ​​the OCT image 100 .

[0056] S5: Post-process the automatic segmentation results. The post-processing methods include:

[0057] S51: Apply morphological operations to smooth the segmentation boundaries.

[0058] S52: Use the U-Net deep learning network structure with an added attention mechanism to fine-tune the segmentation results.

[0059] S6: The automatic recognition model 210 generates a visual image of the segmentation result, including marking each segmented area with a different color.

[0060] The present invention uses an artificial intelligence model to identify and segment high reflection points 150 in the OCT image 100, and can automatically detect high reflection points 150 in the OCT image 100, which can greatly save the time and manpower costs of counting personnel and clinicians, improve accuracy, and thus help provide accurate treatment recommendations for patients.

[0061] The detailed description of step S1 is as follows.

[0062] In the present invention, Figure 1 As shown, the OCT image 100 is a horizontal cross-sectional view generated with the fovea maculae on the retina as the center point. Specifically, the OCT image 100 shows a horizontal section of the retina.

[0063] Preferably, the original OCT image 100 is selected based on a preset diameter range. The preset diameter range is within 12 mm. Figure 1 As shown, the original OCT image is an image within a 12 mm diameter range with the fovea as the center. Preferably, the original OCT image is an image within a 10 mm diameter range with the fovea as the center. Particularly preferably, the original OCT image is an image within a 9 mm diameter range with the fovea as the center. Preferably, Figure 2 In the figure, the original OCT image contains edema areas.

[0064] The detailed description of step S2 is as follows.

[0065] The background image 110, retinal nerve fiber layer 120, inner retina 130, outer retina 140 and high reflective point 150 are determined by annotating the original OCT image at the pixel level. Pixel level means that for each pixel in the OCT two-dimensional image, the annotator needs to assign a semantic category label to each pixel in the image in order to train the model to learn how to accurately perform pixel-level segmentation. "Pixel-level" annotation means that each pixel in the OCT image 100 is classified into one of the above five categories.

[0066] In the original OCT image, the background image 110, the retinal nerve fiber layer 120, the inner retina 130, the outer retina 140 and the high reflection point 150 do not overlap. Therefore, by annotating each area of ​​the original OCT image at the pixel level, it is possible to determine which specific feature area of ​​the background image 110, the retinal nerve fiber layer 120, the inner retina 130, the outer retina 140 and the high reflection point 150 the area in the image belongs to, and generate an annotated OCT image containing the annotation information.

[0067] The appearance of high reflection points is related to inflammation. According to the clinical characteristics of OCT images, the high reflection points 150 related to inflammation mainly exist on the inner retina 130 and the outer retina 140, and their reflection intensity is similar to that of the retinal nerve fiber layer 120, and there is a difference in brightness between the inner retina 130 and the outer retina 140. Therefore, marking the high reflection points 150 with a brightness similar to that of the retinal nerve fiber layer 120 can not only clearly identify the high reflection points 150, but also accurately display the distribution of the high reflection points 150 on the inner retina 130 and the outer retina 140.

[0068] Preferably, according to the clinical characteristics of OCT images, the inner retina 130 and the outer retina 140 are not a single layer structure, but a phenomenon of multiple layers. Therefore, more regions in the OCT image can be annotated at more pixel levels by further training an artificial intelligence model, especially local regions in the inner retina 130 and the outer retina 140 can be annotated at different pixel levels, so that the retinal fovea structure in the OCT image can be cut more detailed.

[0069] According to a preferred embodiment, the pixel-level annotation method includes:

[0070] S21: Setting a pixel brightness threshold, and preliminarily marking pixels exceeding the threshold as high reflection point candidate areas.

[0071] A pixel brightness threshold is set to distinguish the high reflective point 150 from the background image 110. This threshold can be determined according to the specific brightness characteristics of the OCT image 100. For example, if the pixel brightness range of the OCT image 100 is 0 to 255, an intermediate value, such as 200, can be selected as the brightness threshold of the high reflective point 150.

[0072] Specifically, the input OCT image 100 is read and converted into a grayscale image. Each pixel is traversed to determine whether its brightness exceeds a set threshold. For example, the threshold is set to 200. If the brightness of a pixel exceeds 200, the pixel is preliminarily marked as a candidate area of ​​a high reflection point. A binary image can be created and the corresponding position is marked as 1.

[0073] S22: Performing connected domain analysis on the candidate regions, and eliminating regions with an area smaller than a preset threshold to reduce the impact of noise.

[0074] Specifically, a connected domain analysis is performed on the binary image to identify each connected region. The area of ​​each connected region (i.e., the number of pixels in the region) is calculated. A preset area threshold is set, such as 20 pixels. Connected regions with an area less than 20 pixels are eliminated and removed from the binary image (marked as 0).

[0075] S23: Perform morphological processing on the remaining area, such as dilation and corrosion operations, to optimize the shape and size of the high reflection point 150.

[0076] Dilation operation: Use the morphological dilation function (such as cv2.dilate in OpenCV) to process the binary image. Use the morphological dilation operation to expand the high reflection point area. You can use a structural element (such as a 3x3 square) for dilation. Dilation helps fill small holes in the area and connect adjacent small areas.

[0077] Erosion operation: Use the morphological erosion function (such as cv2.erode in OpenCV) to process the binary image. Performing morphological erosion can eliminate the noise that may be introduced during the dilation process. Erosion can reduce the area of ​​high reflection points to make it more accurate.

[0078] Through the expansion and corrosion operations, the shape and size of the high reflection point 150 can be optimized to be more in line with the actual situation.

[0079] Steps S21 and S22 effectively detect and optimize the high reflection point area from the input OCT image 100. This method gradually reduces the noise effect and optimizes the accuracy of the high reflection point 150 by setting pixel brightness thresholds, connected domain analysis and morphological processing, providing a reliable data basis for subsequent image analysis and processing.

[0080] OCT can indicate a variety of characteristic changes related to disease severity, response to treatment and prognosis, such as cystic intraretinal fluid, subretinal fluid, inner retinal structural disorder, high reflective points, etc. In the eye OCT image 100 formed by traditional calculation methods, it is difficult to identify and count the distribution characteristics and number of high reflective points 150 related to inflammation, thereby ignoring the judgment of the inflammatory nature of the disease, increasing the difficulty of diagnosis.

[0081] How to make the high reflection points 150 related to inflammation also be distinguished in the OCT image 100 by brightness difference is an issue that needs to be focused on. The present invention improves the segmentation method of the OCT image so that the high reflection points 150 in the OCT image 100 are displayed with a brightness similar to that of the retinal nerve fiber layer 120. Specifically, the brightness difference between the high reflection point 150 and the retinal nerve fiber layer 120 is less than the preset difference threshold, so that the brightness of the high reflection point 150 is similar to or the same as the brightness of the retinal nerve fiber layer 120, so that the brightness of the high reflection point 150 is significantly different from the brightness of the inner retina 130 and the outer retina 140, so the three can be clearly distinguished and displayed by brightness, which helps doctors to more clearly identify and distinguish the high reflection points 150 in the OCT image 100, thereby providing a direct and reliable basis for the diagnosis of inflammation and reducing the possibility of misdiagnosis and missed diagnosis.

[0082] Preferably, the preset difference threshold is a grayscale value of 20. The preset difference threshold can also be set between 20 and 30. Alternatively, the preset difference threshold is a ratio of the brightness difference value to the brightness of the retinal nerve fiber layer 120. The preset difference threshold is, for example, 20%.

[0083] However, the fact that the high reflection point 150 is displayed with a brightness similar to that of the retinal nerve fiber layer 120 does not mean that the high reflection point 150 has exactly the same brightness as that of the retinal nerve fiber layer 120. There may also be high reflection points 150 on the retinal nerve fiber layer 120. The high reflection points 150 in this part may be related to neurodegeneration (such as glaucoma, optic neuropathy), eye trauma and deposits, structural abnormalities (macular degeneration, epiretinal membrane or posterior vitreous membrane), and other diseases. Therefore, there is a brightness difference between the high reflection point 150 and the retinal nerve fiber layer 120. If necessary, the processing steps of calculating the distribution characteristics and quantity statistics of the high reflection points 150 on the retinal nerve fiber layer 120 can be added according to the diagnostic needs to provide more objective image data for diagnosis.

[0084] The present invention uses pixel features to mark, which helps to effectively divide and accurately identify various areas of the OCT image 100, and avoids incorrect marking of the artificial intelligence model. In the OCT image 100, different brightness is displayed as different degrees of gray.

[0085] According to a preferred embodiment, when the high reflection point 150 is located in the edema area, the high reflection point 150 is classified into one of the inner retina 130 or the outer retina 140 corresponding to the location of the edema area. Identifying the retinal location of the high reflection point 150 helps to determine the stage of the disease. Generally, in the early stage of the disease, the high reflection point 150 is mainly located in the inner retina 130, and in the middle and late stages, it is mainly located in the outer retina 140.

[0086] The detailed description of step S3 is as follows.

[0087] The original OCT images and the annotated OCT images are trained using an artificial intelligence model to form an automatic recognition model 210 .

[0088] Specifically, an artificial intelligence model is established. The artificial intelligence model specifically refers to a deep learning semantic segmentation model, preferably a U-Net deep learning network structure, DeepLab V3+, etc.

[0089] The U-Net deep learning network structure is specially optimized for the identification of high reflectivity points 150. The steps for optimizing the U-Net deep learning network structure include:

[0090] S31: Add an attention mechanism to the encoder part of the U-Net deep learning network structure to improve the sensitivity to small-sized highly reflective points 150. The attention mechanism can dynamically adjust the degree of attention to different areas and strengthen the extraction of important features.

[0091] U-Net consists of an encoder (downsampling path) and a decoder (upsampling path). The encoder extracts image features layer by layer and performs downsampling, while the decoder restores the spatial resolution layer by layer and finally outputs a prediction image of the same size as the input image.

[0092] Commonly used attention mechanisms include channel attention mechanisms (such as SE modules) and spatial attention mechanisms (such as CBAM modules). An attention module is added after each convolutional layer in the encoder part of U-Net. The attention module generates a weight matrix to adjust the weight of each feature map based on the content of the feature map.

[0093] The attention module calculates the weight of each channel or spatial position through a fully connected layer and an activation function (such as Sigmoid). The attention module multiplies the calculated weight with the original feature map to strengthen important features and suppress unimportant features.

[0094] S32: Multi-scale feature fusion is used in the decoder part to better capture high reflection points 150 of different sizes. Multi-scale feature fusion helps to extract and combine features at different resolutions, thereby improving the network's recognition ability for high reflection points 150.

[0095] Specifically, at each upsampling stage of the decoder, feature maps of different scales are retained. The feature maps of the skip connection are obtained from the encoder part, which contain information of different scales. At each upsampling stage, the feature map of the current layer is fused with the feature map of the skip connection.

[0096] The fusion methods include concatenation (concat) and element-by-element addition (add) of feature maps.

[0097] Splicing fusion is to splice feature maps of different scales in the channel dimension, and then extract features through the convolution layer to integrate multi-scale information. Weighted fusion is to perform weighted addition of feature maps of different scales, so that the network can adaptively adjust the importance of features at each scale.

[0098] In the present invention, the upsampling operation is to restore the spatial resolution of the image using a deconvolution or upsampling (such as bilinear interpolation) operation. After each upsampling, multi-scale feature fusion is performed to ensure that high-resolution features are fully combined with low-resolution features. After upsampling, detail features are further extracted through a convolution layer. Multi-scale feature fusion helps to restore details at high resolution and improve the recognition accuracy of high-reflection points 150 of different sizes.

[0099] In the last layer of the decoder, a convolutional layer is used to generate a final prediction map, which combines multi-scale features and can accurately locate and identify high-reflection points 150 of different sizes.

[0100] By introducing the attention mechanism in the encoder part of the U-Net deep learning network structure and using multi-scale feature fusion in the decoder part, the network's ability to recognize high-reflection points 150 can be significantly improved. The attention mechanism enhances the sensitivity to small-sized high-reflection points 150, while multi-scale feature fusion ensures the comprehensive capture of high-reflection points 150 of different sizes. These optimization steps help improve the accuracy and robustness of the network and provide a more effective solution for the detection of high-reflection points 150.

[0101] The automatic recognition model 210 can be used for automatic segmentation and recognition of the background image 110 , the retinal nerve fiber layer 120 , the inner retina 130 , the outer retina 140 and the high reflection points 150 , and can recognize and count the distribution position and number of the high reflection points 150 .

[0102] The detailed description of step S4 is as follows.

[0103] Since the original OCT image is pre-screened and is a horizontal cross-sectional view with the retinal fovea as the center point, when the automatic recognition model 210 is applied, the fovea, the background image 110, the retinal nerve fiber layer 120, the inner retina 130, the outer retina 140 and the high reflection point 150 also need to be automatically recognized.

[0104] Methods for identifying the fovea in the OCT image 100 to be evaluated include:

[0105] S41: Determine the narrowest position of the inner retina 130 by an image processing algorithm related to the graphic morphology.

[0106] The image of the inner retinal area is processed by an image processing algorithm related to the pattern morphology to be an image capable of determining the narrowest position of the inner retina 130 .

[0107] Specifically, according to the inner retinal area automatically segmented by the automatic recognition model 210, the inner retinal area is processed into a binary image, and the number of connected domains in the binary image is determined. If there is only one connected domain in the binary image, the morphological erosion operation is continuously performed using the template se=ones(3,3) until two connected domains appear; if the connected domains are greater than or equal to two, the top two connected domains in terms of area are retained.

[0108] S42: Identify the determined narrowest position as the fovea of ​​the to-be-evaluated OCT image 100. The calculation is helpful to improve the accuracy of the fovea.

[0109] Extract the boundary points of two connected domains, traverse the boundary points, calculate the Euclidean distance between the two points, and take the center point between the two points with the shortest Euclidean distance as the automatically identified central concave point.

[0110] In the case of determining the fovea, the retinal nerve fiber layer 120, the inner retina 130, the outer retina 140 and the high reflective point 150 can be quickly identified based on pixel differences and positional relationships.

[0111] S43: With the fovea as the center point, the preset diameter of the fovea is used to determine the positions of the inner retina 130 and the outer retina 140 and the number of high reflection points 150. Preferably, the preset diameter of the fovea is 6-10 mm. The automatic recognition model 210 locates and counts the high reflection points 150 within this range.

[0112] Specifically, the fovea is marked at the center of the OCT image 100. With the fovea as the center point, a circular region of interest is determined according to a preset diameter of 6 to 10 mm.

[0113] Preferably, the statistical method of high reflection points 150 includes:

[0114] S431: Binarization processing is performed on the high reflection point probability map output by the automatic recognition model 210.

[0115] Specifically, the binarization method is as follows: Initialize a binary image matrix B with the same size as the high reflection point probability map. Each pixel in the high reflection point probability map is P(x, y). If P(x, y)≥T, then B(x, y)=1 (indicating a high reflection point). Otherwise, B(x, y)=0 (indicating a non-high reflection point). This ensures that all pixels have been traversed and assigned values ​​to generate a complete binary image B. T represents the threshold used for binarization.

[0116] S432: Using a connected domain labeling algorithm, adjacent high-reflection point pixels are grouped as one high-reflection point 150 .

[0117] Use the 8-connectivity algorithm to consider the connectivity in the up-down, left-right, and diagonal directions. Create a label matrix L of the same size as the binary image B, with initial values ​​of 0, to record the labels of the connected domain. Initialize the connected domain counter C, with an initial value of 0. Traverse each pixel B(x, y) of the binary image B from left to right and from top to bottom. If B(x, y)=1 and L(x, y)=0, it means this is a new high reflection point 150.

[0118] Mark connected domains: Use depth-first search (DFS) or breadth-first search (BFS) to start from the current pixel and mark all its adjacent pixels with a value of 1 as the same connected domain. For each new connected domain, increase the value of the connected domain counter C and use the counter value to mark all pixels in the connected domain.

[0119] Generate a connected domain label graph: In the final generated connected domain label graph L, each connected domain has a unique label value (from 1 to C).

[0120] S433: Count the number of high reflection points 150 in the inner retina 130 and the outer retina 140 respectively.

[0121] According to the segmentation boundary, the connected domain label map L is divided into the inner retina 130 and the outer retina area.

[0122] Create two mask matrices Minner and Mouter to mark the inner and outer regions respectively.

[0123] For each pixel (x, y), if it is in the inner region, Minner(x, y)=1, otherwise it is 0.

[0124] For each pixel (x, y), if it is in the outer region, Mouser(x, y)=1, otherwise it is 0.

[0125] The number of high reflection points is counted by traversing the connected domain label graph L using the inner mask Minner and the outer mask Mouter respectively. Specifically, the number of high reflection points 150 in the inner retina 130, that is, the number of connected domains within the Minner range, is calculated. The number of high reflection points 150 in the outer retina 140, that is, the number of connected domains within the Mouter range, is calculated.

[0126] The statistical results are summarized to obtain the number of high reflection points 150 in the inner retina 130 and the outer retina 140 respectively.

[0127] For example, the automatic recognition model 210 layers the retina and accurately marks the positions of the inner retina 130 and the outer retina 140. Within the 6 mm diameter range, the automatic recognition model 210 detects 35 high reflection points 150.

[0128] The number of high reflection points 150 distributed on the inner retina 130 is 20. The number of high reflection points 150 distributed on the outer retina 140 is 15. The specific coordinates (in pixels) of the high reflection points 150 are:

[0129] Inner retina 130: [(x1, y1), (x2, y2), ..., (x 20 , y 20 )].

[0130] Outer retina 140: [(x 21 , y 21 ), (x 22 , y 22), ..., (x 35 , y 35 )].

[0131] The present invention can intuitively output or display the distribution of high-reflection points 150 within the fovea centralis, providing important diagnostic reference for clinicians.

[0132] The present invention describes step S5 as follows.

[0133] S51: Apply morphological operations to smooth the segmentation boundaries.

[0134] Specifically, first, the images of the background image 110, the retinal nerve fiber layer 120, the inner retina 130, the outer retina 140 and the high reflection point 150 are dilated to expand the foreground area. Dilation helps to fill small holes in the segmented area and connect adjacent small areas. The binary image is processed using a morphological dilation function (such as cv2.dilate in OpenCV).

[0135] Then, an erosion operation is performed to shrink the foreground area to restore its original size while smoothing the boundaries. The previously dilated image is processed using a morphological erosion function (such as cv2.erode in OpenCV).

[0136] The different layers formed by segmentation usually have jagged boundaries, which may be caused by the quantization of the image or the limitations of the segmentation algorithm. The dilation and erosion operations in the morphological operation can effectively smooth the boundaries of the background image 110, the retinal nerve fiber layer 120, the inner retina 130, the outer retina 140 and the high reflective point 150, making them smoother and more natural, and avoiding irregular and abrupt shapes. In some segmentation tasks, the fovea may be mistakenly segmented into multiple discontinuous small areas. The dilation operation in the morphological operation can expand the foreground area, thereby connecting these broken small areas to form a coherent whole and improving the integrity of the segmentation result. In retinal image segmentation, the closing operation can help fill in small breaks in the vascular area, making the segmentation result more coherent and accurate.

[0137] S52: Add the U-Net deep learning network structure with attention mechanism to fine-tune the segmentation results.

[0138] Specifically, the fine-tuning methods include boundary correction, detail refinement, blurry area processing, artifact elimination and multi-scale information fusion.

[0139] Boundary correction uses the features learned by the model to further correct the segmentation boundaries and improve the accuracy and smoothness of the boundaries. The U-Net deep learning network structure can capture more subtle image structures, thereby refining the details of the segmented area, such as small objects or complex structures. For uncertain or ambiguous areas in the preliminary segmentation results, the U-Net deep learning network structure can perform more accurate classification based on contextual information. The U-Net deep learning network structure can effectively identify and eliminate artifacts or false positive areas in the preliminary segmentation. By using multi-scale feature fusion, the U-Net deep learning network structure can better handle objects of different sizes and improve the comprehensiveness of segmentation.

[0140] The description of step S6 of the present invention is as follows.

[0141] S6: The automatic recognition model 210 generates a visual image of the segmentation result, including marking each segmented area with a different color.

[0142] For example, the labels of the segmented regions formed by the automatic recognition model 210 are as follows:

[0143] Background: label value 0; retinal nerve fiber layer 120 (RNFL): label value 1; inner retina 130: label value 2; outer retina 140: label value 3; high reflective point 150: label value 4.

[0144] In order to visualize these category labels, the automatic recognition model 210 needs to assign a specific grayscale value to each category label. Different grayscale values ​​make each segmented area visually contrasted. The following is a possible way to map the grayscale value of the grayscale image. The shape of the grayscale image is mapped to the segmented area.

[0145] Background image 110: gray value 0 (black); retinal nerve fiber layer 120: gray value 180 (lighter gray); inner retina 130: gray value 100~120 (darker gray); outer retina 140: gray value 120~140 (darker gray); high reflection point 150: gray value 160~170 (light gray).

[0146] Preferably, the inner retina 130 and the outer retina 140 include a multi-layer retinal structure and can be displayed as different grayscale values.

[0147] Superimposing the mapped grayscale image with the original OCT image can show the segmentation results more intuitively. The superposition method usually combines the two images with a certain weight so that the details of the original image and the different segmented areas of the segmentation result can be clearly displayed. Through this mapping, different tissue layers and features will be displayed with different grayscale values ​​on the image, thus forming a clear visual contrast.

[0148] The present invention also provides an OCT image scoring method, the method comprising:

[0149] S110 : Inputting the OCT image 100 to be evaluated into the trained automatic recognition model 210 .

[0150] S120 : The automatic recognition model 210 automatically determines the background image 110 , the retinal nerve fiber layer 120 , the inner retina 130 , the outer retina 140 , and the high reflection points 150 of the input OCT image.

[0151] S130: In the process of automatically determining using the automatic recognition model 210, the fovea of ​​the input OCT image 100 is recognized according to the following steps.

[0152] S131: For the inner retina obtained by automatic segmentation of the model, the inner retina is processed by an image processing algorithm related to the graphic morphology so as to determine the narrowest position of the inner retina.

[0153] S132 : Identify the determined narrowest position as the fovea of ​​the OCT image 100 .

[0154] S140: Taking the fovea centralis as the center point, the positions of the inner retina 130 and the outer retina 140 and the number of the high reflection points 150 are determined according to the preset diameters.

[0155] S150: Count the number of high reflection points 150 in the inner retina 130 and the outer retina 140 respectively.

[0156] S160: Determine the inflammation scores of the inner retina 130 and the outer retina 140 of the input OCT image 100 according to a preset high reflection point quantity threshold.

[0157] S170: Calculate the thickness changes of each retinal layer and take it into account in the scoring.

[0158] S180: A machine learning algorithm is used to combine the number of high-reflection points 150 and the change in retinal thickness to generate a comprehensive score.

[0159] Figure 1 The original image of the OCT image 100 with normal macular structure and the segmentation result formed by the OCT image segmentation method of the present invention are shown. Figure 1 As shown, the number of high reflection points 150 in the inner retina 130 is 47, and the number of high reflection points 150 in the outer retina 140 is 1.

[0160] Figure 2The original image of the OCT image 100 to be evaluated containing the macular edema area and the segmentation and recognition results formed by the OCT image segmentation method of the present invention are shown. Figure 2 As shown, the number of high-reflection points 150 in the inner retina 130 is 67, and the number of high-reflection points 150 in the outer retina 140 is 16.

[0161] For example, for the inner retina 130: 0-5 high reflectivity points are scored as 0 points, 6-10 as 1 point, 11-20 as 2 points, and more than 20 as 3 points.

[0162] For the outer retina 140: 0-2 high reflectivity points were scored as 0 points, 3-5 as 1 point, 6-10 as 2 points, and more than 10 as 3 points.

[0163] The present invention determines the inflammation situation based on the number of high reflection points 150, providing reliable reference information for medical personnel. In addition, some studies believe that during the inflammatory activity, the inflammation-related images develop from the inner layer to the outer layer. The method of the present invention can continuously track the relevant hypothesis of inflammatory changes, and from the perspective of scientific statistics, it has created a new working path for conducting more sophisticated research on fundus lesions and provided accurate basis for the selection of clinical treatment plans.

[0164] The steps to calculate the thickness changes of the retinal layers include:

[0165] S171: Calculate the morphological features of the high reflection point 150, including area, perimeter, circularity, etc.

[0166] Preferably, the area of ​​the high reflection point is calculated by counting the number of pixels in the high reflection point area in the binary image. The boundary of the high reflection point 150 is found by an edge detection algorithm (such as Canny edge detection), and the number of pixels in the boundary is counted. By counting the number of these non-zero value pixels, the perimeter of the high reflection point 150 can be obtained.

[0167] Circularity is an indicator to evaluate how close a shape is to a circle and is defined as circularity = 4π × area / circumference 2 A circularity close to 1 indicates that the shape is close to a circle, while a circularity below 1 indicates that the shape is more irregular.

[0168] S172: Based on the number and morphological characteristics of the high reflection points 150, the severity of the inflammation is more finely graded.

[0169] Preferably, the grading is, for example, mild, moderate and severe inflammation.

[0170] Mild: The number of high-reflective points 150 is small, and the morphological characteristics (area, circularity) are close to the normal range.

[0171] Moderate: The number of high reflective points 150 is moderate, and the morphological characteristics show slight irregularity or enlargement.

[0172] Severe: The number of high-reflection points 150 increases significantly, and the morphological characteristics show significant abnormalities, such as increased area and irregular shape.

[0173] Preferably, the automatic identification model 210 also calculates the thickness of each layer. By comparing these thickness data with baseline data (such as the average thickness value of the healthy control group), the thickness change of each layer can be calculated. The definition of thickness change can be determined by a simple difference calculation, or by more complex statistical methods, such as standard deviation or percentage change. Thickness change is used to indicate the degree of change in retinal structure. For example, the specific scoring criteria for retinal thickness change are as follows: when the thickness change is between 0-1 microns, the score is 0 points; when it is between 2-3 microns, the score is 1 point; when it is between 4-5 microns, the score is 2 points; when it is greater than 5 microns, the score is 3 points.

[0174] For example, in the OCT image 100 of patient A, the number of high reflection points 150 distributed in the inner retina 130 is 20. The number of high reflection points 150 distributed in the outer retina 140 is 15. The retinal thickness variation of the inner retina 130 and the outer retina 140 is: [5 microns, 3 microns]. High reflection point features: [number = 20, area = 0.5 square microns, perimeter = 2 microns, circularity = 0.9].

[0175] Then, the score of the inner retina 130 is 2 points, and the score of the outer retina 140 is 3 points. The score of the high reflection point feature is: Assuming that each feature score has the same weight, the scoring method is as follows.

[0176] Quantity (20): directly used as feature values ​​for standardization.

[0177] Area (0.5 square micrometers): Normalized value is 0.5.

[0178] Circumference (2 microns): Normalized value is 2.

[0179] Circularity (0.9): The normalized value is 0.9.

[0180] Feature score = (20 + 0.5 + 2 + 0.9) / 4 = 5.85.

[0181] Thickness variation score: 3 points.

[0182] It is assumed that the weights are set as follows: the number score weight of the high reflection points 150 in the inner retina 130: 0.3; the number score weight of the high reflection points 150 in the outer retina 140: 0.3; the retinal thickness variation score weight: 0.3; the high reflection point feature score weight: 0.1.

[0183] Then, the comprehensive scoring formula is:

[0184] The comprehensive score = 0.3 × (inner retina score) + 0.3 × (outer retina score) + 0.3 × (thickness change score) + 0.1 × (characteristic score).

[0185] Comprehensive score = 0.3×2+0.3×3+0.3×3+0.1×5.85=2.985.

[0186] Example 2

[0187] This embodiment is a further improvement on Embodiment 1, and the repeated contents will not be repeated here.

[0188] This embodiment illustrates an OCT image segmentation method by way of example.

[0189] S210: Collect SD-OCT images of the fundus of diabetic patients, and select a scanning image in the horizontal direction B of the horizontally cut fovea scan.

[0190] Preferably, the size of the image block is set to Image block (pixel resolution: 12.5 μm / pixel horizontally and 4 μm / pixel vertically).

[0191] S211: Divide the SD-OCT images into a training group and a test group, the training group is used to train the model, and the test group is used to test the performance of the model.

[0192] S212: In the marked group SD-OCT image, five regions (including the background image 110, the retinal nerve fiber layer 120, the inner retina 130, the outer retina 140, and the high reflection point 150) are manually marked.

[0193] S213: Iterate the deep learning model in a continuous learning manner to update the model and expand the data set. The initial training data sample is 30 paired images.

[0194] S214: Using the semantic segmentation model U-Net, model training is performed on five types in the image (including background image 110, retinal nerve fiber layer 120, inner retina 130, outer retina 140, and high reflection point 150).

[0195] For example, the training batch size is 8.

[0196] Use RMSpro optimizer with a fixed learning rate of 10 -6 , the loss function uses a hybrid loss function that combines Dice Loss and cross entropy, which is expressed as:

[0197] ;

[0198] ;

[0199] ;

[0200] Among them, X is the tissue mask, Y is the prediction result, is a smoothing term to prevent the denominator from being 0. This hybrid loss function not only ensures the fast convergence and robustness to category imbalance of the image-level segmentation loss function Dice Loss, but also has the advantages of segmentation smoothness of the pixel-level segmentation loss function.

[0201] S215: Use the trained model to predict new data.

[0202] S216: Correct the prediction results and put the corrected image labels into the training set. Train the model again with the same parameters.

[0203] S217: After 10 iterations, the model was successfully constructed and the final model was trained and validated using the test set. The final training set was 500 images and the test set was 80 images.

[0204] S218: Extract the binary image of the inner retina 130 mask and determine whether it is two connected domains. If there is only one connected domain, perform a morphological corrosion operation on the image. The template used is The ones matrix is ​​constructed until two connected domains appear. If there are more than two connected domains, the largest first two connected domains are retained. For two connected domains, the boundary point coordinates are automatically extracted, the shortest Euclidean distance between the boundaries is calculated, the midpoint is calculated based on the shortest connected line segment, and the midpoint is used as the midpoint of the SD-OCT image according to the specified width (diameter 6.0 mm and 9.0 mm) to determine the two ranges.

[0205] S219: According to the two limited ranges, the number of high reflection points 150 at three levels within the ranges is automatically calculated, and a comprehensive score of the high reflection points 150 is given.

[0206] Example 3

[0207] This embodiment is a further improvement on Embodiment 1, and the repeated contents will not be repeated here.

[0208] The present invention provides an OCT image processing device, such as Figure 4As shown, it includes a processor 200 and a display terminal 300, the processor 200 is in communication connection with the display terminal 300, and an automatic recognition model 210 is set in the processor 200. The automatic recognition model 210 is used to automatically segment the OCT image 100 to be evaluated to automatically determine various feature areas, and the display terminal 300 displays the OCT image 100 annotated with various feature areas. The training method of the automatic recognition model 210 includes: selecting the original OCT image at a horizontal cross-sectional angle through the fovea of ​​the retina. The annotated OCT image annotated with various feature areas is formed by annotating the original OCT image at the pixel level. The original OCT image and the annotated OCT image are trained using an artificial intelligence model to form an automatic recognition model 210.

[0209] The OCT image processing device of the present invention can display an accurately identified OCT image 100 and accurately display the position and number of high reflection points 150, thereby providing accurate information of the high reflection points 150 and inflammation conditions to medical personnel.

[0210] According to a preferred embodiment, the automatic recognition model 210 annotates the background image 110, the retinal nerve fiber layer 120, the inner retina 130, the outer retina 140 and the high reflection points 150 according to the pixel features corresponding to each characteristic area in the OCT image 100 to be evaluated, wherein the high reflection points 150 are displayed with a brightness similar to that of the retinal nerve fiber layer 120, and the brightness of the high reflection points 150 is different from that of the inner retina 130 and the outer retina 140. The processing device of the present invention displays the layer distribution of each part through the brightness difference in the image, and clearly displays the distribution position of the high reflection points 150, so that medical personnel can quickly determine the number of high reflection points 150 and the inflammation situation.

[0211] like Figure 4 As shown, the automatic recognition model 210 in the processor 200 scores the inflammation of the inner retina 130 and the outer retina 140 of the evaluated OCT image 100, and the display terminal 300 displays the processed and annotated OCT image and its score. The scoring method includes: taking the fovea as the center point and the preset diameter of the fovea as the center point to determine the position of the inner retina 130 and the outer retina 140 and the number of high reflection points 150. Counting the number of high reflection points 150 in the inner retina 130 and the outer retina 140. Determining the score of the inflammation of the inner retina 130 and the outer retina 140 of the OCT image 100 to be evaluated according to the preset threshold value of the number of high reflection points 150.

[0212] The processing device of the present invention can count the positions and numbers of high reflection points 150 and determine the inflammation score, and display and provide reference information of eye inflammation conditions to medical personnel through the display terminal 300, thereby reducing the dependence of the identification of high reflection points 150 on experienced medical personnel, so that medical personnel can make accurate judgments on lesions based on the information provided by the processing device.

[0213] It should be noted that the above-mentioned specific embodiments are exemplary, and those skilled in the art can come up with various solutions inspired by the disclosure of the present invention, and these solutions also belong to the disclosure scope of the present invention and fall within the protection scope of the present invention. Those skilled in the art should understand that the present invention specification and its drawings are illustrative and do not constitute a limitation on the claims. The scope of protection of the present invention is defined by the claims and their equivalents. The present invention specification contains multiple inventive concepts, such as "preferably" and "according to a preferred embodiment", which means that the corresponding paragraph discloses an independent concept, and the applicant reserves the right to file a divisional application based on each inventive concept.

Claims

1. A method for segmenting an OCT image, characterized in that: The method comprises: Reading an OCT image (100), wherein the OCT image (100) is a horizontal cross-sectional view generated with the fovea maculae on the retina as the center point; Determining a background image (110), a retinal nerve fiber layer (120), an inner retina (130), an outer retina (140), and a high reflection point (150) by annotating the OCT image (100) at a pixel level, wherein local areas in the inner retina (130) and the outer retina (140) are annotated at different pixel levels; Using the read original OCT image and the annotated OCT image to perform training using an artificial intelligence model, a trained automatic recognition model is obtained (210); Inputting a plurality of the OCT images (100) into the automatic recognition model (210), and using the automatic recognition model (210) to automatically determine the background image (110), the retinal nerve fiber layer (120), the inner retina (130), the outer retina (140), and the high reflection point (150); In the process of automatically determining by using the automatic recognition model (210), the fovea centralis is identified according to the following steps: For the inner retina (130) automatically segmented by the automatic recognition model (210), the inner retina (130) region is processed into a binary image, and the inner retina (130) is processed by an image processing algorithm related to graphic morphology to determine the narrowest position of the inner retina (130); identifying the determined narrowest position as the fovea of ​​the OCT image (100); The positions of the inner retina (130) and the outer retina (140) and the number of the high reflection points (150) are determined by taking the fovea centralis as the center point and using a preset diameter thereof.

2. The OCT image segmentation method according to claim 1, characterized in that: The method further comprises: processing the automatic segmentation result of the automatic recognition model (210), wherein the processing method comprises: applying morphological operation to smooth the segmentation boundary; Use the U-Net deep learning network structure with an added attention mechanism to fine-tune the segmentation results; Generate a visualization of the segmentation results, including color-coding each segmented region.

3. The OCT image segmentation method according to claim 2, characterized in that: The pixel-level annotation method comprises: annotating the background image (110), the retinal nerve fiber layer (120), the inner retina (130), the outer retina (140), and the high reflection point (150) according to the pixel features corresponding to each feature area in the OCT image (100); The brightness difference between the high reflection point (150) and the retinal nerve fiber layer (120) is less than a preset difference threshold, so that the brightness of the high reflection point (150) is similar to or the same as the brightness of the retinal nerve fiber layer (120), and there is a difference between the brightness of the high reflection point (150) and the brightness of the inner retina (130) and the outer retina (140).

4. The OCT image segmentation method according to claim 3, characterized in that: In the case where the high reflection point (150) is located in an edema area, the high reflection point (150) is classified into one of the inner retina (130) or the outer retina (140) corresponding to the position of the edema area.

5. The OCT image segmentation method according to any one of claims 1 to 4, characterized in that: The method further comprises: pixel-level labeling of the high reflection point (150) comprises: Setting a pixel brightness threshold, and preliminarily marking pixels exceeding the pixel brightness threshold as high reflection point candidate areas; Perform connected domain analysis on candidate regions and remove regions with an area smaller than a preset threshold to reduce the impact of noise; The remaining area is subjected to morphological processing to optimize the shape and size of the high reflection point (150).

6. The OCT image segmentation method according to any one of claims 1 to 4, characterized in that: The artificial intelligence model is a U-Net deep learning network structure, and the U-Net deep learning network structure includes: An attention mechanism is added to the encoder part of the U-Net deep learning network structure to improve sensitivity to small-sized high-reflection points (150); Multi-scale feature fusion is used in the decoder part to better capture the high reflection points (150) of different sizes.

7. A method for scoring an OCT image, characterized in that: The method comprises: Inputting the OCT image (100) to be evaluated into the trained automatic recognition model (210); The automatic recognition model (210) automatically determines the background image (110), the retinal nerve fiber layer (120), the inner retina (130), the outer retina (140), and the high reflection point (150) of the input OCT image (100), wherein local areas in the inner retina (130) and the outer retina (140) are annotated at different pixel levels; In the process of automatically determining by using the automatic recognition model (210), the fovea centralis is identified according to the following steps: For the inner retina (130) automatically segmented by the automatic recognition model (210), the inner retina (130) region is processed into a binary image, and the inner retina (130) is processed by an image processing algorithm related to graphic morphology to determine the narrowest position of the inner retina (130); identifying the determined narrowest position as the fovea of ​​the OCT image (100); Taking the fovea centralis as the center point and using a preset diameter thereof, the positions of the inner retina (130) and the outer retina (140) and the number of the high reflection points (150) are determined; Determining inflammation scores of the inner retina (130) and the outer retina (140) of the input OCT image (100) according to a preset high reflection point number threshold; Calculate the thickness changes of each retinal layer; A composite score was generated using a machine learning algorithm combining the number of hyperreflective points (150) and retinal thickness changes.

8. The OCT image scoring method according to claim 7, characterized in that: The statistical method of the high reflection points (150) comprises: Binarizing the high reflection point probability map output by the automatic recognition model (210); Using a connected domain labeling algorithm, adjacent high-reflection point pixels are classified as one high-reflection point (150); The number of the high reflection points (150) in the inner retina (130) and the outer retina (140) are counted respectively.

9. The OCT image scoring method according to claim 8, characterized in that: The method further comprises: Calculating morphological features of the high reflection point (150), wherein the morphological features include area, perimeter and circularity; Based on the number of the high-reflection points (150) and the morphological characteristics, the severity of inflammation is more finely graded.

10. An OCT image processing device, comprising a processor and a display terminal (300), wherein the processor (200) is communicatively connected to the display terminal (300), characterized in that: The processor (200) is provided with an automatic recognition model (210); Inputting a plurality of OCT images (100) into the automatic recognition model (210), the automatic recognition model (210) automatically determining a background image (110), a retinal nerve fiber layer (120), an inner retina (130), an outer retina (140), and a high reflection point (150), wherein local areas in the inner retina (130) and the outer retina (140) are annotated at different pixel levels; In the process of automatically determining by using the automatic recognition model (210), the fovea centralis is identified according to the following steps: For the inner retina (130) automatically segmented by the automatic recognition model (210), the inner retina (130) region is processed into a binary image, and the inner retina (130) is processed by an image processing algorithm related to graphic morphology to determine the narrowest position of the inner retina (130); identifying the determined narrowest position as the fovea of ​​the OCT image (100); The positions of the inner retina (130) and the outer retina (140) and the number of the high reflection points (150) are determined by taking the fovea centralis as the center point and using the preset diameter thereof.

Images