Method and apparatus for identifying thyroid-associated ophthalmopathy based on eye CT images

Abstract

The present invention provides a method and apparatus for identifying thyroid-related ophthalmopathy based on eye CT images. The method includes: obtaining a plurality of slice images of an eye image; screening the plurality of slice images to obtain target CT slice images, where the target CT slice images are CT slice images that can be used to judge thyroid-related ophthalmopathy; segmenting each of the target CT slice images to extract contour location information of the eyeball and the orbit in each of the target CT slice images; and obtaining an identification result of thyroid-related ophthalmopathy based on the contour location information of the eyeball and the orbit. In the embodiments of the present invention, by automatically screening and identifying a plurality of slice images, not only the number of annotations is reduced, the development cost is saved, but also the diagnosis efficiency of thyroid-related ophthalmopathy is improved.

Description

Technical Field

[0001] The present invention relates to image processing technology, and in particular to a method, device, electronic device, computer-readable storage medium, and computer program product for identifying thyroid-related eye diseases based on eye CT images. Background Art

[0002] Thyroid-associated ophthalmopathy is one of the common orbital diseases in adults and is generally considered an organ-specific autoimmune disease related to thyroid diseases. Exophthalmos is one of the symptoms of thyroid-associated ophthalmopathy (TAO), usually accompanied by eyelid retraction, eyelid swelling, conjunctival hyperemia, etc., which not only affects aesthetics but also affects the user's vision, thus reducing the user's quality of life.

[0003] In related technologies, the examinations for thyroid exophthalmos include: routine eye examinations, CT scans, MRI scans, etc. Among them, for the CT images obtained through CT scans, doctors need to manually label and analyze each CT slice in the CT images to determine whether the extraocular muscles of TAO patients are hypertrophied and whether the lesions involve the muscle bellies. However, for different patients, the number of layers of CT images is also different. For CT images with relatively thick layers, it takes a lot of time and energy for doctors to label and diagnose, and there may occasionally be misdiagnosis and missed diagnosis, which not only increases the labeling cost but also leads to the problem of reduced diagnostic efficiency.

[0004] Therefore, how to improve the diagnostic efficiency of thyroid-related eye diseases in CT images is a technical problem to be solved currently. Summary of the Invention

[0005] The present invention provides a method, device, electronic device, computer-readable storage medium, and computer program product for identifying thyroid-related eye diseases based on eye CT images, so as to at least solve the technical problems in related technologies that it takes a lot of time to label and diagnose whether CT images are thyroid-related eye diseases, resulting in high diagnostic costs and low efficiency of CT images. The technical solutions of the present invention are as follows:

[0006] According to the first aspect of the embodiments of the present invention, a method for identifying thyroid-related eye diseases based on eye CT images is provided, including:

[0007] Obtain multiple slice images of an eye image;

[0008] Screen the multiple slice images to obtain a target CT slice image, where the target CT slice image is a CT slice image that can be used to judge thyroid-related eye diseases;

[0009] Segment each of the target CT slice images, and extract the contour location information of the eyeballs and orbits in each of the target CT slice images;

[0010] Obtain the recognition result of thyroid-related ophthalmopathy based on the contour location information of the eyeballs and orbits.

[0011] Optionally, the screening of the multiple slice images to obtain the target CT slice images includes:

[0012] Screen the multiple slice images through a screening model to obtain the target CT slice images, where the screening model includes:

[0013] A four-classification model for identifying whether the slice image is a CT slice image and the shooting position of the CT slice image; and

[0014] A two-classification model for judging whether the CT slice is a target CT slice image based on the shooting position of the CT slice image.

[0015] Optionally, the shooting positions of the CT slice images include: horizontal position, coronal position, and sagittal position.

[0016] Optionally, the two-classification model is further used to judge whether the CT slice is a target CT slice image based on the shooting position of the CT slice image and the corresponding eye tissues.

[0017] Optionally, the segmenting each of the target CT slice images and extracting the contour location information of the eyeballs and orbits in each of the target CT slice images includes: segmenting each of the target CT slice images through a segmentation model and extracting the contour location information of the eyeballs and orbits in each of the target CT slice images, specifically including:

[0018] Detect the circular area in the target CT slice image through a shape detection algorithm to obtain the area where the eyeball is located;

[0019] Analyze the target CT slice image through a binary segmentation algorithm and the highlight features of the orbital area to obtain the skull area in the target CT slice image, and analyze the position and area of the skull area to determine the areas where the left and right orbits are located; and

[0020] Extract the contour location information of the eyeballs and orbits in each of the target CT slice images based on the area where the eyeball is located and the areas where the left and right orbits are located.

[0021] Optionally, the obtaining the recognition result of thyroid-related ophthalmopathy based on the contour location information of the eyeballs and orbits includes:

[0022] Obtain the relative position information of the orbit and the eyeball in the target CT slice image based on the contour positioning information of the eyeball and the orbit;

[0023] Based on the relative position information of the orbit and the eyeball, obtain the recognition result of the thyroid-related ophthalmopathy.

[0024] Optionally, the relative position information of the orbit and the eyeball is the area ratio of the part of the eyeball protruding from the orbit. The obtaining of the relative position information of the orbit and the eyeball in the target CT slice image based on the contour positioning information of the eyeball and the orbit includes:

[0025] Perform image segmentation on the contour positioning information of the orbit and the eyeball, and output the segmented region pictures of the orbit and the eyeball;

[0026] Use the contours of the eyeball and the orbit in the segmented region pictures to determine the direction of the target CT slice image;

[0027] Based on the determined direction of the target CT slice image, connect the highest points of the contours of the left and right orbits in the segmented region picture as a reference line; and

[0028] Obtain the area ratio of the part of the eyeball protruding from the orbit according to the reference line.

[0029] According to the second aspect of the embodiments of the present invention, there is provided a device for identifying thyroid-related ophthalmopathy based on CT images, including:

[0030] An acquisition module, configured to acquire a plurality of slice images of an eye image;

[0031] A screening module, configured to screen the plurality of slice images to obtain a target CT slice image, where the target CT slice image is a CT slice image that can be used to judge thyroid-related eye diseases;

[0032] A segmentation module, configured to segment each target CT slice image and extract the contour positioning information of the eyeball and the orbit in each target CT slice image;

[0033] An identification module, configured to obtain the identification result of thyroid-related ophthalmopathy based on the contour positioning information of the eyeball and the orbit.

[0034] Optionally, the screening module is specifically configured to screen the plurality of slice images through a screening model to obtain a target CT slice image, where the screening model includes:

[0035] A four-classification model, configured to identify whether the slice image is a CT slice image and the shooting position of the CT slice image; and

[0036] A binary classification model for determining whether a CT slice is a target CT slice image based on the shooting position of the CT slice image.

[0037] Optionally, the shooting positions of the CT slice images include: horizontal position, coronal position, and sagittal position.

[0038] Optionally, the binary classification model is further used to determine whether a CT slice is a target CT slice image based on the shooting position of the CT slice image and the corresponding eye tissue.

[0039] Optionally, the segmentation module is specifically configured to segment each target CT slice image through a segmentation model, and extract the contour localization information of the eyeball and orbit in each target CT slice image.

[0040] Optionally, the segmentation module includes:

[0041] A detection module for detecting the circular region in the target CT slice image through a shape detection algorithm to obtain the region where the eyeball is located;

[0042] An analysis module for analyzing the target CT slice image through a binary segmentation algorithm and the highlight feature of the orbital region to obtain the skull region in the target CT slice image;

[0043] A region determination module for analyzing the position and area of the skull region to determine the regions where the left and right orbits are located; and

[0044] An extraction module for extracting the contour localization information of the eyeball and orbit in each target CT slice image based on the region where the eyeball is located and the regions where the left and right orbits are located.

[0045] Optionally, the recognition module includes:

[0046] A position acquisition module for obtaining the relative position information of the orbit and eyeball in the target CT slice image based on the contour localization information of the eyeball and orbit;

[0047] An eye disease recognition module for obtaining the recognition result of the thyroid-related eye disease based on the relative position information of the orbit and eyeball.

[0048] Optionally, the position acquisition module includes:

[0049] An image segmentation module for performing image segmentation on the contour localization information of the orbit and eyeball when the relative position information of the orbit and eyeball is the area ratio of the part where the eyeball protrudes from the orbit, and outputting the segmented region pictures of the orbit and eyeball;

[0050] A direction determination module, configured to determine the direction of the target CT slice image by using the contours of the eyeballs and eye sockets in the segmented region image;

[0051] A connection module, configured to connect the line connecting the highest points of the contours of the left and right eye sockets on the segmented region image as a reference line based on the determined direction of the target CT slice image; and an area ratio acquisition module, configured to obtain the area ratio of the part where the eyeballs protrude from the eye sockets according to the reference line.

[0052] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to execute any one of the above-mentioned thyroid-related eye disease recognition methods based on ocular CT images.

[0053] According to a fourth aspect of an embodiment of the present invention, there is provided a computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is caused to execute any one of the above-mentioned thyroid-related eye disease recognition methods based on ocular CT images.

[0054] According to a fifth aspect of an embodiment of the present invention, there is provided a computer program product, including a computer program or instructions, when the computer program or instructions are executed by a processor, the above-mentioned thyroid-related eye disease recognition method based on ocular CT images is implemented.

[0055] The technical solutions provided by the embodiments of the present invention may at least include the following beneficial effects:

[0056] In the embodiments of the present invention, multiple slice images of an ocular image are obtained; the multiple slice images are screened to obtain a target CT slice image, and the target CT slice image is a CT slice image that can be used to judge thyroid-related eye diseases; each target CT slice image is segmented to extract the contour location information of the eyeballs and eye sockets in each target CT slice image; and a recognition result of thyroid-related eye diseases is obtained based on the contour location information of the eyeballs and eye sockets. That is to say, in the embodiments of the present invention, by screening multiple slice images and extracting and recognizing the contour location information of the eyeballs and eye sockets in the screened target CT slice images, a recognition result of thyroid-related eye diseases is obtained. That is, in this embodiment, by automatically screening and recognizing multiple slice images, not only the number of annotations is reduced, the development cost is saved, but also the diagnosis efficiency of thyroid-related eye diseases is improved, the diagnosis process is clear, and the intermediate results are controllable.

[0057] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings here are incorporated into the description and form a part of this description, showing embodiments consistent with the present invention, and are used together with the description to explain the principles of the present invention, and do not constitute an undue limitation of the present invention.

[0059] Figure 1 It is a flowchart of a method for identifying thyroid-related eye diseases based on eye CT images provided by an embodiment of the present invention.

[0060] Figure 2 It is an example diagram of a horizontal CT slice that can be used for diagnostic analysis provided by an embodiment of the present invention.

[0061] Figure 3 It is a flowchart of modeling and annotation for identifying thyroid-related eye diseases based on eye CT images provided by an embodiment of the present invention.

[0062] Figure 4 It is an application example diagram of a method for identifying thyroid-related eye diseases based on eye CT images provided by an embodiment of the present invention.

[0063] Figure 5 It is a block diagram of a device for identifying thyroid-related eye diseases based on eye CT images provided by an embodiment of the present invention.

[0064] Figure 6 It is a block diagram of a segmentation module provided by an embodiment of the present invention.

[0065] Figure 7 It is a block diagram of an identification module provided by an embodiment of the present invention.

[0066] Figure 8 It is a block diagram of an acquisition module provided by an embodiment of the present invention.

[0067] Figure 9 It is a block diagram of an eye disease identification module provided by an embodiment of the present invention.

[0068] Figure 10 It is a structural block diagram of a device for identifying thyroid-related eye diseases based on eye CT images provided by an embodiment of the present invention. Detailed implementation manners

[0069] In order to enable those of ordinary skill in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0070] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0071] Technical term introduction:

[0072] CT scans a certain thickness of a layer of the human body with an X-ray beam. The X-rays passing through this layer are received by a detector and converted into visible light, then into electrical signals by photoelectric conversion, and then converted into digital signals by an analog / digital converter and input into a computer for processing.

[0073] Figure 1 is a flowchart of a method for identifying thyroid-related eye diseases based on eye CT images shown according to an exemplary embodiment. As Figure 1 shown, the method for identifying thyroid-related eye diseases based on eye CT images is used in a terminal or a server, and includes the following steps:

[0074] Step 101, obtain multiple slice images of the eye image;

[0075] In this step, the terminal or the server first obtains an eye image of the user (such as an eye CT image, etc.); then, the eye image is sliced to obtain multiple slice images of the eye image. Among them, each eye image may include multiple pictures of different sizes. By slicing the eye image, multiple slice images can be obtained. That is to say, the eye image is composed of multiple slice images of different sizes spliced together. Among them, the eye image can be an image taken frontally or an image with little distortion.

[0076] It should be noted that in this embodiment, the slicing of the eye image can be performed by manual slicing or by using a slicing model. Of course, other slicing methods can also be used, and this embodiment does not make any restrictions.

[0077] Among them, in this step, to obtain the eye image of the user, it can be the eye image obtained from the imaging system or the eye image taken from the doctor's mobile phone, and this embodiment does not make any restrictions.

[0078] In this embodiment, after obtaining the eye image, it needs to be segmented. The segmentation process can segment the eye image through a semi-automatic method with human participation. Of course, a trained inner slice recognition model can also be used for segmentation.

[0079] Specifically, the specific segmentation methods include:

[0080] One method is: using a trained inner slice recognition model to recognize the eye image to obtain a recognition result including the number of rows and columns; according to the recognition result, the eye image is divided into blocks to obtain multiple sliced images of the eye image; that is to say, this method is to use the constructed inner slice recognition model of the picture for automatic recognition, and the system segments the picture according to the recognition result.

[0081] Among them, the inner slice recognition model is an automatic recognition model, which can specifically be a multi-label classification model. When training this multi-label classification model, it is input with multiple sliced images, and the number of rows and columns of the target CT sliced image contained therein is predicted. The multi-label classification model can select a commonly used convolutional neural network, followed by a fully connected neural network with an output layer of 2 nodes (representing the number of rows and columns respectively) and including several layers.

[0082] During training, in order to provide richer training data and improve the model performance. In this embodiment, in addition to using the labeled real CT sliced images, CT images randomly spliced from the completed CT sliced images are also used as input. Through this method, the number of samples can be expanded, thereby improving the accuracy of the model.

[0083] Another method is: when detecting an operation instruction for the user to input the number of rows and columns for segmenting the sliced image, segment the eye image according to the instruction operation to obtain multiple sliced images of the eye image. That is to say, in this method, the operator manually inputs the number of rows and columns of the sliced images contained in the eye image, and the program automatically segments according to this information.

[0084] That is to say, the segmentation method of the eye image in this embodiment is not limited to the above two methods in practical applications, and other segmentation methods are also possible.

[0085] Step 102: Screen the multiple sliced images to obtain a target CT sliced image, where the target CT sliced image is a CT sliced image that can be used to judge thyroid-related ophthalmopathy.

[0086] In this step, the terminal or the server inputs the multiple sliced images into a trained screening model, and the screening model screens the multiple sliced images to obtain a target CT sliced image, where the target CT sliced image is a CT sliced image that can be used to judge thyroid-related ophthalmopathy.

[0087] Among them, the screening model includes:

[0088] A four-classification model for identifying whether the slice image is a CT slice image and the shooting position of the CT slice image; and

[0089] A two-classification model for judging whether the CT slice is a target CT slice image based on the shooting position of the CT slice image.

[0090] Among them, the screening model in this embodiment is a pre-constructed screening model, which is specifically constructed through two levels:

[0091] First, construct a four-classification model to distinguish which category the CT slice image belongs to among non-CT slice images, horizontal CT slice images, coronal CT slice images, and sagittal CT slice images. Then, for the shooting position of each CT slice image, construct a two-classification model for whether it can be used for diagnosis, that is, the output is a binary label for whether it can be used to judge thyroid ophthalmopathy. Thus, it is realized to judge whether the input CT slice image is suitable for judging whether there is thyroid ophthalmopathy in the application scenario.

[0092] Step 103: Segment each of the target CT slice images, and extract the contour location information of the eyeballs and orbits in each of the target CT slice images.

[0093] In this step, the terminal or server inputs the target CT slice image into the trained segmentation model, and outputs the contour location information of the eyeballs and orbits in each of the target CT slice images; among them, the segmentation model is a pre-trained model, which is specifically trained in the following manner:

[0094] First, identify the regions where the eyeballs and orbits are located in each of the target CT slice images; then, adjust (or optimize) the annotations of the identified regions where the eyeballs and orbits are located to obtain the target CT slice images with the annotated contours required for judging thyroid-related eye diseases. Using the annotated contour regions as the learning target of the segmentation model, input the target CT slice images into the segmentation model for training to obtain the trained segmentation model, and the output result of the segmentation model is used to judge thyroid-related eye diseases.

[0095] Among them, a method for identifying the regions where the eyeballs and orbits are located in each of the target CT slice images specifically includes:

[0096] Detect the circular regions in the target CT slice image through a shape detection algorithm to obtain the regions where the eyeballs are located; and analyze the target CT slice image through a binary segmentation algorithm and the highlight features of the orbital region to obtain the skull region in the target CT slice image; analyze the position and area of the skull region to determine the regions where the left and right orbits are located.

[0097] That is to say, in this embodiment, if it is very time-consuming and costly to directly and finely label the segmentation area of the eyeball and the orbital area. In this embodiment, the shape detection algorithm is used to obtain the area where the eyeball is located, and the binary segmentation algorithm is used to determine the areas where the left and right orbits are located. Then, the doctor is asked to modify and fine-tune the recognition results. That is to say, after rough labeling, for the segmentation tasks that are not easy to solve through image processing and the errors in automatic processing, targeted fine labeling optimization is carried out to achieve the purpose of reducing the labeling workload, saving a large amount of manpower, reducing the labeling cost, and improving the labeling efficiency.

[0098] Among them, when the eyeball has obvious shape information, in this embodiment, the shape detection algorithm is combined with the Hough transform to detect the circle in the CT slice image, that is, the approximate area of the eyeball; then, the range of this area is appropriately enlarged using the morphological method to ensure that the eyeball target is located within the selected area; edge extraction and contour extraction are performed on the selected area to obtain the contour of the eyeball. Then, equidistant sampling is performed along the contour line of the eyeball, and a series of labeling points about the eyeball can be obtained. That is, the morphological algorithm is used to enlarge the recognized area of the eyeball to ensure that the area of the eyeball is located within the selected area, and edge extraction and labeling are performed on the selected area to obtain the contour of the eyeball in the target CT slice.

[0099] Similarly, the extraction of the orbit in this embodiment is similar. It can be found by observation that the orbital area has obvious high-brightness features. First, the skull area in the target CT slice image can be obtained through the binary segmentation method. The range of this area is appropriately enlarged using the morphological method to ensure that the orbital target is located within the selected area. Edge extraction and contour extraction are performed on the selected area to obtain the contour of the orbit, and the contour of the orbit is extracted, and the positions and areas of different orbital contours are analyzed. The areas where the left and right orbits are located can be obtained. According to a similar method, this embodiment can also obtain the labeling points about the left and right skull areas, which is specifically similar to the above process and will not be elaborated here.

[0100] Step 104, obtain the recognition result of thyroid-related ophthalmopathy based on the contour positioning information of the eyeball and the orbit.

[0101] Perform image recognition on the extracted contour positioning information to obtain the recognition result of thyroid-related ophthalmopathy.

[0102] In this step, one identification method is as follows: the terminal or the server performs image segmentation on the extracted contour location information, and outputs pictures of the segmented areas of the eye sockets and eyeballs; then, using the contours of the eyeballs and eye sockets in the segmented area pictures, the direction of the target CT slice is determined; then, the line connecting the highest points of the contours of the left and right eye sockets in the segmented area pictures is used as a reference line, and the area ratio of the part where the eyeball protrudes from the eye socket is obtained according to the reference line; finally, the identification result for thyroid-related eye diseases is determined according to the area ratio.

[0103] Another identification method is: the terminal or the server performs image segmentation on the extracted contour location information, outputs pictures of the segmented areas of the eye sockets and eyeballs, inputs the pictures of the segmented areas of the eye sockets and eyeballs into a classification model for identification, and obtains the identification result for thyroid-related eye diseases.

[0104] Among them, in this embodiment, based on the structural extraction of image processing, the labeled contours required for judging thyroid eye protrusion can be obtained. On the basis of the extracted labeled contours, through manual fine labeling optimization, manual modification of the processing deviation of the automatic labeling method based on image processing is realized through manual verification to improve the accuracy of labeling. At the same time, compared with general binary labels, the segmentation labeling method in this example contains more information available for diagnosis.

[0105] This model is a pre-established multi-label segmentation model. The structure of this segmentation model can be selected as a deep learning segmentation model, etc., but it is not limited to this. In this embodiment, if the CT slice image is input into this deep learning segmentation model, the output is a segmentation image with three labels: background, eye socket, and eyeball, and the discrimination of thyroid eye protrusion is based on this. However, in the related art, pictures are directly input, and the output is a label indicating whether the patient is ill. Therefore, the basis for judgment in this embodiment is more intuitive and can be visually displayed. The process is closer to the method of manual judgment by doctors. That is to say, the interpretability of the deep learning model in medical scenarios is increased in this embodiment.

[0106] After that, this embodiment can combine the positions of the areas of the eyeballs and eye sockets extracted to judge the direction of the slice. If it is not the direction of the horizontal plane CT slice example as Figure 2 shown, the direction of the CT slice can be adjusted by rotation.

[0107] Then, in this embodiment, the line connecting the highest points of the left and right eye socket contours in the image is used as the required reference line; according to the reference line, the area ratio of the part where the eyeball protrudes from the eye socket can be easily obtained, and the identification result for thyroid-related eye diseases is determined according to the area ratio.

[0108] That is to say, the relative position information of the orbit and the eyeball is the area ratio of the part of the eyeball protruding from the orbit, and the recognition result for thyroid-related ophthalmopathy can be determined according to the area ratio.

[0109] According to the positions of the eyeball and the orbit, the direction of the CT slice can be judged. Then, the line connecting the highest points of the left and right orbital contours in the image is used as the required reference line, and the area ratio of the part of the eyeball protruding from the orbit can be obtained according to this reference line.

[0110] Of course, in this embodiment, a classification model can also be constructed according to the contour location information of the eyeball and the orbit in each of the target CT slice images output by the segmentation model to judge the recognition result of thyroid-related ophthalmopathy.

[0111] That is to say, the recognition result can be used both qualitatively for the diagnosis of thyroid ophthalmopathy and quantitatively for the analysis of the severity of thyroid ophthalmopathy. Specifically, as Figure 2 shown, Figure 2 FIG. is an example diagram of a horizontal CT slice image that can be used for diagnostic analysis provided by an embodiment of the present invention.

[0112] As Figure 2 shown, in the embodiment of the present invention, a horizontal CT slice image is selected as a schematic diagram of the research object. The Figure 2 has the lens visible and is suitable for analyzing the degree of eyeball protrusion from the orbit. And the degree of eyeball protrusion can be judged as shown in Figure 2 One way of judgment is: the line connecting the frontmost ends of the orbit intersects the eyeball, dividing the eyeball into two parts. By calculating the area of the part exceeding the orbit in the eyeball, the degree of eyeball protrusion can be quantitatively diagnosed, but in practical applications, it is not limited to this.

[0113] In an embodiment of the present invention, multiple slice images of an eye are obtained; the multiple slice images are screened to obtain a target CT slice image, where the target CT slice image is a CT slice image that can be used to judge thyroid-related ophthalmopathy; each of the target CT slice images is segmented to extract the contour location information of the eyeball and the orbit in each of the target CT slice images; and an identification result of thyroid-related eye diseases is obtained based on the contour location information of the eyeball and the orbit. That is to say, in the embodiment of the present invention, by screening multiple slice images, and extracting and identifying the contour location information of the eyeball and the orbit in the screened target CT slice images, an identification result of thyroid-related eye diseases is obtained. That is, in this embodiment, by automatically screening and identifying multiple slice images, not only the number of annotations is reduced, the development cost is saved, but also the diagnosis efficiency of thyroid-related ophthalmopathy is improved, making the diagnosis process clear and the intermediate results controllable. That is to say, in the embodiment of the present invention, by screening multiple slice images, and extracting and identifying the contour location information of the eyeball and the orbit in the screened target CT slice images, not only the number of annotations is reduced and the development cost is saved, but also its diagnosis process is clear, the intermediate results are visible and controllable, the model has intuitive interpretability, which is beneficial to verification and promotion in clinical practice.

[0114] Please also refer to Figure 3 , which is a flowchart of modeling and annotation for identifying thyroid-related eye diseases based on eye CT images provided by an embodiment of the present invention. As Figure 3 shown, it includes a modeling process and an annotation process, specifically including:

[0115] Step 301: Obtain a data set, where the data set includes multiple slice images of the eye;

[0116] In this step, the data set may include multiple slice images of different sizes. Of course, it may also be a picture formed by splicing and combining multiple slice images. To enrich the usage scenarios, the data set in this embodiment can be pictures imported from an imaging system, or pictures taken and uploaded by a doctor's mobile phone. However, it needs to be segmented. The pictures can be segmented by a semi-automatic method with human participation, or segmented by a segmentation model.

[0117] Step 302: Coarsely annotate each slice image in the obtained data set to obtain a coarsely annotated slice image. Among them, the coarsely annotated slice image includes: the shooting position of each slice image, and whether each slice image can be used for diagnosis, etc.

[0118] Among them, in this embodiment, the coarse annotation is a classification annotation, that is, a label is assigned to each slice image. The coarse annotation is relatively simple and fast, and can efficiently screen out the required slice images with classification labels, avoiding the burden of full-scale annotation.

[0119] That is to say, in this embodiment, rough annotation means classifying the data features of each slice image, and according to the classification results, labeling the corresponding labels on each slice image. After that, after rough annotation, slice images with classification labels are obtained. Then, in combination with the data situation, the data features of the slice images with classification labels are analyzed, and using a structure extraction method based on image processing, the contour annotation extraction of the data features of the CT slice images with classification labels is carried out to obtain the annotation contours required for judging thyroid ophthalmopathy, that is, the relevant regions are pre-identified and then handed over to the doctor for modification and fine-tuning, saving a large amount of labor costs and improving the annotation efficiency. On this basis, in this embodiment, for segmentation tasks that are not easily solved by image processing and errors in automatic processing, targeted fine annotation optimization is carried out (specific details are described in the following process and will not be elaborated here) to achieve the purpose of reducing the annotation workload.

[0120] Among them, the labels for the rough annotation provided in this embodiment are taken as the following seven categories as an example, but in actual applications, it is not limited to this. Specifically, they include:

[0121] Non-CT images, horizontal CT images that can be used for diagnosis, horizontal CT images that cannot be used for diagnosis, coronal CT images that can be used for diagnosis, coronal CT images that cannot be used for diagnosis, sagittal CT images that can be used for diagnosis, and sagittal CT images that cannot be used for diagnosis.

[0122] It should be noted that for each shooting position, the CT images that can be used for diagnosis can also be divided into multiple subtypes according to different diagnostic bases. However, for the convenience of description, this embodiment does not make distinctions here.

[0123] Step 303: Construct a screening model based on the obtained data set and the slice images after the rough annotation to realize the classification of the data set;

[0124] In this embodiment, through rough annotation, slice images with classification labels (i.e., picture samples) have been obtained. A data set can be constructed through these slice images with classification labels to construct a screening model for diagnostic photos.

[0125] Among them, this embodiment constructs a screening model in a hierarchical manner:

[0126] First, construct a four-class classification model to distinguish which category the picture belongs to among non-CT pictures, horizontal CT pictures, coronal CT pictures, and sagittal CT pictures.

[0127] That is to say, the terminal or the server identifies the slice images with classification labels through the established four-class classification model to obtain the classification to which each slice image belongs; for example: the classification to which the slice image belongs is a non-CT picture, a horizontal CT picture, a coronal CT picture, or a sagittal CT picture, etc.

[0128] Secondly, for each shooting position of the images, a binary classification model for diagnosing availability is constructed, that is, the output is a binary label indicating whether it can be used to judge thyroid ophthalmopathy. Thus, it is realized to judge whether the input CT slice image is suitable for judging whether there is thyroid ophthalmopathy in the application scenario. That is to say, for each shooting position of the images, the terminal passes the classified CT slice images through the established binary classification model, selects the CT slice images with classification labels used to judge thyroid-related eye diseases, and through the selected CT slice images with classification labels, it can be judged whether the input CT slice image is suitable for judging the disease of thyroid ophthalmopathy.

[0129] The screening model is used to classify the data set. On the one hand, it screens out the CT slice images that can be used for diagnosis. On the other hand, it can establish a classification model according to the classification of the data set.

[0130] Step 304: For the CT slice images that can be used for diagnosis screened by the screening model, based on the structure extraction of image processing, the extracted CT slice images are obtained;

[0131] Among them, structure extraction is to initially diagnose the labels such as the contour and positioning of the reference anatomical structure in the CT slice image, which are used to judge the marked contour required for thyroid ophthalmopathy.

[0132] In this embodiment, directly and finely annotating the segmentation area of the eyeball and the area of the eye socket is very time-consuming and the annotation cost is high. Based on this, this embodiment provides a structure extraction method based on image processing, which pre-identifies the relevant areas and then hands them over to the doctor for modification and fine-tuning, saving a large amount of labor costs and improving the annotation efficiency.

[0133] The eyeball has obvious shape information. Here, we use edge detection combined with the Hough transform to detect the circles in the CT slice image, and first obtain the approximate area of the eyeball. Appropriately expand this area range using morphological methods to ensure that the eyeball target is located within the selected area. Edge extraction and contour extraction are performed on the selected area, so that the contour of the eyeball can be obtained. Therefore, in this embodiment, equidistant sampling is performed along the contour line of the eyeball, and a series of annotation points about the eyeball can be obtained.

[0134] The extraction of the eye socket is similar. By observation, it can be found that the eye socket area has obvious high-brightness features. First, in this embodiment, the skull area in the CT slice image can be obtained through binary segmentation. When extracting its contour, analyze the positions and areas of different contours, and the areas where the left and right eye sockets are located can be obtained. According to a similar method, the annotation points about the left and right skull areas can also be obtained.

[0135] Step 305: Refine the annotation of the extracted CT slice images and determine whether the CT slice images are diseased labels;

[0136] In this step, based on the structure extraction of image processing, the annotation contours required for judging thyroid ophthalmopathy are obtained. However, since it is an unsupervised operation and has not been verified manually, if there are deviations, it will lead to a decline in the performance of the model. Therefore, in this embodiment, manual work is used to perform fine annotation optimization on the basis of the previously extracted contours. Through manual verification, the places with deviations in the automatic annotation method based on image processing are manually modified, thereby improving the accuracy of the annotation.

[0137] Step 306: Manually review and correct the CT slice images with diseased labels to obtain the corrected CT slice images;

[0138] The correction process is a process of slightly adjusting the labels such as the contours and positioning of the anatomical structure of the CT slice image.

[0139] Step 307: Input the classified data set and the corrected CT slice images into the constructed segmentation model for training to obtain information such as the target contours and positioning in the CT slice images.

[0140] In this embodiment, based on the classified data set and the corrected CT slice images, a multi-label segmentation model can be constructed, and the segmentation model structure can select a common deep learning segmentation model. The input is a CT slice image (or CT slice image, etc.), and the output is a segmentation image with three labels: background, orbit, and eyeball.

[0141] At this time, in this embodiment, in combination with the positions of the extracted eyeballs and orbits, the direction of the slice can be judged. If it is not in the horizontal direction as shown in Figure 2 it can be rotated to the state in the horizontal direction as shown in Figure 2 The line connecting the highest points of the left and right orbital contours in the image is the required reference line. Thus, in this embodiment, it is possible to easily obtain the area ratio of the part where the eyeball protrudes from the orbit, which can be used both qualitatively for the diagnosis of thyroid ophthalmopathy and quantitatively for analyzing the severity of thyroid ophthalmopathy.

[0142] Step 308: Construct a classification model for judging whether a disease is present based on the target contours and positioning and other information in the CT slice images.

[0143] The classification model constructed in this embodiment can directly distinguish the 7 categories to which the CT slice images belong, thereby improving the accuracy of the classification model for recognition and classification.

[0144] Furthermore, in the embodiments of the present invention, the marked contours are used as the learning targets, and a segmentation model is learned, which can output the segmentation regions of the eyeball and the eye socket. The marked contours all serve to obtain the segmentation model during the training phase. In the actual application process, the segmentation model can be directly used.

[0145] Furthermore, in the embodiments of the present invention, by combining image processing and deep learning, the workload of manual annotation can be effectively reduced, and the development cost can be lowered.

[0146] It should be noted that for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present disclosure is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the present invention.

[0147] Please also refer to Figure 4 , which is an application example diagram of a method for identifying thyroid-related eye diseases based on eye CT images provided by the embodiments of the present invention. The method includes:

[0148] Step 401: Obtain an eye picture.

[0149] In this step, the eye picture can be a 2D image instead of a 3D medical image. In specific embodiments, it has low requirements for the hardware devices of the terminal, which is beneficial to reducing the deployment cost.

[0150] Step 402: Perform slice preprocessing on the eye picture to obtain multiple slice images.

[0151] Among them, the specific slicing method in this step is as detailed above and will not be elaborated here.

[0152] Step 403: Input the multiple slice images into a screening model for screening, and screen out the target CT slice images for diagnosis. Among them, the target CT slice images are CT slice images that can be used to judge thyroid-related eye diseases;

[0153] In this step, for the slice images that cannot be used for diagnosis screened out, they are directly discarded.

[0154] Step 404: Input the screened target CT slice images into a segmentation model for contour localization extraction and segmentation, and extract the target contours and localization information of the eyeball and the eye socket in the target CT slice images.

[0155] Step 405: Input the target contour and positioning information of the eyeball and orbit in the target CT slice image into the classification model for disease judgment, so as to obtain the diagnosis result of whether the disease is present.

[0156] In the embodiment of the present invention, through automatic screening and recognition of multiple slice images, and by combining image processing and deep learning, not only the number of annotations is reduced, the development cost is lowered, but also the diagnosis efficiency of thyroid-related ophthalmopathy is improved. The method provided in this embodiment can quantitatively measure thyroid eye protrusion, which not only meets the need for dynamically monitoring the course of the disease, but also refines the medical requirements.

[0157] Figure 5 It is a block diagram of a device for identifying thyroid-related eye diseases based on ocular CT images shown according to an exemplary embodiment. Refer to Figure 5 and this device includes an acquisition module 501, a screening module 502, a segmentation module 503 and an identification module 504, wherein,

[0158] The acquisition module 501 is used to acquire multiple slice images of the ocular image;

[0159] The screening module 502 is used to screen the multiple slice images to obtain a target CT slice image, and the target CT slice image is a CT slice image that can be used to judge thyroid-related ophthalmopathy;

[0160] The segmentation module 503 is used to segment each target CT slice image and extract the contour positioning information of the eyeball and orbit in each target CT slice image;

[0161] The identification module 504 is used to obtain the identification result of thyroid-related eye diseases based on the contour positioning information of the eyeball and orbit.

[0162] Optionally, in another embodiment, on the basis of the above embodiment, the screening module 502 is specifically used to screen the multiple slice images through a screening model to obtain a target CT slice image, wherein the screening model includes:

[0163] A four-classification model for identifying whether the slice image is a CT slice image and the shooting position of the CT slice image; and

[0164] A two-classification model for judging whether the CT slice is a target CT slice image based on the shooting position of the CT slice image.

[0165] Optionally, in another embodiment, on the basis of the above embodiment, the shooting positions of the CT slice images include: horizontal position, coronal position and sagittal position.

[0166] Optionally, in another embodiment, based on the above embodiment, the binary classification model is further configured to determine whether the CT slice is a target CT slice image based on the shooting position of the CT slice image and the corresponding eye tissue.

[0167] Optionally, in another embodiment, based on the above embodiment, the segmentation module is specifically configured to segment each target CT slice image through a segmentation model, and extract the contour location information of the eyeball and the orbit in each target CT slice image.

[0168] Optionally, in another embodiment, based on the above embodiment, the segmentation module 503 includes: a detection module 601, an analysis module 602, a region determination module 603, and an extraction module 604. The structural schematic diagram is as Figure 6 shown, where

[0169] The detection module 601 is configured to detect the circular region in the target CT slice image through a shape detection algorithm to obtain the region where the eyeball is located;

[0170] The analysis module 602 is configured to analyze the target CT slice image through a binary segmentation algorithm and the highlight feature of the orbital region to obtain the skull region in the target CT slice image;

[0171] The region determination module 603 is configured to analyze the position and area of the skull region to determine the regions where the left and right orbits are located; and

[0172] The extraction module 604 is configured to extract the contour location information of the eyeball and the orbit in each target CT slice image based on the region where the eyeball is located and the regions where the left and right orbits are located.

[0173] Optionally, in another embodiment, based on the above embodiment, the recognition module 504 includes: a position acquisition module 701 and an eye disease recognition module 702. The structural schematic diagram is as Figure 7 shown, where

[0174] The position acquisition module 701 is configured to obtain the relative position information of the orbit and the eyeball in the target CT slice image based on the contour location information of the eyeball and the orbit;

[0175] The eye disease recognition module 702 is configured to obtain the recognition result of the thyroid-related eye disease based on the relative position information of the orbit and the eyeball.

[0176] Optionally, in another embodiment, based on the above embodiment, the obtaining module 501 includes: an image segmentation module 801, a direction determination module 802, a connection module 803, and an area ratio obtaining module 804. The schematic structural diagram is as shown in Figure 8 shown, where

[0177] The image segmentation module 801 is configured to perform image segmentation on the contour location information of the eye socket and the eyeball when the area ratio of the part where the eyeball protrudes from the eye socket is the relative position information of the eye socket and the eyeball, and output the segmented area pictures of the eye socket and the eyeball;

[0178] The direction determination module 802 is configured to determine the direction of the target CT slice image by using the contours of the eyeball and the eye socket in the segmented area picture;

[0179] The connection module 803 is configured to connect the line connecting the highest points of the contours of the left and right eye sockets on the segmented area picture as a reference line based on the determined direction of the target CT slice image;

[0180] The area ratio obtaining module 804 is configured to obtain the area ratio of the part where the eyeball protrudes from the eye socket according to the reference line.

[0181] Optionally, in another embodiment, based on the above embodiment, the eye disease recognition module 702 includes: a connection module 901, an area ratio obtaining module 902, and a result determination module 903. The schematic structural diagram is as shown in Figure 9 shown, where

[0182] The connection module 901 is configured to connect the line connecting the highest points of the contours of the left and right eye sockets on the segmented area picture as a reference line based on the determined direction of the target CT slice image;

[0183] The area ratio obtaining module 902 is configured to obtain the area ratio of the part where the eyeball protrudes from the eye socket according to the reference line; and

[0184] The result determination module 903 is configured to obtain the recognition result of thyroid-related eye diseases according to the area ratio.

[0185] Regarding the device in the above embodiment, the specific manners in which each module performs operations have been described in detail in the embodiment related to the method. For the relevant parts, refer to the partial description of the method embodiment, and no detailed description will be given here.

[0186] The embodiment of the present invention further provides an electronic device, including:

[0187] A processor;

[0188] A memory for storing the processor-executable instructions;

[0189] Wherein, the processor is configured to execute the instructions to implement the method for identifying thyroid-related ophthalmopathy based on eye CT images as described above.

[0190] An embodiment of the present invention also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the method for identifying thyroid-related ophthalmopathy based on eye CT images as described above.

[0191] An embodiment of the present invention also provides a computer program product, including a computer program or instructions. When the computer program or instructions are executed by a processor, the method for identifying thyroid-related ophthalmopathy based on eye CT images as described above is implemented.

[0192] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory including instructions. The above instructions can be executed by a processor of a device to complete the above method. Optionally, the storage medium can be a non-transitory computer-readable storage medium. For example, the non-transitory computer-readable storage medium can be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0193] Figure 10 is a block diagram of a device 1000 for identifying thyroid-related ophthalmopathy based on eye CT images shown according to an exemplary embodiment. For example, the device 1000 can be provided as a server. Referring to Figure 10 , the device 1000 includes a processing component 1022, which further includes one or more processors, and memory resources represented by a memory 1032 for storing instructions executable by the processing component 1022, such as application programs. The application programs stored in the memory 1032 can include one or more modules each corresponding to a set of instructions. In addition, the processing component 1022 is configured to execute instructions to perform the above method for identifying thyroid-related ophthalmopathy based on eye CT images.

[0194] The device 1000 may further include a power supply component 1026 configured to perform power management of the device 1000, a wired or wireless network interface 1050 configured to connect the device 1000 to a network, and an input / output (I / O) interface 1058. The device 1000 can operate based on an operating system stored in the memory 1032, such as Windows ServerTM, MacOS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0195] Other embodiments of the present invention will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include known common knowledge or conventional technical means in the technical field not disclosed herein. The specification and examples are only to be considered exemplary, and the true scope and spirit of the invention are pointed out by the following claims.

[0196] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for identifying thyroid-related eye diseases based on eye CT images, characterized in that, Including: Obtaining multiple slice images of an eye image; Filtering the multiple slice images through a filtering model to obtain a target CT slice image, where the target CT slice image is a CT slice image that can be used to judge thyroid-related ophthalmopathy; wherein, the filtering model includes: a four-classification model for identifying whether the slice image is a CT slice image and the shooting position of the CT slice image; and a two-classification model for judging whether the CT slice is a target CT slice image based on the shooting position of the CT slice image; Segmenting each of the target CT slice images to extract the contour localization information of the eyeball and orbit in each of the target CT slice images; Obtaining an identification result of thyroid-related eye diseases based on the contour localization information of the eyeball and orbit, including: Performing image segmentation on the contour localization information of the orbit and eyeball, and outputting a segmented region image of the orbit and eyeball; using the contours where the eyeball and orbit are located in the segmented region image to determine the direction of the target CT slice image; based on the determined direction of the target CT slice image, connecting the highest points of the contours of the left and right orbits on the segmented region image as a reference line; and obtaining the area ratio of the part where the eyeball protrudes from the orbit according to the reference line, and determining the identification result for thyroid-related eye diseases according to the area ratio.

2. The method for identifying thyroid-related eye diseases based on eye CT images according to claim 1, characterized in that, The shooting positions of the CT slice images include: horizontal position, coronal position, and sagittal position.

3. The identification method according to claim 1, characterized in that, The two-classification model is further used to judge whether the CT slice is a target CT slice image based on the shooting position of the CT slice image and the corresponding eye tissues.

4. The method for identifying thyroid-related eye diseases based on eye CT images according to claim 1, characterized in that, The segmenting each of the target CT slice images to extract the contour localization information of the eyeball and orbit in each of the target CT slice images includes: segmenting each of the target CT slice images through a segmentation model to extract the contour localization information of the eyeball and orbit in each of the target CT slice images, specifically including: Detecting a circular region in the target CT slice image through a shape detection algorithm to obtain the region where the eyeball is located; Analyzing the target CT slice image through a binary segmentation algorithm and the highlight feature of the orbit region to obtain the skull region in the target CT slice image, and analyzing the position and area of the skull region to determine the regions where the left and right orbits are located; and Extracting the contour localization information of the eyeball and orbit in each of the target CT slice images based on the region where the eyeball is located and the regions where the left and right orbits are located.

5. A device for identifying thyroid-related eye diseases based on CT images, characterized in that, Including: An obtaining module for obtaining multiple slice images of an eye image; A filtering module for filtering the multiple slice images through a filtering model to obtain a target CT slice image, where the target CT slice image is a CT slice image that can be used to judge thyroid-related ophthalmopathy; wherein, the filtering model includes: a four-classification model for identifying whether the slice image is a CT slice image and the shooting position of the CT slice image; and a two-classification model for judging whether the CT slice is a target CT slice image based on the shooting position of the CT slice image; A segmentation module, configured to segment each of the target CT slice images and extract the contour location information of the eyeballs and orbits in each of the target CT slice images; An identification module, configured to obtain an identification result of thyroid-related ophthalmopathy based on the contour location information of the eyeballs and orbits; Wherein, the identification module includes: An image segmentation module, configured to perform image segmentation on the contour location information of the orbits and eyeballs and output segmented region pictures of the orbits and eyeballs; A direction determination module, configured to determine the direction of the target CT slice image by using the contours of the eyeballs and orbits in the segmented region pictures; A connection module, configured to connect the highest points of the contours of the left and right orbits on the segmented region pictures as a reference line based on the determined direction of the target CT slice image; and An area ratio acquisition module, configured to obtain the area ratio of the part where the eyeballs protrude from the orbits according to the reference line; An ophthalmopathy identification module, configured to determine an identification result for thyroid-related ophthalmopathy based on the area ratio.

6. An electronic device, characterized in that, Including: A processor; A memory for storing executable instructions of the processor; Wherein, the processor is configured to execute the instructions to implement the method for identifying thyroid-related ophthalmopathy based on ocular CT images according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to execute the method for identifying thyroid-related ophthalmopathy based on ocular CT images according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, the method for identifying thyroid-related ophthalmopathy based on ocular CT images according to any one of claims 1 to 4 is implemented.

Images