Extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation method

Abstract

The invention relates to an extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation method, which comprises the following steps: S1, acquiring medical image data and an image containing coronal plane information of eyeballs and surrounding tissues thereof; s2, performing deep learning prediction on the obtained medical image data by using an ESNet model to obtain an extraocular muscle segmentation result; s3, obtaining an accurate extraocular muscle image according to the extraocular muscle segmentation result; s4, performing extraocular muscle three-dimensional reconstruction by using an MPR algorithm according to the obtained extraocular muscle image, and establishing an extraocular muscle three-dimensional model; and S5, calculating the volume of each extraocular muscle according to the extraocular muscle image obtained in the S3. According to the extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation method, deep learning prediction is carried out on the obtained medical image data through the ESNet model to obtain an accurate extraocular muscle image, three-dimensional reconstruction and volume calculation are carried out according to the extraocular muscle image, the extraocular muscle reconstructed through the method is accurate in reduction degree, and the method is suitable for large-scale popularization and application. Diagnosis of strabismus is assisted, and research of extraocular muscle volume related diseases is facilitated.

Description

Technical Field

[0001] The present application relates to the technical field related to extraocular muscle reconstruction, and in particular to a method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume. Background Art

[0002] In the related art, most patients with strabismus have dysplasia of the orbital apex segment of the extraocular muscles on one side, which is manifested as a reduction in the volume of the orbital apex segment of the eye muscles. If a program for 3D reconstruction of the extraocular muscles is designed to visually express the morphological characteristics of the extraocular muscles, calculate the volume of each eye muscle for intuitive comparison, and display it in segments, it will be helpful for the observation and diagnosis of strabismus.

[0003] It is difficult to directly reconstruct the extraocular muscles in three dimensions based on images with existing technology. If three-dimensional reconstruction is performed directly on medical images, the model obtained will be very poor and the focus will not be clear. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present application provides a method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, which accurately extracts the extraocular muscle part, and then performs three-dimensional reconstruction and volume calculation on it.

[0005] The present application provides a method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, comprising:

[0006] S1, acquiring medical image data, wherein the medical image data includes images of coronal plane information of an eyeball and surrounding tissues;

[0007] S2, using the ESNet model to perform deep learning prediction on the acquired medical imaging data to obtain the extraocular muscle segmentation results;

[0008] S3, obtaining an accurate extraocular muscle image according to the extraocular muscle segmentation result;

[0009] S4, based on the acquired extraocular muscle images, the MPR algorithm is used to perform three-dimensional reconstruction of the extraocular muscles and establish a three-dimensional model of the extraocular muscles;

[0010] S5, calculating the volume of the extraocular muscles according to the extraocular muscle image obtained in S3.

[0011] In the above-mentioned method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, in S1, the method for obtaining the medical image data is to connect the midpoint of the bony orbital opening to the orbital apex, where the connecting line is the central axis of the bony orbit, and make a plane cut perpendicular to the central axis, from the anterior edge of the anterior chamber of the eyeball to the orbital apex with uniform layer thickness and uniform intervals, to obtain a continuous image, and export the image in a fixed format as the medical image data.

[0012] In the above-mentioned method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, in S2, the architecture of the ESNet model includes an input layer, a YOLO detection module, and a semantic segmentation module, wherein the input layer is used to receive the obtained medical image data; the YOLO detection module performs eyeball detection on the input medical image data to determine the preliminary position of the extraocular muscles; the semantic segmentation module uses the semantic segmentation model to perform fine segmentation of the extraocular muscles within the bounding box of the preliminary position of the extraocular muscles provided by the YOLO detection module to obtain the extraocular muscle segmentation result.

[0013] In the above-mentioned method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation, the steps of the ESNet model performing deep learning prediction on the obtained medical image data are as follows:

[0014] S21, inputting the obtained medical imaging data;

[0015] S22, marking the eyeball part in the medical image data, marking the position and outline of the eyeball;

[0016] S23, based on the marked eyeball position and contour, use the YOLO target detection model to train the YOLO model;

[0017] S24, applying the trained YOLO model to the medical image data, automatically detecting and locating the position of the eyeball, and determining the preliminary position of the extraocular muscles; based on the preliminary position, cropping the eyeball portion on the medical image data, removing unnecessary background areas, and obtaining a cropped eyeball image;

[0018] S25, annotating the extraocular muscles on the obtained eyeball image, annotating the contours and position information of the extraocular muscles;

[0019] S26, based on the outline and position information of the annotated extraocular muscles, use a semantic segmentation model to train the semantic segmentation model; within the bounding box provided by the YOLO model, use the semantic segmentation model to perform fine segmentation of the extraocular muscles;

[0020] S27, use the loss function to simultaneously optimize the performance of YOLO target detection and semantic segmentation, determine the appropriate loss weight, and balance the training effects of the YOLO model and the semantic segmentation model.

[0021] In the above-mentioned method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, in S23, a YOLO model is trained, the medical image data is annotated using a data annotation tool, the eyeball part is framed using a box, and then model training is performed.

[0022] In the above-mentioned method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, in S23, the training process of the YOLO model includes initializing weights, inputting the medical imaging data for training for forward propagation to obtain prediction results, and calculating the loss between the prediction and the true annotation; calculating the gradient and updating the weights through back propagation, and adjusting the weights using an optimization algorithm; repeating this process for multiple cycles until the loss converges or reaches a preset number of iterations.

[0023] In the above-mentioned method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, in S2, in the process of prediction using the ESNet model, the input medical image data is first adjusted to a specified size, and a feature map is generated through forward propagation through the network; on the feature map, the ESNet model predicts multiple bounding boxes and their confidences for each grid unit, and calculates the object category probability of each box; the ESNet model filters out high-confidence bounding boxes according to the set confidence threshold, and removes boxes with high overlap through a non-maximum suppression algorithm, and finally outputs the category of each detected object and the corresponding bounding box coordinates.

[0024] In the above-mentioned method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, in S26, a semantic segmentation model is trained, semantic segmentation is realized automatically through a deep learning model, and medical image data sets are collected and labeled so that each pixel has a corresponding category label; a network architecture is selected, the semantic segmentation model extracts image features and restores them to the original size through upsampling, and outputs the category probability of each pixel; during the training process, a loss function is used to evaluate the difference between the output and the true label, and the model weight is updated; in the inference stage, the image to be segmented is input into the trained model, and the category of each pixel is output to realize automatic region segmentation.

[0025] In the above-mentioned method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, in S4, a large number of extraocular muscle images at different levels but on the same acquisition plane obtained after semantic segmentation are superimposed to obtain a three-dimensional model of the part, and an MPR algorithm is used to input an image of a folder and output a three-dimensional model. The three-dimensional model is then interactively visualized through three-dimensional visualization software to obtain a reconstructed three-dimensional model of the extraocular muscles.

[0026] In the above-mentioned method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, in S5, it is assumed that each extraocular muscle has n slices, and the length of the extraocular muscle is evenly divided into multiple segments, and the area of ​​each layer is A i (where i = 1, 2, ..., n), the slice spacing is d, and the volume of each segment Vi is expressed as:

[0027]

[0028] In the formula: i = 1, 2, 3, 4... n is the total number of layers of each extraocular muscle, Aj is the area of ​​the jth layer, and d is the fixed distance between each layer.

[0029] The technical solution provided in the present application may include the following beneficial effects: the method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, through deep learning prediction of the obtained medical imaging data by the ESNet model, obtains an accurate extraocular muscle image, and performs three-dimensional reconstruction and volume calculation based on the extraocular muscle image. The extraocular muscles reconstructed by this method are accurate in restoration, assist in the diagnosis of strabismus, and contribute to the study of diseases related to extraocular muscle volume.

[0030] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other objects, features and advantages of the present application will become more apparent through a more detailed description of exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0032] Figure 1 It is a flow chart of a method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume shown in an embodiment of the present application;

[0033] Figure 2 It is the original image of the medical image data of the method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume shown in the embodiment of the present application;

[0034] Figure 3 It is an image of the eyeball portion obtained by the method for three-dimensional reconstruction of the extraocular muscles and calculation of the extraocular muscle volume shown in the embodiment of the present application;

[0035] Figure 4 The extraocular muscle image obtained by the method for three-dimensional reconstruction of the extraocular muscle and calculation of the extraocular muscle volume shown in the embodiment of the present application;

[0036] Figure 5 It is a three-dimensionally reconstructed extraocular muscle model of the method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume shown in the embodiment of the present application;

[0037] Figure 6 It is a volume comparison diagram of the extraocular muscles after segmentation in the method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation shown in the embodiment of the present application;

[0038] Figure 7 It is a flowchart of the ESNet model prediction of the method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation shown in the embodiment of the present application. DETAILED DESCRIPTION

[0039] The preferred embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0040] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0041] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0042] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0043] Figure 1 It is a flow chart of the method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume shown in the embodiment of the present application.

[0044] See also Figure 1 , a method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation, comprising:

[0045] S1, acquiring medical image data, specifically, the medical image data includes an image of three-dimensional information of an eyeball and surrounding tissues;

[0046] S2, ESNet model performs deep learning prediction. Specifically, the ESNet (EyeSegNet) model is used to predict the acquired medical imaging data to obtain the extraocular muscle segmentation results. ESNet combines the real-time target detection capability of the YOLO model and the pixel-level classification advantage of semantic segmentation to optimize the extraction and 3D modeling process of the extraocular muscle part. This model not only improves the accuracy of detection, but also reduces the demand for computing resources.

[0047] S3, obtaining an accurate extraocular muscle image, specifically, obtaining an accurate extraocular muscle image according to the extraocular muscle segmentation result;

[0048] S4, performing three-dimensional reconstruction using an MPR algorithm. Specifically, based on the acquired extraocular muscle image, performing three-dimensional reconstruction of the extraocular muscle using an MPR algorithm to establish a three-dimensional model of the extraocular muscle;

[0049] S5, calculating the volume of the extraocular muscles. Specifically, calculating the volume of the extraocular muscles according to the extraocular muscle image obtained in S3.

[0050] Among them, in S1, the method for obtaining the medical image data is to connect the midpoint of the bony orbital opening to the orbital apex, where the connecting line is the central axis of the bony orbit, and make a plane cut perpendicular to the central axis, from the anterior edge of the anterior chamber of the eyeball to the orbital apex with uniform layer thickness and uniform intervals to obtain a continuous image, and export the image in a fixed format as the medical image data.

[0051] Preferably, in S2, the architecture of the ESNet model includes an input layer, a YOLO detection module, and a semantic segmentation module, wherein the input layer is used to receive the obtained medical image data. The YOLO detection module performs eyeball detection on the input medical image data to determine the preliminary position of the extraocular muscles. The semantic segmentation module uses a semantic segmentation model to perform fine segmentation of the extraocular muscles within the bounding box of the preliminary position of the extraocular muscles provided by the YOLO detection module to obtain the extraocular muscle segmentation result. Further, YOLO (You Only Look Once) is a real-time target detection system that predicts the target bounding box and category through a single neural network. Use data annotation tools (such as ImgLabel (Labelmg) etc.) to annotate the picture, and it is necessary to manually use a box to frame the eyeball part. The data volume is recommended to be around 20,000 to 50,000 (need to be determined according to the model performance, if the robustness is poor, more data is required), and then the model is trained.

[0052] Specifically, see Figure 2 ,Should Figure 2 The original image is used for target detection through the YOLO detection module to determine the initial position of the extraocular muscles and crop them. The cropped image is Figure 3 , and obtain the eyeball part. After obtaining the eyeball part, the semantic segmentation model is used to perform semantic segmentation on the eyeball part to obtain the extraocular muscle part. Figure 4 Finally, the three-dimensional reconstruction of the extraocular muscles is performed through multiple images to obtain a model, such as Figure 5 The calculation results of the extraocular muscle volume are displayed in segments: the extraocular muscle is evenly divided into 4 segments from front to back, marked as S1, S2, S3, and S4 respectively. The volume size of each segment can be compared intuitively, such as Figure 6 .

[0053] Alternatively, see Figure 7 The steps of the ESNet model performing deep learning prediction on the obtained medical imaging data are as follows:

[0054] S21, obtaining medical imaging data, specifically, inputting the obtained medical imaging data. Usually, these medical imaging data can come from imaging equipment such as CT and MRI, and contain three-dimensional information of the eyeball and its surrounding tissues. Definition of medical imaging data: A line from the midpoint of the bony orbital opening to the orbital apex is used as the central axis of the bony orbit. A plane cut is made perpendicular to the central axis, from the anterior edge of the anterior chamber of the eyeball to the orbital apex with a layer thickness of 1mm and an interval of 1mm to obtain a continuous image, which is exported in a bmp format, and the left and right orbits are cut separately.

[0055] S22, annotating the eyeball part, specifically, annotating the eyeball part in the medical image data, annotating the position and outline of the eyeball. Using an annotation tool (such as ImgLabel (Labelmg)), the position and outline of the eyeball are manually annotated on the original medical image data, providing basic data for subsequent eyeball detection.

[0056] S23, training the YOLO model separately, specifically, training the YOLO model using the YOLO target detection model based on the marked position and contour of the eyeball.

[0057] S24, obtaining cropped eyeball image data, specifically, applying the trained YOLO model to the medical image data, automatically detecting and locating the position of the eyeball, and determining the preliminary position of the extraocular muscles; according to the preliminary position, cropping the eyeball portion on the medical image data, removing unnecessary background areas, and obtaining a cropped eyeball image.

[0058] S25, annotating the extraocular muscles, specifically, annotating the extraocular muscles on the obtained eyeball image, annotating the contour and position information of the extraocular muscles. On the cropped eyeball image, Labelme is used to further annotate the contour and position information of the extraocular muscles. This provides training data for subsequent extraocular muscle segmentation.

[0059] S26, train the semantic segmentation model separately, specifically, based on the outline and position information of the annotated extraocular muscles, use the semantic segmentation model for training, train the semantic segmentation model; in the bounding box provided by the YOLO model, use the semantic segmentation model to perform fine segmentation of the extraocular muscles. Based on the extraocular muscle data annotated in the previous step, use the semantic segmentation model for training. Semantic segmentation can divide the entire image into different semantic areas, which are the extraocular muscles.

[0060] S27, joint training, specifically using the loss function to optimize the performance of YOLO object detection and semantic segmentation at the same time, determine the appropriate loss weight, and balance the training effect of the YOLO model and the semantic segmentation model. In this stage, the YOLO and semantic segmentation modules are optimized at the same time to achieve better collaborative working effects. It is necessary to design a suitable loss function to ensure that the two modules can be effectively integrated.

[0061] Specifically, in S23, the YOLO model is trained, the medical image data is annotated using a data annotation tool, the eyeball part is framed using a box, and then the model training is performed.

[0062] Preferably, in S23, the training process of the YOLO model includes initializing weights, inputting the medical imaging data for training for forward propagation to obtain prediction results, and calculating the loss between the prediction and the true annotation; calculating the gradient and updating the weights through back propagation, and adjusting the weights using an optimization algorithm; repeating this process for multiple cycles until the loss converges or a preset number of iterations is reached.

[0063] In S2, in the process of prediction using the ESNet model, the input medical image data is first adjusted to a specified size, and a feature map is generated through the network through forward propagation; on the feature map, the ESNet model predicts multiple bounding boxes and their confidences for each grid unit, and calculates the object category probability of each box; the ESNet model screens out high-confidence bounding boxes according to the set confidence threshold, and removes the boxes with high overlap through the non-maximum suppression algorithm, and finally outputs the category of each detected object and the corresponding bounding box coordinates. Specifically, the prediction process of ESNet first receives the eyeball image and inputs it. Then, the input image passes through the YOLO module to perform target detection, identify the eyeball position and generate a bounding box and its confidence. Next, the image outside the bounding box is cropped, and the corresponding area is extracted in the bounding box output by the YOLO module and input into the semantic segmentation module for pixel-level classification to generate a segmentation mask for the extraocular muscles. Afterwards, the detection results of the YOLO module and the segmentation results of the semantic segmentation module are post-processed to optimize the bounding box and mask, and finally the detection and segmentation results are comprehensively output, and features are extracted to prepare for subsequent three-dimensional reconstruction. Through this series of steps, ESNet can efficiently achieve accurate detection and segmentation of extraocular muscles.

[0064] In S26, a semantic segmentation model is trained. Semantic segmentation realizes automatic segmentation through a deep learning model, collects and annotates medical image data sets, and makes each pixel have a corresponding category label. A network architecture (such as a specialized network architecture such as DeepLab) is selected. The semantic segmentation model extracts image features and restores them to the original size through upsampling, and outputs the category probability of each pixel. During the training process, a loss function is used to evaluate the difference between the output and the true label, and the model weight is updated. In the inference stage, the image to be segmented is input into the trained model, and the category of each pixel is output to achieve automatic region segmentation. Specifically, semantic segmentation realizes pixel-level classification of images. It can divide an image into multiple blocks according to the similarities and differences of categories. A semantic region refers to a connected area in an image that is marked as a specific category, and each pixel is assigned a category label, such as a building, road, or pedestrian. It is defined by pixel-level labels to ensure that pixels of the same category form a continuous region, while regions between different categories are separated. This definition not only relies on visual features, but also takes into account contextual information to help the model better understand the image content.

[0065] Specifically, the eyeball medical image is input, and then the YOLO model is used to detect the object in the image, identify the position of the eyeball and crop it. Then, DeepLab is used to perform fine segmentation of the extraocular muscles on the YOLO cropped image, thus preparing for subsequent 3D reconstruction.

[0066] During the data preparation stage, annotation tools are used to annotate the eyeballs and extraocular muscles. The recommended data volume is between 40,000 and 100,000. Data enhancement is performed through rotation, flipping, scaling, etc. to improve the robustness of the model.

[0067] After training, the model output is optimized using post-processing technology to ensure that each eyeball has only one detection result, and the extraocular muscles are extracted using image processing technology. Finally, the 3D model of the extraocular muscles is generated using the 3D reconstruction algorithm using the extracted feature information, and optimized to ensure the accuracy and visualization of the model.

[0068] Optionally, in S4, a large number of extraocular muscle images at different levels but in the same acquisition plane obtained after semantic segmentation are superimposed to obtain a three-dimensional model of the part, and an MPR algorithm is used to input an image of a folder, output a three-dimensional model, and then the three-dimensional model is interactively visualized through three-dimensional visualization software to obtain a reconstructed three-dimensional model of the extraocular muscle. MPR can extract and reconstruct the extraocular muscle in a targeted manner based on the two-dimensional segmentation results.

[0069] Based on the extraocular muscles segmented by semantic segmentation, the MPR (multi-planar reconstruction) algorithm is used to reconstruct the extraocular muscle model. The MPR algorithm is a technology that is widely used to analyze computer tomography images. It can be used to reconstruct images in orthogonal planes (coronal, sagittal, axial or oblique, select any plane). In the clinical scanning process, we have a common acquisition plane (coronal plane or sagittal plane or cross-section) for a part, but sometimes we also want to look at it in other planes. At this time, we can use the MPR technology. MPR superimposes all axial images within the scanning range and then reconstructs the coronal, sagittal, and oblique images of the tissues specified by the reconstruction lines marked by certain markings. A large number of extraocular muscle images at different levels but on the same acquisition plane obtained after semantic segmentation are superimposed to obtain a three-dimensional model of the part. The MPR algorithm is required. After inputting the pictures in a folder, a three-dimensional model will be output. Finally, the three-dimensional model will be interactively visualized through three-dimensional visualization software (such as 3DSlicer, Mimics, etc.).

[0070] The above volume calculation formula can be implemented using Python and its commonly used 3D data processing libraries (such as trimesh, open3d, etc.). This can be done by importing the reconstructed model and then importing the 3D data processing library. Another method is to use the area calculation method, because MPR is a three-dimensional model formed by stacking. By fixing the distance between each layer, the volume can be calculated. This method is called "layer-by-layer integration method".

[0071] like:

[0072] The slice spacing between the two plane images is set to 1mm. Next, the area of ​​each layer is calculated. This can be done by using an image processing library (such as OpenCV) to calculate the area of ​​the layer. The volume of each layer can be obtained by multiplying the area of ​​each layer by the slice spacing. Finally, the volumes of all layers are added together to obtain the total volume of the entire 3D model.

[0073] Specifically, in S5, assuming that each extraocular muscle has n slices, the extraocular muscle length is evenly divided into multiple segments, and the area of ​​each layer is A i (where i = 1, 2, ..., n), the slice spacing is d, and the volume of each segment Vi is expressed as:

[0074]

[0075] In the formula: i = 1, 2, 3, 4... n is the total number of layers of each extraocular muscle, A j is the area of ​​the jth layer, and d is the fixed distance between each layer.

[0076] Since strabismus patients are judged to have atrophy by observing the volume of the orbital apex segment of the extraocular muscles, this method of three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume aims to accurately calculate the volume of the orbital apex segment of the extraocular muscles in strabismus patients. This method of three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume divides the entire extraocular muscle into four equal parts according to its length, and performs independent volume calculation on each segment. This method can cleverly solve the problem of positioning and can more accurately calculate which parts of the diseased extraocular muscles are atrophied. The generated bar graph is intuitively compared with the corresponding segments of the extraocular muscles on the opposite side of the orbit.

[0077] The overall length is determined based on the number of 1 mm slice images acquired, thereby achieving an accurate assessment of the volume of that specific area.

[0078] This method of extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation uses the ESNet model to perform deep learning prediction on the acquired medical imaging data to obtain an accurate extraocular muscle image, and performs three-dimensional reconstruction and volume calculation based on the extraocular muscle image. The extraocular muscle reconstructed by this method is accurate in restoration, assists in the diagnosis of strabismus, and helps in the study of diseases related to extraocular muscle volume.

[0079] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for three-dimensional reconstruction of extraocular muscles and calculation of extraocular muscle volume, characterized in that: include: S1, acquiring medical image data, wherein the medical image data includes images of coronal plane information of an eyeball and surrounding tissues; S2, using the ESNet model to perform deep learning prediction on the acquired medical imaging data to obtain the extraocular muscle segmentation results; S3, obtaining an accurate extraocular muscle image according to the extraocular muscle segmentation result; S4, based on the acquired extraocular muscle images, the MPR algorithm is used to perform three-dimensional reconstruction of the extraocular muscles and establish a three-dimensional model of the extraocular muscles; S5, calculating the volume of the extraocular muscles according to the extraocular muscle image obtained in S3.

2. The method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation according to claim 1, wherein: In S1, the method for obtaining the medical image data is to connect the midpoint of the bony orbital opening to the orbital apex, where the connecting line is the central axis of the bony orbit, make a plane cut perpendicular to the central axis, and cut from the anterior edge of the anterior chamber of the eyeball to the orbital apex with uniform layer thickness and uniform intervals to obtain a continuous image, and export the image in a fixed format as the medical image data.

3. The method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation according to claim 1, wherein: In S2, the architecture of the ESNet model includes an input layer, a YOLO detection module, and a semantic segmentation module, wherein the input layer is used to receive the obtained medical image data; the YOLO detection module performs eyeball detection on the input medical image data to determine the preliminary position of the extraocular muscles; the semantic segmentation module uses a semantic segmentation model to perform fine segmentation of the extraocular muscles within the bounding box of the preliminary position of the extraocular muscles provided by the YOLO detection module to obtain an extraocular muscle segmentation result.

4. The method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation according to claim 3, wherein: The steps of performing deep learning prediction on the obtained medical image data by the ESNet model are as follows: S21, inputting the obtained medical imaging data; S22, marking the eyeball part in the medical image data, marking the position and outline of the eyeball; S23, based on the marked eyeball position and contour, use the YOLO target detection model to train the YOLO model; S24, applying the trained YOLO model to the medical image data, automatically detecting and locating the position of the eyeball, and determining the preliminary position of the extraocular muscles; based on the preliminary position, cropping the eyeball portion on the medical image data, removing unnecessary background areas, and obtaining a cropped eyeball image; S25, annotating the extraocular muscles on the obtained eyeball image, annotating the contours and position information of the extraocular muscles; S26, based on the outline and position information of the annotated extraocular muscles, use a semantic segmentation model to train the semantic segmentation model; within the bounding box provided by the YOLO model, use the semantic segmentation model to perform fine segmentation of the extraocular muscles; S27, use the loss function to simultaneously optimize the performance of YOLO target detection and semantic segmentation, determine the appropriate loss weight, and balance the training effects of the YOLO model and the semantic segmentation model.

5. The method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation according to claim 4, characterized in that: In S23, the YOLO model is trained, the medical image data is annotated using a data annotation tool, the eyeball part is framed using a box, and then the model training is performed.

6. The method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation according to claim 5, characterized in that: In S23, the training process of the YOLO model includes initializing weights, inputting the trained medical image data for forward propagation to obtain a prediction result, and calculating a loss between the prediction and the true annotation; The gradient is calculated and the weights are updated through backpropagation, and the weights are adjusted using an optimization algorithm; this process is repeated for multiple cycles until the loss converges or the preset number of iterations is reached.

7. The method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation according to claim 6, characterized in that: In S2, in the process of prediction using the ESNet model, the input medical image data is first adjusted to a specified size, and a feature map is generated through the network through forward propagation; on the feature map, the ESNet model predicts multiple bounding boxes and their confidences for each grid unit, and calculates the object category probability of each box; the ESNet model filters out high-confidence bounding boxes according to the set confidence threshold, and removes boxes with high overlap through a non-maximum suppression algorithm, and finally outputs the category of each detected object and the corresponding bounding box coordinates.

8. The method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation according to claim 7, characterized in that: In S26, a semantic segmentation model is trained. Semantic segmentation realizes automatic segmentation through a deep learning model. Medical imaging data sets are collected and annotated so that each pixel has a corresponding category label. A network architecture is selected. The semantic segmentation model extracts image features and restores them to the original size through upsampling, and outputs the category probability of each pixel. During the training process, a loss function is used to evaluate the difference between the output and the true label, and the model weights are updated. In the inference stage, the image to be segmented is input into the trained model, and the category of each pixel is output to realize automatic region segmentation.

9. The method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation according to claim 8, characterized in that: In S4, a large number of extraocular muscle images at different levels but in the same acquisition plane obtained after semantic segmentation are superimposed to obtain a three-dimensional model of the part. Using the MPR algorithm, after inputting images from a folder, a three-dimensional model is output. The three-dimensional model is then interactively visualized using three-dimensional visualization software to obtain a reconstructed three-dimensional model of the extraocular muscles.

10. The method for extraocular muscle three-dimensional reconstruction and extraocular muscle volume calculation according to claim 9, characterized in that: In S5, it is assumed that each extraocular muscle has n slices, and the extraocular muscle length is evenly divided into multiple segments, and the area of ​​each layer is A i (where i = 1, 2, ..., n), the slice spacing is d, and the volume of each segment Vi is expressed as: In the formula: i = 1, 2, 3, 4... n is the total number of layers of each extraocular muscle, A j is the area of ​​the jth layer, and d is the fixed distance between each layer.

Images