A method and device for segmenting orbital images before and after surgery

Abstract

The present invention discloses a method and device for pre- and post-operative orbital image segmentation. The method can automatically extract the orbit from CT images, which can significantly reduce the workload of doctors. Only a small amount of labeled data from doctors is needed during training to automatically segment the orbit contained in a large amount of unlabeled CT images. The orbital segmentation results can be used to quantitatively analyze the effect of orbital decompression surgery.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image analysis, and in particular to a method and device for segmenting orbital images before and after orbital decompression surgery. Background Art

[0002] Thyroid-associated eye disease is a common autoimmune disease. Protrusion of the eyeball within the orbit is a prominent feature of this disease, caused by hyperplasia and enlargement of the orbital muscles. To alleviate the symptoms of proptosis, orbital decompression surgery is currently considered an effective treatment. However, the effectiveness of orbital decompression surgery currently relies primarily on the surgeon's experience, lacking feasible quantitative analysis methods.

[0003] Accurately segmenting the patient's orbit before and after surgery is an important foundation for quantitative analysis. However, since the size and shape of the orbit vary from person to person, it is not a continuous bony structure and consists of seven bones and five openings.

[0004] Therefore, in order to perform quantitative analysis, doctors need to manually mark key points on the orbital contour. Manual marking is an extremely time-consuming and labor-intensive task, and the quality of the marking also depends on the doctor's experience and skills, which can easily lead to inconsistent outlining results.

[0005] Therefore, how to provide a method for automatically extracting eye sockets from CT images can significantly reduce the workload of doctors. It only needs to be based on a small amount of doctors' labeled data to automatically segment the eye sockets of a large amount of unlabeled data. This is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of the above problems, the present invention provides a method and apparatus for segmenting pre- and post-operative orbital images for overcoming the above problems or at least partially solving the above problems.

[0007] The present invention provides the following solutions:

[0008] A method for segmenting orbital images before and after surgery, comprising:

[0009] Obtaining a target preoperative image and a target postoperative image of a patient to be examined; wherein the target preoperative image and the target postoperative image are both unlabeled CT images;

[0010] The target preoperative image and the target postoperative image are processed using a copy-paste strategy and then input into a target Teacher network to obtain an orbital segmentation result;

[0011] The target Teacher network is generated in the following way:

[0012] Obtaining a first sample pre-operative image and a first sample post-operative image with labels;

[0013] Processing the first sample preoperative image and the first sample postoperative image using a copy-paste strategy and inputting the processed images into a Teacher network, pre-training the Teacher network to obtain a pre-trained Teacher network; the Teacher network includes a VNet main structure;

[0014] Acquire a second sample pre-operative image and a second sample post-operative image with labels, and acquire a third sample pre-operative image and a third sample post-operative image without labels;

[0015] The second sample preoperative image, the second sample postoperative image, the third sample preoperative image, and the third sample postoperative image are processed using a copy-paste strategy and then input into a Student network, and the Student network is trained to obtain a predicted label; the Student network and the Teacher network have the same main structure;

[0016] Inputting the third sample pre-operative image and the third sample post-operative image into the pre-trained Teacher network to obtain two pseudo labels;

[0017] The two pseudo labels are processed using a copy-paste strategy to supervise the predicted labels of the Student network to complete the iterative update of the parameters of the Student network;

[0018] The parameters of the pre-trained Teacher network are updated using the parameters of the Student network to obtain the target Teacher network.

[0019] Preferably, processing the first sample pre-operative image and the first sample post-operative image using a copy-paste strategy includes:

[0020] The first sample preoperative image And the first sample postoperative image Generate two new images using the copy-paste strategy and The two new images and Enter the Teacher network;

[0021] Among them, pre represents random Select a piece as the foreground and randomly select Select a piece as the background; post represents a random Select a piece as the foreground and randomly select A piece is selected as the background; the superscript ll represents that both the fused images have labels.

[0022] Preferably, the copy-paste strategy is used to process the second sample pre-operative image, the second sample post-operative image, the third sample pre-operative image, and the third sample post-operative image, including:

[0023] The second sample preoperative image The second sample postoperative image The third sample preoperative image and the third sample postoperative image Use the copy-paste strategy to generate two new images and The two new images and Input the Student network;

[0024] The superscript lu represents the two images being fused, one with a label and one without a label.

[0025] Preferably, the copy-paste strategy includes:

[0026] Determine the mask The foreground value is 0 and the background value is 1. The range of the foreground part is αH×αW×αL, where α∈(0,1). The fused new image is expressed as follows:

[0027]

[0028]

[0029]

[0030]

[0031] Where, Indicates that the corresponding elements are multiplied one by one, 1∈1 W×H×L is a matrix with value 1.

[0032] Preferably: α=0.75.

[0033] Preferably, the third sample pre-operative image and the third sample post-operative image are input into the pre-trained Teacher network to obtain two pseudo labels; comprising:

[0034] The postoperative image of the third sample and the third sample preoperative image Input the pre-trained Teacher network to obtain the probability map:

[0035]

[0036]

[0037] Where: For, Θ t Parameters for pre-training Teacher network;

[0038] For the segmentation task, pixels with probability greater than 0.5 in the probability map are determined as foreground, and the rest are determined as background.

[0039] Preferably, the step of processing the two pseudo labels using a copy-paste strategy to supervise the predicted labels of the Student network and completing the iterative update of the parameters of the Student network includes:

[0040] The copy-paste strategy is used to supervise the prediction results of the Student network, and the results are as follows:

[0041]

[0042]

[0043] in, and is the prediction result of the Student network and supervision signal.

[0044] Preferably, the loss function of the Student network training is calculated as follows:

[0045]

[0046]

[0047] Where, It is a linear combination of Dice loss and cross entropy loss, β is a hyperparameter, β is 0.5, the predicted label and Calculated by the following formula:

[0048]

[0049]

[0050] Where: For, Θ s are the parameters of the Student network;

[0051] Parameter Θ sStochastic gradient descent is used for updating at each iteration; the total loss function is:

[0052]

[0053] Preferably: in each iteration, the parameters of the pre-trained Teacher network Update in the following ways:

[0054]

[0055] Where λ is the smoothing coefficient, and λ is set to 0.5.

[0056] A device for segmenting orbital images before and after surgery, comprising:

[0057] a target image acquisition unit, configured to acquire a target preoperative image and a target postoperative image of the patient to be examined; the target preoperative image and the target postoperative image are both unlabeled CT images;

[0058] a segmentation result acquisition unit, configured to process the target pre-operative image and the target post-operative image using a copy-paste strategy and then input the processed images into a target Teacher network to obtain an orbital segmentation result;

[0059] The target teacher network generating unit is used to generate the target teacher network in the following manner:

[0060] A first sample image acquisition subunit, configured to acquire a labeled first sample pre-operative image and a first sample post-operative image;

[0061] A Teacher network pre-training subunit is configured to process the first sample pre-operative image and the first sample post-operative image using a copy-paste strategy and input them into a Teacher network, and pre-train the Teacher network to obtain a pre-trained Teacher network; the Teacher network includes a VNet main structure;

[0062] A second sample image acquisition subunit is configured to acquire a second sample pre-operative image and a second sample post-operative image with labels, and acquire a third sample pre-operative image and a third sample post-operative image without labels;

[0063] a Student network training subunit, configured to process the second sample pre-operative image, the second sample post-operative image, the third sample pre-operative image, and the third sample post-operative image using a copy-paste strategy, input the processed images into a Student network, and train the Student network to obtain predicted labels; the Student network and the Teacher network have the same main structure;

[0064] a pseudo-label acquisition subunit, configured to input the third sample pre-operative image and the third sample post-operative image into the pre-trained Teacher network to obtain two pseudo-labels;

[0065] The parameter updating subunit of the Student network is used to process the two pseudo labels using a copy-paste strategy to supervise the predicted labels of the Student network to complete the iterative update of the parameters of the Student network;

[0066] The target teacher network acquisition subunit is used to update the parameters of the pre-trained teacher network using the parameters of the student network to obtain the target teacher network.

[0067] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0068] The embodiments of the present application provide a method and device for pre- and post-operative orbital image segmentation, which can automatically extract the orbits from CT images, significantly reducing the workload of doctors. Only a small amount of labeled data from doctors is required during training to automatically segment the orbits contained in a large amount of unlabeled CT images. The orbital segmentation results can be used to quantitatively analyze the effects of orbital decompression surgery.

[0069] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0071] Figure 1 This is a flow chart of a method for segmenting orbital images before and after surgery provided by an embodiment of the present invention;

[0072] Figure 2 is an overall network diagram of the method provided by an embodiment of the present invention;

[0073] Figure 3 It is a quantitative comparison result between the method provided by the embodiment of the present invention and other advanced methods;

[0074] Figure 4 is a schematic diagram of a visualization comparison result of the method provided by an embodiment of the present invention and other advanced methods in a two-dimensional axial view;

[0075] Figure 5 is a schematic diagram of a visual comparison result of the method provided by an embodiment of the present invention and other advanced methods in a three-dimensional view;

[0076] Figure 6 is a schematic diagram of a pre- and post-operative orbital image segmentation device provided by an embodiment of the present invention;

[0077] Figure 7 Schematic diagram of a pre- and post-operative orbital image segmentation device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0078] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of the present invention.

[0079] See also Figure 1 , is a method for segmenting orbital images before and after surgery provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0080] S101: Acquire a target preoperative image and a target postoperative image of a patient to be examined; the target preoperative image and the target postoperative image are both unlabeled CT images;

[0081] S102: processing the target pre-operative image and the target post-operative image using a copy-paste strategy and inputting the processed images into a target Teacher network to obtain an orbital segmentation result;

[0082] The target Teacher network is generated in the following way:

[0083] Obtaining a first sample pre-operative image and a first sample post-operative image with labels;

[0084] The first sample preoperative image and the first sample postoperative image are processed by copy-paste strategy and then input into Teacher network, and the Teacher network is pre-trained to obtain a pre-trained Teacher network; the Teacher network includes a VNet main structure; in specific implementation, the first sample preoperative image And the first sample postoperative image Generate two new images using the copy-paste strategy and The two new images and Enter the Teacher network;

[0085] Among them, pre represents random Select a piece as the foreground and randomly select Select a piece as the background; post represents a random Select a piece as the foreground and randomly select A piece is selected as the background; the superscript ll represents that both the fused images have labels.

[0086] Acquire a second sample pre-operative image and a second sample post-operative image with labels, and acquire a third sample pre-operative image and a third sample post-operative image without labels;

[0087] The second sample preoperative image, the second sample postoperative image, the third sample preoperative image and the third sample postoperative image are processed using a copy-paste strategy and then input into the Student network, and the Student network is trained to obtain predicted labels; the Student network and the Teacher network have the same main structure; the Teacher network provided in the embodiment of the present application has the same main structure as the Student network, and the pre-trained Teacher network obtained by training the Teacher network using the first sample preoperative image and the first sample postoperative image containing labels can be used when training the Student network, and the Teacher network is used to generate corresponding pseudo labels for unlabeled data.

[0088] In a specific implementation, the second sample preoperative image The second sample postoperative image The third sample preoperative image and the third sample postoperative image Use the copy-paste strategy to generate two new images and The two new images and Input the Student network;

[0089] The superscript lu represents the two images being fused, one with a label and one without a label.

[0090] Inputting the third sample pre-operative image and the third sample post-operative image into the pre-trained Teacher network to obtain two pseudo labels;

[0091] The two pseudo labels are processed using a copy-paste strategy to supervise the predicted labels of the Student network to complete the iterative update of the parameters of the Student network;

[0092] The parameters of the pre-trained Teacher network are updated using the parameters of the Student network to obtain the target Teacher network.

[0093] The present embodiment uses a copy-paste strategy to process the original image and generate a new image for subsequent network training. This strategy can better integrate the features of pre- and post-operative images for training. Specifically, the copy-paste strategy includes:

[0094] Determine the mask The foreground value is 0 and the background value is 1. The range of the foreground part is αH×αW×αL, where α∈(0,1). The fused new image is expressed as follows:

[0095]

[0096]

[0097]

[0098]

[0099] Where, Indicates that the corresponding elements are multiplied one by one, 1∈1 W×H×L is a matrix with value 1.

[0100] Specifically, the embodiment of the present application may provide α=0.75.

[0101] Furthermore, the third sample pre-operative image and the third sample post-operative image are input into the pre-trained Teacher network to obtain two pseudo labels; including:

[0102] The postoperative image of the third sample and the third sample preoperative image Input the pre-trained Teacher network to obtain the probability map:

[0103]

[0104]

[0105] Where: For, Θ t Parameters for pre-training Teacher network;

[0106] For the segmentation task, pixels with probability greater than 0.5 in the probability map are determined as foreground, and the rest are determined as background.

[0107] After processing the two pseudo labels using a copy-paste strategy, supervising the predicted labels of the Student network to complete the iterative update of the parameters of the Student network includes:

[0108] The copy-paste strategy is used to supervise the prediction results of the Student network, and the results are as follows:

[0109]

[0110]

[0111] in, and is the prediction result of the Student network and supervision signal.

[0112] The loss function of the Student network training is calculated as follows:

[0113]

[0114]

[0115] Where, It is a linear combination of Dice loss and cross entropy loss, β is a hyperparameter, β is 0.5, the predicted label and Calculated by the following formula:

[0116]

[0117]

[0118] Where: For, Θ s are the parameters of the Student network;

[0119] Parameter Θ s Stochastic gradient descent is used for updating at each iteration; the total loss function is:

[0120]

[0121] In each iteration, the parameters of the pre-trained Teacher network Update in the following ways:

[0122]

[0123] Where λ is the smoothing coefficient, and λ is set to 0.5.

[0124] The embodiment of the present application provides a method for pre- and post-operative orbital image segmentation, which can automatically extract the orbits from CT images, significantly reducing the workload of doctors. It only requires training based on a small amount of doctors' labeled data to automatically segment the orbits contained in a large amount of unlabeled CT images. The orbital segmentation results can be used to quantitatively analyze the effects of orbital decompression surgery.

[0125] Below, the network training process provided in the embodiment of the present application is introduced in detail, and the method is verified.

[0126] The method provided in the embodiment of the present application is implemented using a deep learning method based on a convolutional neural network. The main structure of the network adopts VNet, and the training process generally includes two steps: pseudo-label generation of unlabeled data and automatic orbital segmentation based on semi-supervision.

[0127] The overall framework of the network is as follows Figure 1 shown.

[0128] In step 1, a preoperative image of a patient is first randomly selected and postoperative images Generate two new images using the copy-paste strategy and

[0129] Among them, pre represents random Select a piece as the foreground and randomly select Select a piece as the background; similarly, post represents a random Select a piece as the foreground and randomly select A piece is selected from the image as the background; the superscript ll indicates that both images being fused have labels. The generated new image is fed into the Teacher network for pre-training, generating pseudo labels for the unlabeled data used in step 2.

[0130] In step 2, we first randomly select the preoperative and postoperative images of two patients, one of which has corresponding labels and the other has no labels. and Generate two new images using the copy-paste strategy and The superscript lu represents the two images being fused, one with a label and one without a label. The generated new image is input into the Student network to complete the training and obtain the predicted label and As training progresses, the parameters of the Student network will also update the Teacher network simultaneously.

[0131] In both steps 1 and 2, the method uses a copy-paste strategy to process the original image and generate a new image for subsequent network training. This strategy can better integrate the features of pre- and post-operative images for training.

[0132] You can set a mask The foreground value is 0 and the background value is 1. The range of the foreground is αH×αW×αL, where α∈(0,1). In this method, α=0.75. The fused new image can be expressed as follows:

[0133]

[0134]

[0135]

[0136]

[0137] in, Indicates that the corresponding elements are multiplied one by one, 1∈1 W×H×L is a matrix with value 1.

[0138] During the training process of the Student network, the supervised labels also adopted the copy-paste strategy, and the data without labels Input the Teacher network to get the probability map,

[0139]

[0140]

[0141] For the segmentation task, we consider pixels with a probability greater than 0.5 in the probability map as foreground, and the rest as background. We also use a copy-paste strategy between the pseudo-labels of the unlabeled data and the gold standard of the labeled data to supervise the network's data results. The results are as follows:

[0142]

[0143]

[0144] in, and is the prediction result of the Student network and supervision signal.

[0145] In the embodiment of the present application, the loss function of network training is calculated as follows:

[0146]

[0147]

[0148] in, It is a linear combination of Dice loss and cross entropy loss, and β is a hyperparameter. β is taken as 0.5 in this method.

[0149] and Calculated by the following formula:

[0150]

[0151] Parameter Θ of the Student network s Stochastic gradient descent is used for updating at each iteration. The total loss function is:

[0152]

[0153] In each iteration, the parameters of the Teacher network Update in the following ways:

[0154]

[0155] Wherein, λ is a smoothing coefficient, and in the embodiment of the application, λ is set to 0.5.

[0156] In order to verify the effectiveness of the method provided in the embodiment of the present application, experiments were conducted on 120 CT data provided by Wenzhou Eye and Vision Hospital.

[0157] These 120 data sets were collected from 60 patients and included preoperative and postoperative images of each patient. The gold standard of the data was annotated by experienced clinicians, and the CT resolution was 512×512×L, where L∈[102,179].

[0158] Attachment Figure 3 , Attachment Figure 4 and attached Figure 5 The quantitative and qualitative comparison results of the method provided in the examples of the present application and the current advanced methods are presented respectively.

[0159] Figure 4 A visual comparison of the method provided in this embodiment with other advanced methods in a 2D axial view. The first row shows the results for 5% labeled data, and the second row shows the results for 10% labeled data. The white lines represent the predictions of each method, with the ground truth corresponding to the gold standard.

[0160] Figure 5Comparison of the 3D visualization of the method provided in the application examples with other advanced methods. The first row shows the results for 5% labeled data, and the second row shows the results for 10% labeled data. Ground Truth corresponds to the gold standard, while the others are predictions from various methods. Regions with significant improvements are magnified.

[0161] In the attached Figure 3 The values ​​are presented as the mean ± standard deviation of the test set. Due to their ability to leverage unlabeled data, all semi-supervised methods demonstrate superior performance compared to supervised methods. Specifically, for the case of 5% labeled data, the method provided in this embodiment outperforms other methods across all metrics, particularly exceeding ComWin by 2.4% in terms of the Dice coefficient.

[0162] Similarly, for the case of 10% labeled data, the method provided by the embodiment of the present application achieved the best performance in all indicators and exceeded URPC by 1.6% in terms of the Dice coefficient.

[0163] In addition, as far as 95HD is concerned, the method provided by the embodiment of the present application is significantly better than all other methods. Figure 4 and attached Figure 5 It can be seen that the predictions of the method provided in the examples of the present application are more similar to the gold standard in terms of overlap rate and anatomical shape.

[0164] See also Figure 6 , the embodiment of the present application can also provide a device for segmenting orbital images before and after surgery, such as Figure 6 As shown, the device may include:

[0165] The target image acquisition unit 601 is used to acquire a target preoperative image and a target postoperative image of the patient to be examined; the target preoperative image and the target postoperative image are both unlabeled CT images;

[0166] A segmentation result acquisition unit 602 is configured to process the target pre-operative image and the target post-operative image using a copy-paste strategy and then input the processed images into a target Teacher network to obtain an orbital segmentation result;

[0167] The target teacher network generating unit is used to generate the target teacher network in the following manner:

[0168] A first sample image acquisition subunit, configured to acquire a labeled first sample pre-operative image and a first sample post-operative image;

[0169] A Teacher network pre-training subunit is configured to process the first sample pre-operative image and the first sample post-operative image using a copy-paste strategy and input them into a Teacher network, and pre-train the Teacher network to obtain a pre-trained Teacher network; the Teacher network includes a VNet main structure;

[0170] A second sample image acquisition subunit is configured to acquire a second sample pre-operative image and a second sample post-operative image with labels, and acquire a third sample pre-operative image and a third sample post-operative image without labels;

[0171] a Student network training subunit, configured to process the second sample pre-operative image, the second sample post-operative image, the third sample pre-operative image, and the third sample post-operative image using a copy-paste strategy, input the processed images into a Student network, and train the Student network to obtain predicted labels; the Student network and the Teacher network have the same main structure;

[0172] a pseudo-label acquisition subunit, configured to input the third sample pre-operative image and the third sample post-operative image into the pre-trained Teacher network to obtain two pseudo-labels;

[0173] The parameter updating subunit of the Student network is used to process the two pseudo labels using a copy-paste strategy to supervise the predicted labels of the Student network to complete the iterative update of the parameters of the Student network;

[0174] The target teacher network acquisition subunit is used to update the parameters of the pre-trained teacher network using the parameters of the student network to obtain the target teacher network.

[0175] The present application also provides a device for segmenting orbital images before and after surgery, the device comprising a processor and a memory.

[0176] The memory is used to store program code and transmit the program code to the processor;

[0177] The processor is used to execute the steps of the above-mentioned method for segmenting pre- and post-operative orbital images according to the instructions in the program code.

[0178] like Figure 7 As shown, an apparatus for segmenting orbital images before and after surgery provided by an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, the memory 11, and the communication interface 12 communicate with each other via the communication bus 13.

[0179] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field programmable gate array, or other programmable logic devices.

[0180] The processor 10 may call a program stored in the memory 11 . Specifically, the processor 10 may execute the operations in the embodiment of the method for segmenting pre- and post-operative orbital images.

[0181] The memory 11 is used to store one or more programs. The program may include program code, and the program code includes computer operating instructions. In the embodiment of the present application, the memory 11 stores at least a program for implementing the following functions:

[0182] Obtaining a target preoperative image and a target postoperative image of a patient to be examined; wherein the target preoperative image and the target postoperative image are both unlabeled CT images;

[0183] The target preoperative image and the target postoperative image are processed using a copy-paste strategy and then input into a target Teacher network to obtain an orbital segmentation result;

[0184] The target Teacher network is generated in the following way:

[0185] Obtaining a first sample pre-operative image and a first sample post-operative image with labels;

[0186] Processing the first sample preoperative image and the first sample postoperative image using a copy-paste strategy and inputting the processed images into a Teacher network, pre-training the Teacher network to obtain a pre-trained Teacher network; the Teacher network includes a VNet main structure;

[0187] Acquire a second sample pre-operative image and a second sample post-operative image with labels, and acquire a third sample pre-operative image and a third sample post-operative image without labels;

[0188] The second sample preoperative image, the second sample postoperative image, the third sample preoperative image, and the third sample postoperative image are processed using a copy-paste strategy and then input into a Student network, and the Student network is trained to obtain a predicted label; the Student network and the Teacher network have the same main structure;

[0189] Inputting the third sample pre-operative image and the third sample post-operative image into the pre-trained Teacher network to obtain two pseudo labels;

[0190] The two pseudo labels are processed using a copy-paste strategy to supervise the predicted labels of the Student network to complete the iterative update of the parameters of the Student network;

[0191] The parameters of the pre-trained Teacher network are updated using the parameters of the Student network to obtain the target Teacher network.

[0192] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required for at least one function (such as a file creation function, a data reading and writing function), etc.; the data storage area can store data created during use, such as initialization data, etc.

[0193] In addition, the memory 11 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0194] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0195] Of course, it needs to be explained that Figure 7 The structure shown does not constitute a limitation on the pre- and post-operative orbital image segmentation device in the embodiment of the present application. In actual applications, the pre- and post-operative orbital image segmentation device may include Figure 7 More or fewer components than shown, or combinations of certain components.

[0196] An embodiment of the present application may further provide a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the steps of the above-mentioned method for segmenting pre- and post-operative orbital images.

[0197] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0198] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present application.

[0199] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0200] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the scope of protection of the present invention.

Claims

1. A method for segmenting orbital images before and after surgery, characterized in that: include: acquiring a target pre-operative image and a target post-operative image of a patient to be examined; The target pre-operative image and the target post-operative image are both unlabeled CT images; The target preoperative image and the target postoperative image are processed using a copy-paste strategy and then input into a target Teacher network to obtain an orbital segmentation result; The target Teacher network is generated in the following way: Obtaining a first sample pre-operative image and a first sample post-operative image with labels; The first sample preoperative image and the first sample postoperative image are processed by a copy-paste strategy and then input into a Teacher network, and the Teacher network is pre-trained to obtain a pre-trained Teacher network; the Teacher network includes a VNet main structure; the first sample preoperative image and the first sample postoperative image are processed by a copy-paste strategy including the first sample preoperative image And the first sample postoperative image Generate two new images using the copy-paste strategy and ; Put two new images and Enter the Teacher network; in, pre Represents random Select a piece as the foreground and randomly select Select a piece as the background; post Represents random Select a piece as the foreground and randomly select Select a piece as the background; superscript ll Indicates that both images being fused have labels; Acquire a second sample pre-operative image and a second sample post-operative image with labels, and acquire a third sample pre-operative image and a third sample post-operative image without labels; The second sample preoperative image, the second sample postoperative image, the third sample preoperative image, and the third sample postoperative image are processed using a copy-paste strategy and then input into a Student network, and the Student network is trained to obtain a predicted label; the Student network and the Teacher network have the same main structure; the second sample preoperative image, the second sample postoperative image, the third sample preoperative image, and the third sample postoperative image are processed using a copy-paste strategy, including: The second sample preoperative image , the second sample postoperative image , the third sample preoperative image and the third sample postoperative image Use the copy-paste strategy to generate two new images and , the two new images and Input the Student network; Among them, the superscript lu Represents the two images being fused, one with a label and one without a label; Inputting the third sample pre-operative image and the third sample post-operative image into the pre-trained Teacher network to obtain two pseudo labels; The two pseudo labels are processed by a copy-paste strategy, and then the predicted labels of the Student network are supervised to complete the iterative update of the parameters of the Student network; the two pseudo labels are processed by a copy-paste strategy, including: Determine the mask , where the foreground value is 0 and the background value is 1; the range of the foreground part is , the fused new image is expressed as follows: Where, Indicates that the corresponding elements are multiplied one by one, is a matrix with value 1; The parameters of the pre-trained Teacher network are updated using the parameters of the Student network to obtain the target Teacher network.

2. The method for segmenting orbital images before and after surgery according to claim 1, wherein: described =0.

75.

3. The method for segmenting orbital images before and after surgery according to claim 1, wherein: Inputting the third sample pre-operative image and the third sample post-operative image into the pre-trained Teacher network to obtain two pseudo labels; include: The postoperative image of the third sample and the third sample preoperative image Input the pre-trained Teacher network to obtain the probability map: Where: 、 Parameters for pre-training Teacher network; For the segmentation task, pixels with probability greater than 0.5 in the probability map are determined as foreground, and the rest are determined as background.

4. The method for segmenting orbital images before and after surgery according to claim 1, wherein: After processing the two pseudo labels using a copy-paste strategy, supervising the predicted labels of the Student network to complete the iterative update of the parameters of the Student network includes: The copy-paste strategy is used to supervise the prediction results of the Student network, and the results are as follows: in, and is the prediction result of the Student network and supervision signal.

5. The method for segmenting orbital images before and after surgery according to claim 4, wherein: The loss function of the Student network training is calculated as follows: Where, is a linear combination of Dice loss and cross entropy loss, is a hyperparameter, Take 0.5, the predicted label and Calculated by the following formula: Where: 、 are the parameters of the Student network; parameter Stochastic gradient descent is used for updating at each iteration; the total loss function is:

6. The method for segmenting orbital images before and after surgery according to claim 5, characterized in that: In each iteration, the parameters of the pre-trained Teacher network Update in the following ways: Where, is the smoothing coefficient, Take 0.

5.

7. A device for segmenting orbital images before and after surgery, characterized in that: The device is used to perform the method for segmenting pre- and post-operative orbital images according to any one of claims 1 to 6, comprising: a target image acquisition unit, configured to acquire a target preoperative image and a target postoperative image of the patient to be examined; the target preoperative image and the target postoperative image are both unlabeled CT images; a segmentation result acquisition unit, configured to process the target pre-operative image and the target post-operative image using a copy-paste strategy and then input the processed images into a target Teacher network to obtain an orbital segmentation result; The target teacher network generating unit is used to generate the target teacher network in the following manner: A first sample image acquisition subunit, configured to acquire a labeled first sample pre-operative image and a first sample post-operative image; A Teacher network pre-training subunit is configured to process the first sample pre-operative image and the first sample post-operative image using a copy-paste strategy and input them into a Teacher network, and pre-train the Teacher network to obtain a pre-trained Teacher network; the Teacher network includes a VNet main structure; A second sample image acquisition subunit is configured to acquire a second sample pre-operative image and a second sample post-operative image with labels, and acquire a third sample pre-operative image and a third sample post-operative image without labels; a Student network training subunit, configured to process the second sample pre-operative image, the second sample post-operative image, the third sample pre-operative image, and the third sample post-operative image using a copy-paste strategy, input the processed images into a Student network, and train the Student network to obtain predicted labels; the Student network and the Teacher network have the same main structure; a pseudo-label acquisition subunit, configured to input the third sample pre-operative image and the third sample post-operative image into the pre-trained Teacher network to obtain two pseudo-labels; The parameter updating subunit of the Student network is used to process the two pseudo labels using a copy-paste strategy to supervise the predicted labels of the Student network to complete the iterative update of the parameters of the Student network; The target teacher network acquisition subunit is used to update the parameters of the pre-trained teacher network using the parameters of the student network to obtain the target teacher network.

Images