A human-computer collaborative dual labeling method

Abstract

The application claims a human-computer collaborative double labeling method, comprising: collecting images by ABUS and dividing them into unlabeled data set N and labeled data set M. The method uses image augmentation processing and normalization, uses a self-supervised framework based on contrast learning to enhance image features. Then, a visual model is constructed using deep learning architecture U-Net and ResNet50, automatically labeling the regions of interest in the image and providing a preliminary estimate. After that, the breast images automatically labeled are manually labeled to realize human-computer double labeling and provide more accurate estimates. Finally, the contrast learning algorithm is used to learn image representation from the unlabeled data set N, and the visual model is updated in combination with the supervision signal in the labeled data set M. The application improves the image recognition performance by comprehensively utilizing the labeling results of artificial intelligence and professional doctors, strengthens the expressiveness of image features, and further enhances the generalization ability of the visual model.

Description

TECHNICAL FIELD

[0001] The present application relates to the fields of image processing, deep learning, etc., and in particular to a human-computer collaborative double labeling method. BACKGROUND

[0002] Breast cancer has become one of the malignant tumors that seriously affects women's health. According to the global cancer statistics data released by the Global Cancer Association in 2023, the incidence of breast cancer in women has exceeded that of lung cancer, accounting for 31% of total cancer cases. Among women worldwide, breast cancer accounts for one-fourth of total cancer cases and one-sixth of total cancer deaths. A large amount of evidence-based medical evidence shows that widespread screening in the population helps early diagnosis and discovery of breast cancer, promotes early treatment of the disease, improves the chances of successful treatment, and thus reduces mortality.

[0003] Traditional breast disease diagnosis mainly relies on doctors' interpretation of medical images, but this method has the problems of subjectivity and insufficient diagnostic accuracy, and when faced with large-scale data sets, manual labeling and analysis is a tedious and time-consuming task. Therefore, automated and semi-automated analysis methods are considered to be the key to solving this problem. In recent years, artificial intelligence and machine learning techniques have been shown to improve the accuracy and efficiency of breast disease detection. However, to ensure the accuracy of diagnosis, large-scale medical image data sets still need to be combined with the professional knowledge of human experts, and therefore a human-computer collaborative double labeling method needs to be explored. SUMMARY

[0004] The present application provides a human-computer collaborative double labeling method to solve the technical problems of inaccurate prediction results of existing automatic labeling methods and low efficiency of manual labeling, while improving the detection effect, robustness and generalization ability of the model, and is suitable for improving the accuracy of diagnosing breast diseases.

[0005] To achieve the above-mentioned goal, the present application proposes a human-computer collaborative double labeling method, and is specifically applied to breast disease diagnosis, and the specific implementation steps are as follows:

[0006] Step one, use an automatic imaging system to generate images and divide the images into an unlabeled data set N and a labeled data set M. In the embodiment, an automatic breast volume ultrasound imaging (ABUS) is used to collect the breast images of patients, and the images are divided into an unlabeled data set N and a labeled data set M;

[0007] Step two, pre-process the labeled data set M; through image augmentation processing, increase the data set, and normalize the input data;

[0008] Step three, learn image representation through a self-supervised framework based on contrastive learning for the pre-processed image, and enhance its image features;

[0009] Step four, based on deep learning and transfer learning, learn the image whose features are enhanced in step three, use U-Net and Residual Network (ResNet) deep learning architecture to construct a visual model, automatically label the regions of interest in the image, and obtain an image dataset M; provide a preliminary estimate of breast diseases (e.g., lumps and lumps);

[0010] Step five, manually label the breast image automatically labeled in step four again, realize double labeling of man and machine, obtain the updated image dataset M, and provide a more accurate estimate of breast diseases;

[0011] Step six, learn image representation from the unlabeled dataset N using a contrastive learning algorithm, extract additional supervision signals such as disease labels or segmentation masks from the labeled dataset M, use a contrastive learning model to learn feature representation, and update the visual model in combination with the supervision signal.

[0012] The specific implementation steps of step one are as follows:

[0013] An automatic breast volume ultrasound imaging system is used, and ultrasonic coupling agent is evenly applied to one side of the breast. A disposable coupling film is covered on the scanner. The operator places the scanner on one side of the breast and gently presses to ensure contact with the skin of the breast. After the scanner is in place, the operator starts scanning. Each side of the breast is repeated three times in the AP, LAT, and MED quadrants to form a complete view of the entire breast. If necessary, additional views can be added as needed. The scanning time in one direction is about 60s, and the interlayer spacing of the collected images is 0.5mm. After scanning is completed, the data is transmitted to the workstation to obtain the automatically reconstructed transverse, coronal and sagittal images; the collected images are divided into an unlabeled dataset N and a labeled dataset M;

[0014] The specific implementation steps of step two are as follows:

[0015] First, through image augmentation processing, the images in the dataset M are rotated by 0°, 90°, 180° and 270° respectively, and then random horizontal or vertical flipping is applied on the images to increase the dataset; the augmented images are used as input;

[0016] Second, normalize the input data, and the conversion function is:

[0017] x new =(x-μ) / σ;

[0018] Where μ is the mean of the sample data, σ is the standard deviation of the sample data, and x newFor the normalized image data.

[0019] The third step is implemented as follows:

[0020] First, the image sequence where T represents the sequence length, D is the number of sequence layers, H and W are the length and width of a single sequence, respectively, data augmentation is performed to generate two different views, denoted as I_1 and I_2.

[0021] Second, a momentum encoder is introduced to provide contrastive supervision for online encoder learning discriminative representations. The enhanced complete view is used as input to preserve the semantic integrity and discriminability of the learned representations. Since the semantic labels of images are ambiguous, unlike the semantic-determined labels in natural language processing, it is logical to use global representations for contrastive learning. The formula is as follows:

[0022]

[0023] where is the input label of the momentum encoder, represents the momentum encoder, represents the output sequence of the momentum encoder, represents the output sequence after global pooling.

[0024] The momentum encoder updates the parameters through exponential moving average (EMA), which can facilitate smoother feature changes for more stable training, and and the parameters of s and θ t , respectively, the parameter update formula is as follows:

[0025] θ t ← μθ t + (1-μ)θ s

[0026] where μ is the moving average hyperparameter.

[0027] Third, in order to align with the output of the momentum encoder, a feature decoder is introduced. The feature decoder has the same structure as the pixel decoder, but it contains fewer transformer blocks and is more lightweight. Both serve different learning goals and do not share parameters. The feature decoder aims to decode the latent space feature z s to the momentum encoder feature space. Similar to the momentum encoder, the output of the feature decoder is also averaged and pooled to serve as the representation of the entire image. This feature is then used for contrastive learning, with the specific form as follows:

[0028]

[0029] Where N is the number of all markers in the set, including visible and invisible markers, is the feature decoder output.

[0030] Fourth, for the contrastive learning objective, an asymmetric contrastive loss function is designed based on InfoNCE, which aims to bring semantically consistent samples in a batch closer and push inconsistent samples away. In order to enhance the learning ability of the model, "projection prediction" and "projection" layers are attached to the feature decoder and momentum encoder respectively. Parameters are also updated by exponential sliding average. Specifically, the output y of the feature decoder is x It will be converted through the "projection prediction" structure Similarly, the output of the momentum encoder is represented by z t , apply the "projection" head to get Among them D p Represents the dimension after projection, and then calculates the cosine similarity ρ between the two:

[0031]

[0032] Next, ρ + It is expressed as the cosine similarity of the positive pair, which is given by and To construct, represents the cosine similarity of the jth negative pair, and is obtained by using different images in a batch through the momentum encoder Construct a negative sample pair, and the loss function is as follows:

[0033]

[0034] Where τ is the temperature coefficient and K represents the batch size.

[0035] Fifth, at the feature level, information transfer in two situations is achieved through self-distillation. The loss form is as follows:

[0036]

[0037] Among them L latent Represents the loss function value.

[0038] Sixth, the loss function of the entire framework can be expressed as:

[0039] L=L r +λ c L c +λ latent L latent

[0040] wherein λ c represents a contrast loss weight, λ latent represents a consistency loss weight.

[0041] The fourth step is specifically implemented as follows:

[0042] First, the visual model constructed includes a classification model ResNet50 and a segmentation model U-Net, which are pre-trained in ImageNet before being constructed.

[0043] Second, the structure of the U-Net network can be divided into two main parts: the contraction path and the expansion path. In the first part, i.e., the contraction path, each block is composed of two consecutive 3x3 convolution layers, followed by a ReLU activation unit and a max-pooling layer. This combination is repeated multiple times to form the architecture of the entire contraction path. The innovation of U-Net lies in the second part, i.e., the expansion path. In each stage of the expansion path, the feature map is upsampled by a 2x2 up-convolution. Then, a feature map of the same size is cropped from the corresponding contraction path layer and connected to the upsampled feature map. Subsequently, two consecutive 3x3 convolution layers and ReLU activation are performed. In the final stage, an additional 1x1 convolution is used to reduce the number of channels of the feature map to the required number, generating the final segmentation image.

[0044] The energy function of the network is given by:

[0045] E=∑ω(x)log(p k(x) (x))

[0046] where p k is a pixel-level SoftMax function applied to the final feature map, defined as:

[0047]

[0048] where a k represents the activation k in the channel.

[0049] Third, the ResNet50 uses a shortcut connection method that directly connects the output of the previous layer to the input of a certain layer in the following layers as a shortcut. Its input function is:

[0050] F(x)=H(x)-x

[0051] where F(x) is the residual, H(x) is the complete output of the previous layer, and x is the input of the previous layer.

[0052] During the training process, the loss function will swing after the parameter update, and the network optimizer can optimize the loss function swing amplitude, which can be specifically implemented by using the root mean square propagation algorithm (RMSProp), and the convergence speed of the function will be further accelerated. The learning rate is a hyperparameter that adjusts the network weights through the gradient of the loss function, and the stepLR decay strategy is adopted. The learning rate is set to 0.001, the maximum iteration step is set to 5000, and the learning rate is decayed by 1 / 2 at steps 2000 and 4000. Since the early layers of CNN are associated with specific image datasets very little, and the relationship becomes closer and closer to the selected dataset and its task target. Therefore, fine-tune from the last layer of ResNet50, and then include more and more layers in the update process.

[0053] The fifth step is specifically implemented as follows:

[0054] After the preliminary labeling of breast images by automatic labeling technology (such as visual models based on U-Net and ResNet50), a plurality of trained human experts manually label using a special labeling tool or software, and update the labeled data to the labeled data set M;

[0055] The sixth step is specifically implemented as follows:

[0056] First, use the contrast learning model to learn image feature representation from the unlabeled data set N, which is the same as step three;

[0057] Second, extract additional supervision signals from the updated labeled data set M, such as disease labels or segmentation masks;

[0058] Third, combine the contrast learning task with the supervision signal to form a multi-task loss function, as follows:

[0059] L = L c + λ s *L s

[0060] Where L c is the contrast loss, L s is the supervision loss, and λ s is the weight of the supervision loss.

[0061] Fourth, use the multi-task loss function to train the visual model, and update the parameters of the contrast learning and supervision learning modules, by minimizing the multi-task loss function, the visual model can learn meaningful feature representation and task-specific knowledge at the same time, thereby improving its generalization ability and accuracy.

[0062] The characteristics and effective effects of the present application are:

[0063] The present application ingeniously combines the essence of automatic labeling and manual labeling, improves the recognizability of images, significantly improves the accuracy of breast disease diagnosis in practical application, and effectively reduces the risk of missed diagnosis and misdiagnosis. At the same time, by efficiently using the contrast learning algorithm, the present application enhances the expressiveness of image features and further enhances the generalization ability of the visual model. On this basis, the present application ingeniously combines advanced deep learning architectures such as U-Net and ResNet50, and is supplemented by transfer learning technology, so that the system can efficiently learn and identify key regions in images, such as key regions in breast images in the embodiments of the present application, greatly reducing the time and effort required for manual labeling, thereby greatly improving the diagnosis efficiency and providing strong technical support for early screening of breast diseases. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 is a flowchart of an embodiment of the present application;

[0065] Figure 2 is a whole U-net architecture;

[0066] Figure 3 is a ResNet-50 structure diagram;

[0067] Figure 4 is a Chinese translation version of the ResNet-50 structure diagram. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described clearly and completely in combination with specific embodiments of the present application and corresponding drawings. It should be understood that the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0069] As shown in one embodiment, Figure 1 the present application proposes a human-computer collaborative double labeling method, and is specifically applied to breast disease diagnosis, and the specific implementation steps are as follows:

[0070] Step one, generate images using an automatic imaging system and divide the images into an unlabeled dataset N and a labeled dataset M. In the embodiment, the breast images of the patients are collected using an automatic breast ultrasound system (ABUS) and divided into an unlabeled dataset N and a labeled dataset M. A new type of breast three-dimensional ultrasound technology is adopted, which can comprehensively observe the internal structure of the breast, overcome the problems of small area of handheld ultrasound, no coronal plane image, and dependence on operator experience, and the specific steps are as follows:

[0071] First, use the automatic breast volume ultrasound imaging system to evenly apply ultrasound coupling agent to one side of the breast, cover the scanner with a disposable coupling film, and press the scanner against the breast skin to ensure contact.

[0072] Second, after the scanner is in place, the operator starts scanning, repeating three times for each side of the breast in the AP, LAT, and MED quadrants to form a complete view of the entire breast. If necessary, additional views can be added. The scanning time for one direction is about 60s, and the interlayer spacing of the collected images is 0.5mm.

[0073] Third, after scanning is complete, the data is transmitted to the workstation to obtain the automatically reconstructed transverse, coronal, and sagittal images. The collected images are divided into an unlabeled dataset N and a labeled dataset M.

[0074] Step two, preprocess the labeled dataset M; through image augmentation processing, increase the sample size while reducing the model's dependence on certain properties, thereby improving the generalization ability and preventing model overfitting. The specific methods include vertical flipping, horizontal flipping, image rotation transformation, and transposition. In addition, normalize the input data to improve the convergence speed of the model, improve the accuracy of the model, and avoid the influence of scale differences between different features on model training. The specific steps are as follows:

[0075] First, through image augmentation processing, rotate the images in dataset M by 0°, 90°, 180°, and 270°, and then apply horizontal flipping or vertical flipping on the images to increase the dataset. Use the augmented images as input;

[0076] Second, normalize the input data, and the transformation function is:

[0077] x new =(x-μ) / σ

[0078] Where μ is the mean of the sample data, σ is the standard deviation of the sample data, and x new is the normalized image data.

[0079] Step 3: Use a self-supervised framework based on contrastive learning to learn image representations for the preprocessed images. This self-supervised learning framework is a method for learning image representations that uses large-scale data without manual annotation for self-training. This method can learn features with good representational power by comparing image samples with their corresponding transformed versions during training. The specific steps for enhancing image features are as follows:

[0080] First, for the image sequence Where T represents the sequence length, D is the number of sequence layers, H and W are the length and width of a single sequence, and data augmentation operations are performed to generate two different views, denoted as I_1 and I_2.

[0081] Second, a momentum encoder is introduced to provide contrastive supervision for the online encoder to learn discriminative representations. The augmented full view is used as input to preserve the semantic integrity and discriminability of the learned representation. Since the semantics of image labels are ambiguous, unlike the semantically determined labels in natural language processing, it is logical to use a global representation for contrastive learning. The formula is as follows:

[0082]

[0083] in is the input flag of the momentum encoder, represents the momentum encoder, represents the momentum encoder output sequence, Represents the output sequence Features after global pooling.

[0084] Third, the momentum encoder updates parameters via exponential moving average (EMA), which can promote smooth feature changes for more stable training. and The parameters are denoted as θ s and θ t , then the parameter update formula is as follows:

[0085] θ t ←μθ t +(1-μ)θ s

[0086] where μ is the sliding average hyperparameter.

[0087] Third, to align with the output of momentum encoder, a feature decoder is introduced. The feature decoder has the same structure as the pixel decoder, but it contains fewer transformer blocks and is more lightweight. The two serve different learning goals and do not share parameters. The feature decoder aims to decode latent space features z s to the momentum encoder feature space. Similar to the momentum encoder, the output of the feature decoder is also averaged-pooled to represent the whole image. Then, the feature is used for contrastive learning, which is as follows:

[0088]

[0089] where N represents the number of all labels in the set, including visible and invisible labels, is the output of the feature decoder.

[0090] Fourth, for the contrastive learning goal, an asymmetric contrastive loss function is designed based on InfoNCE. The loss aims to pull together samples with consistent semantics in a batch and push away inconsistent samples. To enhance the learning ability of the model, a "projection prediction" layer and a "projection" layer are added to the feature decoder and the momentum encoder, respectively. The projection head of the momentum encoder is also updated by exponential moving average. Specifically, the output y s of the feature decoder is converted by the "projection prediction" structure to get Similarly, for the output z t of the momentum encoder, the "projection" head is applied to get where D p represents the dimension of the projection, and then the cosine similarity ρ between them is calculated:

[0091]

[0092] Then, ρ + is expressed as a positive cosine similarity, which is constructed by and , where ρ represents the cosine similarity of the jth negative pair, and the of different images in a batch are used to construct negative sample pairs. The loss function is as follows:

[0093]

[0094] where τ is the temperature coefficient and K represents the batch size.

[0095] Fifth, information transmission is achieved in two scenarios at the feature level through self-distillation. The loss form is as follows:

[0096]

[0097] Among them L latent Represents the loss function value.

[0098] Sixth, the loss function of the entire framework can be expressed as:

[0099] L=L r +λ c L c +λ latent L latent

[0100] where λ c represents the contrast loss weight, λ latent represents the consistency loss weight.

[0101] Step 4: Based on the image tile after feature enhancement in step 3, we learn it based on deep learning combined with transfer learning, build a visual model using the U-Net and residual network (ResNet) deep learning architecture, automatically mark the regions of interest in the image, and obtain an image dataset N; provide a preliminary estimate of breast diseases (e.g., lumps and masses). Figure 2 The UNet shown in the figure is a symmetrical image segmentation algorithm based on convolutional neural networks. Its original design was to solve biomedical image problems. Due to its outstanding effect, it is widely used in various directions of semantic segmentation. Figure 3 As shown in the figure, Res-Net50 is a residual learning framework proposed based on the existing deep network training, which has the advantages of easy optimization and low computational burden. The specific steps are as follows:

[0102] First, the constructed visual models include the classification model ResNet50 and the segmentation model U-Net, which are pre-trained on ImageNet before construction;

[0103] Second, the structure of the U-Net network can be divided into two main parts: the contraction path and the expansion path. In the first part, the contraction path, each block consists of two consecutive 3×3 convolutional layers, followed by a ReLU activation unit and a max-pooling layer. This combination is repeated multiple times to form the entire architecture of the contraction path. The innovation of U-Net lies in the second part, the expansion path. At each stage of the expansion path, the feature map is upsampled through a 2×2 upconvolution. Then, a feature map of the same size is cropped from the corresponding contraction path layer and concatenated to the upsampled feature map. This is followed by two consecutive 3×3 convolutional layers and ReLU activation. In the final stage, the number of channels of the feature map is reduced to the required number through an additional 1×1 convolution to generate the final segmentation map.

[0104] The energy function of the network is given by:

[0105] E =∑ω(x)log(p k(x) (x))

[0106] where p k is a pixel-wise SoftMax function applied to the final feature map, defined as:

[0107]

[0108] where a k denotes the activation k in the channel.

[0109] Third, ResNet50 uses a shortcut connection method that directly crosses the output of the previous layer as the input of a certain layer in the future, and its input function is:

[0110] F(x) = H(x) - x

[0111] where F(x) is the residual, H(x) is the complete output of the previous layer, and x is the input of the previous layer.

[0112] During the training process, the loss function will swing after the parameter update, and the network optimizer can optimize the loss function swing amplitude, which can be specifically implemented by using the root mean square propagation algorithm (RMSProp), and the convergence speed of the function will be further accelerated. The learning rate is a hyperparameter that adjusts the network weights through the gradient of the loss function, and the stepLR decay strategy is adopted. The learning rate is set to 0.001, the maximum iteration step is set to 5000, and the learning rate is decayed by 1 / 2 at steps 2000 and 4000. Since the early layers of CNN are associated with specific image datasets very little, and the relationship becomes more and more close to the selected dataset and its task target. Therefore, fine-tune from the last layer of ResNet50, and then include more and more layers in the update process.

[0113] Step five, the breast images automatically labeled in step four are manually labeled again to achieve double labeling of man and machine, and get the updated image dataset M, which provides more accurate estimates of breast diseases while reducing the workload of doctors;

[0114] After the breast images are preliminarily labeled by automatic labeling technology (such as visual models based on U-Net and ResNet50), a number of trained human experts use specialized labeling tools or software for manual labeling, and update the labeled data to the labeling dataset M.

[0115] Step six, using a contrastive learning algorithm to learn image representations from the unlabeled dataset N, extracting additional supervision signals such as disease labels or segmentation masks from the labeled dataset M, using a contrastive learning model to learn feature representations, and combining the supervision signals to update the visual model to improve the feasibility of the model and further optimize the model.

[0116] First, use the contrastive learning model to learn image feature representations from the unlabeled dataset N, which is the same as step three;

[0117] Second, extract additional supervision signals such as disease labels or segmentation masks from the updated labeled dataset M, and update the visual model to better understand and recognize breast diseases;

[0118] Third, combine the contrastive learning task with the supervision signal to form a multi-task loss function as follows:

[0119] L = L c + λ s · L s

[0120] Where L c is the contrastive loss, L s is the supervision loss, and λ s is the weight of the supervision loss.

[0121] Fourth, use the multi-task loss function to train the visual model, and update the parameters of the contrastive learning and supervision learning modules, by minimizing the multi-task loss function, the visual model can learn meaningful feature representations and task-specific knowledge at the same time, thereby improving its generalization ability and accuracy. The formula for minimizing the multi-task loss function is as follows:

[0122] L_multi = α * L_cls + β * L_seg

[0123] Where L_cls is the loss function for the disease classification task, L_seg is the loss function for the segmentation task, and α and β are the weight coefficients corresponding to them.

[0124] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the examples, those skilled in the art should understand that modifications or equivalent replacements to the technical solutions of the present application do not deviate from the spirit and scope of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A method for human-in-the-loop dual annotation, characterized in that, Comprise the following steps: Step one: generate images using an automatic imaging system and divide the images into an unlabeled data set N and a labeled data set M; Step two: pre-process the labeled data set M; Through image augmentation processing, increase the data set, normalize the input data; Step three: learn image representation through a self-supervised framework based on contrastive learning for the pre-processed images, and enhance their image features; Step four: based on deep learning combined with transfer learning, learn the images after feature enhancement in step three, use U-Net and residual network deep learning architecture to build a visual model, automatically label the regions of interest in the images, and obtain the image data set M; Step five: manually label the images labeled automatically in step four again to obtain the updated image data set M; Step six: learn image representation from the unlabeled data set N using a contrastive learning algorithm, extract additional supervision signals from the labeled data set M, learn feature representation using a contrastive learning model, and update the visual model combined with the supervision signal; The step three comprises the following specific steps: First, take data enhancement operation on the image sequence, generate two different views, denoted as I_1 and I_2; Second, introduce a momentum encoder to provide contrastive supervision for online encoder learning discriminative representation, and use the enhanced complete view as input; Third, introduce a feature decoder to perform average pooling calculation as the representation of the entire image; Fourth, respectively attach "projection prediction" and "projection" layers to the feature decoder and momentum encoder; Fifth, realize information transmission in two cases through self-distillation at the feature level; The step six comprises the following specific steps: First, learn image feature representation from the unlabeled data set N using a contrastive learning model, which is the same as step three; Second, extract additional supervision signals from the updated labeled data set M; Third, combine the contrastive learning task with the supervision signal to form a multi-task loss function; Fourth, use the multi-task loss function to train the visual model, and update the parameters of the contrastive learning and supervised learning modules.

2. The method of claim 1, wherein, The step one includes the following specific steps when using automatic breast volume ultrasound imaging: First, evenly apply ultrasound coupling agent to one side of the breast, cover the scanner with a disposable coupling membrane, and press the scanner against the breast skin to ensure contact; Second, start scanning, repeat three times for each side of the breast, form a complete view of the entire breast, the scanning time in one direction is 60 seconds, and the interlayer distance of the collected images is 0.5mm; Third, after scanning, transfer the data to the workstation to obtain transverse, coronal and sagittal images, and divide the collected images into an unlabeled data set N and a labeled data set M.

3. The method of claim 1, wherein, The step two comprises the following specific steps: First, through image augmentation processing, rotate the images in the data set M by 0°, 90°, 180° and 270°, and then randomly apply horizontal or vertical flipping to the images to increase the data set, and use the augmented images as input; Second, normalize the input data.

4. The method of claim 1, wherein, The step four includes the following specific steps: First, the constructed visual model includes a classification model ResNet50 and a segmentation model U-Net, and before construction, pre-training is performed in ImageNet; Second, the structure of the U-Net network includes: a contraction path and an expansion path; each block in the contraction path is composed of two consecutive 3*3 convolution layers, followed by a ReLU activation unit and a max-pooling layer; in each stage of the expansion path, the feature map is up-sampled by 2*2 up-convolution; then, the same size feature map is cropped from the corresponding contraction path layer and connected to the up-sampled feature map; followed by two consecutive 3*3 convolution layers and ReLU activation; in the final stage, the number of channels of the feature map is reduced to the required number by an additional 1*1 convolution, generating the final segmentation image; Third, ResNet50 is connected by directly crossing several layers as a shortcut link to the input of a certain layer in the next layer.

5. The method of claim 1, wherein, The step five includes the following specific steps: After the image is preliminarily labeled by the automatic labeling technology, a plurality of trained human experts use a special labeling tool or software to manually label, and update the labeled data to the labeling data set M.

Images