Model training method and device, image segmentation method and device and electronic equipment

Generating pseudo-labels and multi-task learning through teacher models improves the efficiency and accuracy of student networks in cross-domain training, solves the problem of low cross-domain training efficiency in the existing technology, and achieves efficient segmentation of target domain ultrasound images.

CN119068289AInactive Publication Date: 2024-12-03SHANGHAI FOURTH PEOPLES HOSPITAL
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411193823.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing neural network models are inefficient when training across domains, especially when segmenting ultrasound images of untrained target domains, and require a lot of time-labelled labels and model training.

Method used

The teacher model is used to generate pseudo-labels to train students' networks, improve the generalization ability of the student model through multi-task learning, reduce training time, and reduce dependence on manual annotation through pseudo-labels.

Benefits of technology

It significantly improves the cross-domain adaptive training efficiency and image segmentation accuracy of the network model on the target domain, reduces training time, and reduces the dependence on manual annotation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068289A_ABST
    Figure CN119068289A_ABST
Patent Text Reader

Abstract

The invention provides a model training method and device, an image segmentation method and device and electronic equipment, and aims to solve the problem of low efficiency of cross-domain training of a network model. The method comprises the steps of obtaining a sample data set, wherein the sample data set comprises a source domain data set and a target domain data set; training a teacher network by using the source domain data set to obtain a teacher model; processing the target domain ultrasonic image in the target domain data set by using a teacher model to obtain a pseudo tag; processing the target domain ultrasonic image in the target domain data set by using a student network to obtain a processing result; and training the student network according to the pseudo tag and the processing result to obtain a student model which is used for processing the image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of deep learning and image segmentation, and specifically, to a model training method, an image segmentation method, a device and an electronic device. Background Art

[0002] The current neural network model segments the target tissue ultrasound image or the regional image on the organ ultrasound image (such as the liver or lungs, etc.) mainly by using a manual feature extraction algorithm. The main process of the algorithm includes: first, the doctor selects the region of interest from the acquired image, and then extracts and segments the region of interest that may contain the target tissue. In the specific practice process, it is found that the manual feature extraction algorithm has a higher accuracy rate for segmenting ultrasound images that have been trained in specific fields (such as the liver or lungs), while the accuracy rate for segmenting ultrasound images of untrained target fields (such as peritoneal effusion) is lower. If the model is retrained using ultrasound images of peritoneal effusion, it will take a lot of time to label, train and fine-tune the model. Therefore, the efficiency of cross-domain training of the current network model is low. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a model training method, an image segmentation method, an apparatus and an electronic device for improving the problem of low efficiency in cross-domain training of network models.

[0004] The embodiment of the present application provides a model training method, including: obtaining a sample data set, the sample data set including: a source domain data set and a target domain data set; using the source domain data set to train a teacher network to obtain a teacher model; using the teacher model to process the target domain ultrasound image in the target domain data set to obtain a pseudo label; using the student network to process the target domain ultrasound image in the target domain data set to obtain a processing result; training the student network according to the pseudo label and the processing result to obtain a student model, the student model is used to process the image. In the implementation process of the above scheme, the student network is trained by using the pseudo label generated by the teacher model according to the target domain ultrasound image in the target domain data set. Since the teacher model can capture more features and patterns, the pseudo label generated by the teacher model can provide richer and more accurate information, helping the student network to learn a more general feature representation, so that the student network can be trained on the target domain data set without labels, which helps to improve the generalization ability of the student model in the target domain, greatly reduces the training time of the student network, and ultimately improves the efficiency of the cross-domain adaptive training of the network model.

[0005] Optionally, in an embodiment of the present application, the source domain data set includes: source domain ultrasound images and source domain segmentation labels; the source domain data set is used to train the teacher network, including: body landmark detection on the source domain ultrasound image to obtain a source domain anatomical structure sub-map; feature extraction of the source domain anatomical structure sub-map using the teacher network to obtain source domain features, and image segmentation of the source domain ultrasound image based on the source domain features to obtain a teacher segmented image; calculating a first segmentation loss between the teacher segmented image and the source domain segmentation label; and using the first segmentation loss to update the weight parameters of the teacher network. In the implementation process of the above scheme, by performing body landmark detection and feature extraction on the source domain, the teacher network can learn more representative anatomical structure features, which not only help to accurately segment on the source domain, but also help the model maintain a high segmentation accuracy when encountering target domain data with similar structures but slight differences.

[0006] Optionally, in an embodiment of the present application, the source domain data set also includes: source domain classification labels and source domain domain labels; using the source domain data set to train the teacher network also includes: using the teacher network to classify and predict the teacher segmented image to obtain a first organ classification result; calculating the first organ classification loss between the first organ classification result and the source domain classification label; using the teacher network to perform domain prediction on the source domain features to obtain a first domain classification result; calculating the first domain classification loss between the first domain classification result and the source domain label; using the first organ classification loss and the first domain classification loss to update the weight parameters of the teacher network. In the implementation process of the above scheme, the teacher network is trained through a multi-task learning (MTL) strategy of image segmentation tasks, organ classification tasks and domain classification tasks, so that the trained teacher model better understands the characteristics of different organ tissues (including organs and peritoneal effusions), and effectively enhances the adaptability of the teacher model between different fields. Among them, the multi-task learning (MTL) strategy means that the model is trained to complete multiple related tasks at the same time. Usually, there is some form of synergy between these tasks (shared feature representation). Specifically in this scheme, both the teacher network and the student network can perform segmentation tasks and classification tasks at the same time. The classification task is based on the segmentation task, that is, the input of the classification task is the output result of the segmentation task.

[0007] Optionally, in an embodiment of the present application, a teacher model is used to process a target domain ultrasound image in a target domain data set, including: performing body landmark detection on the target domain ultrasound image to obtain a target domain anatomical structure subgraph; using the teacher model to perform feature extraction on the target domain anatomical structure subgraph to obtain a first target domain feature, and performing image segmentation on the target domain ultrasound image according to the first target domain feature to obtain a pseudo label. In the implementation process of the above scheme, by generating pseudo labels on the target domain image, the reliance on expensive and time-consuming manual annotation can be significantly reduced, which is particularly important for ultrasound image analysis because professional doctors have limited time and resources, and automatic pseudo label generation can accelerate the model training and evaluation process, and help solve the domain adaptation problem, that is, the performance degradation of the model between different data sets (such as ultrasound images in different devices and different hospital environments), which enables the model to effectively segment the target domain data even when there is no or very little labeled data.

[0008] Optionally, in an embodiment of the present application, the student network is trained according to the pseudo-label and the processing result, including: using the feature extractor in the student network to extract features from the target domain anatomical structure sub-graph to obtain the second target domain feature, and performing image segmentation on the target domain ultrasound image according to the second target domain feature to obtain the student segmented image; calculating the second segmentation loss between the student segmented image and the pseudo-label; using the second segmentation loss to update the weight parameters of the student network. In the implementation process of the above scheme, the weight parameters of the student network are updated by the second segmentation loss between the student segmented image and the pseudo-label generated by the teacher model. This teacher-student network architecture and pseudo-label generation technology help the student network to quickly learn across domains and continuously train and optimize, so that it can achieve good performance in the target domain, effectively improving the image segmentation accuracy on the target domain ultrasound image.

[0009] Optionally, in an embodiment of the present application, the target domain data set also includes: a target domain classification label and a target domain domain label; training the student network according to the pseudo-label and the processing result, and also includes: using the task classifier in the student network to classify and predict the student segmented image to obtain a second organ classification result; calculating the second organ classification loss between the second organ classification result and the target domain classification label; using the domain classifier in the student network to perform domain prediction on the second target domain feature to obtain a second domain classification result; calculating the second domain classification loss between the second domain classification result and the target domain label; using the second organ classification loss and the second domain classification loss to update the weight parameters of the student network. In the implementation process of the above scheme, the student network is trained by a multi-task learning method of image segmentation tasks, organ classification tasks and domain classification tasks, so that the trained student model can better understand the characteristics of different organ tissues (including organs and peritoneal effusions), and effectively enhance the adaptability of the student model between different fields.

[0010] Optionally, in an embodiment of the present application, after obtaining the student model, it also includes: performing body landmark detection on the target domain ultrasound image to be processed to obtain an anatomical structure sub-graph; using the student model to perform feature extraction on the anatomical structure sub-graph to obtain ultrasound image features; using the student model to perform image segmentation on the ultrasound image features to obtain image segmentation results. In the implementation process of the above scheme, by first using the student model to perform feature extraction on the anatomical structure sub-graph detected from the target domain ultrasound image to be processed to obtain ultrasound image features, and then using the trained student model to perform image segmentation on the ultrasound image features, the efficiency and accuracy of ultrasound image segmentation can be improved.

[0011] The embodiment of the present application also provides an image segmentation method, including: performing body landmark detection on the target domain ultrasound image to be processed to obtain an anatomical structure sub-graph; using a student model to extract features from the anatomical structure sub-graph to obtain ultrasound image features; using any of the above student models to perform image segmentation on the ultrasound image features to obtain image segmentation results. In the implementation of the above scheme, by first using a student model to extract features from the anatomical structure sub-graph detected from the target domain ultrasound image to be processed to obtain ultrasound image features, and then using a trained student model to perform image segmentation on the ultrasound image features, the efficiency and accuracy of ultrasound image segmentation can be improved.

[0012] The embodiment of the present application also provides a model training device, including: a sample data acquisition module, used to obtain a sample data set, the sample data set includes: a source domain data set and a target domain data set; a teacher model acquisition module, used to use the source domain data set to train a teacher network to obtain a teacher model; an ultrasound image processing module, used to use the teacher model to process the target domain ultrasound image in the target domain data set to obtain a pseudo label; a processing result acquisition module, used to use the student network to process the target domain ultrasound image in the target domain data set to obtain a processing result; a student model training module, used to train the student network according to the pseudo label and the processing result to obtain a student model, and the student model is used to process the image.

[0013] Optionally, in an embodiment of the present application, the source domain data set includes: a source domain ultrasound image and a source domain segmentation label; a teacher model acquisition module, including: a source domain structure subgraph acquisition submodule, used to perform body landmark detection on the source domain ultrasound image to obtain a source domain anatomical structure subgraph; a feature extraction image segmentation submodule, used to use the teacher network to perform feature extraction on the source domain anatomical structure subgraph to obtain source domain features, and perform image segmentation on the source domain ultrasound image according to the source domain features to obtain a teacher segmented image; a first segmentation loss calculation submodule, used to calculate the first segmentation loss between the teacher segmentation image and the source domain segmentation label; a teacher network weight update submodule, used to update the weight parameters of the teacher network using the first segmentation loss.

[0014] Optionally, in an embodiment of the present application, the source domain dataset also includes: a source domain classification label and a source domain domain label; the teacher model acquisition module also includes: a first organ classification acquisition submodule, which is used to use the teacher network to perform classification prediction on the teacher segmentation image to obtain a first organ classification result; a first organ loss calculation submodule, which is used to calculate the first organ classification loss between the first organ classification result and the source domain classification label; a first domain result acquisition submodule, which is used to use the teacher network to perform domain prediction on the source domain features to obtain a first domain classification result; a first domain loss calculation submodule, which is used to calculate the first domain classification loss between the first domain classification result and the source domain label; a teacher network parameter update submodule, which is used to update the weight parameters of the teacher network using the first organ classification loss and the first domain classification loss.

[0015] Optionally, in an embodiment of the present application, the ultrasound image processing module includes: a target structure sub-graph acquisition sub-module, used to perform body landmark detection on the target domain ultrasound image to obtain a target domain anatomical structure sub-graph; a target feature extraction and segmentation sub-module, used to use a teacher model to perform feature extraction on the target domain anatomical structure sub-graph to obtain a first target domain feature, and perform image segmentation on the target domain ultrasound image according to the first target domain feature to obtain a pseudo label.

[0016] Optionally, in an embodiment of the present application, the student model training module includes: a target extraction image segmentation submodule, which is used to use the feature extractor in the student network to perform feature extraction on the target domain anatomical structure subimage to obtain a second target domain feature, and perform image segmentation on the target domain ultrasound image according to the second target domain feature to obtain a student segmented image; a second segmentation loss calculation submodule, which is used to calculate the second segmentation loss between the student segmented image and the pseudo label; and a student network weight update submodule, which is used to update the weight parameters of the student network using the second segmentation loss.

[0017] Optionally, in an embodiment of the present application, the target domain data set also includes: a target domain classification label and a target domain domain label; the student model training module also includes: a second organ classification acquisition submodule, which is used to use the task classifier in the student network to classify and predict the student segmented image to obtain a second organ classification result; a second classification loss calculation acquisition submodule, which is used to calculate the second organ classification loss between the second organ classification result and the target domain classification label; a second domain classification acquisition submodule, which is used to use the domain classifier in the student network to perform domain prediction on the second target domain feature to obtain a second domain classification result; a second domain loss calculation submodule, which is used to calculate the second domain classification loss between the second domain classification result and the target domain label; a student network parameter update submodule, which is used to update the weight parameters of the student network using the second organ classification loss and the second domain classification loss.

[0018] Optionally, in an embodiment of the present application, the model training device also includes: an anatomical structure sub-graph acquisition module, which is used to perform body landmark detection on the target domain ultrasound image to be processed to obtain an anatomical structure sub-graph; an ultrasound image feature acquisition module, which is used to use the student model to extract features of the anatomical structure sub-graph to obtain ultrasound image features; and an image segmentation result acquisition module, which is used to use the student model to perform image segmentation on the ultrasound image features to obtain image segmentation results.

[0019] The embodiment of the present application also provides an image segmentation device, including: an image body landmark detection module, used to perform body landmark detection on the target domain ultrasound image to be processed to obtain an anatomical structure sub-image; a sub-image feature extraction module, used to use a student model to extract features of the anatomical structure sub-image to obtain ultrasound image features; a feature image segmentation module, used to use any student model of the above-mentioned scheme to perform image segmentation on the ultrasound image features to obtain an image segmentation result.

[0020] An embodiment of the present application further provides an electronic device, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions execute the method described above when executed by the processor.

[0021] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is executed.

[0022] The embodiment of the present application also provides a computer program product, including: a computer program or a computer instruction, and the computer program or the computer instruction executes the method described above when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A flow chart of a model training method provided in an embodiment of the present application is shown;

[0025] Figure 2 A schematic diagram of the process flow of the image segmentation method provided in an embodiment of the present application is shown;

[0026] Figure 3 A schematic diagram of the structure of a model training device provided in an embodiment of the present application is shown;

[0027] Figure 4 A schematic diagram of the structure of an image segmentation device provided in an embodiment of the present application is shown;

[0028] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. It should be understood that the drawings in the embodiment of the present application only serve the purpose of explanation and description, and are not used to limit the protection scope of the embodiment of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in the embodiment of the present application shows the operation implemented according to some embodiments of the embodiment of the present application. It should be understood that the operation of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the embodiment of the present application, or remove one or more operations from the flowchart.

[0030] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments. The components of the embodiments of the present application described and shown in the drawings generally here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the embodiments of the present application claimed, but merely represents the selected embodiments of the embodiments of the present application.

[0031] It is understandable that the "first" and "second" in the embodiments of the present application are used to distinguish similar objects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not necessarily limit the difference. In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that there may be three relationships, such as A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship. The term "multiple" refers to more than two (including two), and similarly, "multiple groups" refers to more than two groups (including two groups).

[0032] It should be noted that the model training method provided in the embodiment of the present application can be executed by an electronic device, where the electronic device refers to a device terminal or server with the function of executing a computer program, such as a smart phone, a personal computer, a tablet computer, a personal digital assistant, or a mobile Internet device. A server refers to a device that provides computing services through a network, such as an x86 server and a non-x86 server, and non-x86 servers include mainframes, minicomputers, and UNIX servers.

[0033] The following is an example of an application scenario in which the model training method is applicable: In ultrasound image segmentation tasks, most of the current neural network models are models that perform image segmentation on organs (such as the liver or lungs, etc.) themselves, but there are almost no neural network models that perform image segmentation on peritoneal effusions. As peritoneal effusions secondary to closed injuries to abdominal organs or other serious diseases tend to accumulate in the lowest part of the abdominal cavity, accurately identifying and segmenting the image of peritoneal effusions in the lowest part of the hepatorenal recess in the supine position is crucial for the assessment of the injury of critically ill patients and is an important indicator for determining whether there is combined abdominal organ injury. However, in practice, it was found that models that perform image segmentation on the organs themselves find it difficult to identify and segment peritoneal effusions.

[0034] Current artificial intelligence faces many challenges in the ultrasound image segmentation of peritoneal effusions, including inconsistent image quality, differences in the shape and size of effusions, artifacts and scattering interference, and structural complexity. In addition, the model for image segmentation of organs itself is difficult to complete the image segmentation of peritoneal effusions across different fields. Training a neural network model for the image segmentation task of peritoneal effusions from scratch is very time-consuming and labor-intensive. It requires collecting training images and spending a lot of time labeling. The process of model training and fine-tuning also takes a lot of time and computing power. Therefore, the efficiency of cross-domain training of current network models is low.

[0035] To improve the above problems, see Figure 1 The flow chart of the model training method provided by the embodiment of the present application is shown; the main idea of ​​the model training method is to generate pseudo labels through a teacher model that can capture more features and patterns. Such pseudo labels can provide richer and more accurate information, helping the student network to learn more general feature representations, so that the student network can be trained on an unlabeled target domain data set, which helps to improve the generalization ability of the student model in the target domain, greatly reduces the training time of the student network, and ultimately improves the efficiency of the cross-domain adaptive training of the network model. The implementation methods of the above-mentioned model training method may include:

[0036] Step S110: obtaining a sample data set, where the sample data set includes: a source domain data set and a target domain data set.

[0037] The sample data set mentioned above may include a source domain data set and a target domain data set, wherein the data may refer to ultrasound images, and may also include segmentation labels, classification labels, domain labels, etc. as label data for training models.

[0038] Step S120: Use the source domain dataset to train the teacher network to obtain a teacher model.

[0039] Since the above-mentioned source domain dataset may include: source domain ultrasound images and source domain segmentation labels, and may also include: source domain classification labels and source domain field labels and other situations, these situations will be described in detail below.

[0040] Step S130: Use the teacher model to process the target domain ultrasound image in the target domain dataset to obtain a pseudo label.

[0041] The above-mentioned teacher network is a deep learning model, which can be used to segment target area images (such as organ area images or peritoneal effusion images) from ultrasound images. The backbone network of the teacher network can adopt an encoder network.

[0042] Step S140: Use the student network to process the target domain ultrasound image in the target domain dataset to obtain a processing result.

[0043] Step S150: Train the student network according to the pseudo labels and the processing results to obtain a student model, which is used to process the image.

[0044] In the implementation process of the above scheme, the student network is trained by using the pseudo labels generated by the teacher model according to the target domain ultrasound images in the target domain dataset. Since the teacher model can capture more features and patterns, the pseudo labels generated by the teacher model can provide richer and more accurate information, helping the student network to learn more general feature representations, so that the student network can be trained on the unlabeled target domain dataset, which helps to improve the generalization ability of the student model in the target domain, greatly reduces the training time of the student network, and ultimately improves the efficiency of the cross-domain adaptive training of the network model.

[0045] As an optional implementation of the above step S120, the above source domain data set may include: a source domain ultrasound image and a source domain segmentation label; the above implementation of using the source domain data set to train the teacher network may include:

[0046] Step S121: perform body landmark detection on the source domain ultrasound image to obtain a source domain anatomical structure sub-image.

[0047] The above-mentioned body marker detection is the process of automatically or semi-automatically identifying and locating specific anatomical landmarks (also known as "landmarks" or "body markers") in medical images such as ultrasound images. These anatomical landmarks are usually structures with unique morphological or positional characteristics, such as: specific parts of bones, intersections of blood vessels, morphological boundaries of organs, etc. This step is crucial for subsequent image segmentation, because body markers provide relatively stable and easy-to-identify anatomical reference points in medical images, effectively reducing segmentation errors caused by inaccurate anatomical structure recognition, thereby helping to improve segmentation accuracy.

[0048] An implementation example of the above step S121 is: using a deep learning model to perform body landmark detection on a source domain ultrasound image to obtain a source domain anatomical structure sub-image. The deep learning model here includes but is not limited to a model with a convolutional neural network (CNN) architecture or a model with a residual network architecture (for example, a model using ResNet50 as the backbone network of the detection network).

[0049] Step S122: Use the teacher network to extract features from the source domain anatomical structure sub-image to obtain source domain features, and perform image segmentation on the source domain ultrasound image according to the source domain features to obtain a teacher segmentation image.

[0050] The implementation method of the above step S122 is, for example: using the feature extractor in the teacher network to extract features from the source domain anatomical structure sub-image to obtain source domain features, and using the image segmenter in the teacher network to segment the source domain ultrasound image according to the source domain features to obtain the teacher segmented image. Among them, the above feature extractor can adopt a visual Transformer (ViT) model, a contrastive language-image pre-training (CLIP) model or a U-Net encoder partial network, etc.; the above image segmentor can adopt a U-Net model, a DeepLab model, a PSPNet model or a RefineNet model.

[0051] Step S123: Calculate the first segmentation loss between the teacher segmentation image and the source domain segmentation label.

[0052] The above segmentation loss is the key difference indicator for evaluating the segmentation results predicted by the teacher model (i.e., the teacher segmentation image) and the actual source domain segmentation labels. By minimizing the segmentation loss, the teacher model can continuously improve its segmentation accuracy.

[0053] Step S124: Update the weight parameters of the teacher network using the first segmentation loss.

[0054] The implementation method of the above steps S123 to S124 is, for example: using the formula Calculate the first segmentation loss between the teacher segmentation image and the source domain segmentation label; where L s Represents the first segmentation loss, X represents the above-mentioned teacher segmentation image, that is, the image segmentation result predicted by the teacher network, and Y represents the source domain segmentation label. Then, the weight parameters of the teacher network are updated using the first segmentation loss until the weight parameters of the teacher network converge. By dynamically updating the weight parameters of the teacher network, the model can adaptively adjust its internal representation to better match the anatomical structure features of the source domain and the potential target domain. This adaptability enables the model to maintain good segmentation performance when facing ultrasound images collected under different conditions or devices.

[0055] In the implementation of the above scheme, by performing body landmark detection and feature extraction in the source domain, the teacher network is able to learn more representative anatomical structure features, which not only help with accurate segmentation in the source domain, but also help the model maintain high segmentation accuracy when encountering target domain data with similar but slightly different structures.

[0056] As an optional implementation of the above step S120, the above source domain data set further includes: a source domain classification label and a source domain field label; the above implementation of using the source domain data set to train the teacher network also includes:

[0057] Step S125: Use the teacher network to perform classification prediction on the teacher segmented image to obtain a first organ classification result.

[0058] An implementation example of the above step S125 is: in the process of using the task classifier in the teacher network to classify and predict the teacher segmented image, the classification prompt technology can be used to classify and predict the teacher segmented image, so as to obtain the first organ classification result. Among them, the classification prompt technology helps the model better understand the characteristics of different tissues by providing preliminary guidance. The core idea of ​​the classification prompt is to use natural language descriptions to synthesize classification weights, so that the model can be applied to new classification tasks without additional training. It is mainly used to guide the pre-trained visual language model to perform classification tasks more efficiently in downstream tasks. The task classifier in the above-mentioned teacher network can adopt the comparative language image pre-training (CLIP) model.

[0059] Step S126: Calculate the first organ classification loss between the first organ classification result and the source domain classification label.

[0060] The implementation method of the above step S126 is, for example, to use the formula Calculate the first organ classification loss between the first organ classification result and the source domain classification label, where L c represents the organ classification loss, M represents the number of categories of the source domain classification label, o represents the category of the sample, c represents the category in the source domain classification label, and y o,c is a binary indicator determined by the source domain classification label. If the category o of the sample belongs to the category c of the source domain classification label, the value of the indicator is 1, otherwise it is 0. o,c It represents the probability that the class o of the sample belongs to the class c of the source domain classification label.

[0061] Step S127: Use the teacher network to perform domain prediction on the source domain features to obtain a first domain classification result.

[0062] It can be understood that the above-mentioned first-domain classification result is used to calculate the first-domain classification loss between it and the source domain label, which can enable the trained teacher model to recognize and adapt to different image domains, thereby reducing the impact of image quality differences caused by factors such as patient body shape, instrument settings and operating level on image segmentation results, thereby improving the accuracy of ultrasound image segmentation.

[0063] Step S128: Calculate the first domain classification loss between the first domain classification result and the source domain label.

[0064] The above-mentioned domain classification loss is intended to enhance the adaptability of the model to images in different domains. Since ultrasound images may have quality differences due to factors such as patient body shape, instrument settings, and operator skill level, the domain classification loss helps to reduce the impact of these differences on the segmentation results, ultimately improving the generalization ability of the teacher model, so that the teacher model can maintain a high segmentation accuracy when segmenting unseen images.

[0065] For example, in the implementation of the above step S128, in addition to using the source domain dataset to train the teacher network, the target domain dataset can also be used to train the teacher network at the same time. Specifically, the formula L can be used. d = -dlogD(F(a))-(1-d)log(1-D(F(b))) calculates the first domain classification loss between the first domain classification result and the source domain label. Among them, L d represents the first domain classification loss, d represents the domain label, d=0 represents the image from the source domain (in this case, it is the source domain label), d=1 represents the image from the target domain (in this case, it is the target domain label), F represents the feature extractor in the teacher network, a represents the source domain anatomical structure subgraph, F(a) represents the source domain feature, D(F(a)) is the prediction probability in the first domain classification result, that is, the prediction probability of the domain classifier in the teacher network for the source domain feature F(a), b represents the target domain anatomical structure subgraph determined by the target domain image in the target domain dataset, F(b) represents the target domain feature, and D(F(b)) is the prediction probability of the domain classifier in the teacher network for the target feature F(b).

[0066] Step S129: Update the weight parameters of the teacher network using the first organ classification loss and the first domain classification loss.

[0067] The implementation method of the above step S129 is as follows: using formula L b =γL c +(1-γ)L d The first organ classification loss and the first field classification loss are calculated to obtain the total classification loss value of the teacher network; where L b represents the total classification loss value of the teacher network, γ represents the ratio hyperparameter between the first organ classification loss and the first field classification loss, which can be set according to the specific situation, L c represents the first organ classification loss, L dRepresents the first domain classification loss. Then, the weight parameters of the teacher network are updated by the total classification loss value of the teacher network until the weight parameters of the teacher network converge. This can be understood as the first stage in the knowledge distillation method. In the second stage of knowledge distillation, the weight parameters of the following teacher model are fixed. The role of the teacher model is to generate pseudo labels and give the pseudo labels to the student network as supervision signals to guide training.

[0068] It can be understood that the classification loss includes: organ classification loss and domain classification loss. The organ classification loss can help the teacher network model better understand the characteristics of different organ tissues, while the domain classification loss can help enhance the adaptability of the teacher model between different domains.

[0069] In the implementation process of the above scheme, the teacher network is trained through the multi-task learning (MTL) strategy of image segmentation task, organ classification task and domain classification task, so that the trained teacher model can better understand the characteristics of different organ tissues (including organs and peritoneal effusion), and effectively enhance the adaptability of the teacher model in different fields. Among them, the multi-task learning (MTL) strategy means that the model is trained to complete multiple related tasks at the same time, and usually there is some form of synergy (shared feature representation) between these tasks. Specifically in this scheme, both the teacher network and the student network can perform segmentation tasks and classification tasks at the same time. The classification task is based on the segmentation task, that is, the input of the classification task is the output result of the segmentation task.

[0070] As an optional implementation of the above step S130, the above implementation of using the teacher model to process the target domain ultrasound image in the target domain dataset may include:

[0071] Step S131: perform body landmark detection on the target domain ultrasound image to obtain the target domain anatomical structure sub-image.

[0072] An implementation example of the above step S131 is: using a deep learning model to perform body landmark detection on the target domain ultrasound image to obtain a target domain anatomical structure sub-map, where the deep learning model includes but is not limited to a model with a convolutional neural network (CNN) architecture or a model with a residual network architecture (for example, a model using ResNet50 as the backbone network of the detection network).

[0073] Step S132: Use the teacher model to extract features of the target domain anatomical structure sub-image to obtain a first target domain feature, and perform image segmentation on the target domain ultrasound image according to the first target domain feature to obtain a pseudo label.

[0074] Optionally, as another optional implementation of the above step S130, the adaptive generation algorithm in the teacher model can also be used to generate and calculate the target domain ultrasound image, so as to dynamically adjust the generation strategy of the pseudo-label according to the performance of the student model, so as to gradually improve the quality of the pseudo-label. Among them, the above-mentioned adaptive generation algorithm includes but is not limited to: Virtual Adversarial Training (VAT) algorithm, Temporal Ensembling algorithm, Confidence-Based Thresholding algorithm, etc. The VAT algorithm is a technology for semi-supervised learning, which improves the robustness of the model to input perturbations through virtual adversarial loss. The above-mentioned time mosaic algorithm uses the output average of multiple model snapshots in the sliding window to generate a more stable pseudo-label, which can reduce the fluctuation of a single model prediction and improve the consistency and accuracy of the pseudo-label. When generating pseudo-labels, the dynamic threshold algorithm based on confidence sets a dynamic threshold according to the prediction confidence of the model. For example, if the model is very confident in certain categories, the threshold for these categories can be lowered to allow more pseudo labels to be generated; conversely, for categories that the model is uncertain about, the threshold can be increased to reduce the generation of pseudo labels to avoid introducing noise.

[0075] As optional implementations of the above step S140, the following are included: In a first implementation, the above pseudo-labels may be segmentation pseudo-labels obtained by the teacher model performing image segmentation on the target domain ultrasound image in the target domain dataset. In a second implementation, the above pseudo-labels may also include the teacher model performing organ classification on the segmentation pseudo-labels segmented by the teacher model to obtain classification pseudo-labels, and the above student network may perform feature extraction, image segmentation and organ classification on the target domain ultrasound image in the target domain dataset, and this process will be described in detail below.

[0076] As an optional implementation of the above step S150, the above implementation of training the student network according to the pseudo labels and the processing results may include:

[0077] Step S151: Use the feature extractor in the student network to extract features from the target domain anatomical structure sub-image to obtain second target domain features, and perform image segmentation on the target domain ultrasound image according to the second target domain features to obtain a student segmented image.

[0078] An implementation example of the above-mentioned step S151 is: using the feature extractor in the student network to extract features from the target domain anatomical structure sub-image to obtain the second target domain features, and using the image segmentor in the student network to segment the target domain ultrasound image according to the second target domain features to obtain the student segmented image; wherein the above-mentioned feature extractor can adopt a visual Transformer (ViT) model, a contrastive language-image pre-training (CLIP) model or an encoder partial network of U-Net, etc.; the above-mentioned image segmentor can adopt a U-Net model, a DeepLab model, a PSPNet model or a RefineNet model.

[0079] Step S152: Calculate the second segmentation loss between the student segmentation image and the pseudo label.

[0080] The above-mentioned segmentation loss is the key difference indicator for evaluating the segmentation result predicted by the student network (i.e., the student segmentation image) and the pseudo-label. By minimizing the segmentation loss, the student network can continuously improve its segmentation accuracy. That is to say, the student network is trained using the pseudo-labels generated by the teacher model, and the prediction effect of the model is evaluated by minimizing the segmentation loss, thereby ensuring that the student model continuously improves its segmentation accuracy and provides high-quality segmentation results.

[0081] Step S153: Use the second segmentation loss to update the weight parameters of the student network.

[0082] The implementation method of the above steps S152 to S153 is, for example, to use the formula Calculate the second segmentation loss between the student segmentation image and the pseudo label. s represents the second segmentation loss, X′ represents the above-mentioned student segmentation image, that is, the image segmentation result predicted by the student network, and Y ′ Represents the segmentation pseudo-label obtained by image segmentation. Then, the weight parameters of the student network are updated using the second segmentation loss until the weight parameters of the student network converge.

[0083] In the implementation process of the above scheme, the weight parameters of the student network are updated through the second segmentation loss between the student segmentation image and the pseudo-label generated by the teacher model. This teacher-student network architecture and pseudo-label generation technology help the student network to quickly learn across domains and continuously train and optimize, so that it can achieve good performance in the target domain, effectively improving the image segmentation accuracy on the target domain ultrasound images.

[0084] It can be understood that, as another optional implementation of the above step S150, the above implementation of training the student network based on the pseudo-label and the processing result can also include: using the feature extractor in the student network to extract features from the target domain anatomical structure sub-image to obtain the second target domain feature, and performing image segmentation on the target domain ultrasound image based on the second target domain feature to obtain the student segmentation image. Then, using the student network to classify organs on the student segmentation image to obtain the student classification result, and calculating the third classification loss value between the student classification result and the classification pseudo-label, and using the third classification loss value to update the weight parameters of the student network.

[0085] As an optional implementation of the above step S150, the above target domain data set may further include: target domain classification labels and target domain field labels; the above implementation of training the student network according to the pseudo labels and the processing results may further include:

[0086] Step S154: Use the task classifier in the student network to perform classification prediction on the student segmented image to obtain a second organ classification result.

[0087] An implementation example of the above-mentioned step S154 is as follows: in the process of using the task classifier in the student network to perform classification prediction on the student segmentation image, the classification prompt technology can be used to perform classification prediction on the student segmentation image, thereby obtaining the second organ classification result; wherein, the task classifier in the above-mentioned student network can adopt a comparative language image pre-training (CLIP) model.

[0088] Step S155: Calculate the second organ classification loss between the second organ classification result and the target domain classification label.

[0089] It can be understood that guiding the student network to learn the characteristics of different tissues through the second organ classification loss not only helps the model distinguish different organs, but also provides a targeted method for feature extraction. For example, by identifying the characteristics of organs such as the liver, kidneys, and uterine adnexa, the feature extractor in the student model can extract features related to these organs, effectively improving the relevance and accuracy of feature extraction.

[0090] The implementation method of the above step S155 is, for example, to use the formula Calculate the second organ classification loss between the second organ classification result and the target domain classification label, where L′ c represents the organ classification loss, M represents the number of categories of the target domain classification label, o represents the category of the sample, c represents the category in the target domain classification label, and y o,cis a binary indicator determined by the target domain classification label. If the category o of the sample belongs to the category c of the target domain classification label, the value of the indicator is 1, otherwise it is 0. o,c It represents the probability that the class o of the sample belongs to the class c of the target domain classification label.

[0091] Step S156: Use the domain classifier in the student network to perform domain prediction on the second target domain features to obtain a second domain classification result.

[0092] It can be understood that the above-mentioned second-domain classification results are used to calculate the second-domain classification loss between them and the target domain label, which can enable the trained student model to recognize and adapt to different image fields, thereby reducing the impact of image quality differences caused by factors such as patient body shape, instrument settings and operating level on image segmentation results, thereby improving the accuracy of ultrasound image segmentation.

[0093] Step S157: Calculate the second domain classification loss between the second domain classification result and the target domain label.

[0094] The above-mentioned domain classification loss is intended to enhance the adaptability of the model to images in different domains. Since ultrasound images may have quality differences due to factors such as patient body shape, instrument settings, and operator skill level, domain classification loss helps to reduce the impact of these differences on the segmentation results, ultimately improving the generalization ability of the student model, so that the student model can maintain a high segmentation accuracy when segmenting unseen images.

[0095] For example, in the implementation of the above step S157, in addition to using the target domain dataset to train the teacher network, the source domain dataset can also be used to train the student network at the same time. Specifically, the formula L′ can be used. d = -dlogD′(F′(a))-(1-d)log(1-D′(F′(b))) calculates the second domain classification loss between the second domain classification result and the target domain label. Where L′ d represents the second domain classification loss, d represents the domain label, d=0 represents the image from the source domain (in this case, it is the source domain label), d=1 represents the image from the target domain (in this case, it is the target domain label), F′ represents the feature extractor in the student network, a represents the source domain anatomical structure subgraph, F′(a) represents the source domain feature, D′(F′(a)) is the prediction probability of the domain classifier in the student network for the source domain feature F′(a), b represents the target domain anatomical structure subgraph determined by the target domain image in the target domain dataset, F′(b) represents the target domain feature, and D′(F′(b)) is the second domain classification result, that is, the prediction probability of the domain classifier in the student network for the target feature F′(b).

[0096] Step S158: Update the weight parameters of the student network using the second organ classification loss and the second domain classification loss.

[0097] The implementation method of the above step S158 is, for example, to use the formula L′ b =γ′L′ c +(1-γ′)L′ d The second organ classification loss and the second field classification loss are calculated to obtain the total classification loss value of the student network; where L′ b represents the total classification loss value of the student network, γ′ represents the ratio hyperparameter between the second organ classification loss and the second field classification loss, which can be set according to the specific situation, and L′ c represents the second organ classification loss, L′ d Represents the first domain classification loss. Then, the weight parameters of the student network are updated by the total classification loss value of the student network mentioned above until the weight parameters of the student network converge.

[0098] In the implementation process of the above solution, the student network is trained through multi-task learning such as image segmentation task, organ classification task and domain classification task, so that the trained student model can better understand the characteristics of different organ tissues (including organs and ascites), and effectively enhance the adaptability of the student model between different fields.

[0099] As an optional implementation of the above model training method, the feature extractor and domain classifier of the student network can also be subjected to adversarial training, which may include:

[0100] Step S160: Reversing the gradient of the feature extractor through the gradient reversal layer of the student network so that the feature extractor and the domain classifier are subjected to adversarial training. The gradient reversal layer of the student network is arranged between the feature extractor and the domain classifier.

[0101] It can be understood that the gradient reversal layer of the student network above reverses the gradient passed to the feature extractor during the back-propagation process, so that the feature extractor essentially performs a maximization operation on its loss function. This maximization operation is actually opposite to the goal of the domain classifier, making the features learned by the feature extractor as indistinguishable as possible to the domain classifier, thereby improving the generalization ability of the model on images in different domains and ensuring that the student network can provide high-quality segmentation results in the target domain.

[0102] In the implementation process of the above scheme, by introducing the gradient reversal layer in the student network, an adversarial training mechanism between the feature extractor and the domain classifier of the student network is realized, which enables the feature extractor to learn features that are difficult for the domain classifier to distinguish. That is, the features learned by the feature extractor are indistinguishable to the domain classifier, which ultimately effectively improves the generalization ability of the student model.

[0103] As an optional implementation of the above model training method, after obtaining the student model, the student model can also be used to perform image segmentation. The implementation may include:

[0104] Step S170: Perform body landmark detection on the target domain ultrasound image to be processed to obtain an anatomical structure sub-image.

[0105] The implementation method and technical principle of the above step S170 are similar to the implementation method and technical principle of the above step S121, so they are not repeated here. If there are any unclear points, please refer to the implementation method of the above step S121.

[0106] Step S180: Use the student model to extract features from the anatomical structure sub-image to obtain ultrasound image features.

[0107] Step S190: Use the student model to perform image segmentation on the ultrasound image features to obtain an image segmentation result.

[0108] The implementation methods and technical principles of the above steps S180 to S190 are similar to the implementation methods and technical principles of the above step S122, so they are not repeated here. If there are any unclear points, please refer to the implementation method of the above step S122.

[0109] See also Figure 2 The flowchart of the image segmentation method provided by the embodiment of the present application is shown; the embodiment of the present application provides an image segmentation method, comprising:

[0110] Step S210: Perform body landmark detection on the target domain ultrasound image to be processed to obtain an anatomical structure sub-image.

[0111] The implementation method and technical principle of the above step S210 are similar to the implementation method and technical principle of the above step S121, so they are not repeated here. If there are any unclear points, please refer to the implementation method of the above step S121.

[0112] Step S220: Use the student model to extract features from the anatomical structure sub-image to obtain ultrasound image features.

[0113] Step S230: Use the student model to perform image segmentation on the ultrasound image features to obtain an image segmentation result.

[0114] The implementation methods and technical principles of the above steps S220 to S230 are similar to the implementation methods and technical principles of the above step S122, so they are not repeated here. If there are any unclear points, please refer to the implementation method of the above step S122.

[0115] In the implementation process of the above scheme, by first using the student model to extract features of the anatomical structure sub-image detected in the target domain ultrasound image to be processed, the ultrasound image features are obtained, and then the trained student model is used to perform image segmentation on the ultrasound image features, the segmentation efficiency and accuracy of the ultrasound image can be improved.

[0116] See also Figure 3 The structural diagram of the model training device provided in the embodiment of the present application is shown; the embodiment of the present application provides a model training device 300, including:

[0117] The sample data acquisition module 310 is used to obtain a sample data set, where the sample data set includes: a source domain data set and a target domain data set.

[0118] The teacher model acquisition module 320 is used to train the teacher network using the source domain dataset to obtain the teacher model.

[0119] The ultrasound image processing module 330 is used to process the target domain ultrasound image in the target domain dataset using the teacher model to obtain a pseudo label.

[0120] The processing result obtaining module 340 is used to process the target domain ultrasound image in the target domain data set using the student network to obtain a processing result.

[0121] The student model training module 350 is used to train the student network according to the pseudo labels and the processing results to obtain the student model, and the student model is used to process the image.

[0122] As an optional implementation of the above device, the source domain data set includes: a source domain ultrasound image and a source domain segmentation label; the teacher model acquisition module includes:

[0123] The source domain structure subgraph acquisition submodule is used to perform body landmark detection on the source domain ultrasound image to obtain the source domain anatomical structure subgraph.

[0124] The feature extraction image segmentation submodule is used to use the teacher network to extract features from the source domain anatomical structure sub-image to obtain source domain features, and to perform image segmentation on the source domain ultrasound image based on the source domain features to obtain a teacher segmentation image.

[0125] The first segmentation loss calculation submodule is used to calculate the first segmentation loss between the teacher segmentation image and the source domain segmentation label.

[0126] The teacher network weight update submodule is used to update the weight parameters of the teacher network using the first segmentation loss.

[0127] As an optional implementation of the above device, the source domain data set further includes: a source domain classification label and a source domain field label; and the teacher model acquisition module further includes:

[0128] The first organ classification obtaining submodule is used to use the teacher network to perform classification prediction on the teacher segmentation image to obtain the first organ classification result.

[0129] The first organ loss calculation submodule is used to calculate the first organ classification loss between the first organ classification result and the source domain classification label.

[0130] The first domain result acquisition submodule is used to use the teacher network to perform domain prediction on the source domain features to obtain the first domain classification result.

[0131] The first domain loss calculation submodule is used to calculate the first domain classification loss between the first domain classification result and the source domain label.

[0132] The teacher network parameter updating submodule is used to update the weight parameters of the teacher network using the first organ classification loss and the first domain classification loss.

[0133] As an optional implementation of the above device, the ultrasound image processing module includes:

[0134] The target structure sub-graph acquisition sub-module is used to perform body landmark detection on the target domain ultrasound image to obtain the target domain anatomical structure sub-graph.

[0135] The target feature extraction and segmentation submodule is used to use the teacher model to extract features from the target domain anatomical structure sub-image to obtain a first target domain feature, and to perform image segmentation on the target domain ultrasound image according to the first target domain feature to obtain a pseudo label.

[0136] As an optional implementation of the above device, the student model training module includes:

[0137] The target extraction image segmentation submodule is used to use the feature extractor in the student network to extract features from the target domain anatomical structure sub-image to obtain second target domain features, and to perform image segmentation on the target domain ultrasound image according to the second target domain features to obtain a student segmentation image.

[0138] The second segmentation loss calculation submodule is used to calculate the second segmentation loss between the student segmentation image and the pseudo label.

[0139] The student network weight update submodule is used to update the weight parameters of the student network using the second segmentation loss.

[0140] As an optional implementation of the above device, the target domain data set further includes: a target domain classification label and a target domain field label; the student model training module further includes:

[0141] The second organ classification acquisition submodule is used to use the task classifier in the student network to perform classification prediction on the student segmentation image to obtain the second organ classification result.

[0142] The second classification loss calculation submodule is used to calculate the second organ classification loss between the second organ classification result and the target domain classification label.

[0143] The second domain classification acquisition submodule is used to use the domain classifier in the student network to perform domain prediction on the second target domain features to obtain the second domain classification results.

[0144] The second domain loss calculation submodule is used to calculate the second domain classification loss between the second domain classification result and the target domain label.

[0145] The student network parameter updating submodule is used to update the weight parameters of the student network using the second organ classification loss and the second domain classification loss.

[0146] As an optional implementation of the above device, the model training device further includes:

[0147] The network gradient reversal module is used to reverse the gradient of the feature extractor through the gradient reversal layer of the student network so that the feature extractor and the domain classifier can be trained adversarially. The gradient reversal layer of the student network is set between the feature extractor and the domain classifier.

[0148] As an optional implementation of the above device, the model training device further includes:

[0149] The anatomical structure sub-graph acquisition module is used to perform body landmark detection on the target domain ultrasound image to be processed to obtain the anatomical structure sub-graph.

[0150] The ultrasound image feature acquisition module is used to extract features from the anatomical structure sub-image using the student model to obtain ultrasound image features.

[0151] The image segmentation result acquisition module is used to perform image segmentation on ultrasound image features using a student model to obtain image segmentation results.

[0152] See also Figure 4 The structure diagram of the image segmentation device provided in the embodiment of the present application is shown; the embodiment of the present application provides an image segmentation device 400, including:

[0153] The image body landmark detection module 410 is used to perform body landmark detection on the target domain ultrasound image to be processed to obtain an anatomical structure sub-image.

[0154] The sub-image feature extraction module 420 is used to extract features from the anatomical structure sub-image using the student model to obtain ultrasound image features.

[0155] The feature image segmentation module 430 is used to perform image segmentation on the ultrasound image features using the above-mentioned student model to obtain an image segmentation result.

[0156] It should be understood that the device corresponds to the above-mentioned model training method embodiment and can perform the various steps involved in the above-mentioned method embodiment. The specific functions of the device can be found in the description above, and the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the device.

[0157] See also Figure 5 An electronic device 500 provided in an embodiment of the present application includes: a processor 510 and a memory 520, wherein the memory 520 stores machine-readable instructions executable by the processor 510, and when the machine-readable instructions are executed by the processor 510, the above method is executed.

[0158] The embodiment of the present application also provides a computer-readable storage medium 530, on which a computer program is stored, and the computer program is executed by the processor 510 to execute the above method. The computer-readable storage medium 530 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk.

[0159] The embodiment of the present application also provides a computer program product, including: a computer program or a computer instruction, and the computer program or the computer instruction executes the method described above when executed by a processor.

[0160] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0161] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device embodiments described above are only schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a part of a module, a program segment or a code, and a part of a module, a program segment or a code includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also be different from the order of occurrence marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which is mainly determined by the functions involved.

[0162] In addition, each functional module of each embodiment in the embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. In addition, in the description of this specification, the description of reference terms "one embodiment", "some embodiments", "example", "specific example", "some examples", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present application. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples without contradiction.

[0163] The above description is only an optional implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the embodiments of the present application, which should be covered within the protection scope of the embodiments of the present application.

Claims

1. A model training method, characterized in that: include: Obtaining a sample data set, wherein the sample data set includes: a source domain data set and a target domain data set; Using the source domain dataset to train the teacher network to obtain a teacher model; Using the teacher model to process the target domain ultrasound image in the target domain dataset to obtain a pseudo label; Using a student network to process the target domain ultrasound image in the target domain dataset to obtain a processing result; The student network is trained according to the pseudo labels and the processing results to obtain a student model, and the student model is used to process the image.

2. The method according to claim 1, characterized in that The source domain data set includes: a source domain ultrasound image and a source domain segmentation label; and the training of the teacher network using the source domain data set includes: Performing body landmark detection on the source domain ultrasound image to obtain a source domain anatomical structure sub-image; Using the teacher network to perform feature extraction on the source domain anatomical structure sub-image to obtain source domain features, and performing image segmentation on the source domain ultrasound image according to the source domain features to obtain a teacher segmentation image; Calculating a first segmentation loss between the teacher segmentation image and the source domain segmentation label; The weight parameters of the teacher network are updated using the first segmentation loss.

3. The method according to claim 2, characterized in that The source domain data set also includes: a source domain classification label and a source domain field label; the training of the teacher network using the source domain data set also includes: Using the teacher network to perform classification prediction on the teacher segmented image to obtain a first organ classification result; Calculating a first organ classification loss between the first organ classification result and the source domain classification label; Using the teacher network to perform domain prediction on the source domain features to obtain a first domain classification result; Calculating a first domain classification loss between the first domain classification result and the source domain label; The weight parameters of the teacher network are updated using the first organ classification loss and the first domain classification loss.

4. The method according to claim 2, characterized in that: The using the teacher model to process the target domain ultrasound image in the target domain dataset includes: Performing body landmark detection on the target domain ultrasound image to obtain a target domain anatomical structure sub-image; The teacher model is used to perform feature extraction on the target domain anatomical structure sub-image to obtain a first target domain feature, and the target domain ultrasound image is segmented according to the first target domain feature to obtain the pseudo label.

5. The method according to claim 4, characterized in that The training of the student network according to the pseudo labels and the processing results includes: Using the feature extractor in the student network to perform feature extraction on the target domain anatomical structure sub-graph to obtain a second target domain feature, and performing image segmentation on the target domain ultrasound image according to the second target domain feature to obtain a student segmentation image; Calculating a second segmentation loss between the student segmentation image and the pseudo label; The weight parameters of the student network are updated using the second segmentation loss.

6. The method according to claim 5, characterized in that The target domain data set further includes: a target domain classification label and a target domain field label; the training of the student network according to the pseudo label and the processing result further includes: Using the task classifier in the student network to perform classification prediction on the student segmented image to obtain a second organ classification result; Calculating a second organ classification loss between the second organ classification result and the target domain classification label; Using the domain classifier in the student network to perform domain prediction on the second target domain feature to obtain a second domain classification result; Calculating a second domain classification loss between the second domain classification result and the target domain label; The weight parameters of the student network are updated using the second organ classification loss and the second field classification loss.

7. An image segmentation method, characterized in that: include: Perform body landmark detection on the target domain ultrasound image to be processed to obtain an anatomical structure sub-image; Using the student model to extract features from the anatomical structure sub-image to obtain ultrasound image features; Use any student model described in claims 1-6 to perform image segmentation on the ultrasound image features to obtain an image segmentation result.

8. A model training device, characterized in that: include: A sample data acquisition module is used to obtain a sample data set, wherein the sample data set includes: a source domain data set and a target domain data set; A teacher model acquisition module, used to train a teacher network using the source domain dataset to obtain a teacher model; An ultrasound image processing module, configured to process the target domain ultrasound image in the target domain dataset using the teacher model to obtain a pseudo label; A processing result obtaining module, used for processing the target domain ultrasound image in the target domain data set using a student network to obtain a processing result; The student model training module is used to train the student network according to the pseudo-label and the processing result to obtain a student model, and the student model is used to process the image.

9. An image segmentation device, characterized in that: include: An image body landmark detection module is used to perform body landmark detection on the target domain ultrasound image to be processed and obtain an anatomical structure sub-image; A sub-image feature extraction module, used to extract features from the anatomical structure sub-image using the student model to obtain ultrasound image features; A feature image segmentation module is used to perform image segmentation on the ultrasound image features using the student model as described in any one of claims 1-6 to obtain an image segmentation result.

10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions are executed by the processor to perform any method according to claims 1 to 8.

Citation Information

Cited By

  • Embryo development artificial intelligence evaluation method based on progressive active passive domain adaptation

    CN119963953A