Image segmentation method and device, and radiotherapy plan generation method and system

Abstract

The invention provides an image segmentation method and device, and a radiotherapy plan generation method and system. The image segmentation method comprises the following steps: acquiring a to-be-processed image of a target object; the to-be-processed image is input into a pre-trained image segmentation model, a segmented image is output, and the segmented image comprises a target area and the category of the target area; wherein the image segmentation model is obtained by pre-training a neural network model comprising an encoder, a first decoder and a second decoder by using a training sample set comprising a first training image carrying a label and a second training image not carrying the label; the pre-training comprises the steps that the first decoder performs parameter updating through a first training image carrying a label and a label generated by the second decoder based on a second training image not carrying the label, and the second decoder performs parameter updating through the first training image carrying the label. According to the method, the efficiency and the accuracy of image segmentation can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image segmentation method, apparatus, radiotherapy plan generation method and system. Background Technology

[0002] Radiation therapy is a common cancer treatment. When administering radiation therapy to cancer patients, doctors need to delineate the tumor and surrounding tissues to assess the safety of the radiation. This process usually begins with doctors manually delineating each medical image, segmenting the target area of ​​the tissues and organs, labeling it, and then developing a treatment plan based on this information.

[0003] Medical images such as CT (Computed Tomography), PET (Positron Emission Tomography), PET-CT, MRI (Magnetic Resonance Imaging), and NMRI (Nuclear Magnetic Resonance Imaging) all have the concept of window width and window level. Taking CT images as an example, CT images can identify density differences of 2000 different gray levels within the human body, while the human eye can only distinguish 16 gray levels. Therefore, the CT value that the human eye can distinguish on a CT image is 125 HU (2000 / 16). That is, the CT values ​​of different tissues in the human body must differ by more than 125 HU for the human eye to recognize them. The CT values ​​of human soft tissues often vary between 20-50 HU, which is difficult for the human eye to discern. Therefore, the method of windowing is usually used for segmented observation. The range of CT values ​​observed is called the window width, and the CT value at the center of the observation is the window level or window center. The width of the CT window directly affects the image's clarity and contrast. A narrow window displays a small range of CT values ​​with strong contrast, suitable for observing tissue structures with similar densities (such as brain tissue). Conversely, a wide window displays a large range of CT values ​​with poor contrast but uniform density, suitable for observing structures with large density differences (such as bone and soft tissue). Ideally, the window width should reflect the range of CT value changes for the tissue or lesion, and the window level should be equal to or close to the CT value to be observed. When doctors develop treatment plans, some difficult-to-identify tissues require separate adjustments to the CT window width and level, followed by individual delineation, which is time-consuming and labor-intensive. Currently, medical image segmentation relies on medical personnel, resulting in low efficiency and accuracy. Summary of the Invention

[0004] The purpose of this specification is to provide an image segmentation method, apparatus, radiotherapy plan generation method, and system to solve problems such as low efficiency and accuracy in medical image segmentation.

[0005] To solve the above-mentioned technical problems, the first aspect of this specification provides an image segmentation method, including:

[0006] Obtain the image of the target object to be processed;

[0007] The image to be processed is input into a pre-trained image segmentation model, which outputs a segmented image, which includes a target region and the category of the target region. The image segmentation model is pre-trained using a training sample set including a first training image with labels and a second training image without labels. The pre-training includes: the first decoder updating its parameters using the first training image with labels and the labels generated by the second decoder based on the second training image without labels; and the second decoder updating its parameters using the first training image with labels.

[0008] In some embodiments of this specification, the pre-training includes:

[0009] The parameters of the first decoder are updated based on the labels carried by the first training image, the first output, and the third output, and / or the parameters of the first decoder are updated based on the second output and the fourth output.

[0010] The parameters of the second decoder are updated based on the third output and the labels carried by the first training image;

[0011] Wherein, the first output is the output of the first decoder corresponding to the first training image;

[0012] The second output is the output of the first decoder corresponding to the second training image;

[0013] The third output is the output of the second decoder corresponding to the first training image;

[0014] The fourth output is the output of the second decoder corresponding to the second training image.

[0015] In some embodiments of this specification, the pre-training includes:

[0016] In response to the second training image: acquire anchor point samples, adjust the anchor point samples based on random display parameters, and obtain the first random sample;

[0017] The encoder processes the anchor point sample and the first random sample to obtain the first anchor point information and the first random information, respectively.

[0018] The first decoder processes the first anchor information and the first random information to obtain the first anchor label and the first random label, respectively; the second decoder processes the first anchor information to obtain the second anchor label.

[0019] Based on the first anchor label, the first random label, and the second anchor label, the first loss of the first decoder is determined;

[0020] The parameters of the first decoder are updated based on the first loss.

[0021] In some embodiments of this specification, the first loss is determined by the following formula:

[0022]

[0023] Among them, Loss a Let L represent the first loss, σ and υ represent the weighting parameters, and L represent the weighting parameters, respectively. o L represents the loss between the first anchor label and the second anchor label. p L represents the loss between the first anchor label and the first random label. n This represents the loss between the first anchor label and the first negative sample label, where the first negative sample label is obtained based on the output of the neural network model corresponding to other training images.

[0024] In some embodiments of this specification, obtaining anchor point samples includes:

[0025] Perform at least one image enhancement process on the current training image, and use the at least one image obtained from the process and the current training image as anchor samples respectively.

[0026] In some embodiments of this specification, the pre-training includes:

[0027] In response to the first training image: acquire anchor point samples, and adjust the anchor point samples based on the reference display parameters corresponding to the labels carried by the anchor point samples to obtain the first reference sample;

[0028] The encoder processes the first reference sample to obtain the first reference information;

[0029] The second decoder processes the first reference information to obtain the first reference tag;

[0030] Based on the labels carried by the anchor samples and the first reference label, the second loss of the second decoder is determined;

[0031] The parameters of the second decoder are updated based on the second loss.

[0032] In some embodiments of this specification, the pre-training includes:

[0033] In response to the first training image: obtain anchor point samples, adjust the anchor point samples based on random display parameters, and obtain a second random sample;

[0034] The encoder processes the anchor point sample and the second random sample to obtain the third anchor point information and the second random information, respectively.

[0035] The first decoder processes the third anchor point information to obtain the third anchor point label;

[0036] The first decoder or the second decoder processes the second random information to obtain the second random label;

[0037] Based on the labels carried by the anchor samples, the third anchor label, the second random label, and the first reference label, the third loss of the first decoder is determined;

[0038] The parameters of the first decoder are updated based on the third loss.

[0039] In some embodiments of this specification, the neural network model further includes a third decoder that updates based on the parameters of the first decoder;

[0040] The third decoder outputs a second negative sample label for the negative sample, and the first decoder also updates its parameters based on the second negative sample label.

[0041] In some embodiments of this specification, the pre-training includes:

[0042] In response to the first training image: acquire anchor samples, adjust the anchor samples based on random display parameters to obtain a third random sample, and adjust the anchor samples based on the second reference display parameters corresponding to the labels carried by the anchor samples to obtain a second reference sample;

[0043] The encoder processes the anchor point sample, the third random sample, and the second reference sample to obtain the fourth anchor point information, the third random information, and the second reference information, respectively.

[0044] The first decoder processes the fourth anchor point information to obtain the fourth anchor point label;

[0045] The second decoder processes the second reference information to obtain the second reference label;

[0046] The first decoder, the second decoder, or the third decoder processes the third random information to obtain a third random tag;

[0047] Based on the labels carried by the anchor samples, the fourth anchor label, the third random label, the second reference label, and the second negative sample label, the fourth loss of the first decoder is determined;

[0048] The parameters of the first decoder are updated based on the fourth loss.

[0049] In some embodiments of this specification, the fourth loss is determined by the following formula:

[0050] Loss a =αL p +β(δL ce +εL fl +ηL cc +L n );

[0051] Among them, Loss a This represents the fourth loss, where α, β, δ, ε, and η represent the weighting parameters; L ce L represents the cross-entropy between the fourth anchor tag and the tag carried by the anchor sample; fl This represents the focal loss between the fourth anchor label and the label carried by the anchor sample; L cc This indicates the loss between the fourth anchor label and the second reference label; L p This represents the loss between the fourth anchor label and the third random label; L n This represents the loss between the fourth anchor label and the second negative sample label.

[0052] In some embodiments of this specification, the third decoder updates its parameters using the following formula:

[0053] θ k =mθ k +(1-m)θ a ;

[0054] Where, θ k θ represents the parameters of the third decoder. a represents the parameters of the first decoder, and m represents the associated parameters.

[0055] In some embodiments of this specification, the first loss of the first decoder is determined based on the first anchor label, the first random label, and the second anchor label, including:

[0056] The first anchor tag is fused with the first anchor information to obtain the fused first anchor tag;

[0057] Based on the fused first anchor label, first random label, and second anchor label, the first loss of the first decoder is determined.

[0058] In some embodiments of this specification, the segmented image includes multiple target regions and multiple categories of target regions;

[0059] The labels carried by the first training image include a specific region in the first training image and the category of the specific region;

[0060] The categories of all specific regions contained in the training sample set correspond at least to the categories of the multiple target regions.

[0061] A second aspect of this specification provides an image segmentation apparatus, comprising:

[0062] The image acquisition module is used to acquire the image of the target object to be processed.

[0063] An image segmentation module is used to input the image to be processed into a pre-trained image segmentation model and output a segmented image, wherein the segmented image includes a target region and the category of the target region; wherein the image segmentation model is obtained by pre-training a neural network model including an encoder, a first decoder and a second decoder using a training sample set including a first training image with labels and a second training image without labels; the pre-training includes: the first decoder updating its parameters using the first training image with labels and the labels generated by the second decoder based on the second training image without labels, and the second decoder updating its parameters using the first training image with labels.

[0064] The third aspect of this specification provides a method for generating a radiotherapy plan, including:

[0065] Obtain the image of the target object to be processed;

[0066] The image to be processed is input into a pre-trained image segmentation model, which outputs a segmented image, which includes the target region and the category of the target region;

[0067] The material information of the target region is determined based on the category of the target region, and a three-dimensional voxel model is established based on the target region and the material information of the target region.

[0068] Based on the three-dimensional voxel model, tissue dose simulation calculations are performed, and a treatment plan is generated.

[0069] The fourth aspect of this specification provides a radiotherapy system, comprising:

[0070] A beam irradiation device used to generate a radiation beam;

[0071] The treatment plan module is used to generate treatment plans;

[0072] The control module is used to control the beam irradiation device to generate a radiation beam according to the treatment plan;

[0073] The treatment plan module includes:

[0074] The image acquisition unit is used to acquire the image of the target object to be processed.

[0075] An image segmentation unit is used to input the image to be processed into a pre-trained image segmentation model and output a segmented image, wherein the segmented image includes the target region and the category of the target region;

[0076] The model building unit is used to determine the material information of the target region based on the category of the target region, and to build a three-dimensional voxel model based on the target region and the material information of the target region.

[0077] The calculation unit is used to perform tissue dose simulation calculations based on the three-dimensional voxel model and generate a treatment plan.

[0078] A fifth aspect of this specification provides an electronic device including a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements the steps of the aforementioned method.

[0079] The sixth aspect of this specification provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the aforementioned method.

[0080] The image segmentation method provided in this specification involves acquiring an image of the target object to be processed; inputting the image to be processed into a pre-trained image segmentation model; and outputting a segmented image, which includes the target region and its category. The image segmentation model is pre-trained using a training sample set comprising a first training image with labels and a second training image without labels. The pre-training includes: the first decoder updating its parameters using the first training image with labels and the second decoder updating its parameters using the first training image with labels, and the second decoder updating its parameters using the first training image with labels. This method requires only a small number of labeled images to achieve weakly supervised training of the neural network model; the pseudo-labels output by the second decoder can guide the unlabeled training of the first decoder, resulting in low dependence on training samples and high model adaptability. Furthermore, by training the neural network model with a training sample set including labeled first training images and unlabeled second training images, an image segmentation model is obtained. Using this model, there is no need to adjust the CT window width and level, or to delineate and label each partition individually; it can automatically and accurately segment images, saving significant time and labor costs and improving the efficiency and accuracy of image segmentation. Based on this, the generation of radiotherapy plans is also more efficient, accurate, and rational, reducing the workload of medical staff and improving the effectiveness of subsequent treatments. Attached Figure Description

[0081] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0082] Figure 1 A schematic diagram of the image segmentation method provided in the embodiments of this specification;

[0083] Figure 2 A schematic diagram of a response to a first training image provided in an embodiment of this specification;

[0084] Figure 3 A schematic diagram of a response to a second training image provided in an embodiment of this specification;

[0085] Figure 4 This is a schematic diagram of the architecture provided for an embodiment of this specification;

[0086] Figure 5 A schematic diagram of the image segmentation apparatus provided in the embodiments of this specification;

[0087] Figure 6 A schematic diagram of the radiotherapy plan generation method provided in the embodiments of this specification;

[0088] Figure 7 A schematic diagram of a radiotherapy system provided in the embodiments of this specification;

[0089] Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0090] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0091] Currently, image segmentation methods mainly include traditional segmentation methods and deep learning-based methods.

[0092] Traditional segmentation methods mainly include thresholding, region growing, edge detection, clustering, and model matching. However, these methods often require significant manual intervention, failing to meet the demands for accuracy and efficiency in medical image segmentation. With the rapid development of deep learning technology, especially the widespread application of Convolutional Neural Networks (CNNs) in image processing, medical image segmentation techniques have seen significant advancements. For example, the U-Net deep learning method, which introduces skip connections and deconvolution, can achieve efficient pixel-level classification while preserving image details. This semantic-based image segmentation is a pixel-level classification task that can detect biological structures and quantify morphological features, quantitatively capture object shapes, and provide a means for high-resolution spatial statistics. Fully Convolutional Networks (FCNs) are fully automatic segmentation techniques that do not require domain experts to select regions of interest (ROIs). The aforementioned deep learning methods involve supervised learning and require training data, such as deep neural networks, shape models, and random forests. However, obtaining large-scale labeled training data is often an expensive and time-consuming task, especially in data-scarce fields like medical imaging, where the performance of deep neural network models is limited. Unsupervised learning methods, on the other hand, rely heavily on the quality and features of the input data. Noise, missing values, or outliers in the data can negatively impact model performance. FCN-based methods face challenges during training and use, including variations in image size, texture, shape, and ROI selection in low-resolution images.

[0093] As can be seen, current medical image segmentation methods rely on medical personnel to perform detailed mapping of a large number of target regions based on available resources. Furthermore, to ensure the accuracy of these methods, the identified target regions need to be accurate to the pixel level. These methods have relatively high requirements for the dataset and often require interaction between medical personnel and the system, such as selecting an initial ROI before performing detailed segmentation, which is time-consuming and tedious. In addition, since this is for medical image segmentation, changing the window width and level during actual use will alter the image, and the choice of window width and level will affect the segmentation results.

[0094] The image segmentation model in this embodiment is pre-trained using a training sample set including a first training image with labels and a second training image without labels. This pre-training process includes: the first decoder updating its parameters using the first training image with labels and the second decoder updating its parameters using the first training image with labels, and the second decoder updating its parameters using the first training image with labels. Furthermore, after obtaining the image of the target object to be processed, the image can be input into the pre-trained image segmentation model, which outputs a segmented image. The segmented image may include the target region and the category of the target region.

[0095] The image segmentation method provided in the embodiments of this specification requires only a small number of labeled images to achieve weakly supervised training of the neural network model. The pseudo-labels output by the second decoder can guide the unlabeled training of the first decoder, resulting in low dependence on training samples and high model adaptability. Furthermore, by training the neural network model with a training sample set including labeled first training images and unlabeled second training images, an image segmentation model is obtained. Using this model, there is no need to adjust the window width and level of CT scans, or to delineate and label each partition individually; it can automatically and accurately segment images, saving significant time and labor costs and improving the efficiency and accuracy of image segmentation. Based on this, the generation of radiotherapy plans is also more efficient, accurate, and reasonable, reducing the burden on medical staff and improving the effectiveness of subsequent treatments.

[0096] The image segmentation method and radiotherapy plan generation method provided in this application can be executed by an electronic device, which refers to an electronic device with data computing, processing, and storage capabilities. This electronic device can be a terminal such as a personal computer (PC), tablet computer, smartphone, wearable device, or intelligent robot; or it can be a server. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0097] The image segmentation method provided in the embodiments of the specification will be described below with reference to the accompanying drawings.

[0098] Figure 1The diagram illustrates an image segmentation method provided in an embodiment of this specification. While this specification provides method operation steps or apparatus structures as shown in the following embodiments or figures, the method or apparatus may include more or fewer operation steps or module units, either combined or without inventive effort, based on conventional methods or without requiring creative work. In steps or structures where there is no logically necessary causal relationship, the execution order of these steps or the module structure of the apparatus is not limited to the execution order or module structure shown in the embodiments or figures of this specification. When the method or module structure is applied in actual devices, servers, or terminal products, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or figures (e.g., in a parallel processor or multi-threaded processing environment, or even in a distributed processing or server cluster implementation environment). Figure 1 As shown, the method may include:

[0099] S101: Obtain the image of the target object to be processed.

[0100] It is understood that the image to be processed can be a medical image to be segmented, such as a CT image, PET image, PET-CT image, MRI image, NMRI image, etc.; the image to be processed can also be an image from other fields to be segmented, and this specification does not limit this.

[0101] S102: Input the image to be processed into a pre-trained image segmentation model and output a segmented image, which includes the target region and the category of the target region.

[0102] The image segmentation model is obtained by pre-training a neural network model including an encoder, a first decoder, and a second decoder using a training sample set including a first training image with labels and a second training image without labels. The pre-training includes: the first decoder updating its parameters using the first training image with labels and the labels generated by the second decoder based on the second training image without labels, and the second decoder updating its parameters using the first training image with labels.

[0103] It is understood that an image segmentation model can be used to segment an image to be processed, so as to obtain a segmented image including the target region and the category of the target region. In some embodiments, the segmented image output by the image segmentation model can be the output of the first decoder. That is, when training the image segmentation model, the first decoder and the second decoder can be trained using a training sample set. When the model is actually applied to perform image segmentation, it can be implemented through the first decoder, while the second decoder plays the role of assisting the training of the first decoder.

[0104] The segmented image output by the image segmentation model may include a target region and its category, or it may include multiple target regions and their categories. In some embodiments of this specification, the segmented image may include a target region and its category, allowing the user to quickly determine the target region in the image to be processed based on the segmented image. In other embodiments of this specification, the segmented image may include multiple target regions and their categories, allowing the user to quickly determine each target region and its category in the image to be processed, facilitating further analysis of each target region or quickly determining a specific region based on user needs.

[0105] Specifically, the labels carried by the first training image in the training sample set can be specific regions in the image and their corresponding categories. In some embodiments, the first training image may include multiple specific regions, and the labels carried by the first training image may be multiple. The segmentation of the image to be processed by the image segmentation model is to determine the target regions in the image to be processed that match the specific regions in the first and second training images, and to determine the category corresponding to the target regions based on the categories of the matching specific regions. In some embodiments of this specification, the categories of all specific regions contained in the training sample set correspond at least to the categories of the multiple target regions. It can be understood that the categories of all feature regions may be more than or equal to the categories of the target regions in the image to be processed. This can be implemented by including only one category of a specific region in a single first training image, and the categories of the specific regions contained in multiple first training images may not all be the same, with the summed categories of the specific regions at least corresponding to the categories of the multiple target regions; or by including multiple categories of specific regions in a single first training image, and the categories of the specific regions contained in multiple first training images may be different or the same, with the summed categories of the specific regions at least corresponding to the categories of the multiple target regions.

[0106] In some embodiments of this specification, a corresponding image segmentation sub-model can be trained for each specific region. Different sub-models can perform image segmentation processing on the image to be processed simultaneously or sequentially, and the output results of multiple sub-models can be fused. Alternatively, an image segmentation model can be trained to identify and segment multiple specific regions. This image segmentation model can segment all categories at once, resulting in higher segmentation efficiency and a wider range of application scenarios.

[0107] In some embodiments of this specification, the pre-training may include:

[0108] The parameters of the first decoder are updated based on the label carried by the first training image, the first output, and the third output, and / or: the parameters of the first decoder are updated based on the second output and the fourth output; the parameters of the second decoder are updated based on the third output and the label carried by the first training image; wherein, the first output is the output of the first decoder corresponding to the first training image; the second output is the output of the first decoder corresponding to the second training image; the third output is the output of the second decoder corresponding to the first training image; and the fourth output is the output of the second decoder corresponding to the second training image.

[0109] It is understood that the decoder output may include pseudo-labels corresponding to the training image output and / or pseudo-labels corresponding to the transformed training image output. For example, for the first training image, during actual training, the first training image may be transformed, such as through data augmentation, or adjusted based on random display parameters to obtain data-augmented or adjusted samples, both of which are considered positive samples. These positive samples can be input into the first decoder along with the corresponding original first training image, and each will produce a corresponding output, i.e., a pseudo-label. Therefore, the first decoder corresponding to the first training image may refer to a pseudo-label corresponding to the first training image, or it may be understood to include multiple pseudo-labels corresponding to other positive samples. Of course, for other decoders or for other training images, the decoder output may also be one or more pseudo-labels, and this specification does not limit this. It is understood that the fourth output may include the label (i.e., pseudo-label) generated by the second decoder based on the second training image without the label, as mentioned above, and may also include the label (i.e., pseudo-label) generated by the second decoder based on other images transformed from the second training image without the label.

[0110] It is understandable that the parameter update methods for the first decoder and the second decoder differ when the input training images are the first training image and the second training image, respectively. Specifically, when the first training image is input, the parameters of the second decoder can be updated using the first training image, that is, the parameters of the second decoder are updated based on the third output and the label carried by the first training image; and the parameters of the first decoder can be updated using the first training image and the pseudo-label output by the second decoder corresponding to the first training image, that is, the parameters of the first decoder are updated based on the label carried by the first training image, the first output, and the third output. When the second training image is input, since the second training image does not carry a label, the second decoder does not update its parameters. The second training image can be input into the second decoder to obtain the pseudo-label output by the second decoder corresponding to the second training image, i.e., the fourth output. Then, the parameters of the first decoder can be updated based on the fourth output and the second output.

[0111] The model training process will be further described below with reference to an example.

[0112] In some embodiments of this specification, the pre-training may include:

[0113] In response to the second training image: acquire anchor samples, adjust the anchor samples based on random display parameters to obtain a first random sample; the encoder processes the anchor samples and the first random sample to obtain first anchor information and first random information, respectively; the first decoder processes the first anchor information and the first random information to obtain a first anchor label and a first random label, respectively; the second decoder processes the first anchor information to obtain a second anchor label; based on the first anchor label, the first random label, and the second anchor label, determine a first loss of the first decoder; update the parameters of the first decoder based on the first loss.

[0114] It can be understood that the second anchor label output by the second decoder is the fourth output, which can be used as a pseudo label for the second training image that does not carry a label. The first decoder can combine this pseudo label to update the parameters.

[0115] It is understood that when the input training image is the second training image, it can be used as an anchor sample. If multiple second training images are input simultaneously during training, one of the images can be used as an anchor sample for training. In some embodiments, the random display parameters can be parameters adjusted when performing random data augmentation on the anchor samples. Random data augmentation can include rotation, translation, noise addition, cropping, etc., and the random display parameters can include at least one of rotation angle parameters, translation direction and compensation, translation position parameters, noise, cropped size parameters, etc. This specification does not limit this. In other embodiments, the random display parameters can also be parameters for adjusting the window width and window level of the image. They can be pre-specified window width or window level, or randomly generated window width or window level, and this specification does not limit this. Anchor samples can be adjusted based on one or more of the above-mentioned random display parameters to obtain a first random sample; or anchor samples can be adjusted based on multiple individual or superimposed random display parameters to obtain multiple first random samples, each of which is trained using the aforementioned method.

[0116] In some embodiments, image samples are input to the encoder, and the encoder performs a feature extraction operation, which can be implemented using conventional techniques in the art. For example, image features of the samples can be extracted by downsampling the samples; that is, the anchor point samples and the first random samples are downsampled to extract their image features as the first anchor point information and the first random information. Of course, in other embodiments, the encoder can also use other means to perform feature extraction, which is beneficial for semantic feature extraction, etc., and can also perform other processing on the anchor point samples and the first random samples, such as data or image conversion, to obtain the first anchor point information and the first random information. This specification does not limit this.

[0117] In some embodiments of this specification, the first loss can be determined by the following formula:

[0118]

[0119] Among them, Loss a Let L represent the first loss, σ and υ represent the weighting parameters, and L represent the weighting parameters, respectively. o L represents the loss between the first anchor label and the second anchor label. p L represents the loss between the first anchor label and the first random label. n This represents the loss between the first anchor label and the first negative sample label, where the first negative sample label is obtained based on the output of the neural network model corresponding to other training images, L. f As an intermediate quantity, it can be omitted from the formula after substitution and transformation, and therefore does not need to be defined. Its meaning is the loss between the first anchor label and the positive and negative samples, that is, the sum of the losses between the first anchor label and the first random label and the first negative sample label.

[0120] In some embodiments of this specification, determining the first loss of the first decoder based on the first anchor label, the first random label, and the second anchor label may include:

[0121] The first anchor point information and the first anchor point label are fused to obtain the fused first anchor point label; based on the fused first anchor point label, the first random label and the second anchor point label, the first loss of the first decoder is determined.

[0122] It's understandable that the first negative sample label can be the output of the first decoder, the output of the second decoder, or any label selected from the queue of outputs from the first and second decoders. Specifically, the output of the first decoder can be stored as a queue, and labels can be extracted from this queue when negative samples are needed for decoder parameter optimization. Similarly, the output of the second decoder can be stored as a queue, and labels can be extracted from this queue when negative samples are needed for decoder parameter optimization. Alternatively, the outputs of both the first and second decoders can be stored together as a queue, and labels can be extracted from this queue when negative samples are needed for decoder parameter optimization. Furthermore, other decoders, such as the third decoder discussed later, can be used to process other training images, and loss calculations can be performed based on the processed negative sample labels. Before further processing the negative sample labels using other decoders, the negative sample labels can be fused with the corresponding sample features before further processing. It is understood that in other embodiments, it may be determined whether to directly select training images from the training sample set based on the current memory usage, and use the negative sample labels obtained after processing by the encoder and decoder to calculate the loss. That is, negative sample labels can be obtained in a variety of ways, and this specification does not limit this.

[0123] In some embodiments, when calculating the first loss, the first negative sample may not be set, and therefore the loss between the first anchor label and the first negative sample label does not need to be calculated. That is, the first loss may include the weighted loss between the first anchor label and the second anchor label, as well as the weighted loss between the first anchor label and the first random label. In this case, formula (1) can be replaced with:

[0124] Loss a =σL o +υL p

[0125] In some embodiments of this specification, in response to a first training image or a second training image, obtaining anchor point samples may include: performing at least one image enhancement process on the current training image, and using the at least one image obtained through the process and the current training image as anchor point samples, respectively. The image enhancement process may include, for example, random rotation, translation, noise addition, cropping, etc., as mentioned above, and this specification does not limit it.

[0126] In some embodiments of this specification, the pre-training may include: in response to the first training image: acquiring anchor samples, adjusting the anchor samples based on reference display parameters corresponding to the labels carried by the anchor samples to obtain a first reference sample; an encoder processing the first reference sample to obtain first reference information; a second decoder processing the first reference information to obtain a first reference label; determining a second loss of the second decoder based on the labels carried by the anchor samples and the first reference label; and updating the parameters of the second decoder based on the second loss.

[0127] It is understandable that reference display parameters can be the display parameters corresponding to the category of a specific region contained in the training image, which may include a specified window width and window level. Taking medical images as an example, different tissue categories have different recommended window widths and window levels. Differences in window width and window level may affect the contrast and brightness of the tissue image, and thus affect the quality of the displayed image. Adjusting the image to a suitable window width and window level based on the reference display parameters can make the tissue image clearer, thereby improving the accuracy of subsequent feature extraction and segmentation processing of the image.

[0128] In some embodiments of this specification, the pre-training may include: in response to the first training image: acquiring anchor samples, adjusting the anchor samples based on random display parameters to obtain second random samples; an encoder processing the anchor samples and the second random samples to obtain third anchor information and second random information, respectively; a first decoder processing the third anchor information to obtain a third anchor label; a first decoder or a second decoder processing the second random information to obtain a second random label; determining a third loss of the first decoder based on the label carried by the anchor samples, the third anchor label, the second random label, and the first reference label; and updating the parameters of the first decoder based on the third loss.

[0129] It is understood that the adjustments made to the anchor samples obtained from the first training image can refer to the enhancement and other processing of the anchor samples corresponding to the second training image described above, and will not be repeated here. The second random label can also be processed by the third decoder described below. That is, the second random information corresponding to the second random sample can be processed by any one of the first decoder, the second decoder, or the third decoder to obtain the second random label. This specification does not limit this.

[0130] In some embodiments of this specification, determining the third loss of the first decoder based on the tag carried by the anchor sample, the third anchor tag, the second random tag, and the first reference tag includes:

[0131] The third anchor point information and the third anchor point label are fused to obtain the fused third anchor point label; based on the fused third anchor point label, the first random label and the second anchor point label, the first loss of the first decoder is determined.

[0132] In some embodiments of this specification, determining the third loss of the first decoder based on the label carried by the anchor sample, the third anchor label, the second random label, and the first reference label may include: calculating the cross-entropy between the third anchor label and the label carried by the anchor sample; calculating the focal loss between the third anchor label and the label carried by the anchor sample; calculating the loss between the third anchor label and the first reference label; calculating the loss between the third anchor label and the second random label; and then weighting and summing the losses calculated above to obtain the third loss. In other embodiments, negative samples may be set, the loss between the negative sample label and the third anchor label may be calculated, and the loss may be weighted and summed with the aforementioned cross-entropy, focal loss, and other losses to obtain the third loss. Here, negative samples are training images other than the current training image, and negative samples may be obtained using the same method as the negative samples corresponding to the first negative sample label mentioned above. The loss between the third anchor label and the second random label may be the mean squared error loss, and the loss between the third anchor label and the negative sample label may be the noise comparison estimation loss value between the third anchor label and the negative sample label.

[0133] All of the above third anchor labels can be replaced with the aforementioned fused third anchor labels to calculate the loss. By fusing the anchor labels with the anchor information, the anchor labels also contain anchor information. This allows the first decoder to consider the original sample features when classifying images. Even if random window widths and levels are applied to the training samples, the model can still perform comparative learning based on the original anchor information F1. Therefore, the trained image segmentation model does not require manual setting of appropriate window widths and levels, and can adaptively perform accurate image segmentation.

[0134] In some embodiments of this specification, the neural network model may further include a third decoder, which may be updated based on the parameters of the first decoder; the third decoder may output a second negative sample label corresponding to a negative sample, and the first decoder may also update its parameters based on the second negative sample label.

[0135] It can be understood that negative samples are training images other than the current training image (i.e., anchor samples or positive samples obtained by transforming anchor samples) or samples obtained by transforming other training images.

[0136] In some embodiments of this specification, training images whose differences from anchor samples exceed a preset threshold can be selected from the training sample set as negative samples. Furthermore, the difference from anchor samples can be characterized by image similarity matching or by the distance between the image and the anchor sample's acquisition location. The distance between the image and the anchor sample's acquisition location can be the sequential distance along the image acquisition path during image acquisition.

[0137] In some embodiments of this specification, the third decoder can be updated using the following formula:

[0138] θ k =mθ k +(1-m)θ a Formula (2)

[0139] Where, θ k θ represents the parameters of the third decoder. a Let m represent the parameters of the first decoder, and m represent the association parameters. In some embodiments, m can take values ​​of [0.8, 1), preferably 0.99.

[0140] It is understandable that the third decoder can slowly update its own parameters according to the parameters of the first decoder based on the conservation of momentum, so that it is less affected by the first decoder, thereby ensuring that the output of the third decoder is basically consistent for the same input, and ensuring the consistency of features in the entire queue.

[0141] In some embodiments of this specification, the pre-training may include: in response to the first training image: acquiring anchor samples; adjusting the anchor samples based on random display parameters to obtain a third random sample; adjusting the anchor samples based on second reference display parameters corresponding to the labels carried by the anchor samples to obtain a second reference sample; an encoder processing the anchor samples, the third random sample, and the second reference sample to obtain fourth anchor information, third random information, and second reference information, respectively; a first decoder processing the fourth anchor information to obtain a fourth anchor label; a second decoder processing the second reference information to obtain a second reference label; a first decoder, a second decoder, or a third decoder processing the third random information to obtain a third random label; determining a fourth loss of the first decoder based on the labels carried by the anchor samples, the fourth anchor label, the third random label, the second reference label, and the second negative sample label; and updating the parameters of the first decoder based on the fourth loss.

[0142] It is understandable that the process of generating the third random sample and the second reference sample can be referred to the previous description of the first random sample and the first reference sample, and will not be repeated here.

[0143] In some embodiments of this specification, the fourth loss can be determined by the following formula:

[0144] Loss a =αL p +β(δL ce +εL fl +ηL cc +L n ) Formula (3)

[0145] Among them, Loss a This represents the fourth loss, where α, β, δ, ε, and η represent the weighting parameters; L ce L represents the cross-entropy between the fourth anchor tag and the tag carried by the anchor sample; fl This represents the focal loss between the fourth anchor label and the label carried by the anchor sample; L cc This indicates the loss between the fourth anchor label and the second reference label; L p This represents the loss between the fourth anchor label and the third random label; L n This represents the loss between the fourth anchor label and the second negative sample label. The loss between the fourth anchor label and the third random label can be the mean squared error loss, and the loss between the fourth anchor label and the second negative sample label can be the noise comparison estimation loss between them.

[0146] In some embodiments of this specification, determining the first loss of the first decoder based on the first anchor label, the first random label, and the second anchor label may include: fusing the first anchor label with the first anchor information to obtain a fused first anchor label; and determining the first loss of the first decoder based on the fused first anchor label, the first random label, and the second anchor label. Specifically, when calculating the loss value, the fused anchor label can be used to participate in the loss value L. n L p The calculation of L o At this point, the anchor labels before fusion can be used to calculate the loss value, while other loss values, such as L... ce L fl L cc L o Alternatively, the loss value can be calculated using the fused anchor labels.

[0147] The following uses training images as medical images, combined with... Figure 2 and Figure 3 The process of updating the decoder parameters in both tagged and untagged cases will be further described.

[0148] Step 1: Extract a batch of image data from the data heap, and use it as anchor point I. o And retrieve all data I in the queue. k During the initial training, the queue is empty. The queue is the queue obtained by storing the labels output by the decoder for the anchor sample set.

[0149] In this process, the images in each batch are randomly extracted, and the size of the data in a batch is set according to the memory of the actual training environment. There is no interaction between the data in a batch. The training of a batch is a step. After a step is completed, the next batch is extracted from the remaining data pile for the next step of training.

[0150] In cases where training is not the first time, the data in the queue is not empty. In one implementation, the data in the queue is filtered, and the data in the queue with a large difference from the anchor image is selected as I2, which is used as negative samples of the anchor sample in training. This filtering operation can avoid negative samples with small differences from interfering with training.

[0151] In one implementation, the training image data includes medical images of at least one patient. Multiple images of each patient are arranged into an image sequence according to their acquisition location or order. Therefore, images in the queue whose sequence distance from the anchor image is greater than 3 can be filtered out and denoted as I. k In this embodiment, the sequence distance 3 is obtained based on the spacing between the Z-axis of the sequence images. Those skilled in the art can also set the filtering conditions to sequence distance 4, 5, 10 or other reasonable minimum sequence distances, which are not limited here.

[0152] Step 2, for anchor point I o Perform random data augmentation, such as random rotation, translation, noise addition, cropping, etc., and then adjust to a more suitable window width and window level to obtain image I′5 (that is, the reference sample mentioned above).

[0153] In one implementation, adjusting the window width and window level specifically involves determining the parameters based on the label carried by the current sample, i.e., the tissue type represented by the label, to adjust the window width and window level of the image to obtain image I′5 (for example, when creating an image I′5 with the suggested window width and window level corresponding to the bone window when the label is bone); and, without determining the parameters according to the label, randomly adjusting the window width and window level of the image to obtain image I′3 with a random window width and window level (i.e., the random sample mentioned above), where I... o I′3 and I′5 can be denoted as positive samples. Random data augmentation and random adjustment of image window width and level can increase the amount of image data, improve the model's generalization and robustness, and avoid network overfitting. The recommended window width and level from various organizations can adopt publicly available information in the field. In some embodiments, the recommendations shown in Table 1 below are used:

[0154] Table 1

[0155] organize Window width (HU) Window position (HU) Lung window 1500 -400 Skull Enhancement 85 40 liver 200 50 kidney 250 30 throat 325 40 pancreas 40 120 Bone window 1250 300 Soft tissue window 400 50

[0156] In the embodiments of this specification, the neural network model may adopt an Actor-Critic architecture, such as... Figure 4 As shown, a critic decoder (i.e., the second decoder mentioned earlier) and an actor decoder (i.e., the first decoder mentioned earlier) are set up. The main role of the critic network is to improve the segmentation ability while ensuring the generalization ability of the actor decoder. Since the network will diverge in various directions without labels, the constraint of the critic decoder ensures that the network develops in a reasonable general direction. It is understandable that other architectures, such as student-teacher networks, can also be used. The key is that two decoders are required, and there is a mentoring relationship between the two decoders. The output feature F′5 of the image I′5 after being processed by the encoder is used as the input of the critic decoder, because relying solely on the presence or absence of labels as the difference between the two decoders is not enough. An appropriate window width and window position have a great impact on the segmentation result. F′3 is the feature corresponding to the image after random window width and window position, which is used as the input of the actor decoder.

[0157] Step 3: Use the critic decoder and actor decoder to process the input features and output the corresponding labels.

[0158] In this embodiment, a weakly supervised self-evaluation network structure is adopted in the encoder-decoder part to facilitate the learning of the decoder using unlabeled data and obtain reliable pseudo-labels from the decoder output. That is, different classification predictions are made from different perspectives, and then the critic network guides the actor network for training. Specifically, the encoder first extracts deep features from the image through downsampling. The number of downsampling layers is determined based on the area of ​​the smallest ROI. Deeper downsampling can easily cause small ROI features such as the lens to disappear. In some embodiments, three to four downsampling layers are used depending on the resolution of the training images. Correspondingly, upsampling is required in the decoder. It is understood that if the encoder does not use upsampling but uses other methods for feature extraction, the operations in the decoder also need to be adjusted accordingly. The adjustment method can refer to conventional techniques in the art. The encoder downsampling is mainly to expand global perception and reduce computational load. In this embodiment, the encoder can be constructed using residual blocks of a ResNet network. In some implementations, a three-decoder structure is used. All three decoders can use the self-attention mechanism of a Transformer network, or can be replaced with other networks. The encoder corresponds to I... oThe corresponding results obtained from I′3, I′5, and I2 can include: anchor feature F1, positive sample feature F′3, specified window width and bit position image feature F′5, and negative sample feature F2. It is understandable that only the first training image with labels has I′5 and F′5, while the second training image without labels, due to its unknown category, cannot obtain the specified window width and bit position, and therefore does not have I′5 and F′5 data. The critic network only performs fully supervised training on labeled data with specified window width and bit position to obtain accurate mask (i.e., pseudo-label) segmentation ability, while the actor network is responsible for semi-supervised training on all data (including labeled and unlabeled data). Its generalization ability is strong, but its mask segmentation ability is weaker than that of the critic network.

[0159] Step 4: For labeled data, anchor feature F1 is used as input to the actor decoder to obtain mask O1, and specified window width bit image feature F′5 is used as input to the critic decoder to obtain mask O′5. Positive sample feature F′3 is input to decoder K to obtain O′3, and negative sample feature F2 is input to decoder K to obtain O′3 and O2. Masks O1, O′3, O2, and O′5 are all output labels of the decoder, i.e., pseudo-labels, which are essentially the predicted category of each pixel.

[0160] Reference Figure 2 The loss of the actor's decoder includes the loss value Lcc of the mask O1 and the critic's decoder output O′5; and the loss of the true label (i.e., Figure 2 The cross-entropy and focus loss values ​​Lce+Lfl of the label in the image; the mask O1 is fused with the original feature F1 to obtain the new O1 and the mean squared error loss value L of the positive sample output O′3. p The loss value Ln of the noise contrastive estimation (NCE) between the output O2 and the negative sample (the specific feature fusion method is an existing technique). The specific calculation formula for the loss of the actor decoder can be referred to formula (3) above. The loss of the critic decoder is the loss value Ln between the output O′5 and the true label. c .

[0161] For unlabeled data, the actor's network primarily relies on the critic's network to form the loss. The input to the actor's decoder is anchor feature F1 and positive sample feature F′3, and the corresponding output mask O is generated after processing. a1 With O3, corresponding to the first anchor label and the first random label mentioned earlier, the input to the critic decoder is F1, and the corresponding output mask after processing is O. c1 This corresponds to the second anchor label mentioned earlier, by comparing the outputs O of the two networks. a1 With O c1 The loss value L is obtained. o Calculate the loss L between the first anchor label and the positive and negative samples. fThe parameters of the actor's home network are updated. The calculation process for the loss of the actor's home decoder can be found in formula (1) above, and the update process can be found in [the relevant section]. Figure 3 .

[0162] Decoder K updates its own parameters slowly according to the parameters of decoder Actor, based on the conservation of momentum, to ensure the consistency of features throughout the queue. The specific update formula for the parameters of decoder K can be found in formula (2) above.

[0163] Step 5: Store the output feature O1 in the queue.

[0164] In specific storage, the first-in-first-out (FIFO) principle can be adopted. The number of queues can be determined according to the training environment. Ideally, the larger the better. Due to memory constraints, the FIFO method is used to maintain the image features of a queue.

[0165] In this embodiment, refer to Figure 2 As shown, when the queue is full with features from 10 batches, the newly generated feature O1 enters the queue, while the first feature to enter, b1, leaves the queue. Features b2 through b10 move forward one position, and O1 occupies the position of b10. The purpose of the queue is to provide sufficient negative samples for comparative learning of the original images. When the queue is empty, only the original images are available for learning, meaning that initially only positive samples are compared and learned. Although this method will have some impact on the training results—that is, it only affects one batch—as the queue becomes full, previous features are removed from the queue, minimizing the impact on the final training results.

[0166] Step 6: After training is complete, use the trained image segmentation model to segment the medical images to obtain the extent, outline, and category of the tissues in the medical images.

[0167] In some implementations, such as Figure 4 As shown, the first training image includes multiple specific regions, and the first training image carries multiple labels. The categories of all specific regions contained in the training sample set correspond at least to the categories of the multiple target regions. An image segmentation model for identifying and segmenting multiple specific regions is trained. Figure 4The image segmentation model automatically fuses multiple target regions in the output segmented image. The output segmentation result includes the fused target regions and their corresponding categories, which are then overlaid and displayed. This image segmentation model can segment all categories at once, resulting in higher segmentation efficiency and a wider range of applications. For example, multiple images of the same patient can be segmented into multiple regions of interest, and then feature fusion can be performed. Different tissue categories are marked with different colors to define their contours, and finally displayed on a single CT image, with the names of the tissue categories represented by the different colors indicated.

[0168] This specification's embodiments improve upon medical image segmentation algorithms. By employing a weakly supervised self-evaluation network structure, it accelerates model convergence, performs different classification predictions from various perspectives, and then guides the actor decoder through unlabeled training via a critic decoder. This method exhibits low dependence on the dataset and good adaptability. Segmentation using this method eliminates the need for individual adjustments to window width and level. Accurate segmentation can be obtained regardless of the appropriateness of the input image's window width and level. This is because the model is trained using both labeled and unlabeled training images according to specific rules. This method integrates the influence of window width and level on image features and segmentation into the model, enabling stable and excellent image segmentation capabilities unaffected by window width and level. It can achieve automatic segmentation of the entire region and improve the stability of the segmentation results. It is understood that the ability to adaptively segment medical images using window width and level is only one aspect of the method's beneficial effects and does not represent its entire value. This method can also produce beneficial effects such as low dependence on training samples, high model adaptability, and accurate segmentation results when applied to other image segmentation fields. Furthermore, the contrastive learning in the model is performed at the pixel level. Pixel-level contrastive learning can improve the cosine similarity between pixels of the same category (i.e., the use of the loss value Ln mentioned earlier). By updating the parameters of the image segmentation model using this method, the cosine similarity between different categories can be reduced, the decoder parameters (especially the actor network) can be optimized, the segmentation ability can be improved, and the accuracy of the segmentation results can be high.

[0169] The image segmentation method in this embodiment can be applied not only to CT images but also to PET-CT images. By training a deep learning model, it automatically segments different structures and tissues in the images (especially including the segmentation of tumor target areas), thereby achieving accurate target delineation. Addressing the impact of window width and level in medical image data on feature recognition in deep learning models, and the limited availability of CT, PET, and other medical image data for training deep learning models, this method uses artificial intelligence algorithms for deep learning and intelligent analysis of medical images. The model is trained using the original image data fused from PET-CT, avoiding image feature extraction biases caused by different window width and level settings, and effectively reducing the amount of training data. Ultimately, it achieves automatic identification of treatment target areas in images under limited sample conditions, assisting doctors in delineation and accelerating treatment planning. It is understood that the image segmentation method in this embodiment is not limited to the aforementioned CT or PET-CT medical images; the method provided in this application embodiment can also be used for segmentation processing of other unlisted types of medical images, such as MRI.

[0170] Based on the image segmentation method described above, one or more embodiments of this specification also provide an image segmentation apparatus. The apparatus may include devices (including distributed systems), software (applications), modules, plug-ins, servers, clients, etc., using the methods described in the embodiments of this specification, combined with necessary hardware implementations. Based on the same innovative concept, the apparatuses in one or more embodiments provided in this specification are as described in the following embodiments. Since the implementation schemes and methods for solving the problem are similar, the implementation of specific apparatuses in the embodiments of this specification can refer to the implementation of the foregoing methods, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the apparatuses described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated. Figure 5 As shown, the image segmentation device 500 may include:

[0171] Image acquisition module 501 is used to acquire the image of the target object to be processed.

[0172] Image segmentation module 502 is used to input the image to be processed into a pre-trained image segmentation model and output a segmented image, wherein the segmented image includes a target region and the category of the target region; wherein the image segmentation model is obtained by pre-training a neural network model including an encoder, a first decoder and a second decoder using a training sample set including a first training image with labels and a second training image without labels; the pre-training includes: the first decoder updating its parameters using the first training image with labels and the labels generated by the second decoder based on the second training image without labels, and the second decoder updating its parameters using the first training image with labels.

[0173] The descriptions and functions of the above modules can be found in the section on image segmentation methods, and will not be repeated here.

[0174] The image acquisition module 501 and the image segmentation module 502 can be independent execution units or they can share a single execution unit. The execution unit may include one or more processors, or it may include one or more processors and one or more memories, with the memory storing instructions that can be executed by the processor.

[0175] Based on the same innovative concept of the image segmentation method described above, one or more embodiments of this specification also provide a method for generating a radiotherapy plan, such as... Figure 6 As shown, it may include:

[0176] S601: Obtain the image to be processed of the target object.

[0177] S602: Input the image to be processed into a pre-trained image segmentation model and output a segmented image, wherein the segmented image includes the target region and the category of the target region.

[0178] It is understood that this step in the radiotherapy plan generation method can employ the image segmentation methods described in the embodiments above in this specification, and can be implemented with reference to the aforementioned image segmentation methods; repeated details will not be elaborated upon. Furthermore, the image segmentation model can also be trained using other model training methods related to the art.

[0179] S603: Determine the material information of the target region based on the category of the target region, and establish a three-dimensional voxel model based on the target region and the material information of the target region.

[0180] S604: Perform tissue dose simulation calculations based on the three-dimensional voxel model and generate a treatment plan. The method for generating the treatment plan in this step can refer to relevant techniques in the art. The target region and its category obtained using the image segmentation method described above are more accurate, therefore the generated treatment plan is more accurate and has better results.

[0181] In the field of radiotherapy, the treatment target area for BNCT (boron neutron capture therapy) differs from that of traditional radiotherapy. Generally, the characteristics of target areas such as GTV (Gross Tumor Volume), CTV (clinical target volume), and PTV (planning target volume) are not defined separately. This method can be applied to BNCT treatment by segmenting PET-CT images fused with PET and CT images containing functional information such as F-BPA drug distribution. By training a deep learning model, it automatically segments different structures and tissues in the images (especially including the segmentation of the tumor target area), thereby achieving accurate target delineation.

[0182] Currently, BNCT (Brain-Nutritive Clinical Trial) is relatively new, and the amount of medical imaging data such as CT and PET available for training deep learning models is limited, making it difficult to obtain large amounts of training data. Labeled data is even scarcer. This method addresses these issues and considers the impact of window width and window level in medical imaging data on feature recognition in deep learning models. It uses artificial intelligence algorithms to perform deep learning and intelligent analysis on medical images. The model can be trained using raw image data fused with PET-CT, avoiding image feature extraction bias caused by different window width and window level settings. At the same time, it effectively reduces the amount of training data required, ultimately achieving automatic, accurate, and reliable identification of treatment target areas under limited sample conditions, assisting doctors in delineation, and accelerating treatment planning.

[0183] PET images with F-BPA drug distribution information are unique to BNCT (Brain Neck Surgery) and display the distribution of boron drug concentrations in the patient's body using high contrast. Although F-BPA and CT may have misalignment and resolution limitations, F-BPA still provides very useful information to help manually delineate the treatment target area on CT. The complementary advantages of F-BPA and CT are crucial for accurate target delineation. This method can use PET-CT fused images and initial CT and PET multimodal image data as input to the image segmentation model, thereby integrating the unique semantic features of PET and CT for deep training to automatically identify the treatment target area and assist doctors in delineation. This method can achieve key segmentation of the nine most relevant major organs at risk (OARs) in head and neck BNCT, including the brainstem, chiasm, mandible, left optic nerve, right optic nerve, left parotid gland, right parotid gland, left mandible, and right mandible.

[0184] Based on the same innovative concept of the above-described radiotherapy plan generation method, one or more embodiments of this specification also provide a radiotherapy system 700. Since the system's solution to the problem is similar to the radiotherapy plan generation method, the specific implementation of the system in the embodiments of this specification can refer to the implementation of the aforementioned method; repeated details will not be elaborated further. Figure 7 As shown, the radiotherapy system 700 may include:

[0185] The beam irradiation device 701 is used to generate a radiation beam.

[0186] Treatment plan module 702 is used to generate treatment plans.

[0187] The control module 703 is used to control the beam irradiation device 701 to generate a radiation beam according to the treatment plan.

[0188] The treatment planning module 702 may include:

[0189] The image acquisition unit 7021 is used to acquire the image of the target object to be processed.

[0190] The image segmentation unit 7022 is used to input the image to be processed into a pre-trained image segmentation model and output a segmented image, wherein the segmented image includes the target region and the category of the target region.

[0191] The model building unit 7023 is used to determine the material information of the target region based on the category of the target region, and to build a three-dimensional voxel model based on the target region and the material information of the target region.

[0192] The calculation unit 7024 is used to perform tissue dose simulation calculations based on the three-dimensional voxel model and generate a treatment plan.

[0193] The treatment planning module 702, control module 703, image acquisition unit 7021, image segmentation unit 7022, model building unit 7023, and calculation unit 7024 can each be an independent execution unit, or they can share an execution unit with one or more of the modules. The execution unit may include one or more processors, or it may include one or more processors and one or more memories, and the memory stores instructions that can be executed by the processor.

[0194] In some embodiments, the radiotherapy system may include a treatment planning system (TPS). In other embodiments, the radiotherapy system may also include a robotic arm control system. In still other embodiments, the radiotherapy system may also include a data management system for interacting with and processing data from other systems and modules. Accordingly, one or more of the treatment planning module 702, control module 703, image acquisition unit 7021, image segmentation unit 7022, model building unit 7023, and calculation unit 7024 may be located in the TPS, the robotic arm control system, the data management system, or in a separately located data processing device. The specific configuration can be set according to actual needs and circumstances, and is not limited here.

[0195] This application also provides an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 801 and a memory 802, wherein the processor 801 and the memory 802 may be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0196] Processor 801 can be a Central Processing Unit (CPU). Processor 801 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0197] The memory 802, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the image segmentation method and / or radiotherapy plan generation method in the embodiments of the present invention. The processor 801 executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory 802, thereby implementing the image segmentation method and / or radiotherapy plan generation method in the above method embodiments.

[0198] The memory 802 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor 801, etc. Furthermore, the memory 802 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 802 may optionally include memory remotely located relative to the processor 801, and these remote memories may be connected to the processor 801 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0199] The one or more modules are stored in the memory 802. In some embodiments, when executed by the processor 801, they perform actions such as... Figure 1 The image segmentation method in the illustrated embodiment. In some embodiments, when executed by the processor 801, the aforementioned radiotherapy plan generation method is performed.

[0200] The specific details of the aforementioned electronic device can be understood by referring to the relevant descriptions and effects in the above method embodiments, and will not be repeated here.

[0201] This specification also provides a computer storage medium storing computer program instructions, which, when executed, implement the steps of the image segmentation method described above. In some embodiments, when executed, the computer program instructions implement the radiotherapy planning generation method described above.

[0202] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0203] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.

[0204] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0205] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute certain parts of the methods of various embodiments of this application.

[0206] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0207] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0208] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments in this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0209] The above description is merely an embodiment of the present specification and is not intended to limit the embodiments of the present specification. For those skilled in the art, various modifications and variations can be made to the embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present specification should be included within the scope of the claims of the embodiments of the present specification.

Claims

1. An image segmentation method, characterized in that, include: Obtain the image of the target object to be processed; The image to be processed is input into a pre-trained image segmentation model, which outputs a segmented image, which includes a target region and the category of the target region. The image segmentation model is pre-trained using a training sample set including a first training image with labels and a second training image without labels. The pre-training includes: the first decoder updating its parameters using the first training image with labels and the labels generated by the second decoder based on the second training image without labels; and the second decoder updating its parameters using the first training image with labels.

2. The method according to claim 1, characterized in that, The pre-training includes: The parameters of the first decoder are updated based on the labels carried by the first training image, the first output, and the third output, and / or the parameters of the first decoder are updated based on the second output and the fourth output. The parameters of the second decoder are updated based on the third output and the labels carried by the first training image; Wherein, the first output is the output of the first decoder corresponding to the first training image; The second output is the output of the first decoder corresponding to the second training image; The third output is the output of the second decoder corresponding to the first training image; The fourth output is the output of the second decoder corresponding to the second training image.

3. The method according to claim 1, characterized in that, The pre-training includes: In response to the second training image: acquire anchor point samples, adjust the anchor point samples based on random display parameters, and obtain the first random sample; The encoder processes the anchor point sample and the first random sample to obtain the first anchor point information and the first random information, respectively. The first decoder processes the first anchor information and the first random information to obtain the first anchor label and the first random label, respectively; the second decoder processes the first anchor information to obtain the second anchor label. Based on the first anchor label, the first random label, and the second anchor label, the first loss of the first decoder is determined; The parameters of the first decoder are updated based on the first loss.

4. The method according to claim 3, characterized in that, The first loss is determined by the following formula: Among them, Loss a Let L represent the first loss, σ and υ represent the weighting parameters, and L represent the weighting parameters, respectively. o L represents the loss between the first anchor label and the second anchor label. p L represents the loss between the first anchor label and the first random label. n This represents the loss between the first anchor label and the first negative sample label, where the first negative sample label is obtained based on the output of the neural network model corresponding to other training images.

5. The method according to claim 1, characterized in that, The pre-training includes: In response to the first training image: acquire anchor point samples, and adjust the anchor point samples based on the reference display parameters corresponding to the labels carried by the anchor point samples to obtain the first reference sample; The encoder processes the first reference sample to obtain the first reference information; The second decoder processes the first reference information to obtain the first reference tag; Based on the labels carried by the anchor samples and the first reference label, the second loss of the second decoder is determined; The parameters of the second decoder are updated based on the second loss.

6. The method according to claim 5, characterized in that, The pre-training includes: In response to the first training image: obtain anchor point samples, adjust the anchor point samples based on random display parameters, and obtain a second random sample; The encoder processes the anchor point sample and the second random sample to obtain the third anchor point information and the second random information, respectively. The first decoder processes the third anchor point information to obtain the third anchor point label; The first decoder or the second decoder processes the second random information to obtain the second random label; Based on the labels carried by the anchor samples, the third anchor label, the second random label, and the first reference label, the third loss of the first decoder is determined; The parameters of the first decoder are updated based on the third loss.

7. The method according to claim 1, characterized in that, The neural network model also includes a third decoder, which updates based on the parameters of the first decoder; The third decoder outputs a second negative sample label for the negative sample, and the first decoder also updates its parameters based on the second negative sample label.

8. The method according to claim 1, characterized in that, The segmented image includes multiple target regions and multiple categories of target regions; The labels carried by the first training image include a specific region in the first training image and the category of the specific region; The categories of all specific regions contained in the training sample set correspond at least to the categories of the multiple target regions.

9. An image segmentation apparatus, characterized in that, include: The image acquisition module is used to acquire the image of the target object to be processed. An image segmentation module is used to input the image to be processed into a pre-trained image segmentation model and output a segmented image, wherein the segmented image includes a target region and the category of the target region; wherein the image segmentation model is obtained by pre-training a neural network model including an encoder, a first decoder and a second decoder using a training sample set including a first training image with labels and a second training image without labels; the pre-training includes: the first decoder updating its parameters using the first training image with labels and the labels generated by the second decoder based on the second training image without labels, and the second decoder updating its parameters using the first training image with labels.

10. A method for generating a radiotherapy plan, characterized in that, include: Obtain the image of the target object to be processed; The image to be processed is input into a pre-trained image segmentation model, which outputs a segmented image, which includes the target region and the category of the target region; The material information of the target region is determined based on the category of the target region, and a three-dimensional voxel model is established based on the target region and the material information of the target region. Based on the three-dimensional voxel model, tissue dose simulation calculations are performed, and a treatment plan is generated.

11. A radiotherapy system, characterized in that, include: A beam irradiation device used to generate a radiation beam; The treatment plan module is used to generate treatment plans; The control module is used to control the beam irradiation device to generate a radiation beam according to the treatment plan; The treatment plan module includes: The image acquisition unit is used to acquire the image of the target object to be processed. An image segmentation unit is used to input the image to be processed into a pre-trained image segmentation model and output a segmented image, wherein the segmented image includes the target region and the category of the target region; The model building unit is used to determine the material information of the target region based on the category of the target region, and to build a three-dimensional voxel model based on the target region and the material information of the target region. The calculation unit is used to perform tissue dose simulation calculations based on the three-dimensional voxel model and generate a treatment plan.

12. An electronic device comprising a processor and a memory for storing processor-executable instructions, characterized in that, When the processor executes the instructions, it implements the steps of the method according to any one of claims 1 to 8 or claim 10.

13. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When executed by a processor, the computer program / instructions implement the steps of the method as described in any one of claims 1 to 8 or claim 10.

Images