A collaborative approach for weakly supervised medical image segmentation and registration based on contrastive learning and spatial encoding

Abstract

The present invention provides a weakly supervised medical image segmentation and registration collaborative method based on contrastive learning and spatial coding, which first constructs an image segmentation model, pre-trains the segmentation model with a small amount of annotated images; obtains medical image data and its organ segmentation annotations, and divides them into floating images and fixed images; uses the segmentation model to predict image segmentation annotations without segmentation annotations; uses the predicted segmentation annotations to calculate the spatial encoding of the image for training the registration network; uses the segmentation annotations to calculate structural correspondence information for training the pixel-level Siamese network; uses the registration model to generate augmented data combined with the Siamese network to retrain the segmentation model, and repeats the training until multiple models converge. Through the above registration and segmentation models, the performance of medical image segmentation and registration tasks in small sample conditions is greatly improved, and the collaborative joint processing between medical image registration and segmentation tasks is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation and image registration, and in particular to a weakly supervised medical image segmentation and registration collaborative method based on contrastive learning and spatial coding. Background Art

[0002] Image segmentation and image registration are two key tasks in image processing, and they are also widely used in the fields of medical assistance and biological research. They can effectively assist doctors in diagnosing lesions and analyzing and processing the organ structure of organisms. At the same time, these two tasks are also highly related and can promote each other. For example, labeled atlas images are used for segmentation through image registration as data augmentation. Segmentation can also provide additional supervision for image registration (in addition to image intensity) and be used to evaluate the registration results.

[0003] After deep learning developed and was widely used in various fields, it brought new development directions to image registration and image segmentation. However, unlike other computer vision fields with tens of thousands of data, data in the biomedical field is difficult to obtain. At the same time, data annotation is time-consuming and labor-intensive and requires expert knowledge, and the amount of data is usually in the hundreds. Therefore, deep learning technology in the biomedical field often needs to be implemented in scenarios with small samples and a small amount of labeled data. Therefore, how to make full use of existing data, while mining the correlation and complementarity between multiple tasks, improving the generalization ability of the model, and ultimately improving the performance of the two tasks has become an urgent problem to be solved. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a weakly supervised medical image segmentation and registration collaborative processing method that greatly improves the performance of image segmentation tasks and image registration tasks in small sample scenarios, improves both the image segmentation accuracy and the image registration accuracy, and provides a new solution for medical image segmentation and registration tasks in small sample scenarios.

[0005] A first aspect of the present invention provides a weakly supervised medical image segmentation and registration collaborative method based on contrastive learning and spatial encoding, the method comprising:

[0006] Acquiring a medical image to be processed;

[0007] A collaborative process of performing registration and segmentation on the medical image is performed according to a pre-trained registration and segmentation model.

[0008] Preferably, the registration and segmentation model is pre-trained and generated based on the following steps:

[0009] S1, build a segmentation model with a small amount of labeled medical image data, perform pre-training, and use the pre-trained segmentation model to predict the segmentation annotations of the remaining images;

[0010] S2, dividing the acquired medical image data into floating images and fixed images according to whether they have manual segmentation annotations;

[0011] S3, using the segmentation model to predict the segmentation annotation of the image, using the predicted segmentation annotation to generate the corresponding spatial encoding, and using the encoding information to train the Voxel-Morph registration model;

[0012] S4, randomly pair the images in the dataset, use the segmentation labels predicted by the two paired images to generate a texture correspondence map, and use the texture correspondence map to train the pixel-level Siamese model;

[0013] S5, using the Voxel-Morph registration model to generate augmented data for the segmentation network, including full supervision data and weak supervision data, using the Siamese network to filter and weight the weak supervision data to reduce the impact of erroneous information, and use this information to retrain the segmentation network;

[0014] S6, repeating steps S3 to S5 until the registration model and the segmentation model converge to generate a registration and segmentation model.

[0015] Preferably, the S1 uses a small amount of labeled medical image data to build a segmentation model, performs pre-training, and uses the pre-trained segmentation model to predict the segmentation annotations of the remaining images, including:

[0016] S11, the segmentation network model adopts the U-net network architecture, and a small number of labeled images are sent to the network for pre-training;

[0017] S12, use the trained segmentation network to perform segmentation prediction on the fixed image to obtain the predicted label of the fixed image.

[0018] Preferably, the step S2, dividing the acquired medical image data into floating images and fixed images according to whether they have manual segmentation annotations, includes:

[0019] S21, let the number of images in the medical image dataset be n, among which only a small number of images contain manual segmentation and annotation of organ structures, which are called floating images, and let their number be x, and the remaining nx images have no segmentation annotation, which are called fixed images;

[0020] S22, randomly pairing the floating image with the fixed image, so that the floating image in each pair of images has a segmentation annotation, and the fixed image has no segmentation annotation.

[0021] Preferably, the step S3, using the segmentation model to predict the segmentation annotation of the image, using the predicted segmentation annotation to generate corresponding spatial encoding, and using the encoding information to train the Voxel-Morph registration model, includes:

[0022] S31, sending the fixed image in each pair of images into the segmentation model to obtain the predicted segmentation annotation;

[0023] S32, converting the segmentation annotations of the images in the image pair into one-hot encoding to facilitate subsequent calculations;

[0024] S33, performing spatial coding calculation on each voxel in the three-dimensional medical image, and the calculation method is as follows:

[0025]

[0026]

[0027]

[0028] As shown in the above formula, the spatial encoding of each voxel consists of three bits, Represents the n-th bit code of the voxel with coordinates (x, y, z) in the three-dimensional image, label one-shot The segmentation annotation of the image in one-hot encoding form is mathematically represented as a four-dimensional matrix, where i, l, w, and h represent the anatomical structure, length, width, and height, respectively;

[0029] S34, connecting the code of each image with the original image to form a four-channel image, wherein the original image occupies one channel and the spatial code occupies three channels;

[0030] S35, the registration model built uses the Voxel-Morph network architecture, and the original 2 input channels are modified to 6 input channels to adapt to the new input type after adding spatial encoding;

[0031] S36, the trained registration model will output a deformation field that can register the floating image to the fixed image, as shown in the following formula:

[0032] Φ=Reger(I mov ,I fix )

[0033]

[0034] Where Reger() represents the registration model, I mov and I′ mov They represent the floating image and the result after the floating image is registered to the fixed image respectively, Φ represents the deformation field output by the network, and the ° symbol represents the deformation operation of the image through the deformation field.

[0035] Preferably, the S4, randomly pairing the images in the data set, using the segmentation labels predicted by the two paired images to generate a texture correspondence map, and using the texture correspondence map to train a pixel-level Siamese model, includes:

[0036] S41, since the processing object of the Siamese network is a pair of images, the medical image dataset used for training is first randomly paired, and the segmentation annotations of the two images are converted into one-hot encoding;

[0037] S42, using the segmentation annotations predicted by the segmentation model to calculate the structural correspondence between the two images, the calculation process is as follows:

[0038]

[0039]

[0040] Among them, P pos and P nat They respectively represent the regions where the structures of the two paired images correspond and the regions where the structures do not correspond; and Represents the one-hot segmentation annotation of the two images respectively;

[0041] S43 uses the supervision information obtained by segmentation annotation to train the Siamese network. Its objective function consists of two parts. The first part is the contrast loss function, which is calculated as follows:

[0042]

[0043] Among them, L represents the final objective function value used for back propagation, d represents the Euclidean distance of the feature vectors of the two images after Siamese network encoding, and margin is an artificially set threshold; the above objective function can constrain the Siamese network to make the encoding results of the structural areas corresponding to the images consistent, otherwise the differences will be enlarged;

[0044] S44, the objective function of the second part is used to constrain the final output of the network, so that we can obtain a confidence map that reflects the texture correspondence at the same position between the input image pairs through the Siamese network. The formula is as follows:

[0045]

[0046] Where N is the number of voxels in the image matrix, P acm Represents the confidence map output by the Siamese network.

[0047] Preferably, S5, using a Voxel-Morph registration model to generate augmented data for the segmentation network, including fully supervised data and weakly supervised data, using a Siamese network to filter and weight the weakly supervised data to reduce the impact of erroneous information, and using this information to retrain the segmentation network, includes:

[0048] S51, the floating image I with segmentation annotation mov and a fixed image I without annotations fix Send it into the registration model to get the registered floating image I′ mov and its segmentation label ′ mov ; then I′ mov and label′ mov The correspondence can be regarded as a pair of fully supervised data, label′ mov and I fix The correspondence can be regarded as a pair of weakly supervised data; the specific process is as follows:

[0049] Φ=Reger(I mov ,I fix )

[0050]

[0051]

[0052] S52, the full supervision data and weak supervision data augmented by the registration model are sent to the segmentation model for further training; when using weak supervision data for training, I mov and I fix Input into the Siamese network to obtain the confidence map of the structural correspondence between the two images. This confidence map can be used to perform weighted screening on the weakly supervised data and remove the erroneous information in the weakly supervised data. The specific usage is as follows:

[0053] L acm =P acm ·label′ mov ·log(S(I fix ))

[0054] In the above formula, S(I fix ) represents the segmentation model. The actual calculation process is to multiply the cross entropy function by the confidence map, and screen the loss value of each voxel. Only the area with correct supervision information is counted into the value for back propagation.

[0055] The second aspect of the present invention provides an electronic device, which includes: one or more processors, and a memory, wherein the memory is used to store one or more computer programs; the computer program is configured to be executed by the one or more processors, and the program includes steps for executing the weakly supervised medical image segmentation and registration collaborative method based on contrastive learning and spatial encoding as described above.

[0056] The third aspect of the present invention provides a storage medium storing a computer program; characterized in that the program is loaded and executed by a processor to implement the steps of the weakly supervised medical image segmentation and registration collaborative method based on contrast learning and spatial encoding as described above.

[0057] In the scheme of the present invention, the medical image is subjected to collaborative processing of registration and segmentation according to a pre-trained registration and segmentation model; the registration and segmentation model is obtained by first constructing an image segmentation model, and pre-training the segmentation model with a small amount of annotated images; obtaining medical image data and its organ segmentation annotations, and dividing them into floating images and fixed images; using the segmentation model to predict the image segmentation annotations without segmentation annotations; using the predicted segmentation annotations to calculate the spatial encoding of the image for training the registration network; using the segmentation annotations to calculate the structural correspondence information for training the pixel-level Siamese network; using the registration model to generate augmented data combined with the Siamese network to retrain the segmentation model, and repeating the training until multiple models converge. Through the above registration and segmentation model, the performance of medical image segmentation and registration tasks in small sample conditions is greatly improved, and collaborative joint processing between medical image registration and segmentation tasks is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 is a flowchart of the steps of pre-training and generating the registration and segmentation model disclosed in the embodiment of the present invention;

[0060] Figure 2 It is a specific generation structure diagram of the embodiment disclosed in the embodiment of the present invention, including the Voxel-Morph unsupervised registration model, the pixel-level Siamese model's predetermined registration and segmentation model. DETAILED DESCRIPTION

[0061] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0062] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0063] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0064] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0065] The implementation details of the technical solution of the embodiment of the present application are described in detail below:

[0066] A weakly supervised medical image segmentation and registration collaborative method based on contrastive learning and spatial coding according to an embodiment of the present invention comprises:

[0067] Acquire a medical image to be processed; and perform collaborative processing of registration and segmentation on the medical image according to a pre-trained registration and segmentation model.

[0068] Preferably, Figure 1 As shown, the registration and segmentation model is pre-trained and generated based on the following steps:

[0069] S1, build a segmentation model with a small amount of labeled medical image data, perform pre-training, and use the pre-trained segmentation model to predict the segmentation annotations of the remaining images;

[0070] S2, dividing the acquired medical image data into floating images and fixed images according to whether they have manual segmentation annotations;

[0071] S3, using the segmentation model to predict the segmentation annotation of the image, using the predicted segmentation annotation to generate the corresponding spatial encoding, and using the encoding information to train the Voxel-Morph registration model;

[0072] S4, randomly pair the images in the dataset, use the segmentation labels predicted by the two paired images to generate a texture correspondence map, and use the texture correspondence map to train the pixel-level Siamese model;

[0073] S5, using the Voxel-Morph registration model to generate augmented data for the segmentation network, including full supervision data and weak supervision data, using the Siamese network to filter and weight the weak supervision data to reduce the impact of erroneous information, and use this information to retrain the segmentation network;

[0074] S6, repeating steps S3 to S5 until the registration model and the segmentation model converge to generate a registration and segmentation model.

[0075] Further, if Figure 2 As shown, it is a specific generation structure diagram of the Voxel-Morph unsupervised registration model, the pixel-level Siamese model, and the segmentation model. ,

[0076] Preferably, the S1 uses a small amount of labeled medical image data to build a segmentation model, performs pre-training, and uses the pre-trained segmentation model to predict the segmentation annotations of the remaining images, including:

[0077] S11, the segmentation network model adopts the U-net network architecture, and a small number of labeled images are sent to the network for pre-training;

[0078] S12, use the trained segmentation network to perform segmentation prediction on the fixed image to obtain the predicted label of the fixed image.

[0079] Preferably, the S2, dividing the acquired medical image data into floating images (Mov images) and fixed images (Fix images) according to whether they have manual segmentation annotations, includes:

[0080] S21, let the number of images in the medical image dataset be n, among which only a small number of images contain manual segmentation and annotation of organ structures, which are called floating images, and let their number be x, and the remaining nx images have no segmentation annotation, which are called fixed images;

[0081] S22, randomly pairing the floating image with the fixed image, so that the floating image in each pair of images has a segmentation annotation, and the fixed image has no segmentation annotation.

[0082] Preferably, the step S3, using the segmentation model to predict the segmentation annotation of the image, using the predicted segmentation annotation to generate corresponding spatial encoding, and using the encoding information to train the Voxel-Morph registration model, includes:

[0083] S31, sending the fixed image in each pair of images into the segmentation model to obtain the predicted segmentation annotation.

[0084] S32, convert the segmentation annotations of the images in the image pair into one-hot encoding to facilitate subsequent calculations.

[0085] S33, performing spatial coding calculation on each voxel in the three-dimensional medical image, and the calculation method is as follows:

[0086]

[0087]

[0088]

[0089] As shown in the above formula, the spatial encoding of each voxel consists of three bits, Represents the n-th bit code of the voxel with coordinates (x, y, z) in the three-dimensional image, label one-shot The segmentation annotation in the one-hot encoding form of the image is mathematically represented as a four-dimensional matrix, where i, l, w, and h represent the anatomical structure, length, width, and height, respectively.

[0090] S34, connecting the encoding of each image with the original image to form a four-channel image, wherein the original image occupies one channel and the spatial encoding occupies three channels.

[0091] S35, the built registration model adopts the Voxel-Morph network architecture, and the original 2 input channels are modified to 6 input channels to adapt to the new input type after adding spatial encoding.

[0092] S36, the trained registration model will output a deformation field that can register the floating image to the fixed image, as shown in the following formula:

[0093] Φ=Reger(I mov ,I fix )

[0094]

[0095] Where Reger() represents the registration model, I mov and I′ movThey represent the floating image and the result after the floating image is registered to the fixed image respectively, Φ represents the deformation field output by the network, and the ° symbol represents the deformation operation of the image through the deformation field.

[0096] Preferably, the S4, randomly pairing the images in the data set, using the segmentation labels predicted by the two paired images to generate a texture correspondence map, and using the texture correspondence map to train a pixel-level Siamese model, includes:

[0097] S41, since the processing object of the Siamese network is a pair of images, the medical image dataset used for training is first randomly paired, and the segmentation annotations of the two images are converted into one-hot encoding;

[0098] S42, using the segmentation annotations predicted by the segmentation model to calculate the structural correspondence between the two images, the calculation process is as follows:

[0099]

[0100]

[0101] Among them, P pos and P nat They respectively represent the regions where the structures of the two paired images correspond and the regions where the structures do not correspond; and Represents the one-hot segmentation annotation of the two images respectively;

[0102] S43 uses the supervision information obtained by segmentation annotation to train the Siamese network. Its objective function consists of two parts. The first part is the contrast loss function, which is calculated as follows:

[0103]

[0104] Among them, L represents the final objective function value used for back propagation, d represents the Euclidean distance of the feature vectors of the two images after Siamese network encoding, and margin is an artificially set threshold; the above objective function can constrain the Siamese network to make the encoding results of the structural areas corresponding to the images consistent, otherwise the differences will be enlarged;

[0105] S44, the objective function of the second part is used to constrain the final output of the network, so that we can obtain a confidence map that reflects the texture correspondence at the same position between the input image pairs through the Siamese network. The formula is as follows:

[0106]

[0107] Where N is the number of voxels in the image matrix, P acm Represents the confidence map output by the Siamese network.

[0108] Preferably, S5, using a Voxel-Morph registration model to generate augmented data for the segmentation network, including fully supervised data and weakly supervised data, using a Siamese network to filter and weight the weakly supervised data to reduce the impact of erroneous information, and using this information to retrain the segmentation network, includes:

[0109] S51, the floating image I with segmentation annotation mov and a fixed image I without annotations fix Send it into the registration model to get the registered floating image I′ mov and its segmentation label ′ mov ; then I′ mov and label′ mov The correspondence can be regarded as a pair of fully supervised data, label′ mov and I fix The correspondence can be regarded as a pair of weakly supervised data; the specific process is as follows:

[0110] Φ=Reger(I mov ,I fix )

[0111]

[0112]

[0113] S52, the full supervision data and weak supervision data augmented by the registration model are sent to the segmentation model for further training; when using weak supervision data for training, I mov and I fix Input into the Siamese network to obtain the confidence map of the structural correspondence between the two images. This confidence map can be used to perform weighted screening on the weakly supervised data and remove the erroneous information in the weakly supervised data. The specific usage is as follows:

[0114] L acm =P acm ·label′ mov ·log(S(I fix ))

[0115] In the above formula, S(I fix ) represents the segmentation model. The actual calculation process is to multiply the cross entropy function by the confidence map, and screen the loss value of each voxel. Only the area with correct supervision information is counted into the value for back propagation.

[0116] In order to verify the effect of the spatial encoding method, this example uses the spatial encoding calculated by segmentation annotation and does not use spatial encoding to train the registration model. The following Table 1 shows the comparison of the registration results of this embodiment with and without using spatial encoding.

[0117] Table 1

[0118]

[0119] It can be seen that after using the spatial encoding as the input of the registration network, the accuracy of the registration model on the two datasets increased from 75.5% to 77.8% and from 73.0% to 74.2%, respectively. This shows that this method improves the performance of the registration-segmentation collaborative method in a small number of manually annotated scenes.

[0120] In order to verify the effectiveness of the Siamese network screening weak supervision information method, this example trains the segmentation model under three conditions: no data augmentation, using augmented data but not using the Siamese network for processing, and using the Siamese network to process the weak supervision data augmented by the registration model. The following Table 2 shows the segmentation result ratios of whether augmentation and Siamese network screening are used in this embodiment.

[0121] Table 2

[0122]

[0123] It can be seen that under the premise of data augmentation, the best segmentation results can be obtained by using Siamese network to process weakly supervised data, which illustrates the necessity and superiority of using double sampling in this method.

[0124] In summary, compared with the prior art, the present invention closely combines spatial coding technology, a pixel-level Siamese network training method based on contrastive learning, and image segmentation-registration collaborative technology, thereby greatly improving the performance of image segmentation tasks and image registration tasks in small sample scenarios, which not only improves the segmentation accuracy of brain images, but also improves the registration accuracy of brain images, providing a new solution for medical image segmentation and registration tasks in small sample scenarios.

[0125] In addition, an embodiment of the present application also discloses an electronic device, which includes: one or more processors, and a memory, wherein the memory is used to store one or more computer programs; characterized in that the computer program is configured to be executed by the one or more processors, and the program includes steps for executing the weakly supervised medical image segmentation and registration collaborative method based on contrast learning and spatial encoding as described above.

[0126] In addition, an embodiment of the present application also provides a storage medium, which stores a computer program; the program is loaded and executed by a processor to implement the steps of the weakly supervised medical image segmentation and registration collaborative method based on contrast learning and spatial encoding as described above.

[0127] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0128] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0129] The units described as separate components may or may not be physically separated. As a unit, it can be appreciated by those skilled in the art that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0130] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0131] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a grid device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0132] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A weakly supervised medical image segmentation and registration collaborative method based on contrastive learning and spatial encoding, characterized in that: The method comprises: Acquire a medical image to be processed; perform collaborative processing of registration and segmentation on the medical image according to a pre-trained registration and segmentation model; The registration and segmentation models are pre-trained based on the following steps: S1, build a segmentation model with a small amount of labeled medical image data, perform pre-training, and use the pre-trained segmentation model to predict the segmentation annotations of the remaining images; S2, dividing the acquired medical image data into floating images and fixed images according to whether they have manual segmentation annotations; S3, using the segmentation model to predict the segmentation annotation of the image, using the predicted segmentation annotation to generate the corresponding spatial encoding, and using the encoding information to train the Voxel-Morph registration model; The S3, using the segmentation model to predict the segmentation annotation of the image, using the predicted segmentation annotation to generate a corresponding spatial encoding, and using the encoding information to train the Voxel-Morph registration model, includes: S31, sending the fixed image in each pair of images into the segmentation model to obtain the predicted segmentation annotation; S32, converting the segmentation annotations of the images in the image pair into one-hot encoding to facilitate subsequent calculations; S33, performing spatial coding calculation on each voxel in the three-dimensional medical image, and the calculation method is as follows: As shown in the above formula, the spatial encoding of each voxel consists of three bits, Represents the n-th bit code of the voxel with coordinates (x, y, z) in the three-dimensional image, label one-shot The segmentation annotation of the image in one-hot encoding form is mathematically represented as a four-dimensional matrix, where i, l, w, and h represent the anatomical structure, length, width, and height, respectively; S34, connecting the code of each image with the original image to form a four-channel image, wherein the original image occupies one channel and the spatial code occupies three channels; S35, the registration model built uses the Voxel-Morph network architecture, and the original 2 input channels are modified to 6 input channels to adapt to the new input type after adding spatial encoding; S36, the trained registration model will output a deformation field that can register the floating image to the fixed image, as shown in the following formula: Φ=Reger(I mov ,I fix ) Where Reger() represents the registration model, I mov and I′ mov They represent the floating image and the result of registering the floating image to the fixed image, respectively. Φ represents the deformation field output by the network. The symbol represents the deformation operation of the image through the deformation field; S4, randomly pairing the images in the data set, using the segmentation labels predicted by the two paired images to generate a texture correspondence map, and using the texture correspondence map to train the pixel-level Siamese model; S4, randomly pairing the images in the data set, using the segmentation labels predicted by the two paired images to generate a texture correspondence map, and using the texture correspondence map to train the pixel-level Siamese model, includes: S41, since the processing object of the Siamese network is a pair of images, the medical image dataset used for training is first randomly paired, and the segmentation annotations of the two images are converted into one-hot encoding; S42, using the segmentation annotations predicted by the segmentation model to calculate the structural correspondence between the two images, the calculation process is as follows: Among them, P pos and P nat They respectively represent the regions where the structures of the two paired images correspond and the regions where the structures do not correspond; and Represents the one-hot segmentation annotation of the two images respectively; S43 uses the supervision information obtained by segmentation annotation to train the Siamese network. Its objective function consists of two parts. The first part is the contrast loss function, which is calculated as follows: L=mean(P pos ·d 2 +P nat ·max(margan-d,0) 2 ) Among them, L represents the final objective function value used for back propagation, d represents the Euclidean distance of the feature vectors of the two images after Siamese network encoding, and margin is an artificially set threshold; the above objective function can constrain the Siamese network to make the encoding results of the structural areas corresponding to the images consistent, otherwise the differences will be enlarged; S44, the objective function of the second part is used to constrain the final output of the network, so that we can obtain a confidence map that reflects the texture correspondence at the same position between the input image pairs through the Siamese network. The formula is as follows: Where N is the number of voxels in the image matrix, P acm Represents the confidence map output by the Siamese network; S5, using the Voxel-Morph registration model to generate augmented data for the segmentation network, including full supervision data and weak supervision data, using the Siamese network to filter and weight the weak supervision data to reduce the impact of erroneous information, and use this information to retrain the segmentation network; S6, repeating steps S3 to S5 until the registration model and the segmentation model converge to generate a registration and segmentation model.

2. According to the weakly supervised medical image segmentation and registration collaborative method based on contrastive learning and spatial encoding in claim 1, the S1 uses a small amount of labeled medical image data to build a segmentation model, performs pre-training, and uses the pre-trained segmentation model to predict the segmentation annotations of the remaining images, including: S11, the segmentation model adopts the U-net network architecture, and a small number of labeled images are sent to the network for pre-training; S12, use the trained segmentation model to perform segmentation prediction on the fixed image to obtain the predicted label of the fixed image.

3. According to the weakly supervised medical image segmentation and registration collaborative method based on contrastive learning and spatial encoding in claim 2, said S2, dividing the acquired medical image data into floating images and fixed images according to whether they have manual segmentation annotations, comprises: S21, let the number of images in the medical image dataset be n, among which only a small number of images contain manual segmentation and annotation of organ structures, which are called floating images, and let their number be x, and the remaining nx images have no segmentation annotation, which are called fixed images; S22, randomly pairing the floating image with the fixed image, so that the floating image in each pair of images has a segmentation annotation, and the fixed image has no segmentation annotation.

4. According to the weakly supervised medical image segmentation and registration collaborative method based on contrastive learning and spatial encoding in claim 1, S5, using the Voxel-Morph registration model to generate augmented data for the segmentation network, including fully supervised data and weakly supervised data, using the Siamese network to filter and weight the weakly supervised data to reduce the impact of erroneous information, and using this information to retrain the segmentation network, including: S51, the floating image I with segmentation annotation mov and a fixed image I without annotations fix Send it into the registration model to get the registered floating image I′ mov and its segmentation label ′ mov ; then I′ mov and label′ mov The correspondence can be regarded as a pair of fully supervised data, label′ mov and I fix The correspondence can be regarded as a pair of weakly supervised data; the specific process is as follows: Φ=Reger(I mov ,I fix ) S52, sending the fully supervised data and weakly supervised data augmented by the registration model into the segmentation model for further training; When using weakly supervised data for training, I mov and I fix Input into the Siamese network to obtain the confidence map of the structural correspondence between the two images. This confidence map can be used to perform weighted screening on the weakly supervised data and remove the erroneous information in the weakly supervised data. The specific usage is as follows: L acm =P acm ·label′ mov ·log(S(I fix )) In the above formula, S(I fix ) represents the segmentation model. The actual calculation process is to multiply the cross entropy function by the confidence map, screen the loss value of each voxel, and only the area with correct supervision information is counted into the value for back propagation.

5. An electronic device, comprising: One or more processors, a memory, wherein the memory is used to store one or more computer programs; characterized in that the computer program is configured to be executed by the one or more processors, and the program includes steps for executing the weakly supervised medical image segmentation and registration collaborative method based on contrast learning and spatial encoding as described in any one of claims 1-4.

6. A storage medium storing a computer program; characterized in that: The program is loaded and executed by a processor to implement the steps of the weakly supervised medical image segmentation and registration collaborative method based on contrastive learning and spatial encoding as described in any one of claims 1 to 4.

Images