Image segmentation, model training method and device, electronic equipment and storage medium

Abstract

The application provides an image segmentation method and device, a model training method and device, an electronic device and a storage medium. The first image and the second image collected from the same object are registered by using a registration model to obtain a registration deformation field. The registration deformation field and a segmentation model of the second image obtained in advance are calculated to obtain a segmentation model of the first image to be segmented. The segmentation and identification of the first image with low tissue resolution can be realized. Compared with the case that a large amount of manual annotation is required and time-consuming and laborious in the supervised segmentation network mode, the image segmentation method provided in the application can quickly obtain a segmentation image of the first image with high accuracy.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image segmentation and model training method, apparatus, electronic device, and storage medium. Background Technology

[0002] Image segmentation technology is widely used in diagnostic imaging. For example, segmenting the heart and coronary arteries can help doctors accurately diagnose heart diseases. Currently, most medical image segmentation techniques, whether traditional algorithms or deep learning-based methods, are based on enhanced CT (Computed Tomography) images, utilizing the effect of contrast agents to achieve relatively good segmentation and recognition. However, in some medical scenarios, image segmentation needs to be performed on plain CT images, which are challenging due to their lower tissue resolution. Summary of the Invention

[0003] In view of this, embodiments of this application provide an image segmentation and model training method, apparatus, electronic device, and storage medium that can quickly obtain segmented images from plain CT images.

[0004] In a first aspect, embodiments of this application provide an image segmentation method, the method comprising:

[0005] Acquire a first image and a second image, wherein the first image and the second image are images captured from the same object;

[0006] The first image and the second image are input into a pre-trained registration model, and the registration model is used to register the first image and the second image to obtain the registration deformation field.

[0007] Based on the registration deformation field and the segmentation model corresponding to the second image, the segmentation model of the first image is calculated, wherein the segmentation model corresponding to the second image is obtained in advance.

[0008] Secondly, embodiments of this application provide a model training method, including:

[0009] Acquire sample images, which include a first sample image and a second sample image, wherein the first sample image and the second sample image are images acquired from the same object;

[0010] The neural network is trained based on the first sample image and the second sample image to obtain a registration model that can register the first sample image and the second sample image.

[0011] Thirdly, embodiments of this application provide an image segmentation apparatus, comprising:

[0012] The acquisition module is configured to acquire a first image and a second image, wherein the first image and the second image are images captured from the same object;

[0013] The registration module is configured to input the first image and the second image into a pre-trained registration model, and use the registration model to register the first image and the second image to obtain a registration deformation field;

[0014] The segmentation module is configured to calculate the segmentation model of the first image based on the registration deformation field and the segmentation model corresponding to the second image, wherein the segmentation model corresponding to the second image is obtained in advance.

[0015] Fourthly, embodiments of this application provide a model training apparatus, comprising:

[0016] The sample acquisition module is configured to acquire sample images, which include a first sample image and a second sample image, wherein the first sample image and the second sample image are images acquired from the same object;

[0017] The model training module is configured to train the neural network based on the first sample image and the second sample image to obtain a registration model capable of registering the first sample image and the second sample image.

[0018] Fifthly, embodiments of this application provide a computationally readable storage medium storing a computer program for executing the image segmentation method described in the first aspect or for executing the model training method described in the second aspect.

[0019] In a sixth aspect, embodiments of this application provide an electronic device, including: a processor; and a memory for storing processor-executable instructions, wherein the processor is used to execute the image segmentation method described in the first aspect above, or to execute the model training method described in the second aspect above.

[0020] This application provides an image segmentation method, apparatus, electronic device, and storage medium for training a model. The method involves registering a first image and a second image acquired from the same object using a registration model to obtain a registration deformation field. This registration deformation field is then used to calculate a segmentation model for the first image to be segmented. This method enables segmentation and recognition of first images with low tissue resolution. Compared to supervised segmentation networks, which require extensive manual annotation and are time-consuming and labor-intensive, the image segmentation method provided in this application can quickly obtain a highly accurate segmented image of the first image. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of an implementation environment provided by an exemplary embodiment of this application.

[0022] Figure 2 This is a flowchart illustrating an exemplary embodiment of the image segmentation method provided in this application.

[0023] Figure 3 This is a flowchart illustrating an image segmentation method provided in another exemplary embodiment of this application.

[0024] Figure 4a This is a plain CT image of the coronary arteries of the heart provided in an exemplary embodiment of this application.

[0025] Figure 4b This is an enhanced CT image of the coronary arteries of the heart provided in an exemplary embodiment of this application.

[0026] Figure 5 This is a schematic flowchart of a registration method for plain CT images and enhanced CT images provided in an exemplary embodiment of this application.

[0027] Figure 6 This is a flowchart illustrating a training method for a model provided in an exemplary embodiment of this application.

[0028] Figure 7 This is a schematic diagram of the structure of an image segmentation apparatus provided in an exemplary embodiment of this application.

[0029] Figure 8 This is a schematic diagram of the structure of a training device for a model provided in an exemplary embodiment of this application.

[0030] Figure 9 This is a block diagram of an electronic device provided in an exemplary embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] Application Overview

[0033] Deep learning achieves artificial intelligence in computing systems by building hierarchical artificial neural networks. Because hierarchical artificial neural networks can extract and filter input information layer by layer, deep learning possesses representation learning capabilities, enabling end-to-end supervised and unsupervised learning. The hierarchical artificial neural networks used in deep learning come in various forms, with the complexity of their hierarchy collectively referred to as "depth." Based on their architecture, deep learning includes multilayer perceptrons, convolutional neural networks, recurrent neural networks, deep belief networks, and other hybrid architectures. Deep learning uses data to update the parameters in its architecture to achieve training objectives; this process is collectively called "learning." Deep learning proposes a method that allows computers to automatically learn pattern features and integrates feature learning into the model building process, thereby reducing the incompleteness caused by manually designed features.

[0034] A neural network is a computational model composed of numerous interconnected nodes (or neurons). Each node corresponds to a policy function, and the connection between any two nodes represents a weighted value for the signal passing through that connection. A neural network typically consists of multiple layers, cascaded together. The output of the i-th layer is connected to the input of the (i+1)-th layer, the output of the (i+1)-th layer is connected to the input of the (i+2)-th layer, and so on. When a sample image is input into the cascaded neural network layers, each layer outputs a result, which becomes the input to the next layer. This process continues, with multiple layers calculating the output. The predicted value of each layer is compared to the actual target value. Based on the difference between the predicted and actual values, the weight matrix and policy function of each layer are adjusted. This adjustment process is repeated continuously using sample images until the predicted value matches the actual target value. This process is called the training process of the neural network. After training, a neural network model is obtained.

[0035] Current cardiac segmentation techniques primarily employ traditional algorithms or deep learning methods based on enhanced CT images. The contrast agent's effect makes the heart and arteries clearly distinguishable from other tissues, thus enabling relatively good segmentation and recognition. However, for plain CT images, high-density bone tissues, such as bone and calcifications, typically have a HU value of 100 or higher, while other tissues, such as muscles, organs, and blood, usually have a HU value below 100. Furthermore, the differentiation between these tissues is very low, making effective segmentation impossible. Directly using conventional supervised segmentation networks requires a large amount of manually labeled data, which is quite difficult.

[0036] To address the aforementioned issues, this application provides an image segmentation method. This method uses a registration model to register a first image and a second image acquired from the same object to obtain a registration deformation field. The registration deformation field is then used to calculate a segmentation model for the first image to be segmented. This method enables segmentation and recognition of first images with low tissue resolution. Compared to supervised segmentation networks, which require extensive and time-consuming manual annotation, this method quickly yields highly accurate segmented images of the first image, facilitating the diagnosis of heart-related diseases by doctors.

[0037] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0038] Exemplary System

[0039] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application. The implementation environment includes a CT scanner 130, a server 120, and a computer device 110. The computer device 110 can acquire multiple sets of medical images from the CT scanner 130 used for X-ray scanning of human tissue. Simultaneously, the computer device 110 can also be connected to the server 120 via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0040] The computer device 110 can be a general-purpose computer or a computer device composed of dedicated integrated circuits, etc., and this application embodiment does not limit this. For example, the computer device 110 can be a mobile terminal device such as a tablet computer, or it can be a personal computer (PC), such as a laptop computer and a desktop computer, etc. Those skilled in the art will know that the number of the above-mentioned computer devices 110 can be one or more, and their types can be the same or different. For example, there can be one computer device 110, or there can be dozens or hundreds of computer devices 110, or more. This application embodiment does not limit the number and type of computer devices 110.

[0041] In some alternative embodiments, computer device 110 acquires multiple sets of medical sample images from CT scanner 130. Each set of medical sample images includes CT enhanced sample images and CT plain scan sample images. Computer device 110 trains a neural network using the multiple sets of medical sample images to obtain a network model for registering CT enhanced sample images and CT plain scan sample images within a set.

[0042] Server 120 is a single server, or a combination of several servers, or a virtualization platform, or a cloud computing service center.

[0043] In some alternative embodiments, computer device 110 sends multiple sets of medical sample images acquired from CT scanner 130 to server 120. A set of medical sample images includes CT enhanced sample images and CT plain scan sample images. Server 120 trains a neural network using the multiple sets of medical sample images to obtain a network model for registering CT enhanced sample images and CT plain scan sample images.

[0044] Exemplary methods

[0045] Figure 2 This is a flowchart illustrating an exemplary embodiment of the image segmentation method provided in this application. Figure 2 The method is executed by a computing device, such as a server. Figure 2 As shown, this image segmentation method includes the following:

[0046] S210: Acquire a first image and a second image, wherein the first image and the second image are images acquired from the same object.

[0047] The first image and the second image are images acquired from the same part of the same object. For example, they can be images of the coronary arteries of the heart or other organs of the same patient. It should be noted that the first image and the second image are different types of medical images. For example, the first image can be a plain CT image or other medical images with low local resolution, while the second image can be an enhanced CT image or other medical images with higher resolution. This embodiment does not impose specific limitations on this.

[0048] S220: Input the first image and the second image into a pre-trained registration model, and use the registration model to register the first image and the second image to obtain a registration deformation field.

[0049] Specifically, medical image registration is a commonly used technique in medical image analysis. It involves transforming the coordinates of one image (moving image) to another image (fixed image) so that corresponding positions in the two images match, resulting in a registered image (Moved). For a medical image, one or more spatial transformations are sought to achieve spatial consistency with corresponding points in another medical image. The registration result should match all anatomical points, or at least all diagnostically significant points and points of surgical interest, in both images. The main task in image registration is to find the optimal spatial transformation and grayscale transformation relationships to achieve optimal alignment between the two images. Spatial transformation is key to achieving accurate registration. Transformations can be divided into rigid transformations and non-rigid transformations. Rigid transformations involve image translation and rotation, which can roughly match images. Non-rigid transformations mainly involve complex transformations such as scaling and affine transformations to process and adjust the image. The registration model in this embodiment can be a rigid model, a non-rigid model, or a combination of both; no specific limitations are imposed.

[0050] The first and second images are registered using a registration model to obtain a registration deformation field, which is a matrix formed by the vectors of pixel displacements in the image. In this embodiment, the first image is a fixed image, and the second image is a moving image. The second image is registered onto the first image using the registration model, that is, the coordinates of the second image are transformed into those of the first image. At this time, the displacement of the second image generates a registration deformation field.

[0051] S230: Based on the registration deformation field and the segmentation model corresponding to the second image, the segmentation model of the first image is calculated, wherein the segmentation model corresponding to the second image is obtained in advance.

[0052] Specifically, the segmentation model corresponding to the second image is obtained in advance. In this embodiment, the second image is an enhanced CT image. Since the enhanced CT image has a high resolution, obvious data features, and easy data annotation, the segmentation network is relatively simple to train. The segmentation model of the enhanced CT image can be easily obtained using the trained segmentation network. In an exemplary embodiment, the segmentation model can be a segmentation mask.

[0053] After obtaining the registration deformation field through the aforementioned steps, the segmentation model of the first image can be obtained by calculation by combining the segmentation model of the second image. This calculation method can realize the transformation of pixel displacement, such as dot product operation. By using the easily obtained segmentation model of the second image, the segmentation model of the first image with lower resolution can be obtained through pixel displacement transformation. Compared with supervised segmentation networks for segmenting the first image, the segmentation method provided in this embodiment is faster and more effective, and can obtain a segmentation model of the first image with higher accuracy, which can assist in the diagnosis and treatment of medical diseases and reduce the reliance on human experience.

[0054] In another embodiment of this application, Figure 3 The method shown is Figure 2 Examples of the methods shown are as follows: Figure 3 The method shown also includes the following.

[0055] S310: Acquire plain CT images and enhanced CT images, wherein the plain CT images and enhanced CT images are cardiac coronary artery images acquired from the same object.

[0056] Plain CT images and enhanced CT images are images of the same part of the same object, such as images of the coronary arteries of the heart. Figure 4a The image shows a plain CT scan of the coronary arteries. Figure 4b The image shows enhanced CT images of the coronary arteries of the heart. Figure 4a and Figure 4b As can be seen from the plain CT images, the HU value of high-density bone tissue is relatively high, while the HU value of general tissues such as muscles, organs and blood is relatively low and cannot be effectively distinguished. However, the resolution of each tissue is relatively high in the enhanced CT images.

[0057] It should be noted that objects of different densities, as well as human organs and tissues of different densities, have different X-ray absorption coefficients, or HU values. The HU value is a unit of measurement for the density of a local tissue or organ, commonly known as the Henle unit. Enhanced CT images, obtained with the help of contrast agents, provide higher resolution CT images and are primarily used to differentiate between vascular and non-vascular lesions, and to clarify the relationship between mediastinal lesions and the heart and great vessels.

[0058] S320: Input the plain CT image and enhanced CT image of the coronary artery into the pre-trained registration model, and use the registration model to register the first image and the second image to obtain the registration deformation field.

[0059] The enhanced CT image is registered to the plain CT image after pixel displacement transformation using a registration model, thus establishing a one-to-one correspondence between pixels in the plain CT image and the enhanced CT image. In this embodiment, the type of registration model is not limited; it can be a deep neural network (DNN), a convolutional neural network (CNN), or a recurrent neural network (RNN), etc.

[0060] S330: Based on the registration deformation field and the segmentation mask of the coronary arteries corresponding to the enhanced CT image, the segmentation mask of the coronary arteries of the plain CT image is calculated, wherein the segmentation mask corresponding to the enhanced CT image is obtained in advance.

[0061] By obtaining the segmentation mask of the coronary arteries in enhanced CT images through a pre-processing segmentation network and combining it with the registration deformation field, the segmentation mask of the coronary arteries in plain CT images can be quickly obtained through dot product operations, which can effectively solve the problem of difficulty in obtaining segmentation models for plain CT images.

[0062] In one embodiment, inputting the first image and the second image into a pre-trained registration model includes: preprocessing the first image and the second image, and inputting the preprocessed first image and the second image into the pre-trained registration model.

[0063] Specifically, the preprocessing in this embodiment is image processing. To obtain the region of interest and reduce computational resource consumption, the second image can be cropped using the minimum bounding box of the segmentation mask, and both the first and second images are resampled to a size of 128×128×128, maintaining isotropy, i.e., keeping the pixel spacing consistent in the x, y, and z scanning directions, with the window width and window level both set to (300, 800). For the first image without a segmentation mask, this invention uses a simple coarse registration method, coarsely registering the segmentation mask of the second image onto the first image, and expanding the minimum bounding box using padding = 30, only needing to obtain the approximate location of the region of interest. After preprocessing the first and second images, the registration accuracy of the registration model can be improved.

[0064] It should be noted that in other embodiments, the above preprocessing methods can be added or replaced. The image processing described above is only an exemplary illustration of a preprocessing method, and this application does not limit it.

[0065] In one embodiment, the registration model includes a first registration model and a second registration model.

[0066] The step of registering the first image and the second image using the registration model to obtain a registration deformation field includes: registering the first image and the second image using a first registration model to obtain a first registration deformation field; registering the first image and the second image using a second registration model based on the first registration deformation field to obtain a second registration deformation field; and merging the first registration deformation field and the second registration deformation field to obtain the registration deformation field.

[0067] In this embodiment, both the first and second registration models are neural network models, representing different registration types: the first is a rigid registration model, and the second is a non-rigid registration model. The first registration model first performs rigid registration on the first and second images, then coarsely matches them through image translation and rotation transformations to obtain the first registration deformation field. The second registration model then processes and adjusts the coarse registration, performing non-rigid registration on the first and second images to obtain the second registration deformation field. Finally, the first and second registration deformation fields are vector-added to obtain the final registration deformation field. Using both models, optimal alignment of the first and second images can be achieved, improving alignment accuracy and providing a foundation for accurately obtaining a segmentation model for the first image.

[0068] It should be noted that in this embodiment, the first registration model uses a Deep Residual Network (ResNet), and the second registration model uses a ResUNet network. The ResUNet network is a combination of ResNet and U-Net networks. Through ResNet and ResUNet networks, feature extraction of medical images can be effectively performed, achieving accurate registration of medical images. In other embodiments, ResNet and ResUNet networks can be replaced with other 3D semantic segmentation networks.

[0069] In one embodiment, based on the first registration deformation field, the first image and the second image are registered using a second registration model to obtain a second registration deformation field, including: performing a dot product operation on the first registration deformation field and the second image to obtain a third image; and registering the first image and the third image using the second registration model to obtain the second registration deformation field.

[0070] Figure 5 The method flow for registering plain CT image 501 and enhanced CT image 502 using a registration model is shown, such as... Figure 5As shown, the registration model includes a first registration model 503 for rigid registration and a second registration model 505 for non-rigid registration. After inputting the plain CT image 501 and the enhanced CT image 502 into the first registration model, a first registration deformation field 504 is obtained. The first registration deformation field 504 and the enhanced CT image 502 are then dot-producted to generate a third image, i.e., a new enhanced CT image. This new enhanced CT image and the plain CT image 501 are then input into the second registration model 505 for registration, resulting in a second registration deformation field 506. Finally, the first registration deformation field 504 and the second registration deformation field 506 are vector-added to obtain the final registration deformation field 507. Through the combined action of the first registration model 503 and the second registration model 505, accurate registration of the enhanced CT image 501 and the plain CT image 502 is achieved, improving the overall registration accuracy.

[0071] Figure 6 This is a flowchart illustrating a training method for a model provided in one embodiment of this application. Figure 6 The method described is executed by a computing device (e.g., a server), but this application embodiment is not limited thereto. The server may be a single server, or a combination of several servers, or a virtualization platform, or a cloud computing service center; this application embodiment does not limit this. Figure 6 As shown, the method includes the following:

[0072] S610: Acquire sample images, the sample images including a first sample image and a second sample image, wherein the first sample image and the second sample image are images acquired from the same object.

[0073] The first and second sample images in this embodiment are the same as those in the previous embodiments, both being images acquired from the same part or organ of the same object. Sample images can include images acquired from different parts or organs of multiple objects. The first and second sample images are of different types; for example, the first sample image can be a plain CT image, while the second sample image can be an enhanced CT image.

[0074] S620: Train the neural network based on the first sample image and the second sample image to obtain a registration model that can register the first sample image and the second sample image.

[0075] The neural network is trained using the first and second sample images collected. The parameters of the registration model are updated through iterative operations until the loss function is minimized and the images are matched, thus obtaining the trained registration model.

[0076] In one embodiment, a loss function value is calculated based on the registration deformation field output by the registration model, the first sample image, and the second sample image, and the parameters of the registration model are updated based on the loss function value.

[0077] To further improve the output accuracy of the registration model, this embodiment also optimizes the overall result by using a pre-set loss function. Specifically, it employs a global loss function and a gradient similarity loss function to optimize the overall result. The global loss function is the Normalized Cross-Correlation loss (NCC loss), which measures the similarity of the registration results by calculating the correlation coefficient. The NCC loss calculation formula is as follows:

[0078]

[0079] Where X and Y represent samples, cov represents covariance, and Var represents variance.

[0080] The gradient similarity loss function constrains the matching degree of image brightness differences, and the calculation formula is as follows:

[0081]

[0082] Where x and y represent samples, x′ and y′ represent gradients, μ represents the mean, σ represents the variance, and C1 and C2 represent constants.

[0083] In one embodiment, the neural network includes a first neural network and a second neural network, and the registration model includes a first registration model and a second registration model;

[0084] The step of training a neural network based on the first sample image and the second sample image to obtain a registration model capable of registering the first sample image and the second sample image includes: training the first neural network based on the first sample image and the second sample image to obtain a first registration model capable of registering the first sample image and the second sample image; performing a dot product operation on the first registration deformation field output by the first registration model and the second sample image to obtain a third sample image; and training a second neural network based on the first sample image and the third sample image to obtain a second registration model capable of registering the first sample image and the third sample image.

[0085] In this embodiment, the first neural network and the second neural network are the ResNet network and ResUnet network in the image segmentation method embodiment, respectively. The first sample image and the second sample image are a plain CT image and an enhanced CT image. The ResNet network is trained using the plain CT image and the enhanced CT image to obtain a first registration model. Then, the first registration deformation field output by the first registration model is multiplied by the second sample image to obtain a third sample image, which is the new enhanced CT image. The ResUnet network is trained using the plain CT image and the third sample image to obtain a second registration model.

[0086] It should be noted that before training the registration model using the first and second sample images, preprocessing is required. Preprocessing includes image processing and data augmentation. The image processing is the same as in the above embodiments and will not be repeated here. To improve the generalization performance of the registration model, data augmentation is also required for the first and second sample images, specifically through random rotation, brightness adjustment, and random small-angle jitter. These data augmentation operations are only illustrative examples; those skilled in the art can add or remove image preprocessing operations according to actual needs, and no limitations are imposed here.

[0087] In one embodiment, training the first neural network based on the first sample image and the second sample image to obtain the first registration model capable of registering the first sample image and the second sample image includes:

[0088] The first sample image and the second sample image are input into the first neural network, and the first registration deformation field is output through the first neural network. Based on the first registration deformation field, the first sample image and the second sample image, a similarity value is obtained by mutual information calculation. The parameters of the first neural network are updated according to the similarity value, wherein the first neural network is a residual network.

[0089] Specifically, the first sample image and the second sample image are input into the first neural network to obtain the first registration deformation field. The third sample image is obtained by dot product operation based on the first registration deformation field and the second sample image. The third sample image is also the registered image. The parameters of the first neural network are adjusted based on the mutual information between the registered third sample image and the first sample image (fixed image). When the similarity exceeds a certain threshold, it can be considered that the similarity is high, and the training of the first neural network is completed to obtain the first registration model.

[0090] The formula for calculating mutual information is as follows:

[0091]

[0092] Where H represents the entropy calculation function, p represents the probability distribution function, and a and b represent samples.

[0093] In one embodiment, training the second neural network based on the first sample image and the third sample image to obtain a second registration model capable of registering the first sample image and the third sample image includes:

[0094] The first sample image and the third sample image are input into the second neural network, and the second registration deformation field is output through the second neural network. The loss function value is calculated based on the second registration deformation field, the third sample image and the first sample image, and the second neural network is updated according to the loss function value. The second neural network is a deep convolutional neural network.

[0095] The first sample image and the third sample image obtained through the first registration are input into the second neural network to register the first sample image and the third sample image. The first sample image is used as a fixed image and the third sample image is used as a moving image. The second registration deformation field is multiplied by the third sample image to obtain the fourth sample image (i.e., the registered image) after the second registration. The second neural network is updated by calculating the loss function value between the first sample image and the fourth sample image to obtain the second registration model.

[0096] For example, the loss function in this embodiment is the Smooth Loss function, and the specific calculation formula is as follows:

[0097]

[0098] Among them, y i and f xi Let represent the true value of the i-th sample and its corresponding predicted value, respectively, where n is the number of samples.

[0099] Exemplary device

[0100] Figure 7 This is a schematic diagram of the structure of an image segmentation apparatus 700 provided in an exemplary embodiment of this application. Figure 7 As shown, the image segmentation apparatus 700 includes:

[0101] The acquisition module 710 is configured to acquire a first image and a second image, wherein the first image and the second image are images acquired from the same object;

[0102] The registration module 720 is configured to input the first image and the second image into a pre-trained registration model, and use the registration model to register the first image and the second image to obtain a registration deformation field;

[0103] The segmentation module 730 is configured to calculate the segmentation model of the first image based on the registration deformation field and the segmentation model corresponding to the second image, wherein the segmentation model corresponding to the second image is obtained in advance.

[0104] It should be understood that the specific working process and functions of the acquisition module 710 to the segmentation module 730 in the above embodiments can be referred to the above. Figures 2 to 5 The description of the image segmentation method provided in the embodiments will not be repeated here to avoid repetition.

[0105] Figure 8 This is a schematic diagram of the structure of a training device 800 for a model provided in an exemplary embodiment of this application. Figure 8 As shown, the training device 800 for this model includes:

[0106] The sample acquisition module 810 is configured to acquire sample images, the sample images including a first sample image and a second sample image, wherein the first sample image and the second sample image are images acquired from the same object;

[0107] The model training module 820 is configured to train a neural network based on the first sample image and the second sample image to obtain a registration model capable of registering the first sample image and the second sample image.

[0108] It should be understood that the specific working process and functions of the sample acquisition module 810 to the model training module 820 in the above embodiments can be referred to the above. Figure 6 The description of the training method for the model provided in the embodiments will not be repeated here to avoid repetition.

[0109] Exemplary electronic devices and computer-readable storage media

[0110] Figure 9 This is a block diagram of an electronic device 900 provided in an exemplary embodiment of this application.

[0111] Reference Figure 9The electronic device 900 includes a processing component 910, which further includes one or more processors, and memory resources represented by memory 920 for storing instructions, such as application programs, that can be executed by the processing component 910. The application programs stored in memory 920 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 910 is configured to execute instructions to perform the methods described in the above embodiments.

[0112] Electronic device 900 may also include a power supply component configured to perform power management of electronic device 900, a wired or wireless network interface configured to connect electronic device 900 to a network, and an input / output (I / O) interface. Electronic device 900 can be operated based on an operating system stored in memory 920, such as Windows Server. TM Mac OSX TM Unix TM Linux TM FreeBSD TM Or similar.

[0113] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the electronic device 900, enables the electronic device 900 to perform the method described in the above embodiments.

[0114] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0115] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0120] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program verification codes, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] It should be noted that in the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0122] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications or equivalent substitutions made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image segmentation method, characterized in that, include: Acquire a first image and a second image, wherein the first image and the second image are images captured from the same part of the same object, and the resolution of the second image is higher than the resolution of at least some areas in the first image; The first image and the second image are input into a pre-trained registration model, and the registration model is used to register the first image and the second image to obtain the registration deformation field. Based on the registration deformation field and the segmentation model corresponding to the second image, the segmentation model of the first image is calculated, wherein the segmentation model corresponding to the second image is obtained in advance.

2. The method according to claim 1, characterized in that, The step of inputting the first image and the second image into a pre-trained registration model includes: The first image and the second image are preprocessed respectively, and the preprocessed first image and the second image are input into the pre-trained registration model.

3. The method according to claim 1, characterized in that, The registration model includes a first registration model and a second registration model. The process of registering the first image and the second image using the registration model to obtain the registration deformation field includes: The first image and the second image are registered using a first registration model to obtain a first registration deformation field; Based on the first registration deformation field, the first image and the second image are registered using the second registration model to obtain the second registration deformation field; The first registration deformation field and the second registration deformation field are merged to obtain the registration deformation field.

4. The method according to claim 3, characterized in that, The step of registering the first image and the second image based on the first registration deformation field using a second registration model to obtain the second registration deformation field includes: A third image is obtained by performing a dot product operation between the first registered deformation field and the second image; The first image and the third image are registered using a second registration model to obtain the second registration deformation field.

5. The method according to claim 1, characterized in that, The segmentation model of the first image is calculated based on the registration deformation field and the segmentation model corresponding to the second image, including: The segmentation model of the first image is obtained by performing a dot product operation on the registration deformation field and the segmentation model corresponding to the second image.

6. A method for training a model, characterized in that, include: Acquire sample images, which include a first sample image and a second sample image, wherein the first sample image and the second sample image are images acquired from the same part of the same object, and the resolution of the second sample image is higher than the resolution of at least some areas in the first sample image; The neural network is trained based on the first sample image and the second sample image to obtain a registration model capable of registering the first sample image and the second sample image. The registration model is used to receive the first sample image and the second sample image collected from the same part of the same object and register the first sample image and the second sample image to obtain a registration deformation field. The registration deformation field and the segmentation model corresponding to the pre-acquired second sample image are used together to calculate the segmentation model of the first sample image.

7. The method according to claim 6, characterized in that, Also includes: The loss function value is calculated based on the registration deformation field output by the registration model, the first sample image, and the second sample image, and the parameters of the registration model are updated based on the loss function value.

8. The method according to claim 6, characterized in that, The neural network includes a first neural network and a second neural network, and the registration model includes a first registration model and a second registration model; The step of training the neural network based on the first sample image and the second sample image to obtain a registration model capable of registering the first sample image and the second sample image includes: The first neural network is trained based on the first sample image and the second sample image to obtain the first registration model that can register the first sample image and the second sample image. The first registration deformation field output by the first registration model and the second sample image are subjected to a dot product operation to obtain the third sample image; The second neural network is trained based on the first sample image and the third sample image to obtain the second registration model that can register the first sample image and the third sample image.

9. The method according to claim 8, characterized in that, The step of training the first neural network based on the first sample image and the second sample image to obtain the first registration model capable of registering the first sample image and the second sample image includes: The first sample image and the second sample image are input into the first neural network, and the first registration deformation field is output through the first neural network. Based on the first registration deformation field, the first sample image, and the second sample image, a similarity value is obtained through mutual information calculation. The parameters of the first neural network are updated based on the similarity value, wherein the first neural network is a residual network.

10. The method according to claim 8, characterized in that, The step of training a second neural network based on the first sample image and the third sample image to obtain a second registration model capable of registering the first sample image and the third sample image includes: The first sample image and the third sample image are input into the second neural network, and the second registration deformation field is output through the second neural network. The loss function value is calculated based on the second registration deformation field, the third sample image, and the first sample image. The second neural network is then updated based on the loss function value, wherein the second neural network is a deep convolutional neural network.

11. An image segmentation apparatus, characterized in that, include: The acquisition module is configured to acquire a first image and a second image, wherein the first image and the second image are images acquired from the same part of the same object, and the resolution of the second image is higher than the resolution of at least a part of the first image; The registration module is configured to input the first image and the second image into a pre-trained registration model, and use the registration model to register the first image and the second image to obtain a registration deformation field; The segmentation module is configured to calculate the segmentation model of the first image based on the registration deformation field and the segmentation model corresponding to the second image, wherein the segmentation model corresponding to the second image is obtained in advance.

12. A training device for a model, characterized in that, include: The sample acquisition module is configured to acquire sample images, which include a first sample image and a second sample image, wherein the first sample image and the second sample image are images acquired from the same part of the same object, and the resolution of the second sample image is higher than the resolution of at least some areas in the first sample image; The model training module is configured to train the neural network based on the first sample image and the second sample image to obtain a registration model capable of registering the first sample image and the second sample image. The registration model is used to receive the first sample image and the second sample image collected from the same part of the same object and register the first sample image and the second sample image to obtain a registration deformation field. The registration deformation field and the segmentation model corresponding to the pre-acquired second sample image are used together to calculate the segmentation model of the first sample image.

13. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the image segmentation method according to any one of claims 1 to 5, and / or for executing the training method of the model according to any one of claims 6 to 10.

14. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions. The processor is used to execute the image segmentation method according to any one of claims 1 to 5, and / or to execute the training method of the model according to any one of claims 6 to 10.

Images