Medical image scanning image processing method and device

Abstract

The embodiments of the present specification provide a medical image scanning image processing method and device, wherein the method includes: determining a first medical image scanning image and a second medical image scanning image of a target part; inputting the first medical image scanning image and the second medical image scanning image into a target image registration model to obtain a displacement field between the first medical image scanning image and the second medical image scanning image, wherein the target image registration model is obtained by optimizing an initial image registration model based on a first training image set, and the initial image registration model is obtained by pre-training based on a second training image set, and the first training image set and the second training image set are different; adjusting the first medical image scanning image according to the displacement field to obtain an adjusted first medical image scanning image; determining the target object position of the target part according to the second medical image scanning image and the adjusted first medical image scanning image.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of image processing technology, and in particular to a method for processing medical image scanning images. Background Art

[0002] With the development of computer technology in the field of medical image processing, digital images play an increasingly important role in pathological analysis and diagnosis. In the process of digital image pathology slice processing, image registration is a very important link. Image registration can be understood as the process of matching and superimposing two or more images. The patient's condition can be analyzed based on the registered images.

[0003] Generally, the registration of digitized images can be achieved using an image registration model. However, due to differences in medical scanning equipment or medical centers, the medical imaging scan images they include also differ greatly. When an image registration model trained using a digitized image data set of a medical scanning device or a medical center is migrated to other medical scanning devices or other medical centers, the differences between the images will lead to reduced robustness and accuracy of the image registration model, poor processing performance, and will also affect the accuracy of the registration results of the medical imaging scan images. Therefore, an effective technical solution is urgently needed to solve the above problems. Summary of the invention

[0004] In view of this, the embodiments of this specification provide two medical image scanning image processing methods. One or more embodiments of this specification also involve two medical image scanning image processing devices, an image registration model training method, an image registration model training device, a computing device, a computer-readable storage medium and a computer program to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a method for processing a medical image scan is provided, comprising:

[0006] Determine a first medical imaging scan image and a second medical imaging scan image of a target part;

[0007] Inputting the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set, and the initial image registration model is obtained by pre-training according to a second training image set, and the first training image set and the second training image set are different;

[0008] Adjusting the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image;

[0009] The target object position of the target part is determined according to the second medical image scan image and the adjusted first medical image scan image.

[0010] According to a second aspect of the embodiments of this specification, there is provided a medical image scanning image processing device, comprising:

[0011] A first determination module is configured to determine a first medical image scan image and a second medical image scan image of a target part;

[0012] an input module, configured to input the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set, and the initial image registration model is obtained by pre-training according to a second training image set, and the first training image set and the second training image set are different;

[0013] an adjustment module, configured to adjust the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image;

[0014] The second determination module is configured to determine the target object position of the target part according to the second medical image scan image and the adjusted first medical image scan image.

[0015] According to a third aspect of an embodiment of this specification, a medical image scanning image processing method is provided, which is applied to a cloud-side device, including:

[0016] Receiving an image registration request, wherein the image registration request carries a first medical image scan image and a second medical image scan image of a target part;

[0017] Inputting the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set, and the initial image registration model is obtained by pre-training according to a second training image set, and the first training image set and the second training image set are different;

[0018] Adjusting the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image;

[0019] The target object position of the target part is determined according to the second medical image scan image and the adjusted first medical image scan image.

[0020] According to a fourth aspect of the embodiments of this specification, a medical image scanning image processing device is provided, which is applied to a cloud-side device, including:

[0021] A receiving module is configured to receive an image registration request, wherein the image registration request carries a first medical image scan image and a second medical image scan image of a target part;

[0022] an input module, configured to input the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set, and the initial image registration model is obtained by pre-training according to a second training image set, and the first training image set and the second training image set are different;

[0023] an adjustment module, configured to adjust the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image;

[0024] The determination module is configured to determine the target object position of the target part according to the second medical image scan image and the adjusted first medical image scan image.

[0025] According to a fifth aspect of an embodiment of this specification, there is provided an image registration model training method, which is applied to a cloud-side device, and includes:

[0026] Determining a first training image set, wherein the first training image set includes a first image sample and a second image sample;

[0027] In the initial image registration model, extracting a first image feature of the first image sample and a second image feature of the second image sample, and processing the first image feature and the second image feature according to a preset resolution to obtain a predicted displacement field between the first image sample and the second image sample;

[0028] adjusting the first image sample according to the predicted displacement field to obtain an adjusted first image sample;

[0029] The initial image registration model is optimized according to the predicted displacement field, the second image samples and the adjusted first image samples to obtain a target image registration model.

[0030] According to a sixth aspect of an embodiment of this specification, there is provided an image registration model training device, which is applied to a cloud-side device, including:

[0031] A determination module is configured to determine a first training image set, wherein the first training image set includes a first image sample and a second image sample;

[0032] a processing module configured to extract, in an initial image registration model, a first image feature of the first image sample and a second image feature of the second image sample, and process the first image feature and the second image feature according to a preset resolution to obtain a predicted displacement field between the first image sample and the second image sample;

[0033] an adjustment module, configured to adjust the first image sample according to the predicted displacement field to obtain an adjusted first image sample;

[0034] The optimization module is configured to optimize the initial image registration model according to the predicted displacement field, the second image samples and the adjusted first image samples to obtain a target image registration model.

[0035] According to a seventh aspect of an embodiment of this specification, there is provided a computing device, including:

[0036] Memory and processor;

[0037] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above method are implemented.

[0038] According to an eighth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and the instructions implement the steps of the above method when executed by a processor.

[0039] According to a ninth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above method.

[0040] One embodiment of the present specification provides a medical image scanning image processing method, which determines a first medical image scanning image and a second medical image scanning image of a target part; inputs the first medical image scanning image and the second medical image scanning image into a target image registration model to obtain a displacement field between the first medical image scanning image and the second medical image scanning image, wherein the target image registration model is obtained by optimizing an initial image registration model based on a first training image set, and the initial image registration model is pre-trained based on a second training image set, and the first training image set and the second training image set are different; according to the displacement field, the first medical image scanning image is adjusted to obtain an adjusted first medical image scanning image; according to the second medical image scanning image and the adjusted first medical image scanning image, the target object position of the target part is determined.

[0041] In the above method, after the initial image registration model is obtained by pre-training according to the second training image set, the initial image registration model is optimized according to the first training image set different from the second training image set, so that the obtained target image registration model can be applied to the field corresponding to the first training image set. This avoids the reduction in robustness and accuracy of the image registration model caused by the difference between the first training image set and the second training image set, further improves the processing performance of the target image registration model, and ensures the accuracy of the registration results of the target image registration model for the medical imaging scan image. Based on this, the patient's condition analysis can be achieved based on the target image registration model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a structural schematic diagram of a medical image scanning image processing system provided by an embodiment of this specification;

[0043] Figure 2 is a flow chart of a medical image scanning image processing method provided by an embodiment of this specification;

[0044] Figure 3 It is a schematic diagram of the training process of a target image registration model in a medical image scanning image processing method provided by an embodiment of this specification;

[0045] Figure 4 It is a schematic diagram of a medical image scanning image processing method provided by an embodiment of this specification applied to the medical field;

[0046] Figure 5 It is a data graph of a medical image scanning image processing method provided by an embodiment of this specification applied to the medical field;

[0047] Figure 6A medical image scanning image processing method provided by an embodiment of this specification is applied to a data graph of a case scan;

[0048] Figure 7 It is a structural schematic diagram of a medical image scanning image processing device provided by an embodiment of this specification;

[0049] Figure 8 is a flowchart of another medical image scanning image processing method provided by an embodiment of this specification;

[0050] Fig. 9 It is a structural schematic diagram of another medical image scanning image processing device provided by an embodiment of this specification;

[0051] Fig.10 is a flow chart of an image registration model training method provided by an embodiment of this specification;

[0052] Fig.11 It is a structural schematic diagram of an image registration model training device provided by an embodiment of this specification;

[0053] Fig.12 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0054] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0055] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0056] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0057] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0058] First, the terms involved in one or more embodiments of this specification are explained.

[0059] CNN: Convolutional Neural Network, a mathematical model or computational model that imitates the structure and function of biological neural networks and can be applied to fields such as image recognition.

[0060] NIO: Neural Instance Optimization, a method to fully adapt deep learning based registration models to the feature distribution of a single pair of images at the inference stage.

[0061] MR: Magnetic resonance imaging is a common imaging examination method in clinical practice. It mainly uses the magnetic resonance phenomenon generated by a strong external magnetic field and hydrogen nuclei in the human body under the action of specific radio frequency pulses, and finally forms an image through professional equipment. Magnetic resonance imaging can be used to examine various parts of the body.

[0062] CT: A clinical medical imaging technique that uses X-rays to scan the patient's body structure.

[0063] Knowledge distillation: It is a model compression method that improves the performance of the student model without changing its structure by guiding the lightweight student model to "imitate" the teacher model with better performance and more complex structure.

[0064] DSC: Dice similarity coefficient, Dice similarity coefficient, is an indicator used to calculate the similarity between two sets.

[0065] In this specification, two medical image scanning image processing methods are provided. This specification also involves two medical image scanning image processing devices, an image registration model training method, an image registration model training device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0066] See also Figure 1 , Figure 1 A schematic diagram of the structure of a medical image scanning image processing system provided according to an embodiment of the present specification is shown. The medical image scanning image processing system 100 may include a client 102 and a server 104;

[0067] The client 102 is used to send a first medical image scan image and a second medical image scan image of a target part to the server 104;

[0068] The server 104 inputs the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set, and the initial image registration model is obtained by pre-training according to a second training image set, and the first training image set and the second training image set are different; according to the displacement field, the first medical image scan image is adjusted to obtain an adjusted first medical image scan image; according to the second medical image scan image and the adjusted first medical image scan image, the target object position of the target part is determined. The first medical image scan image and the second medical image scan image marked with the target object position are sent to the client 102;

[0069] The client 102 is further configured to receive the first medical image scan image and the second medical image scan image marked with the target object position sent by the server 104 .

[0070] In the above system, after the initial image registration model is obtained by pre-training according to the second training image set, the initial image registration model is optimized according to the first training image set different from the second training image set, so that the obtained target image registration model can be applicable to the field corresponding to the first training image set. This avoids the reduction in robustness and accuracy of the image registration model caused by the difference between the first training image set and the second training image set, further improves the processing performance of the target image registration model, and ensures the accuracy of the registration results of the target image registration model for the medical imaging scan image. Based on this, the patient's condition analysis can be achieved based on the target image registration model.

[0071] In practical applications, the medical image scanning image processing system may include multiple clients 102 and a server 104. Multiple clients 102 may establish communication connections through the server 104. In the medical image scanning image processing scenario, the server 104 is used to provide medical image scanning image processing services between multiple clients 102. Multiple clients 102 may serve as senders or receivers, respectively, and achieve communication through the server 104.

[0072] The user can interact with the server 104 through the client 102 to receive data sent by other clients 102, or send data to other clients 102, etc. In the medical image scanning image processing scenario, the user can publish a data stream to the server 104 through the client 102, and the server 104 generates a medical image scanning image according to the data stream, and pushes the target object image to other clients that have established communication.

[0073] The client 102 and the server 104 are connected via a network. The network provides a medium for a communication link between the client 102 and the server 104. The network may include various connection types, such as wired or wireless communication links or optical fiber cables, etc. The data transmitted by the client 102 may need to be encoded, transcoded, compressed, etc. before being released to the server 104.

[0074] The client 102 can be a browser, an APP (Application), or a web application such as an H5 (HyperText Markup Language 5, Hypertext Markup Language Version 5) application, or a light application (also known as a mini-program, a lightweight application) or a cloud application, etc. The client 102 can be based on the software development kit (SDK, Software Development Kit) of the corresponding service provided by the server 104, such as based on the real-time communication (RTC, Real Time Communication) SDK development and acquisition. The client 102 can be connected to a medical scanning device for communication, and the medical scanning device can be, for example, a CT scanning device. The client 102 can be deployed in an electronic device and needs to rely on the device to run or some APPs in the device to run. For example, the electronic device can have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, and other end-side devices. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0075] The server 104 may include servers that provide various services, such as servers that provide communication services to multiple clients, servers for background training that provide support for models used on clients, and servers that process data sent by clients. It should be noted that the server 104 can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server can also be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server (cloud-side device) for basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0076] It is worth noting that the medical image scanning image processing method provided in the embodiments of this specification is generally executed by the server, but in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the medical image scanning image processing method provided in the embodiments of this specification. In other embodiments, the medical image scanning image processing method provided in the embodiments of this specification may also be jointly executed by the client and the server.

[0077] See also Figure 2 , Figure 2 A flowchart of a medical image scanning image processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0078] Step 202: Determine a first medical image scan image and a second medical image scan image of a target part.

[0079] The target part can be understood as a body part of the patient, such as the patient's brain, lungs, and stomach. The first medical image scan image and the second medical image scan image of the target part can be understood as medical image scan images obtained after a medical scan of the patient's body part, such as a medical image scan image obtained after a CT scan of the patient's lungs. The first medical image scan image and the second medical image scan image can be different. For example, the first medical image scan image can be a medical image scan image obtained after a CT scan of the patient's lungs when the patient is inhaling, and the second medical image scan image can be a medical image scan image obtained after a CT scan of the patient's lungs when the patient is exhaling.

[0080] Based on this, a medical scanning device can be used to scan the target part of the patient, and a first medical imaging scan image and a second medical imaging scan image of the target part of the patient sent by the medical scanning device can be received.

[0081] In practical applications, the first medical image scan image may be a moving image, and the second medical image scan image may be a fixed image.

[0082] Step 204: Input the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model based on a first training image set, and the initial image registration model is obtained by pre-training based on a second training image set, and the first training image set and the second training image set are different.

[0083] Specifically, after determining the first medical image scan image and the second medical image scan image of the target part of the patient, the target image registration model can be used to perform registration processing on the first medical image scan image and the second medical image scan image to obtain a displacement field between the first medical image scan image and the second medical image scan image.

[0084] The displacement field between the first medical image scan image and the second medical image scan image can be understood as a deformation field.

[0085] In practical applications, the target image registration model can be a CNN model.

[0086] The following combination Figure 3 , Figure 3 A schematic diagram of the training process of a target image registration model in a medical image scanning image processing method provided according to an embodiment of the present specification is shown, and the specific training steps are as follows.

[0087] Before inputting the first medical image scan image and the second medical image scan image into the target image registration model, the method further includes:

[0088] Step 302: Determine a first training image set, wherein the first training image set includes a first image sample and a second image sample;

[0089] The first image sample and the second image sample may be understood as medical image scanning image samples.

[0090] Step 304: In the initial image registration model, extract the first image feature of the first image sample and the second image feature of the second image sample, and process the first image feature and the second image feature according to a preset resolution to obtain a predicted displacement field between the first image sample and the second image sample.

[0091] Specifically, the initial image registration model here can be understood as an initial image registration model obtained after pre-training that satisfies the training stop condition. Before using the initial image registration model to process the first image sample and the second image sample, the initial image registration model can be pre-trained according to the second training image set, and the specific implementation method is as follows:

[0092] Before extracting the first image feature of the first image sample and the second image feature of the second image sample in the initial image registration model, the method further includes:

[0093] Determining a second training image set, wherein the second training image set includes a third image sample and a fourth image sample;

[0094] Inputting the third image sample and the fourth image sample into an initial image registration model to obtain an initial predicted displacement field output by the initial image registration model;

[0095] The initial image registration model is trained according to the similarity between the third image sample and the fourth image sample and the initial predicted displacement field until an initial image registration model that meets a training stop condition is obtained.

[0096] Among them, the third image sample and the fourth image sample included in the second training image set can be understood as medical image scan images. It can be understood that the field corresponding to the second training image set and the field corresponding to the first training image set can be different. For example, the second training image set can include medical image scan images of multiple body parts obtained by performing medical scans on multiple patients in medical center A. The first training image set may include medical image scan images of multiple body parts obtained by performing medical scans on patients in medical center B. The training stop condition can be understood as the number of training times reaching a preset number threshold and / or the model loss value reaching a preset loss value threshold.

[0097] Specifically, in the initial image registration model, the third image feature of the third image sample and the fourth image feature of the fourth image sample can be extracted, and the third image feature and the fourth image feature can be processed according to a preset resolution to obtain an initial predicted displacement field between the third image sample and the fourth image sample. When pre-training the initial image registration model, the similarity between the third image feature and the fourth image feature can be used as the similarity between the third image sample and the fourth image sample, and the initial image registration model can be pre-trained based on the similarity and the smoothness of the initial predicted displacement field until an initial image registration model that meets the training stop condition is obtained. It can be understood that the pre-training process of the initial image registration model is similar to the subsequent optimization process of the initial image registration model that meets the training stop condition obtained after pre-training, and will not be repeated here.

[0098] In summary, by pre-training the initial image registration model using the second training image set, it is possible to learn the image registration performance of the initial image registration model, so that the initial image registration model that meets the training stop condition can have image registration performance.

[0099] In a specific implementation, the first image feature and the second image feature are processed according to a preset resolution to obtain a predicted displacement field between the first image sample and the second image sample, including:

[0100] The first image feature and the second image feature are downsampled according to a preset resolution to obtain a predicted displacement field between the first image sample and the second image sample.

[0101] In a specific implementation, the first image sample and the second image sample can be input into an initial image registration model. In the initial image registration model, a convolutional network encoder is used to extract features of the first image sample and the second image sample to obtain first image features and second image features. The first image features and the second image features are input into a convolutional network decoder to obtain a predicted displacement field output by the convolutional network decoder.

[0102] In practical applications, the first image samples and the second image samples may be downsampled using trilinear interpolation.

[0103] Further, the initial image registration model includes n network layers, wherein n is greater than 1 and n is an integer;

[0104] Accordingly, the processing of the first image feature and the second image feature according to the preset resolution to obtain a predicted displacement field between the first image sample and the second image sample includes:

[0105] According to the jth preset resolution corresponding to the i-th network layer, the first image feature and the second image feature are processed by the i-th network layer to obtain the i-th predicted displacement field between the first image sample and the second image sample, wherein i and j start from 1, and i and j are integers, and the preset resolution increases as j increases;

[0106] Determine whether i is greater than or equal to n;

[0107] If not, i and j are incremented by 1, and the step of processing the first image feature and the second image feature by using the i-th network layer according to the j-th preset resolution corresponding to the i-th network layer to obtain the i-th predicted displacement field between the first image sample and the second image sample is continued;

[0108] If so, determine a predicted displacement field between the first image sample and the second image sample according to the i-th predicted displacement field.

[0109] Specifically, taking the target image registration model including three network layers as an example, the first image feature and the second image feature can be processed by the first network layer according to the first preset resolution corresponding to the first network layer to obtain the first predicted displacement field between the first image sample and the second image sample. According to the second preset resolution corresponding to the second network layer, the first image feature and the second image feature are processed by the second network layer to obtain the second predicted displacement field between the first image sample and the second image sample. According to the third preset resolution corresponding to the third network layer, the first image feature and the second image feature are processed by the third network layer to obtain the third predicted displacement field between the first image sample and the second image sample. The first predicted displacement field, the second predicted displacement field and the third predicted displacement field are added to obtain the predicted displacement field between the first image sample and the second image sample. Among them, the first preset resolution is less than the second preset resolution and less than the third preset resolution.

[0110] In practical applications, in a medical image scanning image processing method provided by an embodiment of this specification, the target image registration model can use a neural instance optimization method (i.e., NIO). In order to avoid the defects in the instance optimization method, the generated displacement field can be implicitly improved by updating the parameters of the CNN (i.e., the target image registration model). The pyramid image registration network can be parameterized according to the deep Laplace, as shown in the following formula (1).

[0111]

[0112] Among them, F t It can be understood as a fixed image, M tIt can be understood as a moving image, ο represents a composite operator, that is, M t οf θ (F t , M t )=M t (Φ). θ can be understood as the network parameter of the image registration model, u is the displacement field, and f θ is a network layer. In the inference phase of each image pair consisting of a fixed image and a moving image, the parameter θ in the network (i.e., the image registration model) can be initialized using a training dataset including fixed images and moving images, and then θ can be determined by η iterations of gradient descent. * The optimal set of , which enables the model to fit a single image pair in the training dataset.

[0113] In the NIO-based multi-resolution optimization strategy, an image registration network framework can be given, and the image input into the image registration network framework can be downsampled using trilinear interpolation to create an image pyramid for the input image, and each network layer in the image registration network framework can be gradually optimized.

[0114] Specifically, the inherent inductive biases embedded in CNNs, namely weight sharing and locality, implicitly reduce the degrees of freedom of the solution and reduce the search space of the optimization problem. Second, during the optimization process, the prior knowledge of image registration in the training dataset can be transferred to a single test image pair, avoiding suboptimal solutions in instance optimization.

[0115] Step 306: adjusting the first image samples according to the predicted displacement field to obtain adjusted first image samples.

[0116] In a specific implementation, adjusting the first image sample according to the predicted displacement field to obtain the adjusted first image sample includes:

[0117] According to the predicted displacement field, the first image samples are spatially transformed to obtain adjusted first image samples.

[0118] Specifically, performing spatial transformation on the first image sample can be understood as deforming the first image sample, that is, image twisting. In practical applications, the spatial transformation includes but is not limited to scaling, translating, rotating, and nonlinearly transforming the first image sample.

[0119] Step 308: Optimize the initial image registration model according to the predicted displacement field, the second image samples and the adjusted first image samples to obtain a target image registration model.

[0120] Specifically, optimizing the initial image registration model according to the predicted displacement field, the adjusted first image samples and the second image samples to obtain a target image registration model includes:

[0121] Determining the smoothness of the predicted displacement field, and determining the smoothness as a first model loss value;

[0122] Determine a similarity between the second image sample and the adjusted first image sample, and determine the similarity as a second model loss value;

[0123] The initial image registration model is optimized according to the first model loss value and the second model loss value to obtain a target image registration model.

[0124] Specifically, the target model loss value can be determined according to the first model loss value and the second model loss value, and the parameters of the initial image registration model can be adjusted according to the target model loss value to obtain the target image registration model.

[0125] In specific implementation, methods for determining the target model loss value include but are not limited to summing the first model loss value and the second model loss value, performing weighted summation of the first model loss value and the second model loss value, etc., and the embodiments of this specification do not limit this.

[0126] In practical applications, the loss function of the i-th network layer is shown in the following formula (2).

[0127]

[0128] Where L is the number of iterations. λ can be used to normalize the loss function, and its value range is usually (0,1). is a moving image adjusted according to the displacement field.

[0129] In a specific implementation, adjusting the first image sample according to the predicted displacement field to obtain the adjusted first image sample includes:

[0130] Adjusting the first image sample according to the i-th predicted displacement field and the predicted displacement fields output by all network layers before the i-th network layer to obtain an adjusted first image sample;

[0131] Accordingly, the initial image registration model is optimized according to the predicted displacement field, the second image sample and the adjusted first image sample to obtain a target image registration model, including:

[0132] According to the i-th predicted displacement field and the predicted displacement fields output by all network layers before the i-th network layer, the second image samples and the adjusted first image samples, the i-th network layer in the initial image registration model is optimized to obtain a target image registration model.

[0133] Specifically, following the above example, the first image sample can be adjusted according to the first predicted displacement field to obtain the adjusted first image sample, and the first network layer can be optimized according to the first predicted displacement field, the second image sample and the adjusted first image sample until the first network layer meets the optimization stop condition. Afterwards, the first image feature and the second image feature can be processed using the first network layer that meets the optimization stop condition to obtain the first predicted displacement field.

[0134] Then, the first image sample is adjusted according to the sum of the first predicted displacement field and the second predicted displacement field to obtain the adjusted first image sample, and the second network layer is optimized according to the sum of the first predicted displacement field and the second predicted displacement field, the second image sample and the adjusted first image sample, until the second network layer meets the optimization stop condition.

[0135] Finally, the first image sample is adjusted according to the sum of the first predicted displacement field, the second predicted displacement field and the third predicted displacement field to obtain the adjusted first image sample, and the third network layer is optimized according to the sum of the first predicted displacement field, the second predicted displacement field and the third predicted displacement field, the second image sample and the adjusted first image sample until the third network layer meets the optimization stop condition.

[0136] In summary, it is possible to optimize the underlying layer of the target image registration model, and gradually add the upper-layer network of the target image registration model for joint optimization, and realize implicit updating of the displacement field. On the basis of unsupervised learning, the image registration problem is relatively simple in the optimization space of low image resolution (i.e., underlying network processing), and can provide a robust initial solution for higher resolution images, making it easier to determine the optimal solution to the registration problem during the optimization process. In addition, a large amount of calculations are concentrated on the low and medium resolutions of the image, which can reduce the amount of calculation and improve the optimization efficiency.

[0137] Step 206: adjusting the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image.

[0138] Specifically, after obtaining the displacement field between the first medical image scan image and the second medical image scan image output by the target image registration model, the first medical image scan image is spatially transformed according to the displacement field to obtain an adjusted first medical image scan image.

[0139] The spatial transformation of the first medical image scanned image can be understood as deforming the first medical image scanned image, that is, image twisting. In practical applications, the spatial transformation includes but is not limited to scaling, translating, rotating and nonlinear transformation of the first medical image scanned image.

[0140] Step 208: Determine the target object position of the target part according to the second medical image scan image and the adjusted first medical image scan image.

[0141] The target object position of the target part can be understood as the lesion position of the patient's body part.

[0142] In specific implementation, the lesion location can be outlined on the first medical image scan image, and the outlined lesion location can be spatially transformed using the displacement field output by the target image registration model to obtain an adjusted first medical image scan image, thereby obtaining the lesion location in the second medical image scan image corresponding to the first medical image scan image.

[0143] It can be understood that the first medical image scan image can be understood as one of the two medical image scan images, and the second medical image scan image can be understood as the other of the two medical image scan images. Based on this, the second medical image scan image can also be adjusted according to the displacement field to obtain the adjusted second medical image scan image, and the target object position of the target part can be determined based on the first medical image scan image and the adjusted second medical image scan image.

[0144] In summary, in the above method, after the initial image registration model is obtained by pre-training according to the second training image set, the initial image registration model is also optimized according to the first training image set different from the second training image set, so that the obtained target image registration model can be applicable to the field corresponding to the first training image set. This avoids the reduction in robustness and accuracy of the image registration model caused by the difference between the first training image set and the second training image set, further improves the processing performance of the target image registration model, and ensures the accuracy of the registration results of the target image registration model for medical imaging scans. Based on this, the patient's condition analysis can be achieved based on the target image registration model.

[0145] Specifically, the target image registration model provided in the embodiments of this specification is evaluated on the cross-dataset brain atlas registration. As shown in Table 1 below, Table 1 shows other current registration methods and the NIO-based target image registration model provided in this solution on the brain MR scan of the first data set and the brain MR scan of the second data set.

[0146] Table 1

[0147]

[0148] The higher the first performance index and the second performance index, the better the registration performance. The lower the third performance index and the fourth performance index, the better the registration performance. In test 1, the first data set is used as a training data set, the second data set is used as a test data set, and the n scans of the first data set are trained. The images in the second data set are registered to a predefined atlas. In test 2, the second data set is used as a training data set, and the first data set is used as a test data set. In test 3, the second data set is used as both a training data set and a test data set.

[0149] As can be seen from Table 1, the image registration method provided by this scheme performs well in processing brain MR scan images, and significantly improves the registration accuracy and robustness in brain map registration.

[0150] See Table 2 below, which shows the mean ± standard deviation of the registration errors determined by the image registration method provided by the present solution and other image registration methods on multiple cases in the third data set.

[0151] Table 2

[0152]

[0153] It can be seen from Table 2 that the image registration method provided by this scheme has a lower registration error and better performance.

[0154] See Table 3, which is similar to Table 1, and shows the performance indicators of the image registration method provided by this solution and other registration methods when the training data set and the test data set are the same data set.

[0155] Table 3

[0156]

[0157] Among them, in test 4, the training data set and the test data set are both the first data set, and in test 5, the training data set and the test data set are both the second data set. Table 3 shows the quantitative results (within the distribution) of different registration methods for brain MR atlas registration. The results show that this scheme can improve the model performance when the training data and the test data are sampled from the same data set.

[0158] For details, see Figure 4 , Figure 4 A schematic diagram showing a medical image scanning image processing method provided by an embodiment of the present specification applied in the medical field. Figure 4The fixed images and moving images generated by axial, sagittal and coronal MR slices are shown respectively, and the images processed by the current multiple registration methods and the image registration method proposed in this scheme are shown. Figure 5 , Figure 5 A data diagram showing a medical image scanning image processing method provided by an embodiment of the present specification applied to the medical field is shown. Figure 5 (a) and (b) in the figure depict the average similarity scores of each anatomical structure for the current multiple registration methods and the image registration method proposed in this scheme. Specifically, Figure 5 It includes a variety of anatomical structures, such as brainstem structure, cerebellar cortex structure, thalamus structure, brain white matter structure, cerebellar white matter structure and palladium structure. Figure 5 Taking the anatomical structure 1 in the figure as an example, the 1 corresponding to the anatomical structure 1 is the average similarity score of the anatomical structure 1 processed by the first image registration method, 2 is the average similarity score of the anatomical structure 1 processed by the second image registration method, 3 is the average similarity score of the anatomical structure 1 processed by the third image registration method, and 4 is the average similarity score of the anatomical structure 1 processed by the image registration method provided by this scheme. In practical applications, the average similarity score can be, for example, the Dyss similarity coefficient. Figure 6 , Figure 6 A data diagram showing a medical image scanning image processing method provided by an embodiment of the present specification applied to case scanning is shown. It can be understood that Figure 6 The scatter histograms of the target registration errors when the images of Cases 1 to 10 are processed using the three image registration methods are shown in FIG. Figure 6 (a) in FIG. 1 shows the target registration error when the images of cases 1 to 10 are processed according to the first image registration method. Figure 6 (b) in FIG. 1 shows the target registration error when the images of cases 1 to 10 are processed according to the second image registration method. Figure 6 (c) in FIG. 1 shows the target registration error when the images of cases 1 to 10 are processed according to the image registration method proposed in this scheme. Figure 6 In order to facilitate understanding, 10 different colors are used to distinguish Case 1 to Case 10. For example, in (a), Case 1 is represented by blue in the scatter histogram, and Case 2 is represented by orange in the scatter histogram. It can be understood that those skilled in the art can distinguish Case 1 to Case 10 based on the different colors in the scatter histogram.

[0159] It can be seen from the above drawings that the medical image scanning image processing method provided in the embodiments of this specification has good processing performance.

[0160] Corresponding to the above method embodiment, this specification also provides a medical image scanning image processing device embodiment, Figure 7 FIG. 1 is a schematic diagram showing the structure of a medical image scanning image processing device provided by an embodiment of the present specification. Figure 7 As shown, the device comprises:

[0161] A first determination module 702 is configured to determine a first medical image scan image and a second medical image scan image of a target part;

[0162] An input module 704 is configured to input the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set, and the initial image registration model is obtained by pre-training according to a second training image set, and the first training image set and the second training image set are different;

[0163] An adjustment module 706 is configured to adjust the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image;

[0164] The second determination module 708 is configured to determine the target object position of the target part according to the second medical image scan image and the adjusted first medical image scan image.

[0165] In an optional embodiment, the device further includes a training module configured to:

[0166] Determining a first training image set, wherein the first training image set includes a first image sample and a second image sample;

[0167] In the initial image registration model, extracting a first image feature of the first image sample and a second image feature of the second image sample, and processing the first image feature and the second image feature according to a preset resolution to obtain a predicted displacement field between the first image sample and the second image sample;

[0168] adjusting the first image sample according to the predicted displacement field to obtain an adjusted first image sample;

[0169] The initial image registration model is optimized according to the predicted displacement field, the second image samples and the adjusted first image samples to obtain a target image registration model.

[0170] In an optional embodiment, the training module is further configured to:

[0171] The first image feature and the second image feature are downsampled according to a preset resolution to obtain a predicted displacement field between the first image sample and the second image sample.

[0172] In an optional embodiment, the training module is further configured to:

[0173] According to the predicted displacement field, the first image samples are spatially transformed to obtain adjusted first image samples.

[0174] In an optional embodiment, the training module is further configured to:

[0175] Determining the smoothness of the predicted displacement field, and determining the smoothness as a first model loss value;

[0176] Determine a similarity between the second image sample and the adjusted first image sample, and determine the similarity as a second model loss value;

[0177] The initial image registration model is optimized according to the first model loss value and the second model loss value to obtain a target image registration model.

[0178] In an optional embodiment, the training module is further configured to:

[0179] The initial image registration model includes n network layers, where n is greater than 1 and n is an integer;

[0180] According to the jth preset resolution corresponding to the i-th network layer, the first image feature and the second image feature are processed by the i-th network layer to obtain the i-th predicted displacement field between the first image sample and the second image sample, wherein i and j start from 1, and i and j are integers, and the preset resolution increases as j increases;

[0181] Determine whether i is greater than or equal to n;

[0182] If not, i and j are incremented by 1, and the step of processing the first image feature and the second image feature by using the i-th network layer according to the j-th preset resolution corresponding to the i-th network layer to obtain the i-th predicted displacement field between the first image sample and the second image sample is continued;

[0183] If so, determine a predicted displacement field between the first image sample and the second image sample according to the i-th predicted displacement field.

[0184] In an optional embodiment, the training module is further configured to:

[0185] Adjusting the first image sample according to the i-th predicted displacement field and the predicted displacement fields output by all network layers before the i-th network layer to obtain an adjusted first image sample;

[0186] According to the i-th predicted displacement field and the predicted displacement fields output by all network layers before the i-th network layer, the second image samples and the adjusted first image samples, the i-th network layer in the initial image registration model is optimized to obtain a target image registration model.

[0187] In an optional embodiment, the training module is further configured to:

[0188] Determining a second training image set, wherein the second training image set includes a third image sample and a fourth image sample;

[0189] Inputting the third image sample and the fourth image sample into an initial image registration model to obtain an initial predicted displacement field output by the initial image registration model;

[0190] The initial image registration model is trained according to the similarity between the third image sample and the fourth image sample and the initial predicted displacement field until an initial image registration model that meets a training stop condition is obtained.

[0191] In the above device, after the initial image registration model is obtained by pre-training according to the second training image set, the initial image registration model is optimized according to the first training image set different from the second training image set, so that the obtained target image registration model can be applicable to the field corresponding to the first training image set. This avoids the reduction in robustness and accuracy of the image registration model caused by the difference between the first training image set and the second training image set, further improves the processing performance of the target image registration model, and ensures the accuracy of the registration results of the target image registration model for the medical imaging scan image. Based on this, the patient's condition analysis can be achieved based on the target image registration model.

[0192] The above is a schematic scheme of a medical image scanning image processing device of this embodiment. It should be noted that the technical scheme of the medical image scanning image processing device and the technical scheme of the medical image scanning image processing method described above belong to the same concept, and the details not described in detail in the technical scheme of the medical image scanning image processing device can be referred to the description of the technical scheme of the medical image scanning image processing method described above.

[0193] See also Figure 8 , Figure 8 A flowchart of another medical image scanning image processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0194] Step 802: receiving an image registration request, wherein the image registration request carries a first medical image scan image and a second medical image scan image of a target part;

[0195] Step 804: inputting the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set, and the initial image registration model is obtained by pre-training according to a second training image set, and the first training image set and the second training image set are different;

[0196] Step 806: adjusting the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image;

[0197] Step 808: Determine the target object position of the target part according to the second medical image scan image and the adjusted first medical image scan image.

[0198] Specifically, the medical image scanning image processing method provided in the embodiments of this specification can also be applied to the patient's lesion detection, for example, it can be applied to the patient's cancer lesion detection, or tuberculosis lesion detection, etc. The specific implementation method is as follows:

[0199] receiving an image registration request, wherein the image registration request carries a first medical image scan image and a second medical image scan image of the lung;

[0200] Inputting the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set, and the initial image registration model is obtained by pre-training according to a second training image set, and the first training image set and the second training image set are different;

[0201] Adjusting the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image;

[0202] The location of the tuberculosis lesion in the lung is determined according to the second medical image scan image and the adjusted first medical image scan image.

[0203] In practical applications, when the medical image scanning image processing method provided in the embodiments of this specification is applied in the medical field, the patient's CT scan image or MR scan image can be registered through the target image registration model, thereby realizing the patient's lesion location detection and condition analysis.

[0204] Corresponding to the above method embodiment, this specification also provides another medical image scanning image processing device embodiment, Fig. 9 FIG. 2 shows a schematic diagram of the structure of another medical image scanning image processing device provided by an embodiment of the present specification. Fig. 9 As shown, the device comprises:

[0205] A receiving module 902 is configured to receive an image registration request, wherein the image registration request carries a first medical image scan image and a second medical image scan image of a target part;

[0206] An input module 904 is configured to input the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set, and the initial image registration model is obtained by pre-training according to a second training image set, and the first training image set and the second training image set are different;

[0207] An adjustment module 906 is configured to adjust the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image;

[0208] The determination module 908 is configured to determine the target object position of the target part according to the second medical image scan image and the adjusted first medical image scan image.

[0209] In the above device, after the initial image registration model is obtained by pre-training according to the second training image set, the initial image registration model is optimized according to the first training image set different from the second training image set, so that the obtained target image registration model can be applicable to the field corresponding to the first training image set. This avoids the reduction in robustness and accuracy of the image registration model caused by the difference between the first training image set and the second training image set, further improves the processing performance of the target image registration model, and ensures the accuracy of the registration results of the target image registration model for the medical imaging scan image. Based on this, the patient's condition analysis can be achieved based on the target image registration model.

[0210] The above is a schematic scheme of a medical image scanning image processing device of this embodiment. It should be noted that the technical scheme of the medical image scanning image processing device and the technical scheme of the medical image scanning image processing method described above belong to the same concept, and the details not described in detail in the technical scheme of the medical image scanning image processing device can be referred to the description of the technical scheme of the medical image scanning image processing method described above.

[0211] See also Fig.10 , Fig.10 A flowchart of an image registration model training method provided according to an embodiment of the present specification is shown, which is applied to a cloud-side device, and the specific steps are as follows.

[0212] Step 1002: Determine a first training image set, wherein the first training image set includes a first image sample and a second image sample;

[0213] Step 1004: extracting a first image feature of the first image sample and a second image feature of the second image sample in the initial image registration model, and processing the first image feature and the second image feature according to a preset resolution to obtain a predicted displacement field between the first image sample and the second image sample;

[0214] Step 1006: adjusting the first image sample according to the predicted displacement field to obtain an adjusted first image sample;

[0215] Step 1008: Optimize the initial image registration model according to the predicted displacement field, the second image samples and the adjusted first image samples to obtain a target image registration model.

[0216] Specifically, the model optimization process provided in the embodiments of this specification is similar to the aforementioned model optimization process and will not be repeated here.

[0217] Corresponding to the above method embodiment, this specification also provides an image registration model training device embodiment, Fig.11 FIG. 2 shows a schematic diagram of the structure of an image registration model training device provided by an embodiment of the present specification. Fig.11 As shown, the device comprises:

[0218] A determination module 1102 is configured to determine a first training image set, wherein the first training image set includes a first image sample and a second image sample;

[0219] The processing module 1104 is configured to extract a first image feature of the first image sample and a second image feature of the second image sample in the initial image registration model, and process the first image feature and the second image feature according to a preset resolution to obtain a predicted displacement field between the first image sample and the second image sample;

[0220] An adjustment module 1106 is configured to adjust the first image sample according to the predicted displacement field to obtain an adjusted first image sample;

[0221] The optimization module 1108 is configured to optimize the initial image registration model according to the predicted displacement field, the second image samples and the adjusted first image samples to obtain a target image registration model.

[0222] Fig.12 The block diagram of a computing device 1200 according to one embodiment of the present specification is shown. The components of the computing device 1200 include but are not limited to a memory 1210 and a processor 1220. The processor 1220 is connected to the memory 1210 via a bus 1230, and the database 1250 is used to store data.

[0223] The computing device 1200 also includes an access device 1240 that enables the computing device 1200 to communicate via one or more networks 1260. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1240 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0224] In one embodiment of the present application, the above components of the computing device 1200 and Fig.12 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.12 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0225] The computing device 1200 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 1200 may also be a mobile or stationary server.

[0226] The processor 1220 is used to execute the following computer executable instructions, which implement the steps of the above method when executed by the processor.

[0227] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above method.

[0228] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which can implement the steps of the above method when executed by a processor.

[0229] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above method.

[0230] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above method.

[0231] The above is an illustrative solution of a computer program of this embodiment. It should be noted that the technical solution of the computer program and the technical solution of the above method belong to the same concept, and the details not described in detail in the technical solution of the computer program can be referred to the description of the technical solution of the above method.

[0232] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0233] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0234] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0235] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0236] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A medical image scanning image processing method, comprising: Determine a first medical imaging scan image and a second medical imaging scan image of a target part; Inputting the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set and a plurality of incremental preset resolutions, the initial image registration model is obtained by pre-training according to a second training image set, the first training image set and the second training image set have different fields, and the initial image registration model includes n network layers, where n is greater than 1 and is an integer; Adjusting the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image; determining a target object position of the target part according to the second medical image scan image and the adjusted first medical image scan image; Before inputting the first medical image scan image and the second medical image scan image into the target image registration model, the method further includes: Determining a first training image set, wherein the first training image set includes a first image sample and a second image sample; In the initial image registration model, extract a first image feature of the first image sample and a second image feature of the second image sample, and process the first image feature and the second image feature using the i-th network layer according to the j-th preset resolution corresponding to the i-th network layer to obtain an i-th predicted displacement field between the first image sample and the second image sample, wherein i and j start from 1 and are integers, and the preset resolution increases as j increases; Determine whether i is greater than or equal to n; If not, i and j are incremented by 1, and the step of processing the first image feature and the second image feature by using the i-th network layer according to the j-th preset resolution corresponding to the i-th network layer to obtain the i-th predicted displacement field between the first image sample and the second image sample is continued; If so, determining a predicted displacement field between the first image sample and the second image sample according to the i-th predicted displacement field; adjusting the first image sample according to the predicted displacement field to obtain an adjusted first image sample; The initial image registration model is optimized according to the predicted displacement field, the second image samples and the adjusted first image samples to obtain a target image registration model.

2. The method according to claim 1, wherein the first image feature and the second image feature are processed according to a preset resolution to obtain a predicted displacement field between the first image sample and the second image sample, comprising: The first image feature and the second image feature are downsampled according to a preset resolution to obtain a predicted displacement field between the first image sample and the second image sample.

3. The method according to claim 1, wherein adjusting the first image sample according to the predicted displacement field to obtain the adjusted first image sample comprises: According to the predicted displacement field, the first image samples are spatially transformed to obtain adjusted first image samples.

4. The method according to claim 1, wherein the initial image registration model is optimized according to the predicted displacement field, the adjusted first image sample and the second image sample to obtain a target image registration model, comprising: Determining the smoothness of the predicted displacement field, and determining the smoothness as a first model loss value; Determine a similarity between the second image sample and the adjusted first image sample, and determine the similarity as a second model loss value; The initial image registration model is optimized according to the first model loss value and the second model loss value to obtain a target image registration model.

5. The method according to claim 1, wherein adjusting the first image sample according to the predicted displacement field to obtain the adjusted first image sample comprises: Adjusting the first image sample according to the i-th predicted displacement field and the predicted displacement fields output by all network layers before the i-th network layer to obtain an adjusted first image sample; Accordingly, the initial image registration model is optimized according to the predicted displacement field, the second image sample and the adjusted first image sample to obtain a target image registration model, including: According to the i-th predicted displacement field and the predicted displacement fields output by all network layers before the i-th network layer, the second image samples and the adjusted first image samples, the i-th network layer in the initial image registration model is optimized to obtain a target image registration model.

6. The method according to claim 1, before extracting the first image feature of the first image sample and the second image feature of the second image sample in the initial image registration model, further comprising: Determining a second training image set, wherein the second training image set includes a third image sample and a fourth image sample; Inputting the third image sample and the fourth image sample into an initial image registration model to obtain an initial predicted displacement field output by the initial image registration model; The initial image registration model is trained according to the similarity between the third image sample and the fourth image sample and the initial predicted displacement field until an initial image registration model that meets a training stop condition is obtained.

7. A medical image scanning image processing method, applied to a cloud-side device, comprising: Receiving an image registration request, wherein the image registration request carries a first medical image scan image and a second medical image scan image of a target part; Inputting the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set and a plurality of incremental preset resolutions, the initial image registration model is obtained by pre-training according to a second training image set, the first training image set and the second training image set have different fields, and the initial image registration model includes n network layers, where n is greater than 1 and is an integer; Adjusting the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image; determining a target object position of the target part according to the second medical image scan image and the adjusted first medical image scan image; Before inputting the first medical image scan image and the second medical image scan image into the target image registration model, the method further includes: Determining a first training image set, wherein the first training image set includes a first image sample and a second image sample; In the initial image registration model, extract a first image feature of the first image sample and a second image feature of the second image sample, and process the first image feature and the second image feature using the i-th network layer according to the j-th preset resolution corresponding to the i-th network layer to obtain an i-th predicted displacement field between the first image sample and the second image sample, wherein i and j start from 1 and are integers, and the preset resolution increases as j increases; Determine whether i is greater than or equal to n; If not, i and j are incremented by 1, and the step of processing the first image feature and the second image feature by using the i-th network layer according to the j-th preset resolution corresponding to the i-th network layer to obtain the i-th predicted displacement field between the first image sample and the second image sample is continued; If so, determining a predicted displacement field between the first image sample and the second image sample according to the i-th predicted displacement field; adjusting the first image sample according to the predicted displacement field to obtain an adjusted first image sample; The initial image registration model is optimized according to the predicted displacement field, the second image samples and the adjusted first image samples to obtain a target image registration model.

8. An image registration model training method, applied to a cloud-side device, comprising: Determining a first training image set, wherein the first training image set includes a first image sample and a second image sample; In the initial image registration model, extracting a first image feature of the first image sample and a second image feature of the second image sample, and processing the first image feature and the second image feature using the i-th network layer according to the j-th preset resolution corresponding to the i-th network layer, to obtain an i-th predicted displacement field between the first image sample and the second image sample, wherein i and j start from 1, and i and j are integers, and the preset resolution increases as j increases; Determine whether i is greater than or equal to n; If not, i and j are incremented by 1, and the step of processing the first image feature and the second image feature by using the i-th network layer according to the j-th preset resolution corresponding to the i-th network layer to obtain the i-th predicted displacement field between the first image sample and the second image sample is continued; If so, determining a predicted displacement field between the first image sample and the second image sample according to the i-th predicted displacement field, wherein the initial image registration model is pre-trained according to a second training image set, the first training image set and the second training image set are in different fields, and the initial image registration model includes n network layers, where n is greater than 1 and is an integer; adjusting the first image sample according to the predicted displacement field to obtain an adjusted first image sample; The initial image registration model is optimized according to the predicted displacement field, the second image samples and the adjusted first image samples to obtain a target image registration model.

9. A medical image scanning image processing device, comprising: A first determination module is configured to determine a first medical image scan image and a second medical image scan image of a target part; An input module is configured to input the first medical image scan image and the second medical image scan image into a target image registration model to obtain a displacement field between the first medical image scan image and the second medical image scan image, wherein the target image registration model is obtained by optimizing an initial image registration model according to a first training image set and a plurality of incremental preset resolutions, the initial image registration model is obtained by pre-training according to a second training image set, the first training image set and the second training image set have different fields, and the initial image registration model includes n network layers, where n is greater than 1 and is an integer; an adjustment module, configured to adjust the first medical image scan image according to the displacement field to obtain an adjusted first medical image scan image; A second determination module is configured to determine a target object position of the target part according to the second medical image scan image and the adjusted first medical image scan image; The training module is configured to determine a first training image set, wherein the first training image set includes a first image sample and a second image sample; in the initial image registration model, extract a first image feature of the first image sample and a second image feature of the second image sample, and process the first image feature and the second image feature using the i-th network layer according to the j-th preset resolution corresponding to the i-th network layer, so as to obtain an i-th predicted displacement field between the first image sample and the second image sample, wherein i and j start from 1, and i and j are integers, and the preset resolution increases as j increases; determine whether i is greater than or equal to n; if not, i and j are automatically Add 1, continue to execute the step of processing the first image feature and the second image feature according to the jth preset resolution corresponding to the ith network layer using the ith network layer to obtain the ith predicted displacement field between the first image sample and the second image sample; if yes, determine the predicted displacement field between the first image sample and the second image sample according to the ith predicted displacement field; adjust the first image sample according to the predicted displacement field to obtain an adjusted first image sample; optimize the initial image registration model according to the predicted displacement field, the second image sample and the adjusted first image sample to obtain a target image registration model.

10. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images