Image registration quality evaluation method and device, equipment and storage medium
By combining the registration quality evaluation results output by the model and a variety of similarity indicators, comprehensive evaluation indicators are used to evaluate image registration quality, which solves the problem that a single evaluation indicator is susceptible to noise and image quality, and improves the accuracy and efficiency of the evaluation.
Patent Information
- Application Number
- CN202311444096.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-31
- Publication Date
- 2025-05-06
AI Technical Summary
In the prior art, a single evaluation index is susceptible to noise and image quality factors, resulting in errors in image registration quality evaluation.
By combining the registration quality evaluation results output by the model and a variety of similarity indicators corresponding to multimodal medical images, comprehensive evaluation indicators are used to evaluate image registration quality more accurately.
It improves the accuracy and reliability of image registration quality evaluation, reduces labor and time costs, and improves evaluation efficiency.
Smart Images
Figure CN119941603A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to an image registration quality assessment method, device, equipment and storage medium. Background Art
[0002] In the medical field, due to the environment and its equipment, the images collected usually often lack information and cannot meet the doctor's diagnostic needs. By registering images from different types of medical imaging methods, doctors can be provided with comprehensive information from various aspects, as well as a more reliable basis for doctors to make more accurate diagnoses and treatment plans. Among them, the registration quality of the multimodal images to be registered is also an important consideration. The better the registration quality, the more accurate information can be obtained. However, since the multimodal images to be registered mostly come from images acquired at different times and / or under different conditions, the registration quality of the multimodal images to be registered varies. In the prior art, the registration quality of the multimodal images to be registered is often evaluated by a single evaluation index, such as correlation coefficients, but a single evaluation index is easily affected by factors such as noise and image quality, and the evaluation of image registration quality often has errors. Summary of the invention
[0003] In view of this, the purpose of the present invention is to provide an image registration quality assessment method, device, equipment and storage medium, which can more accurately assess the image registration quality through comprehensive assessment indicators by combining the registration quality assessment results output by the model and multiple similarity indicators corresponding to multimodal medical images. The specific scheme is as follows:
[0004] In a first aspect, the present application provides an image registration quality assessment method, comprising:
[0005] Acquire a current multimodal medical image to be registered for a target object;
[0006] Using a preset image registration quality assessment model to evaluate the current multimodal medical image to be registered, so as to obtain an initial registration quality assessment result;
[0007] Determining a comprehensive evaluation index based on the initial registration quality evaluation result and a plurality of similarity indexes corresponding to the current multimodal medical image to be registered;
[0008] A final registration quality evaluation result of the current multimodal medical image to be registered is determined based on the comprehensive evaluation index.
[0009] Optionally, the obtaining of the current multimodal medical image to be registered for the target object includes:
[0010] Acquire an ultrasound image and a computer tomography image to be currently registered for the target object;
[0011] Or, the ultrasound image and the magnetic resonance image to be currently registered of the target object are acquired.
[0012] Optionally, the obtaining of the current multimodal medical image to be registered for the target object includes:
[0013] Acquire an initial multimodal medical image to be registered for the target object;
[0014] The initial multimodal medical image to be registered is preprocessed to obtain the current multimodal medical image to be registered.
[0015] Optionally, preprocessing the initial multimodal medical image to be registered to obtain the current multimodal medical image to be registered includes:
[0016] The initial multimodal medical image to be registered is normalized, resized, and Gaussian filtered to obtain the current multimodal medical image to be registered of the same size.
[0017] Optionally, the normalizing the initial multimodal medical image to be registered includes:
[0018] The pixel values of each modality medical image in the initial multi-modality medical image to be registered are scaled to a preset pixel range.
[0019] Optionally, before evaluating the current multimodal medical image to be registered by using a preset image registration quality assessment model, the method further includes:
[0020] Collecting a plurality of unlabeled multimodal medical images to be registered, and performing data enhancement on the plurality of unlabeled multimodal medical images to obtain a plurality of enhanced multimodal medical images;
[0021] Determining a plurality of target multimodal medical images based on the plurality of enhanced multimodal medical images and the plurality of unlabeled multimodal medical images, and respectively labeling the plurality of target multimodal medical images for registration quality to construct a training set;
[0022] An initial image registration quality assessment model is constructed based on a neural network, and the initial image registration quality assessment model is trained using the training set to obtain the trained preset image registration quality assessment model.
[0023] Optionally, the determining a plurality of target multimodal medical images based on the plurality of enhanced multimodal medical images and the plurality of unlabeled multimodal medical images, and respectively performing registration quality labeling on the plurality of target multimodal medical images to construct a training set includes:
[0024] Preprocessing the plurality of enhanced multimodal medical images and the plurality of unlabeled multimodal medical images respectively to obtain the plurality of target multimodal medical images;
[0025] Annotating the registration transformation parameters and the image registration quality of each of the target multimodal medical images to obtain annotation information corresponding to each of the target multimodal medical images;
[0026] The training set is constructed based on each of the target multimodal medical images and the corresponding annotation information.
[0027] Optionally, the using the training set to train the initial image registration quality assessment model to obtain the trained preset image registration quality assessment model includes:
[0028] The initial image registration quality assessment model is trained using the training set and based on a cross entropy loss function and a stochastic gradient descent algorithm to optimize model parameters and obtain the trained preset image registration quality assessment model.
[0029] Optionally, the several similarity indicators include root mean square error, mutual information and structural similarity index;
[0030] Accordingly, the comprehensive evaluation index is determined based on the initial registration quality evaluation result and a plurality of similarity indicators corresponding to the multimodal medical image to be currently registered, including:
[0031] Determining the root mean square error, the mutual information and the structural similarity index corresponding to the current multimodal medical image to be registered;
[0032] The initial registration quality evaluation result, the root mean square error, the mutual information and the structural similarity index are linearly combined to obtain a corresponding comprehensive evaluation index.
[0033] Optionally, determining the final registration quality evaluation result of the current multimodal medical image to be registered based on the comprehensive evaluation index includes:
[0034] Comparing the comprehensive evaluation index with a preset evaluation threshold to obtain a comparison result;
[0035] A final registration quality assessment result of the current multimodal medical image to be registered is determined based on the comparison result.
[0036] In a second aspect, the present application provides an image registration quality assessment device, comprising:
[0037] An image acquisition module, used for acquiring a multimodal medical image to be registered for a target object;
[0038] An image evaluation module, used to evaluate the multimodal medical image to be currently registered using a preset image registration quality evaluation model to obtain an initial registration quality evaluation result;
[0039] A comprehensive index determination module, used to determine a comprehensive evaluation index based on the initial registration quality evaluation result and a plurality of similarity indicators corresponding to the multimodal medical image to be currently registered;
[0040] A registration quality determination module is used to determine a final registration quality evaluation result of the current multimodal medical image to be registered based on the comprehensive evaluation index.
[0041] In a third aspect, the present application provides an electronic device, including:
[0042] Memory, used to store computer programs;
[0043] A processor is used to execute the computer program to implement the aforementioned image registration quality assessment method.
[0044] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the aforementioned image registration quality assessment method when executed by a processor.
[0045] In the present application, a current multimodal medical image to be registered for a target object is obtained; the current multimodal medical image to be registered is evaluated using a preset image registration quality evaluation model to obtain an initial registration quality evaluation result; a comprehensive evaluation index is determined based on the initial registration quality evaluation result and several similarity indicators corresponding to the current multimodal medical image to be registered; and the final registration quality evaluation result of the current multimodal medical image to be registered is determined based on the comprehensive evaluation index. It can be seen that the present application performs registration quality evaluation on the current multimodal medical image to be registered through the image registration quality evaluation model, and can automatically determine the initial registration quality evaluation result of the current multimodal medical image to be registered, saving a lot of manpower and time costs, and improving the efficiency of image registration quality evaluation; and the present application combines the initial registration quality evaluation result output by the model, and several similarity indicators corresponding to the current multimodal medical image to be registered, so as to more accurately and comprehensively evaluate the image registration quality through the comprehensive evaluation index, thereby improving the reliability and accuracy of the image registration quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0047] Figure 1 A flow chart of an image registration quality assessment method disclosed in this application;
[0048] Figure 2 A training flow chart of an image registration quality assessment model disclosed in this application;
[0049] Figure 3 Construct a flow chart for a training set disclosed in the present application;
[0050] Figure 4 This is a schematic diagram of the structure of an image registration quality assessment device disclosed in this application;
[0051] Figure 5 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION
[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0053] In the prior art, the registration quality of multimodal images to be registered is often evaluated by a single evaluation index, but a single evaluation index is easily affected by factors such as noise and image quality, and the evaluation of image registration quality often has errors. Therefore, the present application provides an image registration quality evaluation method, which combines the registration quality evaluation results output by the model and multiple similarity indicators corresponding to multimodal medical images to more accurately evaluate the image registration quality through a comprehensive evaluation index.
[0054] See also Figure 1 As shown, an embodiment of the present invention discloses an image registration quality assessment method, comprising:
[0055] Step S11: Acquire the current multimodal medical image to be registered for the target object.
[0056] In this embodiment, for obtaining the current multimodal medical image to be registered for the target object, in one case, the ultrasound image and the electronic computed tomography image (CT, Computed Tomography) to be registered for the target object are obtained; in another case, the ultrasound image and the magnetic resonance image (MRI, Magnetic Resonance Imaging) to be registered for the target object are obtained. It should be noted that for the target object, it can be the same part of the same patient, such as the brain of the same patient; it can also be the same part of different patients, such as the brain of patient A and the brain of a typical normal person. For the multimodal medical images to be registered, they can be medical images collected at different times, different imaging devices, different angles, and different backgrounds, and the multimodal medical images to be registered include two or more modality medical images. It should be noted that, in the first case, corresponding single-modality medical images can be acquired by several imaging devices respectively to obtain multi-modality medical images, and then the multi-modality medical images can be transmitted to one of the above-mentioned imaging devices, and the imaging device has a preset image registration quality assessment model built in it for image registration quality assessment of the multi-modality medical images; wherein the imaging devices include ultrasound devices, CT devices, and MRI devices. It should be noted that the above-mentioned imaging devices can be multiple imaging devices of different types, for example, ultrasound devices acquire ultrasound images, CT devices acquire CT images, and MRI devices acquire MRI images; or they can be the same imaging device, for example, ultrasound devices acquire 2D ultrasound images and 3D ultrasound images. In the second case, corresponding single-modality medical images can be acquired by third-party imaging devices respectively, and then each modality medical image can be transmitted to another medical device with a built-in preset image registration quality assessment model, so that the medical device with a built-in preset image registration quality assessment model can obtain multi-modality medical images; wherein, the medical device can be an imaging device, such as an ultrasound diagnostic device.
[0057] Furthermore, for obtaining the current multimodal medical image to be registered for the target object, the initial multimodal medical image to be registered for the target object can be first obtained, and the initial multimodal medical image to be registered can be preprocessed to obtain the current multimodal medical image to be registered for the target object. It should be noted that the initial multimodal medical image to be registered can be preprocessed by a medical device with a built-in preset image registration quality assessment model; wherein the medical device includes an ultrasound device, etc.; the initial multimodal medical image to be registered can also be preprocessed by a third-party device, and then the processed current multimodal medical image to be registered is transmitted to the medical device with a built-in preset image registration quality assessment model. Among them, the third-party device can be other ultrasound equipment, such as ultrasound diagnostic equipment, ultrasound imaging workstation, etc.; it can also be an image processing tool, such as photo editing software, image editor, etc.
[0058] In this embodiment, the preprocessing includes normalization, resizing and Gaussian filtering. Specifically, the initial multimodal medical image to be registered is normalized, resized and Gaussian filtered to obtain the current multimodal medical image to be registered for the target object of the same size. It should be noted that there is no restriction on the order of normalization, resizing and Gaussian filtering. Taking the normalization, resizing and Gaussian filtering of the initial multimodal medical image to be registered as an example, the initial multimodal medical image to be registered is normalized to scale the pixel values of each modality medical image in the initial multimodal medical image to be registered to a preset pixel range, eliminate the brightness difference between each modality medical image, and obtain a normalized multimodal medical image; determine whether the size of the normalized multimodal medical image is consistent. If the size of the normalized multimodal medical image is inconsistent, crop or resize the normalized multimodal medical image to obtain a multimodal medical image of the same size, thereby ensuring the consistency of the multimodal medical image; further, Gaussian filtering is performed on the multimodal medical images of the same size to remove noise in the image to obtain the processed current multimodal medical image to be registered for the target object. In addition, if the sizes of the normalized multimodal medical images are consistent, Gaussian filtering is directly performed on the normalized multimodal medical images to obtain the processed multimodal medical images to be registered for the target object.
[0059] Step S12: using a preset image registration quality assessment model to assess the multimodal medical image to be currently registered, so as to obtain an initial registration quality assessment result.
[0060] Before using the preset image registration quality assessment model to evaluate the current multimodal medical image to be registered, it is necessary to first collect several unlabeled multimodal medical images to be registered as part of the training data, and use data enhancement to increase the diversity and robustness of the training data to obtain a richer training set. At the same time, each training data in the training set has a corresponding registration quality label, so that the model can be trained based on the training set to obtain a trained preset image registration quality assessment model. Further, the initial multimodal medical image to be registered for the target object is obtained, the initial multimodal medical image to be registered is preprocessed, and the processed current multimodal medical image to be registered for the target object is input into the trained preset image registration quality assessment model to perform image registration quality assessment, thereby outputting the corresponding initial registration quality assessment result to preliminarily characterize the registration quality of the current multimodal medical image to be registered. Among them, the initial registration quality assessment result can be characterized by the registration quality score. The higher the registration quality score, the better the image registration quality. Specifically, a preset image registration quality assessment model is used to perform image quality assessment on the current multimodal medical image to be registered, so as to output a registration quality score for characterizing the degree of registration quality.
[0061] It should be noted that, for the image registration quality assessment model, the image registration quality assessment model can be constructed and trained on a certain electronic computer, and then the trained image registration quality assessment model can be placed in a medical device (such as an ultrasound device) to perform registration quality assessment on the current multi-modal medical image to be registered through the medical device; or the image registration quality assessment model can be directly constructed and trained in the medical device, so as to directly use the internally trained image registration quality assessment model to perform registration quality assessment on the current multi-modal medical image to be registered. Among them, medical equipment includes but is not limited to ultrasound diagnostic equipment and ultrasound imaging workstations. Among them, the image registration quality assessment model can adopt a convolutional neural network model (CNN, Convolution Neural Networks), an artificial neural network model (ANN, Artificial Neural Networks), a recurrent neural network model (RNN, Recurrent Neural Networks), etc.
[0062] Step S13: determining a comprehensive evaluation index based on the initial registration quality evaluation result and a plurality of similarity indexes corresponding to the multimodal medical image to be currently registered.
[0063] In this embodiment, several similarity indicators include root mean square error, mutual information and structural similarity indicators. Specifically, by calculating the root mean square error, mutual information and structural similarity indicators corresponding to the processed current multimodal medical image to be registered; and linearly combining the root mean square error, mutual information, structural similarity indicators and the initial registration quality evaluation result output by the model, a comprehensive evaluation indicator corresponding to the current multimodal medical image to be registered is obtained. Specifically, corresponding weights can be set for the root mean square error, mutual information, structural similarity indicators and the initial registration quality evaluation result output by the model, so as to perform weighted calculation on the root mean square error, mutual information and structural similarity indicators corresponding to the processed current multimodal medical image to be registered, and the initial registration quality evaluation result output by the model, so as to obtain the corresponding comprehensive evaluation indicator.
[0064] Among them, for the root mean square error (RMSE) corresponding to the processed current multimodal medical image to be registered, since the sizes of the A-modal medical image and the B-modal medical image in the current multimodal medical image to be registered are consistent, the image length M and image width N corresponding to the A-modal medical image or the B-modal medical image are first determined, and the root mean square error corresponding to the current multimodal medical image to be registered is determined in combination with the pixel value of the A-modal medical image and the pixel value of the B-modal medical image. The formula involved is as follows:
[0065]
[0066] RMSE represents the root mean square error; M and N represent the image length and image width corresponding to any modality of the multimodal medical image to be registered, respectively; A(x, y) represents the pixel value at the coordinate (x, y) on the A modality medical image; B(x, y) represents the pixel value at the coordinate (x, y) on the B modality medical image.
[0067] Among them, for the mutual information (MI) corresponding to the processed current multimodal medical image to be registered, based on the entropies corresponding to the A modality medical image and the B modality medical image in the current multimodal medical image to be registered, and the joint entropy of the A modality medical image and the B modality medical image, the mutual information corresponding to the current multimodal medical image to be registered is determined, and the formula involved is as follows:
[0068] MI = H(A) + H(B) - H(A, B);
[0069] MI represents mutual information, H(A) represents the entropy of the A-modality medical image; H(B) represents the entropy of the B-modality medical image; H(A,B) represents the joint entropy of the A-modality medical image and the B-modality medical image.
[0070] Among them, for the structural similarity index (SSIM, Structural Similarity) corresponding to the processed current multimodal medical image to be registered, the structural similarity index corresponding to the current multimodal medical image to be registered is determined based on the mean value and standard deviation corresponding to the A modality medical image in the current multimodal medical image to be registered, the mean value and standard deviation corresponding to the B modality medical image in the current multimodal medical image to be registered, and the covariance of the A modality medical image and the B modality medical image. The formula involved is as follows:
[0071]
[0072] SSIM represents the structural similarity index, μ x represents the average value of A modality medical image, μ y represents the average value of B-mode medical images, σ x represents the standard deviation of the A modality medical image, σ y represents the standard deviation of B-modality medical images, σ xy represents the covariance of the A-modality medical image and the B-modality medical image, and c1 and c2 represent constants.
[0073] Step S14: determining a final registration quality evaluation result of the current multimodal medical image to be registered based on the comprehensive evaluation index.
[0074] In this embodiment, the comprehensive evaluation index is compared with the preset evaluation threshold to obtain a comparison result; based on the comparison result, the final registration quality evaluation result of the current multimodal medical image to be registered is determined. Specifically, the comprehensive evaluation index is compared with the preset evaluation threshold. If the comprehensive evaluation index is greater than the preset evaluation threshold, it indicates that the final registration quality evaluation result of the current multimodal medical image to be registered is good registration quality; if the comprehensive evaluation index is less than or equal to the preset evaluation threshold, it indicates that the final registration quality evaluation result of the current multimodal medical image to be registered is poor registration quality.
[0075] Among them, for the training of the preset image registration quality assessment model, such as Figure 2As shown, step S21: collect several unlabeled multimodal medical images to be registered, and perform data enhancement on the several unlabeled multimodal medical images to obtain several enhanced multimodal medical images; step S22: determine several target multimodal medical images based on the several enhanced multimodal medical images and the several unlabeled multimodal medical images, and perform registration quality annotation on the several target multimodal medical images respectively to construct a training set; step S23: construct an initial image registration quality assessment model based on a neural network, and train the initial image registration quality assessment model using the training set to obtain a trained preset image registration quality assessment model. It can be understood that several unlabeled multimodal medical images to be registered of different types and different difficulties are collected so that the model can learn the differences under different registration transformations during the training of the image registration quality assessment model; and use random translation, scaling, selection, flipping and other data enhancement operations to process several unlabeled multimodal medical images to generate more abundant enhanced multimodal medical images. Several target multimodal medical images determined based on several enhanced multimodal medical images and several unlabeled multimodal medical images are respectively labeled for registration quality to construct a training set; an initial image registration quality assessment model is constructed based on a neural network, and then the initial image registration quality assessment model is trained using the training set based on a cross entropy loss function and a stochastic gradient descent algorithm to optimize the model parameters and obtain a trained preset image registration quality assessment model.
[0076] It should be noted that the initial image registration quality assessment model is iteratively trained using the training set and based on the cross entropy loss function and the stochastic gradient descent algorithm to optimize the model parameters and obtain the trained image registration quality assessment model. As to whether the trained image registration quality assessment model can be determined as the trained preset image registration quality assessment model, the trained image registration quality assessment model can be determined as the trained preset image registration quality assessment model when the model iteration round reaches the preset iteration round; or the trained image registration quality assessment model can be determined as the trained preset image registration quality assessment model when the model registration quality assessment accuracy reaches the preset assessment accuracy threshold.
[0077] Furthermore, for the construction of a training set for a preset image registration quality assessment model, such as Figure 3As shown, step S31: pre-processing several enhanced multimodal medical images and several unlabeled multimodal medical images respectively to obtain several target multimodal medical images; step S32: labeling the registration transformation parameters and image registration quality of each target multimodal medical image respectively to obtain the labeling information corresponding to each target multimodal medical image; step S33: constructing a training set based on each target multimodal medical image and the corresponding labeling information. It can be understood that by normalizing, resizing and Gaussian filtering each enhanced multimodal medical image and each unlabeled multimodal medical image respectively, the pixel values of the obtained target multimodal medical images can be within the preset pixel range, and the sizes of the obtained target multimodal medical images can be consistent. In this way, by adopting normalization, the brightness difference between each modality of medical images in the multimodal medical image can be eliminated, which is helpful to improve the convergence and stability of the image registration quality assessment model training; by resizing, the consistency of the multimodal medical images in the input image registration quality assessment model can be ensured, and the complexity of the image registration quality assessment model performance can be reduced; by adopting Gaussian filtering, the noise in the image can be removed, and the influence of noise on the registration quality assessment of the image registration quality assessment model can be reduced.
[0078] Among them, by annotating the registration transformation parameters and image registration quality of each target multimodal medical image, a training set can be constructed based on each target multimodal medical image and the corresponding annotation information. Among them, the registration transformation parameters reflect that the A-modal medical image in the target multimodal medical image can achieve the highest similarity with the B-modal medical image in the target multimodal medical image after being transformed by the registration transformation parameters; the image registration quality reflects the binary classification problem, including good image registration quality and poor image registration quality. Specifically, the registration transformation parameters include image translation parameters, image scaling parameters, and image rotation parameters; for example, after the A-modal medical image is translated to the right by x units, rotated clockwise by y degrees, and scaled to half of the original registration transformation, it is found that the similarity between the A-modal medical image and the B-modal medical image reaches the highest, that is, the registration effect is the best.
[0079] Taking the ultrasound image and the computer tomography image to be registered as an example, the ultrasound image and the computer tomography image to be registered are normalized, resized and Gaussian filtered to scale the pixel values of the ultrasound image and the computer tomography image to be registered to a preset pixel range, thereby obtaining processed ultrasound images and computer tomography images of the same size. The processed ultrasound image and the computer tomography image are input into the trained preset image registration quality assessment model to perform image registration quality assessment, thereby outputting the corresponding initial registration quality assessment result, such as the registration quality score. The root mean square error, mutual information and structural similarity index corresponding to the processed ultrasound image and the computer tomography image are calculated, thereby linearly combining the root mean square error, mutual information, structural similarity index and the registration quality score to obtain the comprehensive evaluation index corresponding to the ultrasound image and the computer tomography image to be registered. The final registration quality assessment result of the ultrasound image and the computer tomography image to be registered is determined based on the comparison result of the comprehensive evaluation index and the preset evaluation threshold.
[0080] It can be seen that the present application can reduce the noise in the multimodal medical images by preprocessing the multimodal medical images to be registered; and by performing registration quality assessment on the preprocessed multimodal medical images through the image registration quality assessment model, the initial registration quality assessment results of the multimodal medical images to be registered can be automatically determined, thereby saving a lot of manpower and time costs and improving the efficiency of image registration quality assessment; in addition, the present application combines the initial registration quality assessment results output by the model, as well as the root mean square error, mutual information and structural similarity indicators corresponding to the preprocessed multimodal medical images, to more accurately and comprehensively evaluate the image registration quality through comprehensive evaluation indicators, thereby improving the reliability and accuracy of the image registration quality assessment.
[0081] See also Figure 4 As shown, an embodiment of the present invention discloses an image registration quality assessment device, comprising:
[0082] An image acquisition module 11 is used to acquire a multimodal medical image to be registered for a target object;
[0083] An image evaluation module 12 is used to evaluate the multimodal medical image to be currently registered using a preset image registration quality evaluation model to obtain an initial registration quality evaluation result;
[0084] A comprehensive index determination module 13, configured to determine a comprehensive evaluation index based on the initial registration quality evaluation result and a plurality of similarity indexes corresponding to the multimodal medical image to be currently registered;
[0085] The registration quality determination module 14 is used to determine the final registration quality evaluation result of the multimodal medical image to be registered based on the comprehensive evaluation index.
[0086] It can be seen that the present application performs registration quality assessment on the current multimodal medical image to be registered through the image registration quality assessment model, and can automatically determine the initial registration quality assessment result of the current multimodal medical image to be registered, saving a lot of manpower and time costs, and improving the efficiency of image registration quality assessment; and, the present application combines the initial registration quality assessment result output by the model and several similarity indicators corresponding to the current multimodal medical image to be registered, so as to more accurately and comprehensively evaluate the image registration quality through comprehensive evaluation indicators, thereby improving the reliability and accuracy of image registration quality assessment.
[0087] In some specific embodiments, the image acquisition module 11 includes:
[0088] A first image acquisition unit, configured to acquire an ultrasound image and a computer tomography image to be currently registered for the target object;
[0089] The second image acquisition unit is used to acquire the ultrasound image and the magnetic resonance image currently to be registered for the target object.
[0090] In some specific embodiments, the image acquisition module 11 includes:
[0091] An initial image acquisition unit, used to acquire an initial multimodal medical image to be registered for the target object;
[0092] The image processing submodule is used to preprocess the initial multimodal medical image to be registered to obtain the current multimodal medical image to be registered.
[0093] In some specific embodiments, the image processing submodule includes:
[0094] The image processing unit is used to normalize, resize and Gaussian filter the initial multimodal medical image to be registered to obtain the current multimodal medical image to be registered of the same size.
[0095] In some specific embodiments, the image processing unit is specifically configured to scale the pixel values of each modality medical image in the initial multi-modality medical image to be registered to a preset pixel range.
[0096] In some specific embodiments, the image registration quality assessment device further includes:
[0097] A data enhancement unit, used for collecting a plurality of unlabeled multimodal medical images to be registered, and performing data enhancement on the plurality of unlabeled multimodal medical images to obtain a plurality of enhanced multimodal medical images;
[0098] A training set construction module, used to determine a number of target multimodal medical images based on the number of enhanced multimodal medical images and the number of unlabeled multimodal medical images, and to perform registration quality annotation on the number of target multimodal medical images respectively, so as to construct a training set;
[0099] The model training module is used to construct an initial image registration quality assessment model based on a neural network, and use the training set to train the initial image registration quality assessment model to obtain the trained preset image registration quality assessment model.
[0100] In some specific embodiments, the training set construction module includes:
[0101] a target image acquisition unit, configured to preprocess the plurality of enhanced multimodal medical images and the plurality of unlabeled multimodal medical images respectively to obtain the plurality of target multimodal medical images;
[0102] An image annotation unit, used to annotate the registration transformation parameters and image registration quality of each of the target multimodal medical images, so as to obtain annotation information corresponding to each of the target multimodal medical images;
[0103] A training set construction unit is used to construct the training set based on each of the target multimodal medical images and the corresponding annotation information.
[0104] In some specific embodiments, the model training module includes:
[0105] A model training unit is used to train the initial image registration quality assessment model using the training set and based on a cross entropy loss function and a stochastic gradient descent algorithm to optimize model parameters and obtain the trained preset image registration quality assessment model.
[0106] In some specific embodiments, the several similarity indicators include root mean square error, mutual information and structural similarity index;
[0107] Accordingly, the comprehensive indicator determination module 13 includes:
[0108] An index determination unit, used to determine the root mean square error, the mutual information and the structural similarity index corresponding to the current multimodal medical image to be registered;
[0109] The linear combination unit is used to linearly combine the initial registration quality evaluation result, the root mean square error, the mutual information and the structural similarity index to obtain a corresponding comprehensive evaluation index.
[0110] In some specific embodiments, the registration quality determination module 14 includes:
[0111] A comparison result determination unit, used to compare the comprehensive evaluation index with a preset evaluation threshold to obtain a comparison result;
[0112] A final result determination unit is used to determine a final registration quality assessment result of the current multimodal medical image to be registered based on the comparison result.
[0113] Furthermore, the present application also discloses an electronic device. Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be considered as any limitation on the scope of use of the present application.
[0114] Figure 5 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the image registration quality assessment method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer, a medical diagnostic device, and a medical imaging workstation, wherein the medical diagnostic device includes an ultrasonic diagnostic device, etc., and the medical imaging workstation includes an ultrasonic imaging workstation, etc.
[0115] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.
[0116] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0117] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the image registration quality assessment method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.
[0118] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned disclosed image registration quality assessment method. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, and no further description will be given here.
[0119] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0120] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0121] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0122] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0123] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A method for evaluating image registration quality, characterized in that: include: Acquire a current multimodal medical image to be registered for a target object; Using a preset image registration quality assessment model to evaluate the current multimodal medical image to be registered, so as to obtain an initial registration quality assessment result; Determining a comprehensive evaluation index based on the initial registration quality evaluation result and a plurality of similarity indexes corresponding to the current multimodal medical image to be registered; A final registration quality evaluation result of the current multimodal medical image to be registered is determined based on the comprehensive evaluation index.
2. The image registration quality assessment method according to claim 1, characterized in that: The step of obtaining a current multimodal medical image to be registered for the target object comprises: Acquire an ultrasound image and a computer tomography image to be currently registered for the target object; Or, the ultrasound image and the magnetic resonance image to be currently registered of the target object are acquired.
3. The image registration quality assessment method according to claim 1, characterized in that: The step of obtaining a current multimodal medical image to be registered for the target object comprises: Acquire an initial multimodal medical image to be registered for the target object; The initial multimodal medical image to be registered is preprocessed to obtain the current multimodal medical image to be registered.
4. The image registration quality assessment method according to claim 3, characterized in that: The preprocessing of the initial multimodal medical image to be registered to obtain the current multimodal medical image to be registered includes: The initial multimodal medical image to be registered is normalized, resized, and Gaussian filtered to obtain the current multimodal medical image to be registered of the same size.
5. The image registration quality assessment method according to claim 4, characterized in that: The normalizing the initial multimodal medical image to be registered includes: The pixel values of each modality medical image in the initial multi-modality medical image to be registered are scaled to a preset pixel range.
6. The image registration quality assessment method according to claim 1, characterized in that: Before evaluating the current multimodal medical image to be registered by using a preset image registration quality assessment model, the method further includes: Collecting a plurality of unlabeled multimodal medical images to be registered, and performing data enhancement on the plurality of unlabeled multimodal medical images to obtain a plurality of enhanced multimodal medical images; Determining a plurality of target multimodal medical images based on the plurality of enhanced multimodal medical images and the plurality of unlabeled multimodal medical images, and respectively labeling the plurality of target multimodal medical images for registration quality to construct a training set; An initial image registration quality assessment model is constructed based on a neural network, and the initial image registration quality assessment model is trained using the training set to obtain the trained preset image registration quality assessment model.
7. The image registration quality assessment method according to claim 6, characterized in that: The determining of a plurality of target multimodal medical images based on the plurality of enhanced multimodal medical images and the plurality of unlabeled multimodal medical images, and respectively labeling the plurality of target multimodal medical images with registration quality to construct a training set includes: Preprocessing the plurality of enhanced multimodal medical images and the plurality of unlabeled multimodal medical images respectively to obtain the plurality of target multimodal medical images; Annotating the registration transformation parameters and the image registration quality of each of the target multimodal medical images to obtain annotation information corresponding to each of the target multimodal medical images; The training set is constructed based on each of the target multimodal medical images and the corresponding annotation information.
8. The image registration quality assessment method according to claim 6, characterized in that: The using the training set to train the initial image registration quality assessment model to obtain the trained preset image registration quality assessment model includes: The initial image registration quality assessment model is trained using the training set and based on a cross entropy loss function and a stochastic gradient descent algorithm to optimize model parameters and obtain the trained preset image registration quality assessment model.
9. The image registration quality assessment method according to claim 1, characterized in that: The several similarity indicators include root mean square error, mutual information and structural similarity index; Accordingly, the comprehensive evaluation index is determined based on the initial registration quality evaluation result and a plurality of similarity indicators corresponding to the multimodal medical image to be currently registered, including: Determining the root mean square error, the mutual information and the structural similarity index corresponding to the current multimodal medical image to be registered; The initial registration quality evaluation result, the root mean square error, the mutual information and the structural similarity index are linearly combined to obtain a corresponding comprehensive evaluation index.
10. The image registration quality assessment method according to any one of claims 1 to 9, characterized in that: The determining, based on the comprehensive evaluation index, a final registration quality evaluation result of the current multimodal medical image to be registered comprises: Comparing the comprehensive evaluation index with a preset evaluation threshold to obtain a comparison result; A final registration quality assessment result of the current multimodal medical image to be registered is determined based on the comparison result.
11. An image registration quality assessment device, characterized in that: include: An image acquisition module, used for acquiring a multimodal medical image to be registered for a target object; An image evaluation module, used to evaluate the multimodal medical image to be currently registered using a preset image registration quality evaluation model to obtain an initial registration quality evaluation result; A comprehensive index determination module, used to determine a comprehensive evaluation index based on the initial registration quality evaluation result and a plurality of similarity indicators corresponding to the multimodal medical image to be currently registered; A registration quality determination module is used to determine a final registration quality evaluation result of the current multimodal medical image to be registered based on the comprehensive evaluation index.
12. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the image registration quality assessment method according to any one of claims 1 to 10.
13. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the image registration quality assessment method according to any one of claims 1 to 10.