Image registration quality assessment model training method, device and computer equipment
By calculating the criticality error information in the image registration quality evaluation model and updating the network parameters, the problem of low accuracy in the image registration quality evaluation is solved and the accuracy of the evaluation model is improved.
Patent Information
- Application Number
- CN202011308476.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-20
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2040-11-20
AI Technical Summary
In the prior art, the evaluation results of image registration quality are relatively low, and it is difficult to effectively evaluate the quality of image registration.
By obtaining training image samples, including training reference images and corresponding target reference keyity information, as well as training registration images and corresponding target registration keyity information, the keyity error information is calculated, and the initial keyity information extraction network and image matching are updated. Quasi-quality evaluation network to improve the accuracy of evaluation models.
It improves the accuracy of image registration quality evaluation, enhances the performance of the trained image registration quality evaluation model, and can more accurately evaluate image registration quality.
Smart Images

Figure CN114519729B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to an image registration quality assessment model training, an image registration quality assessment method, an apparatus, a computer device and a storage medium. Background Art
[0002] With the development of artificial intelligence technology, image registration technology has emerged. Image registration is the process of matching and superimposing two or more images acquired at different times, with different sensors (imaging equipment) or under different conditions (weather, illumination, camera position and angle, etc.). It has been widely used in remote sensing data analysis, computer vision, image processing and other fields. As image registration is widely used, the evaluation of image registration quality is becoming more and more important. At present, the evaluation of image registration quality is usually carried out using evaluation indicators. However, the evaluation of image registration quality using evaluation indicators has the problem of low accuracy of evaluation results. Summary of the invention
[0003] Based on this, it is necessary to provide an image registration quality assessment model training, image registration quality assessment method, device, computer equipment and storage medium that can improve the accuracy of image registration quality assessment in response to the above technical problems.
[0004] A method for training an image registration quality assessment model, the method comprising:
[0005] Acquire training image samples and target image registration quality assessment results, the training image samples include a training reference image and corresponding target reference criticality information and a training registration image and corresponding target registration criticality information, and the training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image;
[0006] Inputting the training reference image and the training registration image into the initial critical information extraction network in the initial image registration quality assessment model to obtain the training reference critical information and the training registration critical information;
[0007] Calculating reference criticality error information between training reference criticality information and corresponding target reference criticality information, and calculating registration criticality error information between training registration criticality information and corresponding target registration criticality information;
[0008] Determining target criticality error information based on the reference criticality error information and the registration criticality error information, and updating an initial criticality information extraction network based on the target criticality error information;
[0009] Inputting the training reference image, the training registration image, the training reference criticality information and the training registration criticality information into the initial image registration quality assessment network in the initial image registration quality assessment model to obtain the initial image registration quality assessment result;
[0010] Calculate the error information between the initial image registration quality evaluation result and the target image registration quality evaluation result;
[0011] The initial image registration quality assessment network is updated based on the error information of the assessment results. When the training is completed, an image registration quality assessment model is obtained.
[0012] An image registration quality assessment model training device, the device comprising:
[0013] A training sample acquisition module is used to obtain training image samples and target image registration quality assessment results. The training image samples include a training reference image and corresponding target reference criticality information, and a training registration image and corresponding target registration criticality information. The training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image.
[0014] A training information extraction module is used to input the training reference image and the training registration image into the initial critical information extraction network in the initial image registration quality assessment model to obtain the training reference critical information and the training registration critical information;
[0015] A criticality error calculation module, used to calculate reference criticality error information between training reference criticality information and corresponding target reference criticality information, and calculate registration criticality error information between training registration criticality information and corresponding target registration criticality information;
[0016] An extraction network updating module, used to determine target criticality error information based on reference criticality error information and registration criticality error information, and to update an initial criticality information extraction network based on the target criticality error information;
[0017] An initial quality assessment module is used to input the training reference image, the training registration image, the training reference criticality information and the training registration criticality information into the initial image registration quality assessment network in the initial image registration quality assessment model to obtain an initial image registration quality assessment result;
[0018] An evaluation error calculation module is used to calculate evaluation result error information between an initial image registration quality evaluation result and a target image registration quality evaluation result;
[0019] The model acquisition module is used to update the initial image registration quality assessment network based on the error information of the evaluation result. When the training is completed, the image registration quality assessment model is obtained.
[0020] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0021] Acquire training image samples and target image registration quality assessment results, the training image samples include a training reference image and corresponding target reference criticality information and a training registration image and corresponding target registration criticality information, and the training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image;
[0022] Inputting the training reference image and the training registration image into the initial critical information extraction network in the initial image registration quality assessment model to obtain the training reference critical information and the training registration critical information;
[0023] Calculating reference criticality error information between training reference criticality information and corresponding target reference criticality information, and calculating registration criticality error information between training registration criticality information and corresponding target registration criticality information;
[0024] Determining target criticality error information based on the reference criticality error information and the registration criticality error information, and updating an initial criticality information extraction network based on the target criticality error information;
[0025] Inputting the training reference image, the training registration image, the training reference criticality information and the training registration criticality information into the initial image registration quality assessment network in the initial image registration quality assessment model to obtain the initial image registration quality assessment result;
[0026] Calculate the error information between the initial image registration quality evaluation result and the target image registration quality evaluation result;
[0027] The initial image registration quality assessment network is updated based on the error information of the assessment results. When the training is completed, an image registration quality assessment model is obtained.
[0028] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:
[0029] Acquire training image samples and target image registration quality assessment results, the training image samples include a training reference image and corresponding target reference criticality information and a training registration image and corresponding target registration criticality information, and the training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image;
[0030] Inputting the training reference image and the training registration image into the initial critical information extraction network in the initial image registration quality assessment model to obtain the training reference critical information and the training registration critical information;
[0031] Calculating reference criticality error information between training reference criticality information and corresponding target reference criticality information, and calculating registration criticality error information between training registration criticality information and corresponding target registration criticality information;
[0032] Determining target criticality error information based on the reference criticality error information and the registration criticality error information, and updating an initial criticality information extraction network based on the target criticality error information;
[0033] Inputting the training reference image, the training registration image, the training reference criticality information and the training registration criticality information into the initial image registration quality assessment network in the initial image registration quality assessment model to obtain the initial image registration quality assessment result;
[0034] Calculate the error information between the initial image registration quality evaluation result and the target image registration quality evaluation result;
[0035] The initial image registration quality assessment network is updated based on the error information of the assessment results. When the training is completed, an image registration quality assessment model is obtained.
[0036] The above-mentioned image registration quality assessment model training method, device, computer equipment and storage medium obtain training reference critical information and training registration critical information by inputting training reference image and training registration image into the initial critical information extraction network in the initial image registration quality assessment model; then calculating the target critical error information, using the target critical error information to update the initial critical information extraction network, and simultaneously inputting the training reference image, training registration image, training reference critical information and training registration critical information into the initial image registration quality assessment network in the initial image registration quality assessment model to obtain the initial image registration quality assessment result. Then the evaluation result error information is calculated, and the initial image registration quality assessment network is updated using the evaluation result error information. When the training is completed, the image registration quality assessment model is obtained, that is, the training reference critical information and training registration critical information obtained from the initial critical information extraction network are used as the input of the initial image registration quality assessment network for training, so as to improve the accuracy of the trained image registration quality assessment network, thereby improving the accuracy of the trained image registration quality assessment model.
[0037] A method for evaluating image registration quality, the method comprising:
[0038] Acquire a reference image and a registration image, wherein the registration image is an image obtained by performing image registration on the image to be registered based on the reference image;
[0039] Extract the key points corresponding to the reference image and the registered image respectively to obtain the reference key points and the registered key points;
[0040] Calculating Gaussian distribution information corresponding to the reference key point, and determining reference criticality information corresponding to the reference image based on the Gaussian distribution information corresponding to the reference key point and the position of the pixel point in the reference image;
[0041] Calculating Gaussian distribution information corresponding to the registration key points, and determining registration criticality information corresponding to the registration image based on the Gaussian distribution information corresponding to the registration key points and the positions of the pixels in the registration image;
[0042] Image registration quality assessment is performed based on the reference image, the registered image, the reference criticality information and the registration criticality information to obtain image registration quality assessment results corresponding to the reference image and the registered image.
[0043] In one embodiment, before acquiring the reference image and the registration image, the method further includes:
[0044] A reference image and an image to be registered are obtained, and the reference image and the image to be registered are input into an image registration model to obtain an output registered image. The image registration model is trained based on a deep neural network algorithm using training samples.
[0045] In one embodiment, calculating Gaussian distribution information corresponding to the reference key point, and determining reference criticality information corresponding to the reference image based on the Gaussian distribution information corresponding to the reference key point and the position of the pixel point in the reference image, includes:
[0046] The position coordinates of the reference key points are taken as the mean, and a two-dimensional Gaussian distribution calculation is performed based on the mean and the preset covariance matrix to obtain the Gaussian distribution information corresponding to the reference key points;
[0047] Calculate the reference key degree corresponding to the pixel point in the reference image based on the Gaussian distribution information corresponding to the reference key point and the position coordinates of the pixel point in the reference image;
[0048] The reference key information corresponding to the reference image is obtained based on the reference key corresponding to the pixel point in the reference image.
[0049] In one embodiment, calculating Gaussian distribution information corresponding to the registration key points, and determining registration criticality information corresponding to the registration image based on the Gaussian distribution information corresponding to the registration key points and the positions of pixels in the registration image, includes:
[0050] The position coordinates of the key points of registration are taken as the mean, and two-dimensional Gaussian distribution calculation is performed according to the mean and the preset covariance matrix to obtain Gaussian distribution information corresponding to the key points of registration;
[0051] The registration criticality corresponding to the pixel point in the registration image is calculated based on the Gaussian distribution information corresponding to the registration key point and the position coordinates of the pixel point in the registration image;
[0052] The registration criticality information corresponding to the registered image is obtained based on the registration criticality corresponding to the pixel points in the registered image.
[0053] An image registration quality assessment device, the device comprising:
[0054] An image acquisition module is used to acquire a reference image and a registration image, wherein the registration image is an image obtained by performing image registration on the image to be registered based on the reference image;
[0055] A key point extraction module is used to extract key points corresponding to the reference image and the registration image respectively to obtain reference key points and registration key points;
[0056] A reference information determination module, used to calculate Gaussian distribution information corresponding to the reference key point, and determine reference critical information corresponding to the reference image based on the Gaussian distribution information corresponding to the reference key point and the position of the pixel point in the reference image;
[0057] A registration information determination module, used to calculate Gaussian distribution information corresponding to the registration key points, and determine the registration criticality information corresponding to the registration image based on the Gaussian distribution information corresponding to the registration key points and the positions of the pixels in the registration image;
[0058] The quality assessment module is used to perform image registration quality assessment based on the reference image, the registered image, the reference criticality information and the registration criticality information to obtain the image registration quality assessment results corresponding to the reference image and the registered image.
[0059] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0060] Acquire a reference image and a registration image, wherein the registration image is an image obtained by performing image registration on the image to be registered based on the reference image;
[0061] Extract the key points corresponding to the reference image and the registered image respectively to obtain the reference key points and the registered key points;
[0062] Calculating Gaussian distribution information corresponding to the reference key point, and determining reference criticality information corresponding to the reference image based on the Gaussian distribution information corresponding to the reference key point and the position of the pixel point in the reference image;
[0063] Calculating Gaussian distribution information corresponding to the registration key points, and determining registration criticality information corresponding to the registration image based on the Gaussian distribution information corresponding to the registration key points and the positions of the pixels in the registration image;
[0064] Image registration quality assessment is performed based on the reference image, the registered image, the reference criticality information and the registration criticality information to obtain image registration quality assessment results corresponding to the reference image and the registered image.
[0065] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:
[0066] Acquire a reference image and a registration image, wherein the registration image is an image obtained by performing image registration on the image to be registered based on the reference image;
[0067] Extract the key points corresponding to the reference image and the registered image respectively to obtain the reference key points and the registered key points;
[0068] Calculating Gaussian distribution information corresponding to the reference key point, and determining reference criticality information corresponding to the reference image based on the Gaussian distribution information corresponding to the reference key point and the position of the pixel point in the reference image;
[0069] Calculating Gaussian distribution information corresponding to the registration key points, and determining registration criticality information corresponding to the registration image based on the Gaussian distribution information corresponding to the registration key points and the positions of the pixels in the registration image;
[0070] Image registration quality assessment is performed based on the reference image, the registered image, the reference criticality information and the registration criticality information to obtain image registration quality assessment results corresponding to the reference image and the registered image.
[0071] The above-mentioned image registration quality assessment method, device, computer equipment and storage medium extract reference key points and registration key points corresponding to the reference image and the registered image, then calculate the Gaussian distribution information corresponding to the reference key points to determine the reference criticality information corresponding to the reference image, and calculate the Gaussian distribution information corresponding to the registration key points to determine the registration criticality information corresponding to the registered image, and finally use the reference image, the registered image, the reference criticality information and the registration criticality information to perform image registration quality assessment, and obtain the image registration quality assessment results corresponding to the reference image and the registered image. That is, by using not only the reference image and the registered image, but also the reference criticality information and the registration criticality information when performing image registration quality assessment, the accuracy of the image registration quality assessment results is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0072] Figure 1 A diagram of an application environment of an image registration quality assessment method in one embodiment;
[0073] Figure 2 A schematic diagram of a flow chart of a method for training an image registration quality assessment model in one embodiment;
[0074] Figure 3 A schematic diagram of a process of pre-training a criticality information extraction network in one embodiment;
[0075] Figure 4 A schematic diagram of a process of obtaining a tag in one embodiment;
[0076] Figure 5 A schematic diagram of a process for obtaining target reference criticality information in one embodiment;
[0077] Figure 6 A schematic diagram of a process for obtaining target registration criticality information in one embodiment;
[0078] Figure 7 A schematic diagram of a flow chart of a method for training an image registration quality assessment model in a specific embodiment;
[0079] Figure 8 A schematic diagram of a flow chart of an image registration quality assessment method in one embodiment;
[0080] Fig. 9 A schematic diagram of a flow chart of model evaluation in one embodiment;
[0081] Fig.10 A schematic diagram of a process for obtaining reference criticality information in one embodiment;
[0082] Fig.11 A schematic diagram of a process for obtaining registration criticality information in one embodiment;
[0083] Fig.12 is a schematic diagram of a key point image in a specific embodiment;
[0084] Fig.13 A schematic diagram of a training framework of an image registration quality assessment model training method in one embodiment;
[0085] Fig.14 A diagram showing an application scenario of an image registration quality assessment method in one embodiment;
[0086] Fig.15 It is a structural block diagram of an image registration quality assessment model training device in one embodiment;
[0087] Fig.16 is a structural block diagram of an image registration quality assessment device in one embodiment;
[0088] Fig.17FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0089] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0090] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing so that computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.
[0091] The solution provided in the embodiments of the present application involves technologies such as artificial intelligence computer vision technology, which is specifically described by the following embodiments:
[0092] The image registration quality assessment model training method provided in this application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The server 104 receives the model training instruction sent by the terminal 102, and obtains the training image sample and the target image registration quality evaluation result according to the model training instruction. The training image sample includes the training reference image and the corresponding target reference critical information and the training registration image and the corresponding target registration critical information. The training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image; the server 104 inputs the training reference image and the training registration image into the initial critical information extraction network in the initial image registration quality evaluation model to obtain the training reference critical information and the training registration critical information; the server 104 calculates the reference critical error information between the training reference critical information and the corresponding target reference critical information, and calculates the training registration critical information. The server 104 determines the target key error information based on the reference key error information and the registration key error information, and updates the initial key information extraction network based on the target key error information; the server 104 inputs the training reference image, the training registration image, the training reference key information and the training registration key information into the initial image registration quality assessment network in the initial image registration quality assessment model to obtain the initial image registration quality assessment result; calculates the evaluation result error information between the initial image registration quality assessment result and the target image registration quality assessment result; the server 104 updates the initial image registration quality assessment network based on the evaluation result error information, and when the training is completed, the image registration quality assessment model is obtained. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.
[0093] In one embodiment, Figure 2 As shown in FIG, a method for training an image registration quality assessment model is provided, and the method is applied to Figure 1 The server in the example is used for explanation. It can be understood that the method can also be applied to the terminal. In this embodiment, the method includes the following steps:
[0094] Step 202, obtaining training image samples and target image registration quality assessment results, the training image samples include a training reference image and corresponding target reference criticality information and a training registration image and corresponding target registration criticality information, the training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image.
[0095] Among them, the training image sample refers to the image sample used to train the image registration quality assessment model. The target image registration quality assessment result refers to the image registration quality assessment result between the training reference image and the training registration image in the training image. The training reference image refers to the reference image when training the image registration quality assessment model. The training registration image refers to the registration image corresponding to the training reference image when training the image registration quality assessment model. The target reference criticality information is used to indicate the criticality of the training reference image to the target image registration quality assessment result. The target registration criticality information is used to indicate the criticality of the training registration image to the target image registration quality assessment result.
[0096] Specifically, the server can obtain training image samples and corresponding target image registration quality assessment results from the database. The server can also collect training image samples from the Internet and obtain target image registration quality assessment results. In one embodiment, the server can also collect training reference images and training registration images, and then extract key points in the training reference images and the training registration images, and calculate the target reference criticality information, target registration criticality information and target image registration quality assessment results based on the key points in the training reference images and the training registration images.
[0097] Step 204: input the training reference image and the training registration image into the initial criticality information extraction network in the initial image registration quality assessment model to obtain the training reference criticality information and the training registration criticality information.
[0098] The initial critical information extraction network refers to the critical information extraction network initialized with neural network parameters, the training reference critical information refers to the critical information corresponding to the training reference image extracted using the initial critical information extraction network, and the training registration critical information refers to the critical information corresponding to the training registration image extracted using the initial critical information extraction network.
[0099] Specifically, the server pre-sets the initial image registration quality assessment model to be trained, that is, initializes the model parameters in the image registration quality assessment model. Then, the training reference image and the training registration image are input into the initial critical information extraction network in the initial image registration quality assessment model to obtain the training reference critical information and the training registration critical information output by the initial critical information extraction network.
[0100] Step 206 , calculating reference criticality error information between the training reference criticality information and the corresponding target reference criticality information, and calculating registration criticality error information between the training registration criticality information and the corresponding target registration criticality information.
[0101] The reference criticality error information is used to indicate the error between the reference criticality information, and the registration criticality error information is used to indicate the error between the registration criticality information.
[0102] Specifically, the server can use a pre-set loss function to calculate the reference criticality error information between the training reference criticality information and the corresponding target reference criticality information, and calculate the registration criticality error information between the training registration criticality information and the corresponding target registration criticality information. Among them, the loss function can be an average error loss function, a mean absolute value error, a smoothed mean absolute error, a logarithmic hyperbolic cosine loss function, a quantile loss, and the like.
[0103] Step 208: determining target criticality error information based on the reference criticality error information and the registration criticality error information, and updating the initial criticality information extraction network based on the target criticality error information.
[0104] The target criticality error information is used to represent the criticality error information of the training sample images as a whole.
[0105] Specifically, the server calculates the sum of the reference critical error information and the registration critical error information to obtain the target critical error information, and then uses a gradient descent algorithm to update the network parameters in the initial critical information extraction network based on the target critical error information.
[0106] Step 210: input the training reference image, the training registration image, the training reference criticality information and the training registration criticality information into the initial image registration quality assessment network in the initial image registration quality assessment model to obtain the initial image registration quality assessment result.
[0107] The initial image registration quality assessment result refers to the image registration quality assessment result obtained using the initial image registration quality assessment network.
[0108] Specifically, the server simultaneously inputs the training reference image and the training registration image, as well as the training reference critical information and the training registration critical information output by the initial critical information extraction network into the initial image registration quality assessment network in the initial image registration quality assessment model to obtain the initial image registration quality assessment result.
[0109] Step 212: Calculate the error information between the initial image registration quality evaluation result and the target image registration quality evaluation result.
[0110] Step 214, updating the initial image registration quality assessment network based on the assessment result error information, and when the training is completed, obtaining the image registration quality assessment model.
[0111] The evaluation result error information is used for the error of the output result of the initial image registration quality evaluation model.
[0112] Specifically, the server calculates the difference between the initial image registration quality assessment result and the target image registration quality assessment result to obtain the assessment result error information, and then uses the gradient descent algorithm to update the network parameters in the initial image registration quality assessment network based on the assessment result error information. When the training completion condition is not met, the updated critical information extraction network is used as the initial critical information extraction network, and the updated image registration quality assessment network is used as the initial image registration quality assessment network, and then returns to step S204 to continue iterative execution. When the training completion condition is met, the last updated critical information extraction network and image registration quality assessment network are used as the trained image registration quality assessment model. Among them, the training completion condition is pre-set, including at least one of the training reaching the maximum number of iterations, the assessment result error information is less than a pre-set threshold, and the model parameters no longer change.
[0113] In the above-mentioned image registration quality assessment model training, the training reference critical information and the training registration critical information are obtained by inputting the training reference image and the training registration image into the initial critical information extraction network in the initial image registration quality assessment model; then the target critical error information is calculated, and the initial critical information extraction network is updated using the target critical error information, and the training reference image, the training registration image, the training reference critical information and the training registration critical information are input into the initial image registration quality assessment network in the initial image registration quality assessment model to obtain the initial image registration quality assessment result. Then the evaluation result error information is calculated, and the initial image registration quality assessment network is updated using the evaluation result error information. When the training is completed, the image registration quality assessment model is obtained, that is, the training reference critical information and the training registration critical information obtained from the initial critical information extraction network are used as the input of the initial image registration quality assessment network for training, so as to improve the accuracy of the trained image registration quality assessment network, thereby improving the accuracy of the trained image registration quality assessment model.
[0114] In one embodiment, Figure 3As shown, before step 202, before obtaining the training image sample and the target image registration quality evaluation result, the training image sample includes the training reference image and the corresponding target reference critical information and the training registration image and the corresponding target registration critical information, and the training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image, it also includes:
[0115] Step 302: obtaining pre-training image samples, where the pre-training image samples include pre-training reference images and corresponding target pre-training reference criticality information, and pre-training registration images and corresponding target pre-training registration criticality information.
[0116] Among them, the pre-trained image samples refer to the image samples used when pre-training the critical information extraction network. The pre-trained reference images refer to the reference images used when pre-training the critical information extraction network. The pre-trained registration images refer to the registration images used when pre-training the critical information extraction network. The target pre-trained reference critical information refers to the target reference critical information used when pre-training the critical information extraction network. The target pre-trained registration critical information refers to the target registration critical information used when pre-training the critical information extraction network.
[0117] Specifically, the server may collect pre-trained image samples from the Internet. The server may also directly obtain pre-trained image samples from a database. The server may also collect pre-trained reference images and pre-trained registration images from the Internet, and then extract key points from the pre-trained reference images and pre-trained registration images, and calculate target pre-trained reference criticality information and target pre-trained registration criticality information based on the key points.
[0118] Step 304: input the pre-trained reference image and the pre-trained registration image into a pre-trained criticality information extraction network to obtain pre-trained reference criticality information and pre-trained registration criticality information.
[0119] Among them, the pre-trained critical information extraction network refers to the critical information extraction network that needs to be trained in advance. The pre-trained reference critical information refers to the reference critical information corresponding to the pre-trained reference image obtained by using the pre-trained critical information extraction network. The pre-trained registration critical information refers to the registration critical information corresponding to the pre-trained registration image obtained by using the pre-trained critical information extraction network.
[0120] Specifically, the server obtains a critical information extraction network that needs to be pre-trained, and the network parameters in the pre-trained critical information extraction network are initialized. Then, the pre-trained reference image and the pre-trained registration image are simultaneously input into the pre-trained critical information extraction network to extract critical information, and obtain pre-trained reference critical information and pre-trained registration critical information.
[0121] In one embodiment, the server may input the pre-trained reference image and the pre-trained registration image into a pre-trained key point extraction network to extract key points, and calculate the pre-trained reference criticality information and the pre-trained registration criticality information based on the extracted key points.
[0122] Step 306 , calculating pre-trained reference criticality error information between the pre-trained reference criticality information and the corresponding target pre-trained reference criticality information, and calculating pre-trained registration criticality error information between the pre-trained registration criticality information and the corresponding target pre-trained registration criticality information.
[0123] Step 308 : determining target pre-trained criticality error information based on the pre-trained reference criticality error information and the pre-trained registration criticality error information, and updating the pre-trained criticality information extraction network based on the target pre-trained criticality error information.
[0124] The pre-training reference criticality error information refers to the error information of the reference criticality information during pre-training. The pre-training registration criticality error information refers to the error information of the registration criticality information during pre-training. The target pre-training criticality error information refers to the error information of the criticality information during pre-training.
[0125] Specifically, the server uses a pre-set loss function to calculate the difference between the pre-trained reference critical information and the corresponding target pre-trained reference critical information to obtain the pre-trained reference critical error information, and simultaneously calculates the difference between the pre-trained registration critical information and the corresponding target pre-trained registration critical information to obtain the pre-trained registration critical error information. Then, the sum of the pre-trained reference critical error information and the pre-trained registration critical error information is calculated to obtain the target pre-trained critical error information. Then, the server updates the pre-trained critical information extraction network based on the target pre-trained critical error information using a gradient descent algorithm.
[0126] Step 310: When the pre-training is completed, the pre-trained criticality information extraction network is used as the initial criticality information extraction network in the initial image registration quality assessment model.
[0127] Specifically, when the pre-training does not reach the pre-training completion condition, the server uses the updated pre-trained critical information extraction network as the initial pre-training critical information extraction network, and then returns to step 304 to continuously iterate until the pre-training completion condition is reached, and the pre-trained critical information extraction network is used as the initial critical information extraction network in the initial image registration quality assessment model. The pre-training completion condition is pre-set, including at least one of the pre-training reaching the maximum number of iterations, the target pre-training critical error information is less than a pre-set threshold, and the network parameters no longer change.
[0128] In the above embodiment, by using the pre-trained criticality information extraction network as the initial criticality information extraction network in the initial image registration quality assessment model, and then further training the image registration quality assessment model, the trained image registration quality assessment model can be updated accurately.
[0129] In one embodiment, Figure 4 As shown, step 302, obtaining training image samples and target image registration quality assessment results, the training image samples include training reference images and corresponding target reference criticality information and training registration images and corresponding target registration criticality information, including:
[0130] Step 402: Obtain a training reference image and corresponding reference key point labels as well as a training registration image and corresponding registration key point labels.
[0131] Among them, the reference key point label refers to the label obtained according to the reference key point corresponding to the training reference image, and the registration key point label refers to the label obtained according to the registration key point corresponding to the training registration image.
[0132] Specifically, the server can collect training reference images and corresponding reference key point labels and training registration images and corresponding registration key point labels from the Internet. The server can collect training reference images and training registration images from the Internet, and then use a key point extraction algorithm to extract reference key points corresponding to the training reference images and registration key points corresponding to the training registration images, thereby obtaining reference key point labels and registration key point labels.
[0133] Step 404 , calculate the Gaussian distribution information corresponding to the reference key point label, and determine the target reference criticality information corresponding to the training reference image based on the Gaussian distribution information corresponding to the reference key point label and the position of the pixel point in the training reference image.
[0134] Specifically, the server uses the position coordinates of the reference key point label to calculate the Gaussian distribution information, and then uses the Gaussian distribution information corresponding to the reference key point label and the position coordinates of each pixel in the training reference image to calculate the Gaussian distribution value corresponding to each pixel in the training reference image. Then, the Gaussian distribution value corresponding to each pixel in the training reference image is used as the target reference key information.
[0135] Step 406 , calculate the Gaussian distribution information corresponding to the registration key point label, and determine the target registration criticality information corresponding to the training registration image based on the Gaussian distribution information corresponding to the registration key point label and the position of the pixel point in the training registration image.
[0136] Specifically, the server uses the position coordinates of the registration key point labels to calculate the Gaussian distribution information, and then uses the Gaussian distribution information corresponding to the registration key point labels and the position coordinates of each pixel in the training registration image to calculate the Gaussian distribution value corresponding to each pixel in the training registration image. Then, the Gaussian distribution value corresponding to each pixel in the training registration image is used as the target registration key information.
[0137] Step 408 , calculate the key point label distance between the reference key point label and the registration key point label, and use the key point label distance as the target image registration quality assessment result.
[0138] Among them, the key point label distance is used to indicate the distance between key points. The closer the distance is, the closer the key points are, which means the better the image registration quality is.
[0139] Specifically, the server uses a distance algorithm to calculate the key point label distance between the reference key point label and the corresponding registration key point label, and uses the key point label distance as the target image registration quality evaluation result, wherein the distance algorithm may include at least one of a Euclidean distance algorithm, an L2 (second-order norm) distance algorithm, an L1 (first-order norm) distance algorithm, and a cosine distance algorithm. In a specific embodiment, the key point label distance may be calculated using formula (1) as shown below.
[0140]
[0141] Where S represents the target image registration quality evaluation result. K represents the number of label pairs consisting of reference key point labels and registration key point labels. L FK L represents the Kth reference keypoint label in the reference image. WK Represents the Kth registration keypoint label in the registered image. Represents the L2 distance. That is, the L2 distance value of each reference key point label and the registration key point label is calculated by using formula (1), and then the key point label distance is obtained by calculating the ratio of the sum of the L2 distance values to the number of label pairs, and then the distance is used as the target image registration quality evaluation result.
[0142] In the above embodiment, by calculating the target reference criticality information, the target registration criticality information, and the key point label distance between the reference key point label and the registration key point label, the target image registration quality assessment result is obtained, so that the data used in training is more accurate, and can be directly used in the subsequent training of the image registration quality assessment model, thereby improving the training efficiency.
[0143] In one embodiment, the reference key point labels include at least two; Figure 5As shown, step 404, i.e., calculating the Gaussian distribution information corresponding to the reference key point label, and determining the target reference criticality information corresponding to the training reference image based on the Gaussian distribution information corresponding to the reference key point label and the position of the pixel point in the training reference image, includes:
[0144] Step 502, obtain the position coordinates corresponding to each reference key point label, take the position coordinates as the mean, and perform two-dimensional Gaussian distribution calculation based on the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to each reference key point label.
[0145] Specifically, the server obtains the size of the reference image, and determines the position coordinates corresponding to each reference key point label in the reference image according to the size of the reference image. Then, the position coordinates corresponding to each reference key point label are used as the mean, and the two-dimensional Gaussian distribution function corresponding to each reference key point is generated with the preset covariance matrix, and the generated two-dimensional Gaussian distribution function is used as the Gaussian distribution information corresponding to each reference key point label. Each reference key point generates a corresponding Gaussian distribution information.
[0146] Step 504 , calculating each reference key degree corresponding to the pixel point in the training reference image based on the Gaussian distribution information corresponding to each reference key point label and the position coordinates of the pixel point in the training reference image.
[0147] Specifically, the server calculates the reference key degrees corresponding to each pixel point in the training reference image based on the Gaussian distribution information corresponding to each reference key point label using the position coordinates of the pixel point in the training reference image. That is, the server can input the position coordinates of the pixel point in the training reference image into the Gaussian distribution function corresponding to the reference key point label for calculation, and obtain the reference key degree corresponding to each pixel point in the training reference image corresponding to the reference key point label.
[0148] Step 506, select the target reference key from each reference key corresponding to the pixel point, obtain the target reference key corresponding to the pixel point in the training reference image, and obtain the target reference key information corresponding to the training reference image based on the target reference key corresponding to the pixel point in the training reference image.
[0149] Specifically, the server compares the sizes of the reference key values corresponding to each pixel point, selects the largest reference key value as the target reference key value corresponding to the pixel point, and obtains the target reference key value information corresponding to the training reference image.
[0150] In one embodiment, the registration keypoint tags include at least two; Figure 6As shown, step 406, i.e., calculating the Gaussian distribution information corresponding to the registration key point label, and determining the target registration criticality information corresponding to the training registration image based on the Gaussian distribution information corresponding to the registration key point label and the position coordinates of the pixel points in the training registration image, includes:
[0151] Step 602, obtaining the position coordinates corresponding to each registration key point label, taking the position coordinates as the mean, and performing two-dimensional Gaussian distribution calculation based on the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to each registration key point label.
[0152] Specifically, the server obtains the size of the registration image, and determines the position coordinates corresponding to each registration key point label in the registration image according to the size of the registration image. Then the position coordinates corresponding to each registration key point label are used as the mean, and the two-dimensional Gaussian distribution function corresponding to each registration key point is generated with the preset covariance matrix, and the generated two-dimensional Gaussian distribution function is used as the Gaussian distribution information corresponding to each registration key point label. Each registration key point generates a corresponding Gaussian distribution information.
[0153] Step 604 , calculating each registration key degree corresponding to the pixel point in the training registration image based on the Gaussian distribution information corresponding to each registration key point label and the position coordinates of the pixel point in the training registration image.
[0154] Specifically, the server calculates the registration criticality corresponding to each pixel point in the training registration image based on the Gaussian distribution information corresponding to each registration key point label using the position coordinates of the pixel points in the training registration image. That is, the server can input the position coordinates of the pixel points in the training registration image into the Gaussian distribution function corresponding to the registration key point label for calculation, and obtain the registration criticality corresponding to each pixel point in the training registration image corresponding to the registration key point label.
[0155] Step 606, select the target registration key from each registration key corresponding to the pixel point, obtain the target registration key corresponding to the pixel point in the training registration image, and obtain the target registration key information corresponding to the training registration image based on the target registration key corresponding to the pixel point in the training registration image.
[0156] Specifically, the server compares the sizes of the various registration key degrees corresponding to each pixel point, selects the largest registration key degree as the target registration key degree corresponding to the pixel point, and obtains the target registration key degree information corresponding to the training registration image. The server generates a corresponding Gaussian distribution function according to each registration key point label, calculates the key degree corresponding to each pixel point in the training registration image according to each Gaussian distribution function, and obtains the target registration key degree information corresponding to each Gaussian distribution function.
[0157] In the above embodiment, by taking the key point position coordinates as the mean and generating two-dimensional Gaussian distribution information with the preset covariance matrix, the Gaussian distribution obtained according to the two-dimensional Gaussian distribution information is filled into a matrix of the image size, and the target registration criticality information corresponding to the training registration image is obtained, thereby improving the accuracy of the target registration criticality information.
[0158] In one embodiment, step 206, calculating reference criticality error information between training reference criticality information and corresponding target reference criticality information, includes:
[0159] Calculate the sum of squared errors between the training reference key corresponding to each pixel in the training reference key information and the target reference key corresponding to each pixel in the target reference key information; calculate the product of the width of the reference image and the height of the reference image, and calculate the ratio of the sum of squared errors to the product to obtain the reference key error information.
[0160] Specifically, the server calculates the square of the error between the training reference criticality corresponding to each pixel point and the target reference criticality, then adds the error squares of all pixels to obtain the sum of squared errors, and then calculates the product of the width of the reference image and the height of the reference image according to the size of the reference image. Finally, the server calculates the ratio of the sum of squared errors to the product to obtain the reference criticality error information.
[0161] In one embodiment, when there are multiple training image samples, the sum of the reference critical error information of all training image samples is calculated, and then the ratio of the sum to the total number of training image samples is calculated to obtain the target reference critical error information.
[0162] In a specific embodiment, the target reference criticality error information may be calculated using formula (2) shown below.
[0163]
[0164] Where L1 = L MSE Indicates that the MSE average error loss function is used to calculate the target registration critical error information. b_s represents the number of training image samples in a batch, w represents the width of the training image, and h represents the height of the training image. p b (x, y) represents the training reference information corresponding to the b-th training image at position (x, y), which is predicted. final_map b (x, y) represents the target reference information corresponding to the b-th training image at position (x, y), and the target reference information is a label. The training image can be a reference image or a registration image.
[0165] In one embodiment, step 206, calculating the registration criticality error information between the training registration criticality information and the corresponding target registration criticality information, includes:
[0166] Calculate the sum of square errors between the training registration criticality corresponding to each pixel point in the training registration criticality information and the target registration criticality corresponding to each pixel point in the target registration criticality information; calculate the product of the width of the registration image and the height of the registration image, and calculate the ratio of the sum of square errors to the product to obtain the registration criticality error information.
[0167] Specifically, the server calculates the square of the error between the training registration criticality corresponding to each pixel and the target registration criticality, then adds the error squares of all pixels to obtain the sum of squared errors, and then calculates the product of the width of the registration image and the height of the registration image according to the size of the registration image. Finally, the ratio of the sum of squared errors to the product is calculated to obtain the registration criticality error information.
[0168] In one embodiment, when there are multiple training image samples, the sum of the registration critical error information of all training image samples is calculated, and then the ratio of the sum to the total number of training image samples is calculated to obtain the target registration critical error information. In a specific embodiment, formula (2) can also be used to calculate the target registration critical error information. The sum of the target registration critical error information and the target reference critical error information can be used to update the initial critical information extraction network.
[0169] In one embodiment, step 212, i.e., calculating the evaluation result error information between the initial image registration quality evaluation result and the target image registration quality evaluation result, comprises the steps of:
[0170] The absolute difference between the initial image registration quality assessment result and the target image registration quality assessment result is calculated to obtain the assessment result error information.
[0171] Specifically, the server calculates the absolute difference between the initial image registration quality evaluation result and the target image registration quality evaluation result to obtain the evaluation result error information. When there are multiple training image samples, the evaluation result error information of each training image sample is calculated, and then the ratio of the sum of the error information of each evaluation result to the total number of training image samples is calculated to obtain the target evaluation result error information, and the target evaluation result error information can be used to update the initial image registration quality evaluation result. For example, the target evaluation result error information can be calculated using the formula (3) shown below.
[0172]
[0173] Among them, L2 represents the error information of the target evaluation result calculated using the L1 norm as the loss function.b represents the target image registration quality assessment result corresponding to the b-th training image sample, represents the initial image registration quality assessment result corresponding to the b-th training image sample, where the training image sample can be a reference image or a registered image.
[0174] In a specific embodiment, Figure 7 As shown, the image registration quality assessment model training method specifically includes the following steps:
[0175] Step 702: obtaining pre-training image samples, where the pre-training image samples include a pre-training reference image and corresponding target pre-training reference criticality information, and a pre-training registration image and corresponding target pre-training registration criticality information.
[0176] Step 704, inputting the pre-trained reference image and the pre-trained registration image into a pre-trained criticality information extraction network to obtain pre-trained reference criticality information and pre-trained registration criticality information;
[0177] Step 706, calculating target pre-training criticality error information based on the pre-training reference criticality information and the target pre-training reference criticality information, and the pre-training registration criticality information and the target pre-training registration criticality information;
[0178] Step 708, updating the pre-trained criticality information extraction network based on the target pre-trained criticality error information, and when the pre-training is completed, using the pre-trained criticality information extraction network as the initial criticality information extraction network.
[0179] Step 710, obtaining training image samples and target image registration quality assessment results, the training image samples include a training reference image and corresponding target reference criticality information, and a training registration image and corresponding target registration criticality information, and the training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image.
[0180] Step 712, inputting the training reference image and the training registration image into the initial criticality information extraction network in the initial image registration quality assessment model to obtain the training reference criticality information and the training registration criticality information;
[0181] Step 714, calculating reference criticality error information between the training reference criticality information and the corresponding target reference criticality information, and calculating registration criticality error information between the training registration criticality information and the corresponding target registration criticality information;
[0182] Step 716, determining target criticality error information based on the reference criticality error information and the registration criticality error information, and updating the initial criticality information extraction network based on the target criticality error information;
[0183] Step 718, inputting the training reference image, the training registration image, the training reference criticality information and the training registration criticality information into the initial image registration quality assessment network in the initial image registration quality assessment model to obtain the initial image registration quality assessment result;
[0184] Step 720, calculate the evaluation result error information between the initial image registration quality evaluation result and the target image registration quality evaluation result; update the initial image registration quality evaluation network based on the evaluation result error information, and when the training is completed, obtain the first image registration quality evaluation model.
[0185] In this specific embodiment, the computer device can train and obtain the first image registration quality assessment model by executing the following instruction steps:
[0186] for epoch=1; epoch<=10; epoch++do
[0187]
[0188] Where N is the total number of training image samples, and batch_size is the number of training image samples in each batch during the training process. i is the reference image in the i-th training batch, W i is the registered image in the i-th training batch. f represents the criticality information extraction network. f(·) represents the criticality information output by the criticality information extraction network, and final_map is the final criticality heat map. Represents the final criticality heat map corresponding to the i-th reference image. represents the final criticality heat map corresponding to the i-th registered image. max_epoch represents the maximum number of iterations of training. G represents the image registration quality assessment network, and g(·) represents the image registration quality assessment result output by the image registration quality assessment network. S i represents the image registration quality assessment result labels corresponding to all training image samples in the i-th batch. i ) represents the critical information corresponding to the reference image output by the critical information extraction network. f(W i ) represents the criticality information corresponding to the registered image output by the criticality information extraction network.
[0189] The trained image registration quality assessment model is then deployed and used. When in use, the image registration quality assessment method specifically includes the following steps:
[0190] A reference image and a registered image are obtained, wherein the registered image is an image obtained by performing image registration on the image to be registered based on the reference image; the reference image and the registered image are input into a first image registration quality assessment model, and the first image registration quality assessment model uses a criticality information extraction network to extract criticality information from the reference image and the registered image, and obtains reference criticality information corresponding to the reference image and registration criticality information corresponding to the registered image; the reference image, the registered image, the reference criticality information and the registration criticality information are input into an image registration quality assessment network, and image registration quality assessment results corresponding to the reference image and the registered image are obtained.
[0191] In one embodiment, Figure 8 As shown in FIG. 1 , a method for evaluating the quality of image registration is provided. The method is applied to Figure 1 The server in the example is used as an example. It can be understood that the method can also be applied to a terminal. In this embodiment, the method includes the following steps:
[0192] Step 802 , obtaining a reference image and a registered image, wherein the registered image is an image obtained by performing image registration on the image to be registered based on the reference image.
[0193] The reference image refers to the image that needs to be registered during image registration, that is, the image to be registered needs to be registered based on the reference image during registration. The image to be registered refers to the image that needs to be registered. The registered image refers to the image obtained after image registration of the image to be registered.
[0194] Specifically, the server can obtain the saved reference image and the registered image from the database. The server can use the reference image to perform image registration on the image to be registered in advance, and then save the obtained registered image and the reference image to the server database. The server can also obtain the reference image and the registered image uploaded by the user through the terminal. The server can also obtain the reference image and the registered image directly from the Internet. The server can also obtain the reference image and the registered image from a third-party server, which can be a server for image registration.
[0195] Step 804 , extract key points corresponding to the reference image and the registered image respectively to obtain reference key points and registered key points.
[0196] Among them, key points refer to points or blocks in the image that contain rich local information and can be represented in an invariant form. Key points often appear at the corners of the image and in places where the texture changes dramatically. Reference key points refer to key points in the reference image, and registration key points refer to key points in the registration image. There can be only one key point in an image or multiple key points.
[0197] Specifically, the server can use a key point extraction algorithm to extract the key points corresponding to the reference image and the registration image respectively, and obtain reference key points and registration key points, wherein the key point extraction algorithm can be a SIFT (Scale-invariant feature transform) algorithm, a SURF (Speeded Up Robust Features) algorithm, a FAST (Features from Accelerated Segment Test) algorithm, an ORB (Oriented FAST and Rotated BRIEF, ORB uses the FAST algorithm to detect feature points and the BRIEF (Binary Robust Independent Elementary Features) algorithm to calculate feature point descriptions) algorithm, a deep learning algorithm, and the like.
[0198] Step 806, calculating Gaussian distribution information corresponding to the reference key point, and determining reference criticality information corresponding to the reference image based on the Gaussian distribution information corresponding to the reference key point and the position of the pixel point in the reference image.
[0199] Among them, Gaussian distribution information refers to the Gaussian distribution generated by taking the position coordinates of the key point as the mean, and Gaussian distribution information corresponding to the reference key point refers to the Gaussian distribution generated by taking the position coordinates of the reference key point as the mean. Criticality refers to the criticality of a pixel in an image. Reference criticality refers to the criticality of a pixel in a reference image. The higher the reference criticality of a pixel in a reference image, the more important the pixel is when evaluating the quality of image registration. The reference key point is the pixel with the highest criticality in the reference image. Reference criticality information is used to indicate the criticality of all pixels in the reference image.
[0200] Specifically, the server can use the position coordinates of the reference key points as the mean, and then generate a Gaussian distribution according to a preset covariance matrix. Then, the reference key corresponding to the pixel in the reference image is calculated according to the Gaussian distribution and the position coordinates of the pixel in the reference image, thereby obtaining the reference key information corresponding to the reference image.
[0201] In one embodiment, when the reference image includes multiple reference key points, each reference key point is used to calculate the reference key information corresponding to the reference image, and then the final reference key information corresponding to the reference image is obtained by screening according to each reference key information.
[0202] Step 808: Calculate Gaussian distribution information corresponding to the registration key points, and determine registration criticality information corresponding to the registration image based on the Gaussian distribution information corresponding to the registration key points and the positions of the pixels in the registration image.
[0203] Among them, the Gaussian distribution information corresponding to the registration key point refers to the Gaussian distribution generated by taking the position coordinates of the registration key point as the mean. Registration criticality refers to the criticality of the pixel in the registration image. The higher the registration criticality, the more important the pixel is when evaluating the image registration quality. The registration key point is the pixel with the highest criticality in the registration image. The registration criticality information is used to indicate the criticality of all pixels in the registration image.
[0204] Specifically, the server can use the position coordinates of the registration key points as the mean, and then generate a Gaussian distribution according to a preset covariance matrix. Then, the registration key corresponding to the pixel in the registration image is calculated according to the Gaussian distribution and the position coordinates of the pixel in the registration image, thereby obtaining the registration key information corresponding to the registration image.
[0205] In one embodiment, when the registered image includes multiple registration key points, each registration key point is used to calculate the registration key information corresponding to the registered image, and then the final registration key information corresponding to the registered image is obtained by screening according to each registration key information.
[0206] Step 808 , performing image registration quality assessment based on the reference image, the registered image, the reference criticality information and the registration criticality information, and obtaining image registration quality assessment results corresponding to the reference image and the registered image.
[0207] Among them, the image registration quality assessment is used to evaluate the registration quality of the reference image and the registered image. The image registration quality assessment result is used to represent the registration quality of the reference image and the registered image, which can be expressed as a quality score.
[0208] Specifically, the server can use the reference image, the registered image, the reference key information and the registered key information to perform image registration quality assessment based on a deep learning algorithm to obtain image registration quality assessment results corresponding to the reference image and the registered image. The deep learning algorithm can be a convolutional neural network algorithm, a linear regression algorithm, a recurrent neural network algorithm, a feedforward neural network algorithm, and the like.
[0209] The above-mentioned image registration quality assessment method extracts reference key points and registration key points corresponding to the reference image and the registered image, then calculates Gaussian distribution information corresponding to the reference key points to determine the reference criticality information corresponding to the reference image, and calculates Gaussian distribution information corresponding to the registration key points to determine the registration criticality information corresponding to the registered image, and finally uses the reference image, the registered image, the reference criticality information and the registration criticality information to perform image registration quality assessment, and obtains the image registration quality assessment results corresponding to the reference image and the registered image. That is, by using not only the reference image and the registered image, but also the reference criticality information and the registration criticality information when performing image registration quality assessment, the accuracy of the image registration quality assessment results is improved.
[0210] In one embodiment, before acquiring the reference image and the registration image, the method further includes:
[0211] A reference image and an image to be registered are obtained, and the reference image and the image to be registered are input into an image registration model to obtain an output registered image. The image registration model is trained based on a deep neural network algorithm using training samples.
[0212] Among them, the image registration model is used to register images.
[0213] Specifically, the server obtains a large number of training samples, each of which includes a training reference image, a training image to be registered, and a corresponding label registration image. When training the image registration model, the server inputs the training sample into the image registration model with initialized model parameters, obtains the output training registration image, and calculates the error between the training registration image and the label registration image. For example, the error can be determined by calculating the similarity between the training registration image and the label registration image. The similarity can be obtained by calculating the distance between the key points in the training registration image and the key points in the label registration image. The initialized model parameters are then updated using the calculated error, and then the loop iteration is continued until the error between the training registration image and the label registration image is less than a preset error threshold, thereby obtaining a trained image registration model. Among them, the deep neural network algorithm can be a convolutional neural network algorithm,
[0214] The server can deploy and use the trained image registration model. That is, the server obtains the reference image and the image to be registered, which can be the reference image and the image to be registered uploaded to the server by the user through the terminal, or directly obtained from the server database. Then the reference image and the image to be registered are input into the image registration model to obtain the output registered image. Then the obtained registered image and reference image are used to evaluate the image registration quality. In a specific embodiment, the reference image and the image to be registered can be obtained from the public FIRE (Fundus Image Registration Dataset) dataset, and then the SyN (symmetric image normalization method) registration algorithm is used for registration. In a specific embodiment, the image to be registered and the reference image can be registered using formula (4) shown below.
[0215] (F,W)=R(F,M) Formula (4)
[0216] Among them, F refers to the reference image, W refers to the registered image, R represents the registration process, which can be various registration algorithms, and M refers to the image to be registered.
[0217] In one embodiment, Fig. 9 As shown, the registration image quality assessment method also includes:
[0218] Step 902: Input the reference image and the registered image into the first image registration quality assessment model. The first image registration quality assessment model uses a critical information extraction network to extract critical information from the reference image and the registered image to obtain reference critical information corresponding to the reference image and registration critical information corresponding to the registered image. Input the reference image, the registered image, the reference critical information and the registration critical information into the image registration quality assessment network to obtain image registration quality assessment results corresponding to the reference image and the registered image.
[0219] In step 904, the first image registration quality assessment model is trained using target criticality information and target image registration quality assessment results, wherein the target criticality information is determined based on the Gaussian distribution information corresponding to the key point labels of the training image and the positions of the pixels in the training image, and the target image registration quality assessment results are obtained based on the distance between the key point labels corresponding to the training reference image and the key point labels corresponding to the training registration image.
[0220] Among them, the target criticality information refers to the key point information label used by the image registration quality assessment model during training. The target image registration quality assessment result refers to the image registration quality assessment result label used by the image registration quality assessment model during training. The training image refers to the image used to train the image registration quality assessment model, including the training reference image and the training registration image. The training registration image is the image obtained by registering the training image to be registered using the training reference image as the target image. The training image key point label refers to the label obtained using the key points in the training image. The key point label corresponding to the training reference image refers to the label used during training obtained based on the key points in the training reference image. The key point label corresponding to the training registration image refers to the label used during training obtained based on the key points in the training registration image.
[0221] Specifically, the critical information extraction network is trained based on a convolutional neural network algorithm using a reference image and a registered image as inputs and using target critical information as labels. The image registration quality assessment network is trained based on a residual network algorithm using the reference critical information and registration critical information output by the critical information extraction network as well as the reference image and the registered image as inputs and using the target image registration quality assessment results as labels. Based on the trained critical information extraction network and image registration quality assessment network, a first image registration quality assessment model is obtained, and the first image registration quality assessment model is deployed to a server for use.
[0222] When the server receives a registration quality assessment request for a reference image and a registered image, the server inputs the reference image and the registered image into a first image registration quality assessment model, and the first image registration quality assessment model uses a critical information extraction network to extract critical information from the reference image and the registered image, obtains reference critical information corresponding to the reference image and registration critical information corresponding to the registered image, inputs the reference image, the registered image, the reference critical information, and the registration critical information into the image registration quality assessment network, obtains image registration quality assessment results corresponding to the reference image and the registered image, and the first image registration quality assessment model outputs the obtained image registration quality assessment results. The server may save the image registration quality assessment results output by the first image registration quality assessment model.
[0223] In the above embodiment, by using the first image registration quality assessment model to perform image registration quality assessment, the efficiency and accuracy of obtaining the image registration quality assessment result are improved.
[0224] In one embodiment, step 804, respectively extracting key points corresponding to the reference image and the registered image to obtain reference key points and registered key points, includes the steps of:
[0225] The key point detection model is used to perform key point detection on the reference image and the registered image to obtain the reference key points corresponding to the reference image and the registration key points corresponding to the registered image. The key point detection model is trained using a deep neural network algorithm based on the key point training image and the corresponding key point labels.
[0226] Among them, the key point training image refers to the image used to train the key point detection model, and the key point label is the label used when training the key point detection model based on the key points in the key point training image. The key point detection model is used to detect key points in the reference image and the registration image.
[0227] Specifically, the server pre-uses the key point training image as the input of the key point detection model for initializing the model parameters, and trains the corresponding key point label as the label of the key point detection model for initializing the model parameters. When the training is completed, the key point detection model is obtained, and the key point detection model is deployed for use. When a request for key point detection of a reference image and a registered image is received, the reference image and the registered image are simultaneously input into the key point detection model, and the reference key points corresponding to the reference image and the registered key points corresponding to the registered image are obtained as output.
[0228] In the above embodiment, the key point detection model is used to detect the key points in the reference image and the registered image, thereby improving the accuracy and efficiency of key point detection.
[0229] In one embodiment, step 810, i.e., performing image registration quality assessment based on the reference image, the registered image, the reference criticality information, and the registration criticality information to obtain image registration quality assessment results corresponding to the reference image and the registered image, includes the steps of:
[0230] The reference image, the registered image, the reference critical information and the registration critical information are input into the second image registration quality assessment model to obtain the image registration quality assessment results corresponding to the output reference image and the registered image. The second image registration quality assessment model is trained using a deep neural network algorithm using the training reference image, the training registered image, the target reference critical information and the target registration critical information.
[0231] The training reference image refers to the reference image used in training. The training registration image refers to the registration image corresponding to the reference image used in training. The second image registration quality assessment model uses the reference image, the registration image, the reference criticality information corresponding to the reference image, and the registration criticality information corresponding to the registration image to assess the image registration quality.
[0232] Specifically, the server obtains the critical information corresponding to the training reference image and the training registered image in advance, and then uses the training reference image and the training registered image and the critical information corresponding to the training reference image and the training registered image as the input of the second image registration quality assessment model for model parameter initialization, and uses the target reference critical information and the target registration critical information as labels for training. When the training is completed, the second image registration quality assessment model is obtained, and the server deploys and uses the second image registration quality assessment model. When the server receives an image registration quality assessment request for the reference image and the registered image, the reference critical information corresponding to the reference image and the registration critical information corresponding to the registered image are obtained, wherein the server can extract the key points corresponding to the reference image and the registered image respectively in advance, and then calculate the reference critical information and the registration critical information according to the key points corresponding to the reference image and the registered image respectively using steps 806 and 808. Then the reference image, the registered image, the reference critical information and the registration critical information are input into the second image registration quality assessment model to obtain the output image registration quality assessment result corresponding to the reference image and the registered image.
[0233] In the above embodiment, the server directly uses the second image registration quality assessment model to assess the image registration quality of the reference image and the registered image, thereby improving the efficiency of obtaining the image registration quality assessment result.
[0234] In one embodiment, Fig.10 As shown, step 806, calculating Gaussian distribution information corresponding to the reference key point, and determining reference criticality information corresponding to the reference image based on the Gaussian distribution information corresponding to the reference key point and the position of the pixel point in the reference image, includes:
[0235] Step 1002, taking the position coordinates of the reference key points as the mean, and performing two-dimensional Gaussian distribution calculation based on the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to the reference key points.
[0236] Specifically, the preset covariance matrix refers to a pre-set covariance matrix used to generate a Gaussian distribution. The server uses the position coordinates of the reference key point as the mean, generates a two-dimensional Gaussian distribution function based on the mean and the preset covariance matrix, and obtains the Gaussian distribution information corresponding to the reference key point. For example, when the position coordinates of the reference key point are (x1, y1), the preset covariance matrix is The σ is a hyperparameter and can be set to 5. A two-dimensional Gaussian distribution function is generated according to the position coordinates of the reference key point and the preset covariance matrix to obtain the Gaussian distribution information corresponding to the reference key point.
[0237] Step 1004 , calculating the reference key degree corresponding to the pixel point in the reference image based on the Gaussian distribution information corresponding to the reference key point and the position coordinates of the pixel point in the reference image.
[0238] Step 1006: Obtain reference key information corresponding to the reference image based on the reference key corresponding to the pixel point in the reference image.
[0239] Specifically, the server obtains the size of the reference image, and determines the position coordinates of each pixel in the reference image according to the size of the reference image. For example, the size of the reference image is (w, h), where w represents the width of the reference image and h represents the height of the reference image, and then the position coordinates of each pixel in the reference image are determined according to the size of the reference image. The position coordinates of the pixel in the reference image are input into the two-dimensional Gaussian distribution function corresponding to the reference key point for calculation to obtain the reference key degree corresponding to each pixel. Among them, the pixel with the largest reference key degree is the reference key point. Then, a reference key degree matrix is obtained according to the reference key degree corresponding to each pixel, and the reference key degree matrix is used as the reference key degree information corresponding to the reference image. In a specific embodiment, the reference key point information is visualized to obtain a heat map corresponding to the reference key degree information.
[0240] In the above embodiment, by using the position coordinates of the reference key points and the preset covariance matrix to calculate the Gaussian distribution, the reference key information corresponding to the reference image is obtained, and the reference key of each pixel in the reference image can be obtained, thereby making the obtained reference key information more accurate.
[0241] In one embodiment, when there are at least two reference key points, the position coordinates of each reference key point are taken as the mean, and a two-dimensional Gaussian distribution calculation is performed based on each mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to each reference key point, and the reference key of the pixel in the reference image corresponding to each reference key point is calculated based on the Gaussian distribution information corresponding to each reference key point and the position coordinates of the pixel in the reference image, and then the reference key of the same pixel in the reference image corresponding to each reference key point is compared, and the largest reference key is selected as the final reference key of the pixel, and the reference key corresponding to each pixel in the reference image is compared to determine the final reference key corresponding to each pixel, and the reference key information corresponding to the reference image is obtained according to the final reference key corresponding to each pixel. In a specific embodiment, after obtaining the reference key of the pixel in the reference image corresponding to each reference key point, the reference key of the pixel in the reference image corresponding to each reference key point is visualized to obtain each reference key heat map, for example, when there are 10 reference key points, 10 reference key heat maps can be obtained. Then, the final reference criticality heat map corresponding to the reference image can be calculated using formula (5) as shown below.
[0242] final_map(x,y)=max{map1(x,y),map2(x,y),...,map K (x,y)}Formula (5)
[0243] Among them, final_map(x,y) represents the value at position (x, y) in the final criticality heat map. map1,map2,...,map K Represents each key degree heat map, and K represents the total number of key degree heat maps. K (x, y) represents the value of the Kth critical heat map at position (x, y). map{·} represents taking the maximum value of all elements in the set.
[0244] In one embodiment, Fig.11 As shown, step 808, i.e., calculating the Gaussian distribution information corresponding to the registration key points, and determining the registration criticality information corresponding to the registration image based on the Gaussian distribution information corresponding to the registration key points and the positions of the pixels in the registration image, includes:
[0245] Step 1102 , taking the position coordinates of the registration key points as the mean, and performing two-dimensional Gaussian distribution calculation based on the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to the registration key points.
[0246] Specifically, the preset covariance matrix refers to a pre-set covariance matrix used to generate a Gaussian distribution, which is the same as the covariance matrix used to calculate the Gaussian distribution information corresponding to the reference key points. The server uses the position coordinates of the registration key points as the mean, generates a two-dimensional Gaussian distribution function based on the mean and the preset covariance matrix, and obtains the Gaussian distribution information corresponding to the registration key points.
[0247] Step 1104 , calculating the registration criticality corresponding to the pixel point in the registered image based on the Gaussian distribution information corresponding to the registration key point and the position coordinates of the pixel point in the registered image.
[0248] Step 1106: obtaining registration criticality information corresponding to the registered image based on the registration criticality corresponding to the pixel points in the registered image.
[0249] Specifically, the server obtains the size of the registration image, which is consistent with the size of the reference image, and determines the position coordinates of each pixel in the registration image according to the size of the registration image. Then, the position coordinates of the pixel in the registration image are input into the two-dimensional Gaussian distribution function corresponding to the registration key point for calculation, and the registration criticality corresponding to each pixel is obtained. Among them, the pixel with the largest registration criticality is the registration key point. Then, the registration criticality matrix is obtained according to the registration criticality corresponding to each pixel, and the registration criticality matrix is used as the registration criticality information corresponding to the registration image. In a specific embodiment, the registration key point information is visualized to obtain a heat map corresponding to the registration criticality information.
[0250] In one embodiment, when there are at least two registration key points, the position coordinates of each registration key point are taken as the mean, and a two-dimensional Gaussian distribution calculation is performed based on each mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to each registration key point, and the registration criticality of the pixel point in the reference image corresponding to each registration key point is calculated based on the Gaussian distribution information corresponding to each registration key point and the position coordinates of the pixel point in the registration image, and then the sizes of the various registration criticalities of the same pixel point in the reference image corresponding to each registration key point are compared, and the largest registration criticality is selected as the final registration criticality of the pixel point, and the various registration criticalities corresponding to each pixel point in the registration image are compared to determine the final registration criticality corresponding to each pixel point, and the registration criticality information corresponding to the registration image is obtained according to the final registration criticality corresponding to each pixel point. In a specific embodiment, after obtaining the registration criticality of each pixel point in the registration image corresponding to each registration key point, the registration criticality of each pixel point in the registration image corresponding to each registration key point is visualized to obtain the registration criticality heat map corresponding to each registration key point, and then the final registration criticality heat map corresponding to the registration image can be calculated using formula (5). In a specific embodiment, Fig.12 The left side of the figure is a schematic diagram of each key point in the retinal fundus image. The key degree heat map corresponding to the retinal fundus image can be calculated using each key point in the retinal fundus image. Fig.12 As shown in b on the right side of the figure, the criticality of each key point is the maximum value in the criticality heat map corresponding to each key point, and then the criticality of the heat map gradually decays from the position of each key point to the outside, and the figure shows that it gradually decays from the black key point to the outside (the attenuation process is not shown in the figure).
[0251] In the above embodiment, the criticality corresponding to the pixel in the image is calculated by using the Gaussian distribution information corresponding to the key point and the position coordinates of the pixel in the image, thereby obtaining the criticality information corresponding to the image, thereby improving the accuracy of the obtained criticality information.
[0252] The present application also provides an application scenario, which applies the above-mentioned image registration quality assessment model training method and image registration quality assessment method. Specifically, the application of the image registration quality assessment model training method and image registration quality assessment method in the application scenario is as follows:
[0253] like Fig.13 As shown in FIG. 1 , a schematic diagram of an image registration quality assessment model training method and an application framework of an image registration quality assessment method is shown. Specifically:
[0254] Train the retinal fundus image registration quality assessment model in the cloud server. Specifically:
[0255] A training image sample is obtained, which includes a retinal fundus registration image and a retinal fundus reference image. The retinal fundus registration image and the retinal fundus reference image are input into the critical information extraction network in the image registration quality assessment model to obtain an output heat map, and then the retinal fundus reference image is used to calculate the heat map label, and the error information is calculated by using the MSE loss function, and then the error information is used to update the critical information extraction network. At the same time, the retinal fundus registration image, the retinal fundus reference image, and the heat map output by the critical information extraction network are input into the image registration quality assessment network in the image registration quality assessment model to obtain the output image quality assessment score, and then the L1 norm loss function is used to calculate the error information between the output image registration quality assessment score and the true image registration quality score, and the error information is used to update the image registration quality assessment network. When the training is completed, the image registration quality assessment model is obtained.
[0256] like Fig.14 As shown in the figure, the process of image registration quality assessment in the cloud server includes:
[0257] The cloud server obtains the medical image registration results uploaded by the user through terminal A, such as the retinal fundus registration image and the retinal fundus reference image. The cloud server inputs the retinal fundus registration image and the retinal fundus reference image into the critical information extraction network in the image registration quality assessment model to obtain the output registration heat map and reference heat map, and inputs the retinal fundus registration image, the retinal fundus reference image registration heat map and the reference heat map into the image registration quality assessment network to obtain the predicted image quality assessment score. The predicted image quality assessment score is then sent to terminal B for display.
[0258] In this embodiment, the image registration quality assessment model is tested using the FIRE dataset, and then evaluated using evaluation indicators, wherein the Pearson correlation coefficient (PLCC), root mean square error (RMSE), mean square error (MSE), and mean absolute error (MAE) are selected as evaluation indicators. The evaluation indicator table of the medical registration image quality assessment model is shown in Table 1 below:
[0259] Method PLCC(↑) RMSE(↓) MSE(↓) MAE(↓) Traditional Algorithm 1 0.9198 0.4749 0.2255 0.3612 Traditional Algorithm 2 0.6940 0.8789 0.7725 0.6790 This application 0.9222 0.4676 0.2186 0.3521
[0260] Among them, (↑) means the higher the better, and (↓) means the lower the better. It can be clearly seen that the evaluation indicators tested by this application are significantly better than those of the traditional algorithm. Therefore, using this application to evaluate the image registration quality can improve the evaluation effect of the image registration quality.
[0261] It should be understood that although Figure 2-12 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-12 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.
[0262] In one embodiment, Fig.15As shown, an image quality assessment model training device 1500 is provided. The device can adopt a software module or a hardware module, or a combination of the two to become a part of a computer device. The device specifically includes: a training sample acquisition module 1502, a training information extraction module 1504, a criticality error calculation module 1506, an extraction network update module 1508, an initial quality assessment module 1510, an assessment error calculation module 1512 and a model acquisition module 1514, wherein:
[0263] The training sample acquisition module 1502 is used to obtain training image samples and target image registration quality assessment results. The training image samples include a training reference image and corresponding target reference criticality information and a training registration image and corresponding target registration criticality information. The training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image.
[0264] The training information extraction module 1504 is used to input the training reference image and the training registration image into the initial criticality information extraction network in the initial image registration quality assessment model to obtain the training reference criticality information and the training registration criticality information;
[0265] A key error calculation module 1506 is used to calculate reference key error information between training reference key information and corresponding target reference key information, and calculate registration key error information between training registration key information and corresponding target registration key information;
[0266] An extraction network updating module 1508, configured to determine target criticality error information based on the reference criticality error information and the registration criticality error information, and to update an initial criticality information extraction network based on the target criticality error information;
[0267] An initial quality assessment module 1510 is used to input the training reference image, the training registration image, the training reference criticality information and the training registration criticality information into an initial image registration quality assessment network in the initial image registration quality assessment model to obtain an initial image registration quality assessment result;
[0268] An evaluation error calculation module 1512 is used to calculate evaluation result error information between an initial image registration quality evaluation result and a target image registration quality evaluation result;
[0269] The model acquisition module 1514 is used to update the initial image registration quality assessment network based on the assessment result error information, and when the training is completed, the image registration quality assessment model is obtained.
[0270] In one embodiment, the image quality assessment model training apparatus 1500 further includes:
[0271] The pre-training module is used to obtain pre-training image samples, which include pre-training reference images and corresponding target pre-training reference critical information and pre-training registration images and corresponding target pre-training registration critical information; input the pre-training reference images and pre-training registration images into a pre-training critical information extraction network to obtain pre-training reference critical information and pre-training registration critical information; calculate pre-training reference critical error information between the pre-training reference critical information and the corresponding target pre-training reference critical information, and calculate pre-training registration critical error information between the pre-training registration critical information and the corresponding target pre-training registration critical information; determine the target pre-training critical error information based on the pre-training reference critical error information and the pre-training registration critical error information, and update the pre-training critical information extraction network based on the target pre-training critical error information; when the pre-training is completed, use the pre-trained critical information extraction network as the initial critical information extraction network in the initial image registration quality assessment model.
[0272] In one embodiment, the training sample acquisition module 1502 includes:
[0273] A label acquisition unit, used to acquire a training reference image and corresponding reference key point labels and a training registration image and corresponding registration key point labels;
[0274] A reference information calculation unit, used to calculate Gaussian distribution information corresponding to the reference key point label, and determine the target reference criticality information corresponding to the training reference image based on the Gaussian distribution information corresponding to the reference key point label and the position of the pixel point in the training reference image;
[0275] A registration information calculation unit, used to calculate Gaussian distribution information corresponding to the registration key point label, and determine the target registration criticality information corresponding to the training registration image based on the Gaussian distribution information corresponding to the registration key point label and the position of the pixel point in the training registration image;
[0276] The result calculation unit is used to calculate the key point label distance between the reference key point label and the registration key point label, and use the key point label distance as the target image registration quality evaluation result.
[0277] In one embodiment, the reference key point labels include at least two; the reference information calculation unit is also used to obtain the position coordinates corresponding to each reference key point label, take the position coordinates as the mean, and perform two-dimensional Gaussian distribution calculation based on the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to each reference key point label; based on the Gaussian distribution information corresponding to each reference key point label and the position coordinates of the pixel points in the training reference image, calculate each reference key degree corresponding to the pixel point in the training reference image; select a target reference key degree from each reference key degree corresponding to the pixel point to obtain a target reference key degree corresponding to the pixel point in the training reference image, and obtain the target reference key degree information corresponding to the training reference image based on the target reference key degree corresponding to the pixel point in the training reference image.
[0278] In one embodiment, the registration key point labels include at least two; the registration information calculation unit is also used to: obtain the position coordinates corresponding to each registration key point label, take the position coordinates as the mean, and perform two-dimensional Gaussian distribution calculation based on the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to each registration key point label; calculate each registration key degree corresponding to the pixel point in the training registration image based on the Gaussian distribution information corresponding to each registration key point label and the position coordinates of the pixel point in the training registration image; select the target registration key degree from the various registration key degrees corresponding to the pixel point to obtain the target registration key degree corresponding to the pixel point in the training registration image, and obtain the target registration key degree information corresponding to the training registration image based on the target registration key degree corresponding to the pixel point in the training registration image.
[0279] In one embodiment, the criticality error calculation module 1506 is also used to calculate the sum of square errors between the training reference criticality corresponding to each pixel point in the training reference criticality information and the target reference criticality corresponding to each pixel point in the target reference criticality information; calculate the product of the width of the reference image and the height of the reference image, and calculate the ratio of the sum of square errors to the product to obtain the reference criticality error information.
[0280] In one embodiment, the evaluation error calculation module 1512 is further used to calculate the absolute difference between the initial image registration quality evaluation result and the target image registration quality evaluation result to obtain evaluation result error information.
[0281] In one embodiment, Fig.16 As shown, an image quality assessment device 1600 is provided. The device can adopt a software module or a hardware module, or a combination of the two to become a part of a computer device. The device specifically includes: an image acquisition module 1602, a key point extraction module 1604, a reference information determination module 1606, a registration information determination module 1608 and a quality assessment module 1610, wherein:
[0282] An image acquisition module 1602 is used to acquire a reference image and a registration image, wherein the registration image is an image obtained by performing image registration on the image to be registered based on the reference image;
[0283] A key point extraction module 1604 is used to extract key points corresponding to the reference image and the registration image respectively to obtain reference key points and registration key points;
[0284] A reference information determination module 1606, configured to calculate Gaussian distribution information corresponding to the reference key point, and determine reference criticality information corresponding to the reference image based on the Gaussian distribution information corresponding to the reference key point and the position of the pixel point in the reference image;
[0285] A registration information determination module 1608 is used to calculate Gaussian distribution information corresponding to the registration key points, and determine the registration criticality information corresponding to the registration image based on the Gaussian distribution information corresponding to the registration key points and the positions of the pixels in the registration image;
[0286] The quality assessment module 1610 is used to perform image registration quality assessment based on the reference image, the registered image, the reference criticality information and the registration criticality information, and obtain image registration quality assessment results corresponding to the reference image and the registered image.
[0287] In one embodiment, the image quality assessment apparatus 1600 further includes:
[0288] The registration module is used to obtain a reference image and an image to be registered, and input the reference image and the image to be registered into an image registration model to obtain an output registered image. The image registration model is trained based on a deep neural network algorithm using training samples.
[0289] In one embodiment, the image quality assessment apparatus 1600 further includes:
[0290] The model evaluation module is used to input the reference image and the registered image into the first image registration quality evaluation model. The first image registration quality evaluation model uses a critical information extraction network to extract critical information from the reference image and the registered image to obtain reference critical information corresponding to the reference image and registration critical information corresponding to the registered image. The reference image, the registered image, the reference critical information and the registration critical information are input into the image registration quality evaluation network to obtain image registration quality evaluation results corresponding to the reference image and the registered image. The first image registration quality evaluation model is trained using target critical information and target image registration quality evaluation results. The target critical information is determined based on Gaussian distribution information corresponding to key point labels of the training image and the positions of pixels in the training image. The target image registration quality evaluation results are obtained based on the distance between key point labels corresponding to the training reference image and key point labels corresponding to the training registered image.
[0291] In one embodiment, the key point extraction module 1604 is also used to perform key point detection on the reference image and the registered image using a key point detection model to obtain reference key points corresponding to the reference image and registration key points corresponding to the registered image. The key point detection model is trained using a deep neural network algorithm based on the key point training image and the corresponding key point labels.
[0292] In one embodiment, the quality assessment module 1610 is also used to input the reference image, the registered image, the reference critical information and the registration critical information into a second image registration quality assessment model to obtain the image registration quality assessment results corresponding to the output reference image and the registered image. The second image registration quality assessment model is trained using a deep neural network algorithm using training reference images, training registered images, target reference critical information and target registration critical information.
[0293] In one embodiment, the reference information determination module 1606 is also used to take the position coordinates of the reference key point as the mean, and perform two-dimensional Gaussian distribution calculation based on the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to the reference key point; calculate the reference key degree corresponding to the pixel point in the reference image based on the Gaussian distribution information corresponding to the reference key point and the position coordinates of the pixel point in the reference image; and obtain the reference key degree information corresponding to the reference image based on the reference key degree corresponding to the pixel point in the reference image.
[0294] In one embodiment, the reference information determination module 1606 is also used to take the position coordinates of the registration key points as the mean, and perform two-dimensional Gaussian distribution calculation based on the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to the registration key points; calculate the registration criticality corresponding to the pixel points in the registration image based on the Gaussian distribution information corresponding to the registration key points and the position coordinates of the pixel points in the registration image; and obtain the registration criticality information corresponding to the registration image based on the registration criticality corresponding to the pixel points in the registration image.
[0295] For the specific limitations of the image quality assessment device and the image quality assessment model training device, please refer to the limitations of the image quality assessment method and the image quality assessment model training method in the above text, which will not be repeated here. The various modules in the above-mentioned image quality assessment device and image quality assessment model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0296] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.17As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image sample data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image quality assessment method and an image quality assessment model training method are implemented.
[0297] Those skilled in the art will understand that Fig.17 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0298] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.
[0299] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0300] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.
[0301] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0302] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0303] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A method for training an image registration quality assessment model, characterized in that: The method comprises: Acquire training image samples and target image registration quality assessment results, wherein the training image samples include a training reference image and corresponding target reference criticality information and a training registration image and corresponding target registration criticality information, wherein the training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image; Inputting the training reference image and the training registration image into an initial criticality information extraction network in an initial image registration quality assessment model to obtain training reference criticality information and training registration criticality information; Calculating reference criticality error information between the training reference criticality information and the corresponding target reference criticality information, and calculating registration criticality error information between the training registration criticality information and the corresponding target registration criticality information; Determining target criticality error information based on the reference criticality error information and the registration criticality error information, and updating the initial criticality information extraction network based on the target criticality error information; Inputting the training reference image, the training registration image, the training reference criticality information and the training registration criticality information into an initial image registration quality assessment network in an initial image registration quality assessment model to obtain an initial image registration quality assessment result; Calculating evaluation result error information between the initial image registration quality evaluation result and the target image registration quality evaluation result; The initial image registration quality assessment network is updated based on the assessment result error information, and when the training is completed, an image registration quality assessment model is obtained.
2. The method according to claim 1, characterized in that: In the step of obtaining the training image sample and the target image registration quality assessment result, the training image sample includes a training reference image and corresponding target reference critical information and a training registration image and corresponding target registration critical information, and the training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image, the step further includes: Acquire a pre-trained image sample, wherein the pre-trained image sample includes a pre-trained reference image and corresponding target pre-trained reference criticality information and a pre-trained registration image and corresponding target pre-trained registration criticality information; Inputting the pre-trained reference image and the pre-trained registration image into a pre-trained criticality information extraction network to obtain pre-trained reference criticality information and pre-trained registration criticality information; Calculating pre-trained reference criticality error information between the pre-trained reference criticality information and the corresponding target pre-trained reference criticality information, and calculating pre-trained registration criticality error information between the pre-trained registration criticality information and the corresponding target pre-trained registration criticality information; Determining target pre-trained criticality error information based on the pre-trained reference criticality error information and the pre-trained registration criticality error information, and updating the pre-trained criticality information extraction network based on the target pre-trained criticality error information; When the pre-training is completed, the pre-trained criticality information extraction network is used as the initial criticality information extraction network in the initial image registration quality assessment model.
3. The method according to claim 1, characterized in that: The obtaining of training image samples and target image registration quality assessment results, wherein the training image samples include training reference images and corresponding target reference criticality information and training registration images and corresponding target registration criticality information, includes: Acquire the training reference image and the corresponding reference key point labels and the training registration image and the corresponding registration key point labels; Calculating Gaussian distribution information corresponding to the reference key point label, and determining the target reference criticality information corresponding to the training reference image based on the Gaussian distribution information corresponding to the reference key point label and the position of the pixel point in the training reference image; Calculating Gaussian distribution information corresponding to the registration key point label, and determining the target registration criticality information corresponding to the training registration image based on the Gaussian distribution information corresponding to the registration key point label and the position of the pixel point in the training registration image; And calculate the key point label distance between the reference key point label and the registration key point label, and use the key point label distance as the target image registration quality evaluation result.
4. The method according to claim 3, characterized in that The reference key point labels include at least two; The calculating the Gaussian distribution information corresponding to the reference key point label, and determining the target reference criticality information corresponding to the training reference image based on the Gaussian distribution information corresponding to the reference key point label and the position of the pixel point in the training reference image, includes: Obtaining the position coordinates corresponding to each reference key point label, taking the position coordinates as the mean, and performing a two-dimensional Gaussian distribution calculation based on the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to each reference key point label; Calculate each reference key degree corresponding to the pixel point in the training reference image based on the Gaussian distribution information corresponding to each reference key point label and the position coordinates of the pixel point in the training reference image; A target reference criticality is selected from each reference criticality corresponding to the pixel point to obtain the target reference criticality corresponding to the pixel point in the training reference image, and the target reference criticality information corresponding to the training reference image is obtained based on the target reference criticality corresponding to the pixel point in the training reference image.
5. The method according to claim 3, characterized in that: The registration key point labels include at least two; The calculating the Gaussian distribution information corresponding to the registration key point label, and determining the target registration criticality information corresponding to the training registration image based on the Gaussian distribution information corresponding to the registration key point label and the position coordinates of the pixel points in the training registration image, includes: Obtaining the position coordinates corresponding to each registration key point label, taking the position coordinates as the mean, and performing a two-dimensional Gaussian distribution calculation based on the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to each registration key point label; Calculate each registration key degree corresponding to the pixel point in the training registration image based on the Gaussian distribution information corresponding to each registration key point label and the position coordinates of the pixel point in the training registration image; The target registration criticality is selected from the various registration criticalities corresponding to the pixel points to obtain the target registration criticality corresponding to the pixel points in the training registration image, and the target registration criticality information corresponding to the training registration image is obtained based on the target registration criticality corresponding to the pixel points in the training registration image.
6. The method according to claim 1, characterized in that The calculating reference criticality error information between the training reference criticality information and the corresponding target reference criticality information includes: Calculating the sum of squares of errors between the training reference keyness corresponding to each pixel point in the training reference keyness information and the target reference keyness corresponding to each pixel point in the target reference keyness information; The product of the width of the reference image and the height of the reference image is calculated, and the ratio of the error sum of squares to the product is calculated to obtain the reference criticality error information.
7. The method according to claim 1, characterized in that The calculating of the evaluation result error information between the initial image registration quality evaluation result and the target image registration quality evaluation result includes: The absolute difference between the initial image registration quality assessment result and the target image registration quality assessment result is calculated to obtain the assessment result error information.
8. A method for evaluating image registration quality, characterized in that: The method comprises: Acquire a reference image and a registration image, wherein the registration image is an image obtained by performing image registration on the image to be registered based on the reference image; Respectively extracting key points corresponding to the reference image and the registration image to obtain reference key points and registration key points; Calculating Gaussian distribution information corresponding to the reference key point, and determining reference criticality information corresponding to the reference image based on the Gaussian distribution information corresponding to the reference key point and the position of the pixel point in the reference image; Calculating Gaussian distribution information corresponding to the registration key points, and determining registration criticality information corresponding to the registration image based on the Gaussian distribution information corresponding to the registration key points and positions of pixels in the registration image; Based on the reference image, the registered image, the reference critical information and the registration critical information, image registration quality assessment is performed through a second image registration quality assessment model to obtain image registration quality assessment results corresponding to the reference image and the registered image. The second image registration quality assessment model is trained through a deep neural network algorithm using a training reference image, a training registered image, target reference critical information and target registration critical information.
9. The method according to claim 8, characterized in that Before obtaining the reference image and the registration image, the method further includes: The reference image and the image to be registered are obtained, and the reference image and the image to be registered are input into an image registration model to obtain an output registered image, wherein the image registration model is obtained by training using training samples based on a deep neural network algorithm.
10. The method according to claim 8, characterized in that The method further comprises: Inputting the reference image and the registered image into a first image registration quality assessment model, the first image registration quality assessment model uses a critical information extraction network to extract critical information from the reference image and the registered image, obtaining reference critical information corresponding to the reference image and registration critical information corresponding to the registered image, inputting the reference image, the registered image, the reference critical information and the registration critical information into an image registration quality assessment network, and obtaining image registration quality assessment results corresponding to the reference image and the registered image; The first image registration quality assessment model is trained using target criticality information and target image registration quality assessment results, wherein the target criticality information is determined based on Gaussian distribution information corresponding to the key point labels of the training image and the positions of the pixels in the training image, and the target image registration quality assessment results are obtained based on the distance between the key point labels corresponding to the training reference image and the key point labels corresponding to the training registration image.
11. The method according to claim 8, characterized in that The step of respectively extracting key points corresponding to the reference image and the registration image to obtain reference key points and registration key points includes: A key point detection model is used to perform key point detection on the reference image and the registered image to obtain reference key points corresponding to the reference image and registration key points corresponding to the registered image. The key point detection model is trained using a deep neural network algorithm based on key point training images and corresponding key point labels.
12. The method according to claim 8, characterized in that The performing image registration quality assessment based on the reference image, the registered image, the reference criticality information and the registration criticality information to obtain image registration quality assessment results corresponding to the reference image and the registered image includes: The reference image, the registered image, the reference critical information and the registration critical information are input into a second image registration quality assessment model to obtain the output image registration quality assessment results corresponding to the reference image and the registered image. The second image registration quality assessment model is trained using a deep neural network algorithm using a training reference image, a training registered image, target reference critical information and target registration critical information.
13. The method according to claim 8, characterized in that The calculating the Gaussian distribution information corresponding to the reference key point, and determining the reference criticality information corresponding to the reference image based on the Gaussian distribution information corresponding to the reference key point and the position of the pixel point in the reference image, includes: Taking the position coordinates of the reference key points as the mean, and performing two-dimensional Gaussian distribution calculation according to the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to the reference key points; Calculate the reference key degree corresponding to the pixel point in the reference image based on the Gaussian distribution information corresponding to the reference key point and the position coordinates of the pixel point in the reference image; Reference criticality information corresponding to the reference image is obtained based on the reference criticality corresponding to the pixel points in the reference image.
14. The method according to claim 8, characterized in that The calculating the Gaussian distribution information corresponding to the registration key point, and determining the registration criticality information corresponding to the registration image based on the Gaussian distribution information corresponding to the registration key point and the position of the pixel point in the registration image, includes: Taking the position coordinates of the key points of the registration as the mean, and performing two-dimensional Gaussian distribution calculation according to the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to the key points of the registration; Calculate the registration criticality corresponding to the pixel point in the registration image based on the Gaussian distribution information corresponding to the registration key point and the position coordinates of the pixel point in the registration image; The registration criticality information corresponding to the registered image is obtained based on the registration criticality corresponding to the pixel points in the registered image.
15. An image registration quality assessment model training device, characterized in that: The device comprises: A training sample acquisition module is used to acquire training image samples and target image registration quality assessment results, wherein the training image samples include a training reference image and corresponding target reference criticality information and a training registration image and corresponding target registration criticality information, wherein the training registration image is an image obtained by performing image registration on the training image to be registered based on the training reference image; A training information extraction module, used for inputting the training reference image and the training registration image into an initial criticality information extraction network in an initial image registration quality assessment model to obtain training reference criticality information and training registration criticality information; A criticality error calculation module, used to calculate reference criticality error information between the training reference criticality information and the corresponding target reference criticality information, and calculate registration criticality error information between the training registration criticality information and the corresponding target registration criticality information; An extraction network updating module, configured to determine target criticality error information based on the reference criticality error information and the registration criticality error information, and to update the initial criticality information extraction network based on the target criticality error information; An initial quality assessment module, used for inputting the training reference image, the training registration image, the training reference criticality information and the training registration criticality information into an initial image registration quality assessment network in an initial image registration quality assessment model to obtain an initial image registration quality assessment result; An evaluation error calculation module, used to calculate evaluation result error information between the initial image registration quality evaluation result and the target image registration quality evaluation result; The model acquisition module is used to update the initial image registration quality assessment network based on the assessment result error information, and obtain the image registration quality assessment model when the training is completed.
16. The device according to claim 15, characterized in that The device further comprises: A pre-training module is used to obtain pre-training image samples, wherein the pre-training image samples include a pre-training reference image and a corresponding target pre-training reference critical information and a pre-training registration image and a corresponding target pre-training registration critical information; input the pre-training reference image and the pre-training registration image into a pre-training critical information extraction network to obtain pre-training reference critical information and pre-training registration critical information; calculate pre-training reference critical error information between the pre-training reference critical information and the corresponding target pre-training reference critical information, and calculate pre-training registration critical error information between the pre-training registration critical information and the corresponding target pre-training registration critical information; determine the target pre-training critical error information based on the pre-training reference critical error information and the pre-training registration critical error information, and update the pre-training critical information extraction network based on the target pre-training critical error information; when pre-training is completed, use the pre-trained critical information extraction network as the initial critical information extraction network in the initial image registration quality assessment model.
17. The device according to claim 15, characterized in that The training sample acquisition module comprises: A label acquisition unit, used to acquire the training reference image and the corresponding reference key point labels and the training registration image and the corresponding registration key point labels; A reference information calculation unit, configured to calculate Gaussian distribution information corresponding to the reference key point label, and determine the target reference critical information corresponding to the training reference image based on the Gaussian distribution information corresponding to the reference key point label and the position of the pixel point in the training reference image; A registration information calculation unit, used to calculate Gaussian distribution information corresponding to the registration key point label, and determine the target registration criticality information corresponding to the training registration image based on the Gaussian distribution information corresponding to the registration key point label and the position of the pixel point in the training registration image; The result calculation unit is used to calculate the key point label distance between the reference key point label and the registration key point label, and use the key point label distance as the target image registration quality evaluation result.
18. The device according to claim 17, characterized in that The reference key point labels include at least two; The reference information calculation unit is further used to obtain the position coordinates corresponding to each reference key point label, take the position coordinates as the mean, and perform two-dimensional Gaussian distribution calculation according to the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to each reference key point label; calculate each reference key degree corresponding to the pixel point in the training reference image based on the Gaussian distribution information corresponding to each reference key point label and the position coordinates of the pixel point in the training reference image; A target reference criticality is selected from each reference criticality corresponding to the pixel point to obtain the target reference criticality corresponding to the pixel point in the training reference image, and the target reference criticality information corresponding to the training reference image is obtained based on the target reference criticality corresponding to the pixel point in the training reference image.
19. The device according to claim 17, characterized in that The registration key point labels include at least two; The registration information calculation unit is further used to obtain the position coordinates corresponding to each registration key point label, take the position coordinates as the mean, and perform two-dimensional Gaussian distribution calculation according to the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to each registration key point label; calculate each registration key degree corresponding to the pixel point in the training registration image based on the Gaussian distribution information corresponding to each registration key point label and the position coordinates of the pixel point in the training registration image; The target registration criticality is selected from the various registration criticalities corresponding to the pixel points to obtain the target registration criticality corresponding to the pixel points in the training registration image, and the target registration criticality information corresponding to the training registration image is obtained based on the target registration criticality corresponding to the pixel points in the training registration image.
20. The device according to claim 15, characterized in that The key error calculation module is also used to calculate the sum of square errors between the training reference key corresponding to each pixel point in the training reference key information and the target reference key corresponding to each pixel point in the target reference key information; calculate the product of the width of the reference image and the height of the reference image, and calculate the ratio of the sum of square errors to the product to obtain the reference key error information.
21. The device according to claim 15, characterized in that The evaluation error calculation module is also used to calculate the absolute difference between the initial image registration quality evaluation result and the target image registration quality evaluation result to obtain the evaluation result error information.
22. An image registration quality assessment device, characterized in that: The device comprises: An image acquisition module, used to acquire a reference image and a registration image, wherein the registration image is an image obtained by performing image registration on the image to be registered based on the reference image; A key point extraction module, used to extract key points corresponding to the reference image and the registration image respectively, to obtain reference key points and registration key points; A reference information determination module, used to calculate Gaussian distribution information corresponding to the reference key point, and determine reference critical information corresponding to the reference image based on the Gaussian distribution information corresponding to the reference key point and the position of the pixel point in the reference image; A registration information determination module, used to calculate Gaussian distribution information corresponding to the registration key points, and determine the registration criticality information corresponding to the registration image based on the Gaussian distribution information corresponding to the registration key points and the positions of the pixels in the registration image; A quality assessment module is used to perform image registration quality assessment based on the reference image, the registered image, the reference critical information and the registration critical information through a second image registration quality assessment model to obtain image registration quality assessment results corresponding to the reference image and the registered image, wherein the second image registration quality assessment model is obtained by training with a deep neural network algorithm using a training reference image, a training registered image, target reference critical information and target registration critical information.
23. The device according to claim 22, characterized in that The device further comprises: The registration module is used to obtain the reference image and the image to be registered, input the reference image and the image to be registered into an image registration model to obtain an output registered image, and the image registration model is trained based on a deep neural network algorithm using training samples.
24. The device according to claim 22, characterized in that The device further comprises: A model evaluation module is used to input the reference image and the registered image into a first image registration quality evaluation model, the first image registration quality evaluation model uses a critical information extraction network to extract critical information from the reference image and the registered image to obtain reference critical information corresponding to the reference image and registration critical information corresponding to the registered image, and the reference image, the registered image, the reference critical information and the registration critical information are input into the image registration quality evaluation network to obtain image registration quality evaluation results corresponding to the reference image and the registered image; the first image registration quality evaluation model is trained using target critical information and target image registration quality evaluation results, the target critical information is determined based on Gaussian distribution information corresponding to key point labels of training images and positions of pixels in training images, and the target image registration quality evaluation results are obtained based on the distance between key point labels corresponding to training reference images and key point labels corresponding to training registered images.
25. The device according to claim 22, characterized in that The key point extraction module is also used to perform key point detection on the reference image and the registered image using a key point detection model to obtain reference key points corresponding to the reference image and registration key points corresponding to the registered image. The key point detection model is trained using a deep neural network algorithm based on key point training images and corresponding key point labels.
26. The device according to claim 22, characterized in that The quality assessment module is also used to input the reference image, the registered image, the reference critical information and the registration critical information into a second image registration quality assessment model to obtain the output image registration quality assessment results corresponding to the reference image and the registered image. The second image registration quality assessment model is trained using a deep neural network algorithm using a training reference image, a training registered image, target reference critical information and target registration critical information.
27. The device according to claim 22, characterized in that The reference information determination module is further used to take the position coordinates of the reference key points as the mean, and perform two-dimensional Gaussian distribution calculation according to the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to the reference key points; Calculate the reference key degree corresponding to the pixel point in the reference image based on the Gaussian distribution information corresponding to the reference key point and the position coordinates of the pixel point in the reference image; Reference criticality information corresponding to the reference image is obtained based on the reference criticality corresponding to the pixel points in the reference image.
28. The device according to claim 22, characterized in that The reference information determination module is further used to take the position coordinates of the registration key points as the mean, and perform two-dimensional Gaussian distribution calculation according to the mean and a preset covariance matrix to obtain Gaussian distribution information corresponding to the registration key points; Calculate the registration criticality corresponding to the pixel point in the registration image based on the Gaussian distribution information corresponding to the registration key point and the position coordinates of the pixel point in the registration image; The registration criticality information corresponding to the registered image is obtained based on the registration criticality corresponding to the pixel points in the registered image.
29. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 14 are implemented.
30. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented.
31. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented.
