Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium, the method is applied to the field of image processing, and the method comprises the steps: inputting an original image and a template image into a preset image registration network for image registration, and obtaining a first registration image corresponding to the original image; if the spatial similarity of the first registration image and the template image is smaller than a preset similarity threshold value, obtaining a cross-correlation loss value and a consistency loss value of the first registration image and the template image; optimizing the image registration network based on the cross-correlation loss value and the consistency loss value; and performing image registration on the first registration image based on the optimized image registration network to obtain a second registration image of the original image. According to the scheme, the image registration precision of the image registration network is improved, and it is ensured that the obtained registration image can meet actual application requirements.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to image processing methods, devices, electronic devices, and storage media. Background Art

[0002] In related technologies, deep learning-based image processing networks only use a single loss function to optimize the parameters of the image processing network when registering images, but do not optimize the loss function based on the image spatial structure information. As a result, the image processing network cannot ensure the global and local structural consistency of image registration, resulting in technical problems such as local image mismatch or structural mismatch. Summary of the Invention

[0003] This application provides an image processing method, device, electronic device, and storage medium, aiming to improve the image registration accuracy of an image registration network. The technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0005] Inputting the original image and the template image into a preset image registration network for image registration to obtain a first registered image corresponding to the original image;

[0006] If the spatial similarity between the first registered image and the template image is less than a preset similarity threshold, obtaining a cross-correlation loss value and a consistency loss value between the first registered image and the template image;

[0007] Optimize the image registration network based on the cross-correlation loss value and consistency loss value;

[0008] The first registered image is registered based on the optimized image registration network to obtain a second registered image of the original image.

[0009] In a second aspect, an embodiment of the present application provides an image processing device, comprising:

[0010] An image registration unit, configured to input the original image and the template image into a preset image registration network for image registration, and obtain a first registered image corresponding to the original image;

[0011] a loss calculation unit, configured to obtain a mutual correlation loss value and a consistency loss value between the first registered image and the template image if the spatial similarity between the first registered image and the template image is less than a preset similarity threshold;

[0012] A network optimization unit is used to optimize the image registration network based on the cross-correlation loss value and the consistency loss value;

[0013] The network calling unit is used to perform image registration on the first registered image based on the optimized image registration network to obtain a second registered image of the original image.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, any of the above image processing methods is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, any of the above image processing methods is implemented.

[0016] In the above technical solution, the original image and the template image are initially registered through the image registration network to generate a first registered image. The spatial similarity between the first registered image and the template image is then calculated. If the similarity does not reach the preset threshold, the mutual correlation loss value and consistency loss value of the first registered image and the template image are further calculated. The parameters of the registration network are optimized and adjusted for the first registered image and the template image. The optimized network again performs refined registration processing on the first registered image and outputs a second registered image with higher accuracy. The entire registration process is continued in this iterative optimization manner. Each iteration optimizes the network performance based on the previous registration results and loss calculation, thereby gradually improving the image registration accuracy of the image registration network, and ultimately ensuring that the obtained registered image can meet the needs of actual applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is a schematic diagram of a scenario of an image processing method provided in an embodiment of the present application;

[0019] Figure 2 This is a flowchart of an image processing method provided by an embodiment of the present application;

[0020] Figure 3 This is a flowchart of an image processing method provided by an embodiment of the present application;

[0021] Figure 4 This is a flowchart of an image processing method provided by an embodiment of the present application;

[0022] Figure 5 This is a flowchart of an image processing method provided by an embodiment of the present application;

[0023] Figure 6 This is a flowchart of an image processing method provided by an embodiment of the present application;

[0024] Figure 7 This is a flowchart of an image processing method provided by an embodiment of the present application;

[0025] Figure 8 This is a schematic diagram of a scenario of an image processing method provided in an embodiment of the present application;

[0026] Figure 9 is a structural diagram of an image processing device provided in an embodiment of the present application;

[0027] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] To make the features and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0029] The following will clearly and thoroughly describe the technical solutions in this application in conjunction with the accompanying drawings. In the description of the embodiments of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more than two.

[0030] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0031] The embodiment of the present application provides an image processing method, the execution subject of which is an image processing device or an electronic device with an image processing device. The following is a detailed description. It should be noted that the description order of the following embodiments does not limit the preferred order of the embodiments. Figure 1 , Figure 1 Schematic diagram of a scene of an image processing method provided in an embodiment of the present application. The specific process of the image processing method can be as follows:

[0032] See also Figure 1 , Figure 1 This is a scene diagram of an image processing method provided by an embodiment of the present application. Figure 1 As shown, the original image and template image are images manually selected by the user. The original image and template image are respectively input into the image registration network for network registration, wherein the template image refers to the reference image used as the spatial alignment target when aligning with the original image. The features extracted by the image registration network and the generated spatial transformation matrix guide the original image to adjust its geometric structure. The image registration network is a neural network based on deep learning. The image registration network takes the original image and template image manually selected by the user as input, automatically extracts multi-scale features of the original image and template image to generate the optimal spatial transformation matrix. The spatial transformation matrix is ​​used to guide the original image to perform spatial transformation so as to align the original image with the template image at the voxel level. Compared with traditional methods, the image registration network can efficiently handle images with complex deformations and adapt to different modalities.

[0033] In actual scenarios, there are scenarios where it is not possible to match the spatial features of the original image with the spatial features of the template image through a single registration. Therefore, after spatially transforming the pixels of the original image based on the spatial transformation matrix, it is necessary to calculate the spatial similarity between the first registered image corresponding to the original image and the template image. If it is detected that the spatial similarity between the first registered image and the template image is less than the preset similarity threshold, it indicates that the network parameters in the image registration network need to be further optimized.

[0034] To ensure the optimization of the image registration network, it is necessary to obtain the cross-correlation loss and consistency loss between the first registered image and the template image. The cross-correlation loss represents the global correlation between the template and the first registered image, reflecting the degree of spatial alignment between the two images. The consistency loss indicates the geometric matching of the three-dimensional structures of the template and the first registered image. A pre-set joint loss function is then used to perform a weighted fusion of the cross-correlation and consistency losses of the first registered image and the template image to obtain a joint loss. Based on the backpropagation mechanism, automatic differentiation is used to calculate the gradient of the joint loss with respect to the weights and biases of the image registration network. The contribution of each loss term to the network parameter update is adjusted according to the pre-set weight coefficients. The optimizer is then called to update the network parameters of the image registration network based on the gradients. The learning rate is used to control the parameter adjustment amplitude and simultaneously adjust the weights and biases to minimize the total loss.

[0035] After optimizing the image registration network, the first registered image is used as the original image and the image registration process is repeated through the optimized image registration network to obtain a second registered image. If the spatial similarity between the second registered image and the original image is greater than or equal to a preset similarity threshold, the second registered image is output to the user. If the spatial similarity between the second registered image and the original image is still less than the preset similarity threshold, the image registration network optimization step is executed again, and the image registration process continues with the second registered image as the original image.

[0036] In an embodiment of the present application, the original image and the template image are initially registered through the image registration network to generate a first registered image. The spatial similarity between the first registered image and the template image is then calculated. If the similarity does not reach the preset threshold, the mutual correlation loss value and consistency loss value of the first registered image and the template image are further calculated. The first registered image and the template image are used to optimize and adjust the parameters of the registration network. The optimized network again performs a refined registration process on the first registered image and outputs a second registered image with higher accuracy. The entire registration process is continued in this iterative optimization manner. Each iteration optimizes the network performance based on the previous registration result and loss calculation, thereby gradually improving the image registration accuracy of the image registration network, and ultimately ensuring that the obtained registered image can meet the needs of actual applications.

[0037] based on Figure 1 The scene diagram shown below will be combined with Figure 2-Figure 8 , an image processing method provided in an embodiment of the present application is introduced in detail.

[0038] Based on the above situation, the present application embodiment proposes an image processing method. Figure 2 , Figure 3 This is a flow chart of an image processing method provided by an embodiment of the present application. Figure 2 As shown, the method of the embodiment of the present application may include the following steps S101 to S104.

[0039] S101: Input the original image and the template image into a preset image registration network for image registration to obtain a first registered image corresponding to the original image.

[0040] Specifically, the original image and template image are images manually selected by the user. Before inputting the original image into the image registration network, the original image needs to be brightness normalized to standardize the brightness distribution of the original image. Furthermore, the original image is tested to see if the spacing between each voxel meets a preset spacing value. If not, the original image needs to be resampled to ensure that the voxel spacing of the original image meets the preset spacing value. Furthermore, the original image size is tested to see if it meets the preset size. If not, the original image needs to be cropped to the preset size.

[0041] The cropped original image is input into the image registration network, where the image registration network is a neural network based on deep learning. The image registration network takes the original image and the template image as input, automatically extracts the multi-scale features of the original image and the template image to generate the optimal spatial transformation matrix.

[0042] It should be noted that the spatial transformation matrix is ​​used to define the geometric transformation relationship of each voxel in the original image in space, and adjust the structure of the original image through linear or nonlinear mapping, so that the original image and the template image are matched in position in the three-dimensional voxel space; in the registration process, the spatial transformation matrix controls the direction and amplitude of the deformation of the original image, and repositions each voxel of the original image to the transformed coordinate position through interpolation calculation, thereby determining the first registered image after registration.

[0043] S102: If the spatial similarity between the first registered image and the template image is less than a preset similarity threshold, a mutual correlation loss value and a consistency loss value between the first registered image and the template image are obtained.

[0044] In the embodiment of the present application, spatial similarity refers to an indicator that measures the consistency of the first registered image and the template image in the spatial dimension by quantifying the degree of matching between the first registered image and the template image in terms of geometric structure, feature distribution or pixel arrangement.

[0045] Specifically, the spatial features of the first registered image and the template image are extracted respectively, and a similarity index is generated by using a structural similarity algorithm or a geometric matching method. The spatial alignment degree of the first registered image and the template image is comprehensively evaluated by using parameters such as statistical feature matching, overlapping area ratio or spatial distribution difference, and finally the spatial similarity is output. Among them, the higher the spatial similarity, the better the matching of the geometric structure and content distribution of the first registered image and the template image.

[0046] If it is detected that the spatial similarity between the first registered image and the template image is less than the preset similarity threshold, it is necessary to obtain the cross-correlation loss value and consistency loss value of the first registered image and the template image based on the preset loss function, wherein the cross-correlation loss value is used to represent the global correlation between the template image and the first registered image, reflecting the degree of spatial alignment between the template image and the first registered image, and the consistency loss value represents the geometric matching degree of the structure of the template image and the first registered image in three-dimensional space.

[0047] S103, optimizing the image registration network based on the cross-correlation loss value and the consistency loss value.

[0048] Specifically, a preset joint loss function is called to perform a weighted fusion of the cross-correlation loss and consistency loss of the first registered image and the template image to obtain a joint loss. Furthermore, based on the backpropagation mechanism, a preset differential tool is called to calculate the gradient of the joint loss with respect to the weights and biases of the image registration network. The contribution of each loss term to the network parameter update is adjusted based on the preset weight coefficient. The optimizer is further called to update the network parameters of the image registration network based on the gradient. The learning rate is used to control the parameter adjustment amplitude, and the weights and biases of the image registration network are simultaneously corrected to minimize the total loss.

[0049] S104: Perform image registration on the first registered image based on the optimized image registration network to obtain a second registered image.

[0050] Specifically, after optimizing the image registration network, the first registered image is used as a new input to the image registration network, and the registration process is re-executed through the optimized image registration network to generate a second registered image. At this time, the spatial similarity between the second registered image and the original image needs to be evaluated: if the spatial similarity between the second registered image and the original image is greater than or equal to a preset similarity threshold, then the registration result of the second registered image is determined to meet the user's actual requirements, and the second registered image is directly output for the user to review in a timely manner; if the spatial similarity between the second registered image and the original image is less than the preset similarity threshold, then the second registered image is used as a new input image, and the optimization step of the image registration network is returned to optimize the network parameters of the image registration network again. Based on the optimized image registration network, the registration process for the second registered image is re-executed. This iterative mechanism can obtain a registered image that is consistent with the template image in spatial distribution by iteratively optimizing the network parameters of the image registration network.

[0051] In an embodiment of the present application, the original image and the template image are initially registered through the image registration network to generate a first registered image. The spatial similarity between the first registered image and the template image is then calculated. If the similarity does not reach the preset threshold, the mutual correlation loss value and consistency loss value of the first registered image and the template image are further calculated. The first registered image and the template image are used to optimize and adjust the parameters of the registration network. The optimized network again performs a refined registration process on the first registered image and outputs a second registered image with higher accuracy. The entire registration process is continued in this iterative optimization manner. Each iteration optimizes the network performance based on the previous registration result and loss calculation, thereby gradually improving the image registration accuracy of the image registration network, and ultimately ensuring that the obtained registered image can meet the needs of actual applications.

[0052] To improve the image processing speed. Figure 3 , Figure 3 This is a flow chart of an image processing method provided by an embodiment of the present application. Figure 3 As shown, the method of the embodiment of the present application may include the following steps S201-S202.

[0053] S201 , obtaining spatial difference features of an original image and a template image, and determining a spatial transformation matrix for the original image based on the spatial difference features.

[0054] Specifically, the geometric features or pixel distribution differences between the original image and the template image are compared to extract spatial difference features. This process involves detecting key points in the original and template images, such as edges, corners, or texture features, calculating the positional offset or shape change of these key points, or quantifying the grayscale distribution differences between the images through methods such as mutual information and normalized cross-correlation. Based on the extracted spatial difference features, an optimization algorithm is used to calculate the optimal spatial transformation parameters, ultimately generating a spatial transformation matrix for correcting the original image.

[0055] S202: Perform spatial transformation on the original image based on the spatial transformation matrix to obtain a first registered image.

[0056] Specifically, each pixel of the original image is traversed, the new coordinate position of each pixel is calculated according to the matrix parameters of the transformation matrix, and each pixel is moved from the original coordinate position to the new coordinate position, thereby obtaining the first registered image.

[0057] In this application, the optimal spatial transformation matrix is ​​automatically generated by analyzing the spatial differences between the original image and the template image, achieving precise geometric correction of the original image. By combining feature matching with spatial transformation, it is possible to adaptively handle rotation, scaling, and nonlinear deformation without manual intervention, significantly improving the accuracy and efficiency of image registration.

[0058] Because traditional methods cannot simultaneously take into account the accuracy of local feature matching and the rationality of global structure, the registration results are prone to local alignment but overall distortion. Therefore, it is necessary to provide a design method for the loss function that takes into account both local and global considerations. Figure 4 , Figure 4 This is a flow chart of an image processing method provided by an embodiment of the present application. Figure 4 As shown, the method of the embodiment of the present application may include the following steps S301-S302.

[0059] S301, calling a preset joint loss function, and obtaining a joint loss value based on the cross-correlation loss value and the consistency loss value.

[0060] In the embodiment of the present application, the specific formula of the joint loss function is expressed as follows:

[0061] GLSC(I,J)=α·GCC(I,J)+β·3DSC(I,j)

[0062] Where I is the template image, j is the first registered image, α and β are preset weight coefficients, GCC(I,J) represents the cross-correlation loss between the first registered image and the template image, and 3DSC(I,J) represents the consistency loss between the first registered image and the template image. The cross-correlation loss represents the global correlation between the template image and the first registered image, reflecting the degree of spatial alignment between the template image and the first registered image. The consistency loss indicates the geometric match between the template image and the first registered image in three-dimensional space.

[0063] Specifically, the joint loss GLSC(I,J) is obtained by multiplying the cross-correlation loss GCC(I,J) and the consistency loss 3DSC(I,J) between the template image I and the registered image J by the weight coefficients α and β respectively and then adding them together.

[0064] S302: performing gradient updates on the weights and biases of the image registration network based on the joint loss value.

[0065] Specifically, after obtaining the joint loss value based on step S301 above, the gradient of the joint loss value with respect to the weights and biases of the image registration network is calculated using an automatic differentiation tool based on the backpropagation mechanism. The contribution of each loss term to the parameter update is adjusted based on the weight coefficients α and β. An optimizer, such as Adam, is further used to update the network parameters of the image registration network based on the gradient. The learning rate is used to control the parameter adjustment amplitude, and the weights and biases are simultaneously corrected to minimize the total loss. This process is repeated iteratively to gradually adapt the image registration network to the task requirements.

[0066] In an embodiment of the present application, by fusing the cross-correlation loss and consistency loss into a joint loss function, it is possible to simultaneously optimize the local similarity and global structural consistency of image registration. The cross-correlation loss enhances the robustness of feature matching and reduces the interference of local deformations; the consistency loss constrains the smoothness of the transformation and avoids excessive distortion. The synergistic effect of the two enables the network to converge more stably during training and improves registration accuracy. The gradient update mechanism ensures that the model can adaptively adjust weights and biases so that the registration results meet reasonable topological constraints while maintaining local details.

[0067] Due to the global semantic inconsistency problem caused by the dominance of local features in existing methods in segmentation or detection tasks, it is necessary to improve network performance through long-range contextual dependencies. Figure 5 , Figure 5 This is a flow chart of an image processing method provided by an embodiment of the present application. Figure 5 As shown, the method of the embodiment of the present application may include the following steps S401-S402.

[0068] S401 , obtaining a first partial image of a first registration image and a second partial image of a template image.

[0069] Specifically, the original image and the template image are divided into a plurality of preset numbers of local sub-images, wherein the first local image is any local sub-image of the original image, and the second local image is any local sub-image of the template image.

[0070] S402 , calling a preset loss function to determine the similarity between the first partial image and the second partial image, and determining a cross-correlation loss value between the first registered image and the template image based on the similarity.

[0071] In the embodiment of the present application, the preset loss function is the global normalized cross-correlation loss function (Global Correlation Consistency Loss, GCC Loss) and the specific function form is:

[0072]

[0073] Where I is the template image, J is the first registered image, E[] represents the mathematical expectation of the image voxels, which can be set according to actual needs, and ε is a very small constant with a value range of 10 -10 ~10 -5 , preferably 10 in the embodiment of the present application -8 as the value of ε.

[0074] Specifically, the covariance of the pixel values ​​of the first and second partial images is calculated, where the covariance reflects the synchronization of pixel changes. The standard deviation of the pixels of the first and second partial images is then calculated, where the standard deviation reflects the degree of dispersion of the pixel distribution. The covariance is divided by the product of the standard deviations and a constant ε is added to obtain a cross-correlation loss value between the first and second partial images. The cross-correlation loss value ranges from 0 to 1. The closer the cross-correlation loss value is to 1, the higher the similarity between the first registered image and the template image. The closer the cross-correlation loss value is to 0, the lower the similarity between the first registered image and the template image.

[0075] The above process is applied to the remaining local sub-images of the original image and the template image to determine the cross-correlation loss value of each local sub-image. The obtained cross-correlation loss values ​​are averaged to obtain the overall cross-correlation loss value.

[0076] In an embodiment of the present application, by extracting local areas of the first registration image and the template image and calculating their similarity, the method can finely evaluate the local alignment effect between images, thereby generating a more accurate cross-correlation loss value; this local similarity measurement not only enhances the sensitivity to small deformations and detail differences, but also effectively suppresses global noise interference, so that the registration network pays more attention to the matching accuracy of key areas during the optimization process, and ultimately improves the local consistency and structural fidelity of the overall registration results, which is particularly suitable for image registration tasks with complex local deformations or non-uniform texture features.

[0077] Since traditional registration methods rely on grayscale consistency, the same anatomical structure in different modalities may present completely different intensity distributions, resulting in registration failure. Therefore, how to robustly align images from different modalities or imaging conditions without relying on direct matching of pixel or voxel intensities has become a technical problem that needs to be solved. Figure 6 , Figure 6 This is a flow chart of an image processing method provided by an embodiment of the present application. Figure 6 As shown, the method of the embodiment of the present application may include the following steps S501-S503.

[0078] S501 : Acquire a first gradient value set of a first registration image and a second gradient value set of a template image.

[0079] Specifically, a three-dimensional edge extraction algorithm is called to calculate the gradient values ​​of the first registered image and the template image along the three directions of the three-dimensional coordinate system X, Y, and Z, respectively, where the gradient values ​​in the three directions of X, Y, and Z represent the local structural features of the first registered image and the template image respectively.

[0080] The first gradient value set of the first registered image is obtained by integrating the gradient values ​​of the first registered image in the X, Y, and Z directions, and the second gradient value set is obtained by integrating the gradient values ​​of the template image in the X, Y, and Z directions.

[0081] It should be noted that the three-dimensional edge extraction algorithm used in the embodiment of the present application may include the Sobel operator (Sobel), the Roberts Cross Operator (Prewitt) and the Canny Edge Detector (Canny). The specific three-dimensional edge extraction algorithm used can be set according to the actual application scenario and is not specifically limited here.

[0082] S502 : Perform difference calculation on the gradient values ​​in the same preset coordinate direction in the first gradient value set and the second gradient value set to obtain a gradient loss value set.

[0083] Specifically, the first gradient value set and the second gradient value set are traversed separately. The gradient values ​​in the X direction in the first gradient value set are calculated as the difference with the gradient values ​​in the X direction in the second gradient value set. The gradient values ​​in the Y direction in the first gradient value set are calculated as the difference with the gradient values ​​in the Y direction in the second gradient value set. The gradient values ​​in the Z direction in the first gradient value set are calculated as the difference with the gradient values ​​in the Z direction in the second gradient value set. The gradient differences in the X, Y, and Z directions are integrated to obtain a gradient loss value set.

[0084] For example, the first gradient value set is {X=0.5, Y=0.3, Z=0.2}, and the second gradient value set is {X=0.4, Y=0.2, Z=0.1}. The difference between the first and second gradient value sets is calculated: 0.5-0.4=0.1 in the X direction; 0.3-0.2=0.1 in the Y direction; and 0.2-0.1=0.1 in the Z direction, ultimately generating the gradient loss value set {0.1, 0.1, 0.1}.

[0085] S503 , summing up the gradient differences in the gradient loss value set to obtain a consistency loss value between the first registered image and the template image.

[0086] Specifically, the gradient loss value set is substituted into the preset three-dimensional spatial structure consistency loss function (3DStructural Consistency Loss, 3DSC Loss) to obtain the consistency loss value of the first registered image and the template image, as follows:

[0087]

[0088] in, and They represent the gradient values ​​of the template image and the first registered image in direction d, I represents the template image, and J represents the first registered image.

[0089] In an embodiment of the present application, by calculating the gradient difference between the first registered image and the template image in the same coordinate direction and summing them to obtain the consistency loss value, this method can effectively capture the structural differences between images and ensure that the registered image maintains geometric consistency in high-frequency features such as edges and textures; this gradient-based constraint not only enhances the ability to maintain the local structure of the image during the registration process, but also avoids artifacts or distortion caused by excessive deformation, thereby improving the rationality of the registration results.

[0090] To improve the processing speed of the image registration network. Figure 7 , Figure 7 This is a flow chart of an image processing method provided by an embodiment of the present application. Figure 7 As shown, the method of the embodiment of the present application may include the following steps S601-S604.

[0091] S601, obtaining the pixel mean and pixel standard deviation of the original image.

[0092] Specifically, all pixels of the original image are traversed and the arithmetic mean and standard deviation of the grayscale value of each pixel are calculated, where the mean reflects the overall brightness level of the original image and the standard deviation measures the degree of discreteness of the pixel value of each pixel in the original image.

[0093] For example, the mean value of the original image is 120, for example, the standard deviation is 30, indicating that the original image is brighter overall and has a moderate contrast; if the standard deviation is lower, for example, the standard deviation is 10, it indicates that the dynamic range of the original image is narrow.

[0094] S602: Perform brightness normalization processing on the original image based on the pixel mean and pixel standard deviation.

[0095] In the embodiment of the present application, the pixel mean and standard deviation calculated in S501 are used to perform global adjustment on the original image so that its brightness distribution tends to be standardized.

[0096] Specifically, a statistical normalization method is used to make the overall brightness of the adjusted original image concentrated in a preset brightness range while keeping the relative contrast unchanged.

[0097] For example, if the original image has a dark grayscale distribution and low contrast, normalization will stretch or compress the original image's brightness range to more closely match the statistical characteristics of the template image. This process helps subsequent similarity metrics more accurately assess the degree of match between images and avoid registration errors caused by inconsistent brightness.

[0098] S603: resample the original image to adjust the voxel spacing of the original image to a preset spacing.

[0099] Specifically, the current voxel spacing of the original image (i.e., the actual physical size of each voxel in 3D space) is determined by reading the metadata of the original image or by using the correspondence between the image size and the physical size of the original image. Based on the target voxel spacing set by the user, the required scaling ratio for resampling is calculated.

[0100] Based on a preset interpolation method, a new voxel value is generated by weighted averaging neighboring voxels. Based on this new voxel value, the original image is spatially transformed. This involves reallocating the positions of the voxels in the original image and calculating the value of each new voxel. During this process, the image's orientation must be consistent with the original data to avoid errors introduced by rotation or flipping. After resampling, the voxel spacing of the output original image is verified to meet the preset value, and the image quality is checked to ensure there is no excessive blur or artifacts. The final result is the original image with adjusted voxel spacing.

[0101] For example, if the original pitch of the original image is 2 mm×2 mm×2 mm, the adjusted target pitch is 2.0833 mm×2.0690 mm×2.0000 mm.

[0102] S604: Crop the resampled original image to a preset size.

[0103] In the embodiment of the present application, the preset size can be set according to the actual scene requirements and is not specifically limited here.

[0104] Specifically, the 3D initial size of the original image is compared with the 3D preset size. If it is detected that the 3D initial size of the original image is larger than the 3D preset size, the original image is cropped from the 3D initial size to the 3D preset size.

[0105] For example, the original image has an initial 3D size of 256×256×256, while the preset 3D size is 128×128×128. In this case, 64 layers are cut from the center to both sides along each dimension (i.e., 128 layers are retained in the center area), and the original image is finally cropped from 256×256×256 to 128×128×128.

[0106] In an embodiment of the present application, by resampling the original image so that the voxel spacing is adjusted to a preset standard value, this step can effectively eliminate the spatial resolution differences between different image data, thereby improving the accuracy and reliability of the registration. Due to the different image acquisition equipment and parameters, the physical size of the voxels of the original data often differ significantly. If the registration is performed directly, these resolution differences will cause deviations in spatial alignment, affecting the accuracy of subsequent analysis. Resampling unifies the image to the same voxel spacing through interpolation or downsampling methods to ensure that all data are registered at a consistent scale. This not only optimizes the stability of the registration algorithm, but also reduces the errors introduced by resolution mismatch, making the calculation process of multimodal image fusion and deformation registration more efficient and accurate. In addition, the standardized voxel spacing helps to reduce computational complexity, improve the convergence speed of the algorithm, and provide a more reliable basis for subsequent quantitative analysis and clinical applications.

[0107] In a feasible embodiment, the method of the embodiment of the present application may include: if the spatial similarity between the second registered image and the template image is greater than or equal to a similarity threshold, then determining the second registered image as the target registered image for response; if the spatial similarity between the second registered image and the template image is less than the similarity threshold, then executing the step of optimizing the image registration network.

[0108] Specifically, after optimizing the image registration network, the first registered image is used as the original image, and the image registration process is executed again through the optimized image registration network to obtain a second registered image. If it is detected that the spatial similarity between the second registered image and the original image is greater than or equal to the preset similarity threshold, the second registered image is output to the user. If the spatial similarity between the second registered image and the original image is still less than the preset similarity threshold, the optimization step of the image registration network is executed, and the second registered image is used as the original image to continue the above-mentioned image registration and the steps of optimizing the image registration network. For the steps of image registration and optimizing the image registration network, please refer to the above-mentioned steps S101-S104, which will not be repeated here.

[0109] Please also refer to Figure 8 , Figure 8 This is a flow chart of an image processing method provided by an embodiment of the present application. Figure 8 As shown, there is a deviation in the spatial distribution between the first registered image and the template image, that is, the first registered image is represented obliquely, while the template image is represented positively. The second registered image is obtained by taking the first registered image as the original image and registering it through the optimized image registration network. At this time, the second registered image presents a positive representation consistent with the template image, and it is considered that the second registered image meets the actual needs of the user.

[0110] In an embodiment of the present application, the image registration network is optimized through iterative optimization. Each iteration optimizes the network performance based on the previous registration result and loss calculation, thereby gradually improving the image registration accuracy of the image registration network, and ultimately ensuring that the obtained registered image can meet the actual application needs.

[0111] based on Figure 1 The following is a schematic diagram of the scene. Figure 9 , the image processing device provided by the embodiment of the present application is introduced in detail. It should be noted that, Figure 9 The image processing device in the present application is used to execute Figure 2-Figure 8 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 2-Figure 8 In the embodiment shown, the image processing apparatus 700 may include an image registration unit 701, a loss calculation unit 702, a network optimization unit 703, and a network calling unit 703, specifically as follows:

[0112] An image registration unit 701 is configured to input the original image and the template image into a preset image registration network for image registration to obtain a first registered image corresponding to the original image;

[0113] a loss calculation unit 702 for obtaining a mutual correlation loss value and a consistency loss value between the first registered image and the template image if the spatial similarity between the first registered image and the template image is less than a preset similarity threshold;

[0114] A network optimization unit 703 is used to optimize the image registration network based on the cross-correlation loss value and the consistency loss value;

[0115] The network calling unit 703 is configured to perform image registration on the first registered image based on the optimized image registration network to obtain a second registered image of the original image.

[0116] In some embodiments, the loss calculation unit 702 further includes a matrix acquisition unit and a space transformation unit.

[0117] a matrix acquisition unit, configured to acquire spatial difference features between the original image and the template image, and determine a spatial transformation matrix for the original image based on the spatial difference features;

[0118] The spatial transformation unit is used to perform spatial transformation on the original image based on the spatial transformation matrix to obtain a first registered image.

[0119] In some embodiments, the network optimization unit 703 further includes a function calling unit and a network parameter updating unit.

[0120] A function calling unit, configured to call a preset joint loss function and obtain a joint loss value based on the cross-correlation loss value and the consistency loss value;

[0121] The network parameter update unit is used to perform gradient updates on the weights and biases of the image registration network based on the joint loss value.

[0122] In some embodiments, the loss calculation unit 702 further includes a first loss calculation unit and a second loss calculation unit.

[0123] a first loss calculation unit, configured to obtain a similarity difference value between the first registered image and the template image, and perform normalization processing on the similarity difference value to obtain a cross-correlation loss value between the first registered image and the template image;

[0124] The second loss calculation unit is used to obtain the gradient loss value of the first registered image and the template image, and obtain the consistency loss value of the first registered image and the template image based on the gradient loss value.

[0125] In some embodiments, the loss calculation unit 702 further includes an image acquisition unit, a difference value calculation unit, and a difference value processing unit.

[0126] an image acquisition unit, configured to acquire a first partial image of the first registration image and a second partial image of the template image;

[0127] a difference value calculation unit, configured to determine a degree of matching between local features of the first partial image and the second partial image, and determine a similarity difference value between the first registered image and the template image based on the degree of matching between the local features;

[0128] The difference value processing unit is used to call a preset loss function to normalize the similarity difference value to obtain a cross-correlation loss value between the first registered image and the template image.

[0129] In some embodiments, the loss calculation unit 702 further includes a gradient value acquisition unit, a difference calculation unit, and a difference summation unit.

[0130] a gradient value acquisition unit, configured to acquire a first gradient value set of the first registration image and a second gradient value set of the template image, wherein both the first gradient value set and the second gradient value set include gradient values ​​in a preset coordinate direction;

[0131] a difference calculation unit, configured to perform difference calculation on the gradient values ​​of the first gradient value set and the second gradient value set in the same preset coordinate direction to obtain a gradient loss value set;

[0132] The difference summing unit is used to sum the gradient differences in the gradient loss value set to obtain a consistency loss value between the first registered image and the template image.

[0133] In some embodiments, the image registration unit 701 further includes a pixel parameter acquisition unit and a normalization processing unit.

[0134] A pixel parameter acquisition unit, used to obtain the pixel mean and pixel standard deviation of the original image;

[0135] The normalization processing unit is used to perform brightness normalization processing on the original image based on the pixel mean and the pixel standard deviation.

[0136] In some embodiments, the image registration unit 701 further includes a sampling unit and an image cropping unit.

[0137] A sampling unit, configured to resample the original image to adjust the voxel spacing of the original image to a preset spacing;

[0138] The image cropping unit is used to crop the resampled original image into a preset size.

[0139] In some embodiments, the network calling unit 704 further includes an image detection unit and an image processing subunit.

[0140] an image detection unit, configured to output a second registered image if the spatial similarity between the second registered image and the template image is greater than or equal to a similarity threshold;

[0141] The image processing subunit is used to determine the second registered image as the first registered image if the spatial similarity between the second registered image and the template image is less than the similarity threshold, and to proceed to the step of obtaining the cross-correlation loss value and consistency loss value of the first registered image and the template image.

[0142] In an embodiment of the present application, the original image and the template image are initially registered through the image registration network to generate a first registered image. The spatial similarity between the first registered image and the template image is then calculated. If the similarity does not reach the preset threshold, the mutual correlation loss value and consistency loss value of the first registered image and the template image are further calculated. The first registered image and the template image are used to optimize and adjust the parameters of the registration network. The optimized network again performs a refined registration process on the first registered image and outputs a second registered image with higher accuracy. The entire registration process is continued in this iterative optimization manner. Each iteration optimizes the network performance based on the previous registration result and loss calculation, thereby gradually improving the image registration accuracy of the image registration network, and ultimately ensuring that the obtained registered image can meet the needs of actual applications.

[0143] In addition, the image processing device provided in the above embodiment and an image processing method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment and will not be repeated here.

[0144] The serial numbers of the embodiments of the present application are for descriptive purposes only and do not represent the merits of the embodiments. In some cases, the actions or steps described in the claims may be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0145] See Figure 10 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 10 As shown, the electronic device 800 includes a processor 801 and a memory 802. The processor 801 is electrically connected to the memory 802.

[0146] The processor 801 is the control center of the electronic device 800 and may include one or more processing cores. The processor 801 utilizes various interfaces and lines to connect the various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or calling computer programs stored in the memory 802, as well as calling data stored in the memory 802, thereby performing overall management and control of the electronic device. Optionally, the processor 801 may be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 801 may integrate one or a combination of a CPU, a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user pages, and applications; the GPU is responsible for rendering and drawing display content; and the modem is used to handle wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 801 and may be implemented separately through a communication chip.

[0147] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the computer programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store an operating system, computer programs required for at least one function, etc.; the data storage area may store data generated based on the use of the electronic device.

[0148] In addition, the memory 802 may include a high-speed random access memory and a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0149] In the embodiment of the present application, the processor 801 in the electronic device 800 loads instructions corresponding to one or more computer program processes into the memory 802 according to the following steps, and the processor 801 runs the computer program stored in the memory 802 to implement various functions as follows:

[0150] Inputting the original image and the template image into a preset image registration network for image registration to obtain a first registered image corresponding to the original image;

[0151] If the spatial similarity between the first registered image and the template image is less than a preset similarity threshold, obtaining a cross-correlation loss value and a consistency loss value between the first registered image and the template image;

[0152] Optimize the image registration network based on the cross-correlation loss value and consistency loss value;

[0153] The first registered image is registered based on the optimized image registration network to obtain a second registered image of the original image.

[0154] Optionally, the processor 801 inputs the original image and the template image into a preset image registration network for image registration to obtain a first registered image corresponding to the original image, and specifically performs: obtaining spatial difference features of the original image and the template image, and determining a spatial transformation matrix for the original image based on the spatial difference features; and performing spatial transformation on the original image based on the spatial transformation matrix to obtain a first registered image.

[0155] Optionally, the processor 801 performs optimization of the image registration network based on the cross-correlation loss value and the consistency loss value, specifically performing: calling a preset joint loss function, and obtaining a joint loss value based on the cross-correlation loss value and the consistency loss value; and performing gradient updates on the weights and biases of the image registration network based on the joint loss value.

[0156] Optionally, when the processor 801 obtains the cross-correlation loss value and consistency loss value between the first registered image and the template image, it specifically performs: obtaining the similarity difference value between the first registered image and the template image, normalizing the similarity difference value to obtain the cross-correlation loss value between the first registered image and the template image; obtaining the gradient loss value between the first registered image and the template image, and obtaining the consistency loss value between the first registered image and the template image based on the gradient loss value.

[0157] Optionally, the processor 801 obtains the similarity difference value between the first registered image and the template image, normalizes the similarity difference value, and obtains the cross-correlation loss value between the first registered image and the template image. Specifically, the following steps are performed: obtaining a first local image of the first registered image and a second local image of the template image; calling a preset loss function to determine the similarity between the first local image and the second local image, and determining the cross-correlation loss value between the first registered image and the template image based on the similarity.

[0158] Optionally, the processor 801 obtains the gradient loss values ​​of the first registered image and the template image, and obtains the consistency loss value of the first registered image and the template image based on the gradient loss values, specifically performing the following steps: obtaining a first gradient value set of the first registered image and a second gradient value set of the template image, wherein both the first gradient value set and the second gradient value set contain gradient values ​​in a preset coordinate direction; performing difference calculation on the gradient values ​​in the same preset coordinate direction in the first gradient value set and the second gradient value set to obtain a gradient loss value set; and summing the gradient differences in the gradient loss value set to obtain the consistency loss value of the first registered image and the template image.

[0159] Optionally, before the processor 801 inputs the original image and the template image into a preset image registration network for image registration to obtain a first registered image corresponding to the original image, it specifically performs the following steps: obtaining the pixel mean and pixel standard deviation of the original image; and performing brightness normalization processing on the original image based on the pixel mean and pixel standard deviation.

[0160] Optionally, after performing brightness normalization processing on the original image based on the pixel mean and pixel standard deviation, the processor 801 specifically performs: resampling the original image to adjust the voxel spacing of the original image to a preset spacing; and cropping the resampled original image to a preset size.

[0161] Optionally, after the processor 801 performs image registration on the first registered image based on the optimized image registration network to obtain the second registered image of the original image, it specifically performs the following steps: if the spatial similarity between the second registered image and the template image is greater than or equal to the similarity threshold, then the second registered image is output; if the spatial similarity between the second registered image and the template image is less than the similarity threshold, then the second registered image is determined as the first registered image, and the step of obtaining the cross-correlation loss value and consistency loss value of the first registered image and the template image is executed.

[0162] In an embodiment of the present application, the original image and the template image are initially registered through the image registration network to generate a first registered image. The spatial similarity between the first registered image and the template image is then calculated. If the similarity does not reach the preset threshold, the mutual correlation loss value and the consistency loss value of the first registered image and the template image are further calculated. The first registered image and the template image are used to optimize and adjust the parameters of the registration network. The optimized network again performs a refined registration process on the first registered image and outputs a second registered image with higher accuracy. The entire registration process is continued in this iterative optimization manner. Each iteration optimizes the network performance based on the previous registration result and loss calculation, thereby gradually improving the image registration accuracy of the image registration network, and ultimately ensuring that the obtained registered image can meet the needs of actual applications.

[0163] In addition, the device provided in the embodiment of the present application can specifically be a chip, component or module, and the chip may include a connected processor and memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute an image processing method provided in the above embodiment.

[0164] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer, the computer executes the above-mentioned related method steps to implement an image processing method provided by the above-mentioned embodiment.

[0165] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement an image processing method provided by the above-mentioned embodiment.

[0166] Among them, the device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0167] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0168] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between the related ones shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0169] The above content is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: Inputting the original image and the template image into a preset image registration network for image registration to obtain a first registered image corresponding to the original image; If the spatial similarity between the first registered image and the template image is less than a preset similarity threshold, obtaining a mutual correlation loss value and a consistency loss value between the first registered image and the template image; Optimizing the image registration network based on the cross-correlation loss value and the consistency loss value; Image registration is performed on the first registered image based on the optimized image registration network to obtain a second registered image of the original image.

2. The method according to claim 1, characterized in that Inputting the original image and the template image into a preset image registration network for image registration to obtain a first registered image corresponding to the original image, including: Acquire spatial difference features between the original image and the template image, and determine a spatial transformation matrix for the original image based on the spatial difference features; The original image is spatially transformed based on the spatial transformation matrix to obtain the first registered image.

3. The method according to claim 1, characterized in that The optimizing the image registration network based on the cross-correlation loss value and the consistency loss value includes: Calling a preset joint loss function, and obtaining a joint loss value based on the cross-correlation loss value and the consistency loss value; The weights and biases of the image registration network are gradient updated based on the joint loss value.

4. The method according to claim 1, wherein The obtaining of the cross-correlation loss value and the consistency loss value between the first registered image and the template image includes: Obtaining a similarity difference value between the first registered image and the template image, and performing normalization processing on the similarity difference value to obtain a cross-correlation loss value between the first registered image and the template image; Gradient loss values ​​of the first registered image and the template image are obtained, and a consistency loss value of the first registered image and the template image is obtained based on the gradient loss values.

5. The method according to claim 4, characterized in that The obtaining of a similarity difference value between the first registered image and the template image, and normalizing the similarity difference value to obtain a cross-correlation loss value between the first registered image and the template image includes: Acquire a first partial image of the first registration image and a second partial image of the template image; A preset loss function is called to determine the similarity between the first partial image and the second partial image, and a cross-correlation loss value between the first registered image and the template image is determined based on the similarity.

6. The method according to claim 4, characterized in that The obtaining of the gradient loss value of the first registered image and the template image, and obtaining a consistency loss value of the first registered image and the template image based on the gradient loss value, includes: Acquire a first gradient value set of the first registration image and a second gradient value set of the template image, wherein both the first gradient value set and the second gradient value set include gradient values ​​in a preset coordinate direction; performing difference calculation on the gradient values ​​in the same preset coordinate direction in the first gradient value set and the second gradient value set to obtain a gradient loss value set; The gradient differences in the gradient loss value set are summed to obtain a consistency loss value between the first registered image and the template image.

7. The method according to claim 1, characterized in that Before inputting the original image and the template image into a preset image registration network for image registration to obtain a first registered image corresponding to the original image, the method further includes: Obtaining a pixel mean and a pixel standard deviation of the original image; Brightness normalization processing is performed on the original image based on the pixel mean and the pixel standard deviation.

8. The method according to claim 7, characterized in that After performing brightness normalization processing on the original image based on the pixel mean and the pixel standard deviation, the method further includes: resampling the original image to adjust the voxel spacing of the original image to a preset spacing; The resampled original image is cropped to a preset size.

9. The method according to claim 1, characterized in that After performing image registration on the first registered image based on the optimized image registration network to obtain a second registered image of the original image, the method further includes: If the spatial similarity between the second registered image and the template image is greater than or equal to the similarity threshold, outputting the second registered image; If the spatial similarity between the second registered image and the template image is less than the similarity threshold, the second registered image is determined as the first registered image, and the step of obtaining the cross-correlation loss value and consistency loss value between the first registered image and the template image is executed.

10. An image processing device, characterized in that: include: An image registration unit, configured to input the original image and the template image into a preset image registration network for image registration, and obtain a first registered image corresponding to the original image; a loss calculation unit, configured to obtain a mutual correlation loss value and a consistency loss value between the first registered image and the template image if the spatial similarity between the first registered image and the template image is less than a preset similarity threshold; A network optimization unit, configured to optimize the image registration network based on the cross-correlation loss value and the consistency loss value; A network calling unit is used to perform image registration on the first registered image based on the optimized image registration network to obtain a second registered image of the original image.

11. An electronic device, characterized in that: The electronic device comprises: a memory for storing executable program code; A processor is configured to call and run the executable program code from the memory, so that the electronic device executes the image processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed, the image processing method according to any one of claims 1 to 9 is implemented.

Images