Health image data analysis processing assisted diagnosis method

Abstract

The present application relates to the field of health care informatics, and discloses a health image data analysis processing auxiliary diagnosis method. The initial deformation field is predicted by the feature encoder and the deformable convolution layer through the acquisition of the to-be-registered image and the reference image of the same subject at different time points. The local gray histogram features of the two are extracted to calculate the gray distribution offset, which is converted into a contrast compensation coefficient. In the back propagation stage, the contrast compensation coefficient is applied to the initial deformation field gradient, and the contrast adaptive correction is performed on the spatial displacement vector to generate the target deformation field. The spatial transformation of the to-be-registered image is performed by using the target deformation field, the pathological evolution features are extracted by inputting the time difference network, and the disease progression evaluation result is output. The present scheme corrects the deviation of the spatial displacement vector of the deformation field caused by the inconsistent gray, eliminates the registration misplacement caused by the difference of the acquisition parameters, and ensures the accuracy of the pathological evolution feature extraction.

Description

Technical Field

[0001] This invention relates to the field of healthcare informatics and discloses a method for assisting diagnosis through health image data analysis and processing. Background Technology

[0002] Assisted diagnosis of healthy image sequences from different time points for the same subject relies on flexible registration networks to achieve spatial alignment. The closest conventional approach employs a feature encoder to extract feature maps from the image to be registered and a reference image, then inputs these feature maps into a deformable convolutional layer to predict the initial deformation field. During the prediction of the initial deformation field, existing methods construct a network containing spatial displacement vectors and use a single mutual information or mean squared error as the loss function to optimize the network weights. This process entirely depends on the feature similarity between the two image sequences to guide the iterative update of the deformation field, enabling the extraction of subsequent pathological evolution features and disease assessment.

[0003] When the equipment model or scanning exposure parameters change between two image acquisitions, grayscale differences arise between the image to be registered and the reference image. Because conventional techniques directly calculate the similarity of the original grayscale features or uncalibrated feature maps when calculating the deformation field, these grayscale differences interfere with the registration network's recognition of tissue structures. This interference manifests as the loss function guiding the update of the spatial displacement vector of the deformation field in the wrong direction, leading to a calculation deviation in the initial deformation field. This deviation directly causes spatial registration misalignment of subsequent temporal images, resulting in inaccurate extraction of pathological evolution features and ultimately, failure of assisted diagnosis. Summary of the Invention

[0004] The purpose of this invention is to provide a method for auxiliary diagnosis through health image data analysis and processing, which can effectively solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for assisting diagnosis by analyzing and processing health image data includes: acquiring health image sequences of the same subject at different time points, and dividing the health image sequences into images to be registered and reference images;

[0007] The image to be registered and the reference image are input into a preset feature encoder to extract the first multimodal feature map of the image to be registered and the second multimodal feature map of the reference image;

[0008] The first multimodal feature map and the second multimodal feature map are input into a deformable convolutional layer to predict the initial deformation field of the image to be registered relative to the reference image.

[0009] Extract the local grayscale histogram features of the image to be registered and the reference image at corresponding spatial locations, calculate the grayscale distribution offset between the image to be registered and the reference image based on the local grayscale histogram features, and convert the grayscale distribution offset into a contrast compensation coefficient.

[0010] The contrast compensation coefficient is applied to the initial deformation field gradient calculated by the deformable convolutional layer during the backpropagation stage, and the spatial displacement vector contained in the initial deformation field is adaptively corrected for contrast to generate the target deformation field.

[0011] The target deformation field is used to perform spatial transformation on the image to be registered, and the registered target image sequence is output.

[0012] The target image sequence is input into a temporal difference network to extract pathological evolution features and output disease progression assessment results.

[0013] Preferably, the step of inputting the image to be registered and the reference image into a preset feature encoder to extract the first multimodal feature map of the image to be registered and the second multimodal feature map of the reference image includes: the feature encoder includes three convolutional blocks connected in sequence, each of the convolutional blocks including cascaded dilated convolutional layers and residual connection branches;

[0014] For the i-th convolutional block, the input feature map is input into the dilated convolutional layer with a dilation rate of d and the residual connection branch, where the value of d increases exponentially with the increase of i.

[0015] The output feature map of the dilated convolutional layer and the output feature map of the residual connection branch are concatenated along the channel dimension. The concatenated feature map is then input into the first convolutional kernel for dimensionality reduction to generate the output feature map of the i-th convolutional block.

[0016] The output feature map of the third convolutional block is used as the first multimodal feature map and the second multimodal feature map, respectively.

[0017] Preferably, the step of inputting the first multimodal feature map and the second multimodal feature map into a deformable convolutional layer to predict the initial deformation field of the image to be registered relative to the reference image includes: concatenating the first multimodal feature map and the second multimodal feature map in the channel dimension to generate a fused feature map;

[0018] The fused feature map is input into the second convolutional kernel to generate an offset prediction map and a modulation coefficient prediction map;

[0019] For each sampling point in the fused feature map, the three-dimensional offset vector corresponding to the sampling point is extracted from the offset prediction map, and the scalar modulation weight corresponding to the sampling point is extracted from the modulation coefficient prediction map.

[0020] Based on the three-dimensional offset vector, the neighborhood offset sampling points of the sampling point are determined in the fused feature map. The feature vector of the sampling point and the feature vector of the neighborhood offset sampling points are weighted and summed according to the scalar modulation weight. The weighted summation result is recombined into the initial deformation field containing the set of three-dimensional spatial displacement vectors.

[0021] Preferably, the step of extracting the local grayscale histogram features of the image to be registered and the reference image at corresponding spatial positions, and calculating the grayscale distribution offset between the image to be registered and the reference image based on the local grayscale histogram features, includes: delineating a cube window with a side length of N pixels, centered on the same preset coordinate point in the image to be registered and the reference image.

[0022] The pixel frequency of the image to be registered in each gray-level interval within the cube window is counted to generate a first local gray-level histogram feature vector. The pixel frequency of the reference image in each gray-level interval within the cube window is counted to generate a second local gray-level histogram feature vector.

[0023] Calculate the Wasserstein distance between the first local gray-level histogram feature vector and the second local gray-level histogram feature vector, and use the Wasserstein distance as the gray-level distribution offset at the preset coordinate point.

[0024] Preferably, converting the grayscale distribution offset into a contrast compensation coefficient includes: constructing a multilayer perceptron containing two fully connected layers;

[0025] The grayscale distribution offset is input into the first fully connected layer of the multilayer perceptron, and the first fully connected layer maps the input grayscale distribution offset to the hidden layer feature space.

[0026] The output vector of the hidden layer feature space is input into the second fully connected layer of the multilayer perceptron, and the second fully connected layer maps the output vector to the real number interval between 0 and 1 through the Sigmoid activation function;

[0027] The real number output by the Sigmoid activation function is used as the contrast compensation coefficient corresponding to the gray-scale distribution offset, wherein the value of the contrast compensation coefficient increases monotonically as the gray-scale distribution offset increases.

[0028] Preferably, applying the contrast compensation coefficient to the initial deformation field gradient calculated by the deformable convolutional layer during the backpropagation stage, and performing contrast adaptive correction on the spatial displacement vectors contained in the initial deformation field, includes: obtaining the partial derivative matrix of the loss function with respect to each of the spatial displacement vectors in the initial deformation field during the backpropagation process of the deformable convolutional layer.

[0029] The contrast compensation coefficient is extended into a diagonal weight matrix with the same dimension as the partial derivative matrix;

[0030] Calculate the product of the diagonal weight matrix and the partial derivative matrix to generate the modified partial derivative matrix;

[0031] The partial derivative matrix is ​​replaced by the corrected partial derivative matrix, and the gradient descent update process of the network weights in the deformable convolutional layer is continued. The forward propagation is then performed again with the updated network weights to generate the target deformation field.

[0032] Preferably, before performing the channel-dimensional concatenation operation on the output feature map of the dilated convolutional layer and the output feature map of the residual connection branch, the method further includes: inputting the output feature map of the dilated convolutional layer into the channel attention calculation branch, wherein the channel attention calculation branch performs a global average pooling operation on the output feature map of the dilated convolutional layer to generate a one-dimensional channel descriptor;

[0033] The one-dimensional channel descriptor is sequentially input into a dimensionality-reducing fully connected layer and an dimensionality-increasing fully connected layer. The output vector of the dimensionality-increasing fully connected layer is then input into the Sigmoid function to generate a channel attention weight vector.

[0034] The channel attention weight vector is multiplied channel by channel by channel by the output feature map of the residual connection branch, and the feature map after multiplication is used as the output feature map of the residual connection branch participating in the splicing operation.

[0035] Preferably, after recombining the weighted summation result into the initial deformation field containing a set of three-dimensional spatial displacement vectors, the method further includes: extracting the displacement component maps of the initial deformation field in the three directions of the X-axis, Y-axis and Z-axis;

[0036] For each direction of the displacement component map, calculate the displacement gradient value between adjacent pixels in the displacement component map, sum the absolute values ​​of the displacement gradient values, and generate a spatial smoothing penalty term.

[0037] Calculate the absolute value of the determinant of the Jacobian matrix corresponding to the initial deformation field, and use the number of pixels whose absolute value is less than a preset constant as the deformation folding penalty term.

[0038] The spatial smoothing penalty term and the deformation folding penalty term are added together according to a preset weight to generate a regularization loss term, which is then superimposed on the total loss function of the deformable convolutional layer.

[0039] Preferably, after using the Wasserstein distance as the grayscale distribution offset at the preset coordinate point, the method further includes: determining whether there is a spatial overlap region between the cube windows corresponding to adjacent preset coordinate points;

[0040] If the spatial overlap region exists, the gray values ​​of the pixels located in the spatial overlap region in the image to be registered are weighted according to the spatial inverse distance from the adjacent preset coordinate points to the pixels, and an updated first local gray-level histogram feature vector and an updated second local gray-level histogram feature vector are generated.

[0041] The grayscale distribution offset is recalculated based on the updated feature vector, and the recalculated grayscale distribution offset is input into the multilayer perceptron to generate the target contrast compensation coefficient.

[0042] A three-dimensional Gaussian filtering operation is performed on the target contrast compensation coefficient to generate the filtered target contrast compensation coefficient.

[0043] Preferably, the step of replacing the partial derivative matrix with the modified partial derivative matrix and continuing to perform the gradient descent update process of the network weights in the deformable convolutional layer includes: calculating the L2 norm of each element in the modified partial derivative matrix, scaling the elements whose L2 norm is greater than a preset pruning threshold proportionally to the preset pruning threshold, and generating a pruning modified partial derivative matrix.

[0044] The gradient descent update process of the network weights is performed using the cropped partial derivative matrix.

[0045] The step of inputting the target image sequence into a temporal difference network, extracting pathological evolution features, and outputting disease progression assessment results includes: inputting the reference image and the target image after spatial transformation using the target deformation field into the feature alignment layer of the temporal difference network, calculating the absolute difference feature map between the reference image features and the target image features, and inputting the absolute difference feature map into a three-dimensional convolutional layer to extract the pathological evolution features.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0047] 1. This scheme calculates the gray-level distribution offset by extracting local gray-level histogram features from the image to be registered and the reference image. A multilayer perceptron is then used to convert the gray-level distribution offset into a contrast compensation coefficient. During the backpropagation stage, this contrast compensation coefficient is applied as a diagonal weight matrix to the initial deformation field gradient, thus solving the technical problem of gray-level differences interfering with the accuracy of spatial registration of the deformation field. The intervention of the contrast compensation coefficient directly changes the magnitude and direction of the gradient, correcting the deviation of the spatial displacement vector of the deformation field caused by gray-level inconsistencies. Spatial transformation using the corrected target deformation field eliminates registration misalignment caused by differences in acquisition parameters, ensuring the accuracy of pathological evolution feature extraction.

[0048] 2. The feature encoder employs dilated convolutions with exponentially increasing dilation rates to expand the receptive field. Combined with channel attention calculation branches, the feature weights of residual connection branches are adjusted, improving the expressive power of the extracted multimodal feature maps. In the spatially overlapping regions of the cubic window, a weighted allocation based on spatial inverse distance is used to calculate histogram features, smoothing the compensation coefficients of adjacent regions and avoiding abrupt local compensation changes. The regularization loss term constrains the spatial smoothness of the initial deformation field and suppresses deformation folding. L2-norm clipping is performed on the corrected partial derivative matrix to prevent gradient anomalies during backpropagation, collectively maintaining the computational stability of the registration network. Attached Figure Description

[0049] Figure 1 This is an overall flowchart of the present invention;

[0050] Figure 2 This is a flowchart of the feature extraction process within the feature encoder of the present invention;

[0051] Figure 3 This is a flowchart of the initial deformation field prediction and regularization process of the present invention;

[0052] Figure 4 This is a flowchart of the grayscale distribution offset calculation process of the present invention;

[0053] Figure 5 This is a flowchart of the contrast compensation coefficient calculation and filtering process of the present invention;

[0054] Figure 6 This is a flowchart of the gradient correction and disease progression assessment output of the present invention. Detailed Implementation

[0055] Please refer to the attached document. Figure 1This embodiment provides a diagnostic aid method for health image data analysis and processing disclosed in this solution. Based on health image sequences of the same subject at different time points, spatial registration and pathological evolution feature extraction are completed, providing quantifiable diagnostic evidence for assessing disease progression. Health image sequences of the same subject at different time points are acquired. These health image sequences are medical image datasets containing three-dimensional spatial information, and their modalities include, but are not limited to, computed tomography (CT) images, magnetic resonance imaging (MRI) images, and positron emission tomography (PET) images. The different time points correspond to the image acquisition times of the subject's baseline examination and at least one follow-up examination. The health image sequences are divided into images to be registered and reference images. The reference images are image data acquired at the baseline examination time, and the images to be registered are image data acquired at the follow-up examination time that have the same anatomical structure coverage as the reference images.

[0056] Please refer to the attached document. Figure 2 The image to be registered and the reference image are input into a preset feature encoder to extract a first multimodal feature map of the image to be registered and a second multimodal feature map of the reference image. The feature encoder is a pre-trained 3D convolutional neural network, and its training process is completed using a publicly available medical image registration dataset. The training objective is to minimize the structural similarity loss between the registered image and the reference image. The network weights of the pre-trained feature encoder are fixed or participate in fine-tuning during subsequent registration processes.

[0057] The feature encoder takes normalized images to be registered and reference images as input. Normalization uses a linear mapping method to map the grayscale values ​​of the original images to a range of 0 to 1, eliminating interference from differences in the dynamic range of grayscale values ​​between different image acquisition devices on feature extraction. The output of the feature encoder is a multi-channel 3D feature map. The 3D feature map output for the image to be registered is the first multimodal feature map, and the 3D feature map output for the reference image is the second multimodal feature map. The first and second multimodal feature maps have the same spatial dimensions and number of channels, ensuring dimensionality matching in subsequent feature fusion and deformation field prediction processes.

[0058] The first and second multimodal feature maps are input into a deformable convolutional layer to predict the initial deformation field of the image to be registered relative to the reference image. The deformable convolutional layer is a three-dimensional deformable convolutional structure used to learn the spatial deformation mapping relationship between the image to be registered and the reference image. The initial deformation field is a three-dimensional tensor with the same spatial dimensions as the image to be registered. Each spatial position in the initial deformation field corresponds to a three-dimensional spatial displacement vector, which includes... axis, shaft and The displacement components in the three axes are used to describe the spatial offset of the corresponding pixel in the image to be registered relative to the pixel at the same spatial position in the reference image. The input of the deformable convolutional layer is the fusion feature of the first multimodal feature map and the second multimodal feature map, and the output is the initial deformation field. The numerical range of the initial deformation field is constrained within a preset displacement range to avoid displacement prediction results that exceed the reasonable deformation range of the anatomical structure.

[0059] Local gray-level histogram features of the image to be registered and the reference image at corresponding spatial locations are extracted. Based on these local gray-level histogram features, the gray-level distribution offset between the two images is calculated, and the offset is converted into a contrast compensation coefficient. The extraction of local gray-level histogram features involves defining a fixed-size local window centered on a spatial coordinate point in the image, statistically analyzing the gray-level value distribution within the window, and generating a corresponding feature vector.

[0060] The image to be registered and the reference image use the exact same window size, gray-level interval division rules, and statistical methods to ensure the comparability of the extracted local gray-level histogram features. The gray-level distribution offset is used to quantify the difference in local gray-level distribution between the image to be registered and the reference image at the same spatial location; its value is positively correlated with the degree of gray-level distribution difference. The contrast compensation coefficient is a real number between 0 and 1, used to adjust the gradient update magnitude of the initial deformation field during the backpropagation stage. Its value is monotonically increasing with the gray-level distribution offset; that is, the greater the gray-level distribution difference, the larger the contrast compensation coefficient, and the higher the corresponding gradient adjustment magnitude.

[0061] Table 1. Gray-level interval division and pixel frequency statistics rules of local gray-level histogram.

[0062] 1 [0.00,0.05) The number of pixels falling within the corresponding interval of the window, excluding pixels truncated at the edges. Gray values ​​below 0.00 are uniformly classified into this range and are not included in the statistical frequency outliers. 2 [0.05,0.15) The number of pixels falling within the corresponding interval of the window, excluding pixels truncated at the edges. none 3 [0.15,0.30) The number of pixels falling within the corresponding interval of the window, excluding pixels truncated at the edges. none 4 [0.30,0.45) The number of pixels falling within the corresponding interval of the window, excluding pixels truncated at the edges. none 5 [0.45,0.55) The number of pixels falling within the corresponding interval of the window, excluding pixels truncated at the edges. none 6 [0.55,0.70) The number of pixels falling within the corresponding interval of the window, excluding pixels truncated at the edges. none 7 [0.70,0.85) The number of pixels falling within the corresponding interval of the window, excluding pixels truncated at the edges. none 8 [0.85,0.95) The number of pixels falling within the corresponding interval of the window, excluding pixels truncated at the edges. none 9 [0.95,1.00] The number of pixels falling within the corresponding interval of the window, excluding pixels truncated at the edges. Gray values ​​higher than 1.00 are uniformly classified into this range and are not included in the statistical frequency outliers.

[0063] The grayscale interval division rules specified in Table 1 are uniformly executed globally during the local grayscale histogram feature extraction process of the image to be registered and the reference image. The low and high grayscale intervals use narrower interval widths, while the intermediate grayscale intervals use wider interval widths. This adapts to the grayscale distribution characteristics of anatomical structures and background regions in medical images, improving the quantification accuracy of grayscale distribution differences. During pixel frequency statistics, truncated pixels at window edges are excluded to avoid statistical errors caused by window boundaries crossing different anatomical structures. The outlier handling rules unify the processing method for grayscale values ​​exceeding the normalization range, eliminating statistical biases caused by residual errors in image normalization.

[0064] The contrast compensation coefficient is applied to the initial deformation field gradient calculated by the deformable convolutional layer during the backpropagation phase. This performs contrast adaptive correction on the spatial displacement vectors contained in the initial deformation field, generating the target deformation field. The training process of the deformable convolutional layer employs an iterative approach of alternating forward and backward propagation. During the forward propagation phase, the initial deformation field is predicted based on the input fused feature map, and the corresponding total loss function is calculated. During the backpropagation phase, the partial derivatives of the total loss function with respect to each spatial displacement vector in the initial deformation field are calculated, i.e., the initial deformation field gradient. Gradient descent updates of the network weights are then performed based on this initial deformation field gradient.

[0065] The contrast compensation coefficient intervenes during the backpropagation stage to weight and correct the initial deformation field gradient, adjusting the gradient update amplitude at different spatial locations. The gradient update amplitude is amplified for locations with significant grayscale distribution differences, while it is retained or reduced for locations with smaller differences, correcting the deformation field prediction bias caused by inconsistent grayscale distribution. The corrected gradient is used to update the network weights of the deformable convolutional layer. The updated network weights then re-execute the forward propagation process to generate the corrected target deformation field. The 3D spatial displacement vector at each spatial location of the target deformation field undergoes adaptive contrast adjustment to accommodate the grayscale distribution differences at different spatial locations.

[0066] The target deformation field is used to spatially transform the image to be registered, outputting a registered target image sequence. The spatial transformation employs a three-dimensional spatial transformation algorithm, specifically a trilinear interpolation algorithm. Based on the three-dimensional spatial displacement vector of each spatial location in the target deformation field, the corresponding coordinates of each pixel in the image to be registered in the reference image coordinate system are calculated. The grayscale value of the corresponding coordinates is then calculated using trilinear interpolation to generate the registered target image. The use of the trilinear interpolation algorithm avoids grayscale value discretization errors during the spatial transformation process, ensuring the grayscale continuity of the registered target image. The registered target image and the reference image are in the same spatial coordinate system, with the spatial positions of identical anatomical structures perfectly aligned. This eliminates spatial misalignment caused by changes in subject position, respiratory movements, etc., and corrects registration deviations caused by differences in grayscale distribution.

[0067] The target image sequence is input into a temporal difference network to extract pathological evolution features and output disease progression assessment results. The temporal difference network is a pre-trained 3D convolutional neural network, trained using a longitudinal medical image dataset labeled with disease progression information. The training objective is to minimize the classification and regression losses between the disease progression assessment results and the gold standard label. The input to the temporal difference network is a target image sequence composed of registered target and reference images, and the output is pathological evolution features and the corresponding disease progression assessment results. Pathological evolution features are feature vectors that quantitatively describe changes in anatomical structures and lesion regions between two image acquisition points. Disease progression assessment results include, but are not limited to, lesion volume change rate, pathological grade changes, and disease progression risk level, providing quantifiable auxiliary evidence for clinical diagnosis.

[0068] In this embodiment, the gray-level distribution difference between the image to be registered and the reference image is quantified using local gray-level histogram features, generating a corresponding contrast compensation coefficient. During the backpropagation stage, the gradient update process of the initial deformation field is corrected using the contrast compensation coefficient, eliminating the interference of gray-level distribution differences on deformation field prediction. Pathological evolution features are extracted based on the registered target image sequence, ensuring the reliability of the disease progression assessment results.

[0069] In a preferred embodiment, the feature encoder includes three convolutional blocks connected in sequence, each convolutional block comprising a cascaded dilated convolutional layer and a residual connection branch. For the i-th convolutional block, the input feature map is input to both the dilated convolutional layer with a dilation rate of d and the residual connection branch, where the value of d increases exponentially with increasing i. The output feature map of the dilated convolutional layer and the output feature map of the residual connection branch are concatenated along the channel dimension, and the concatenated feature map is input to the first convolutional kernel for dimensionality reduction to generate the output feature map of the i-th convolutional block. The output feature map of the third convolutional block is used as both the first multimodal feature map and the second multimodal feature map.

[0070] For 3D input feature maps ,in , , These are the height, width, and depth of the feature map, respectively. Given the number of input channels, the kernel size of the dilated convolutional layer is... The void ratio is Then the actual sampling interval of the convolution kernel on the feature map is The corresponding receptive field size is Output feature map of dilated convolutional layer In the middle, position The formula for calculating the eigenvalue at a given location is:

[0071]

[0072] in, For output channel index, For input channel index, For the convolution kernel weight tensor, For the kernel bias term, , , This is the spatial index of the convolution kernel.

[0073] Hollow rate of three convolutional blocks Follow The increase in void ratio is exponential, and the void ratio of the first convolutional block is... =1, the hole rate of the second convolutional block =2, the hole rate of the 3rd convolutional block =4, that is The value of satisfies ,in The sequence number of the convolutional block. =1,2,3. This exponentially increasing dilatation rate setting gradually expands the receptive field of the convolutional layer without increasing the number of convolutional kernel parameters or computational cost, enabling the feature encoder to capture anatomical features at different scales, from local detailed texture features to global organ structure features, thus improving the expressive power of multimodal feature maps.

[0074] The residual connection branch contains a 1×1×1 three-dimensional convolutional layer, whose input is the input features of the corresponding convolutional block. Figure 1 The number of channels in the output feature map is consistent with that of the dilated convolutional layer, preserving the original information of the input feature map, alleviating the gradient vanishing problem in deep convolutional networks, and improving the training stability of the feature encoder. The output feature map of the dilated convolutional layer and the output feature map of the residual connection branch are concatenated along the channel dimension. The number of channels in the concatenated feature map is the sum of the output channels of the dilated convolutional layer and the residual connection branch. The concatenated feature map is then input into the first convolutional kernel, a 1×1×1 three-dimensional convolutional layer. Its function is to reduce the dimensionality of the concatenated high-channel feature map, compressing the number of channels to a preset output channel number, generating the output feature map of the i-th convolutional block. The number of output channels of the first convolutional kernel increases with the convolutional block number: 32 for the first convolutional block, 64 for the second, and 128 for the third, gradually increasing the channel dimension of the feature map and encoding richer semantic features.

[0075] Further, the output feature map of the dilated convolutional layer is input into the channel attention calculation branch. The channel attention calculation branch performs global average pooling on the output feature map of the dilated convolutional layer to generate a one-dimensional channel descriptor. The one-dimensional channel descriptor is then sequentially input into a dimensionality-reducing fully connected layer and an dimensionality-increasing fully connected layer. The output vector of the dimensionality-increasing fully connected layer is input into a sigmoid function to generate a channel attention weight vector. The channel attention weight vector is then multiplied channel-by-channel by a scalar multiplication operation with the output feature map of the residual connection branch. The feature map resulting from the multiplication operation is used as the output feature map of the residual connection branch participating in the concatenation operation.

[0076] For the output feature map of the dilated convolutional layer ,in The number of output channels for the dilated convolutional layer is given. The channel attention calculation branch first... Perform a global average pooling operation to generate a one-dimensional channel descriptor. , of which The formula for calculating the descriptor value of each channel is:

[0077]

[0078] in, For channel index, , , This is the spatial coordinate index of the feature map.

[0079] One-dimensional channel descriptor Input to a dimension-reducing fully connected layer; the dimension-reducing fully connected layer reduces the number of channels from... Compress to ,in The dimensionality reduction ratio is preset to 16; the output vector of the dimensionality-reducing fully connected layer is input to the dimensionality-upgrading fully connected layer, which then reduces the number of channels from... Restore to The output vector of the upgraded fully connected layer is input into the Sigmoid activation function to generate the channel attention weight vector. The calculation formula is as follows:

[0080]

[0081] in, , The weight matrix and bias terms of the dimension-reduced fully connected layer. , For the weight matrix and bias terms of the upgraded fully connected layer, It is a linear rectified activation function. This is the Sigmoid activation function.

[0082] Channel attention weight vector Output feature map of the residual connection branch Perform channel-by-channel scalar multiplication; the feature map after multiplication. The calculation formula is:

[0083]

[0084] in, For channel index, , , This is the spatial coordinate index of the feature map.

[0085] After the channel-by-channel scalar multiplication operation is completed, the feature map Each channel's feature value is weighted by its corresponding channel attention weight. Channels containing effective anatomical structural information receive higher weights, while channels containing noise or redundant information receive lower weights. This enhances the expressive power of the residual connection branch's output features and suppresses interference from invalid features. (Weighted feature map) As the final output of the residual connection branch, it is compared with the output feature map of the dilated convolutional layer. Perform channel-level splicing operations.

[0086] Table 2. Structural parameters of each convolutional block in the feature encoder.

[0087] 1 1 3×3×3 1×1×1 16 32 2 2 3×3×3 1×1×1 16 64 3 4 3×3×3 1×1×1 16 128

[0088] The structural parameters specified in Table 2 are executed sequentially in the three convolutional blocks of the feature encoder. The dilation rate increases exponentially with the convolutional block number, gradually expanding the receptive field of the convolutional layer without increasing the size of the convolutional kernel parameters. The residual connection branches uniformly use 1×1×1 convolutional kernels to match the number of channels without changing the feature map spatial size, preserving the original information of the input features. The channel attention branches use a fixed dimensionality reduction ratio to achieve channel-dimensional feature weighting while controlling computational cost, increasing the weight ratio of effective features. The number of output channels in the convolutional blocks increases with the sequence number, gradually improving the semantic encoding capability of the features and providing highly discriminative multimodal feature maps for subsequent deformation field prediction.

[0089] In this embodiment, the feature encoder adopts a three-cascaded convolutional block structure, expands the receptive field by setting an exponentially increasing hole rate, retains the original feature information by combining residual connection branches, and introduces a channel attention calculation branch to perform weighted optimization of residual features, thereby improving the feature representation ability and discriminativeness of the multimodal feature map, and providing a reliable feature foundation for the accurate prediction of the deformation field in the future.

[0090] In a preferred embodiment, the first multimodal feature map and the second multimodal feature map are input into a deformable convolutional layer to predict the initial deformation field of the image to be registered relative to the reference image. Specifically, this includes: concatenating the first and second multimodal feature maps along the channel dimension to generate a fused feature map; inputting the fused feature map into a second convolutional kernel to generate an offset prediction map and a modulation coefficient prediction map; for each sampling point in the fused feature map, extracting the corresponding three-dimensional offset vector from the offset prediction map and the corresponding scalar modulation weight from the modulation coefficient prediction map; determining neighboring offset sampling points of the sampling point in the fused feature map based on the three-dimensional offset vector; weighting and summing the feature vectors of the sampling point and the neighboring offset sampling points according to the scalar modulation weights; and recombining the weighted sum into the initial deformation field containing a set of three-dimensional spatial displacement vectors.

[0091] For fused feature maps ,in , , To fuse the spatial dimensions of the feature maps, To determine the number of channels in the fused feature map, the number of sampling points in a deformable convolutional layer is: , , This is the size of a standard convolutional kernel, preset to 3. For the location in the fused feature map... The sampling points, whose sampling grid of regular convolution is... ,in For relative to The normal sampling offset has a value range of -1, 0, 1×-1, 0, 1×-1, 0, 1.

[0092] Deformable convolutional layers generate offset prediction maps through a second convolutional kernel. With modulation coefficient prediction graph In this context, each spatial location in the offset prediction map corresponds to... Each of the three-dimensional offset vectors contains offset components in the X, Y, and Z axes; each spatial location in the modulation coefficient prediction map corresponds to... Each scalar modulation weight has a value between 0 and 1.

[0093] Deformable convolutional layers at position The formula for calculating the output feature value at point is:

[0094]

[0095] in, For the first The three-dimensional offset vector corresponding to each sampling point is extracted from the offset prediction map; For the first The scalar modulation weights corresponding to each sampling point are extracted from the modulation coefficient prediction map; For deformable convolutional layers at position The output feature value at that location.

[0096] The second convolutional kernel is a 3×3×3 three-dimensional convolutional layer, whose input is the fused feature map and whose output channel number is... ,in Each channel corresponds to an offset prediction map. Each channel corresponds to a modulation coefficient prediction map. The output value of the offset prediction map is not processed by the activation function, retaining positive and negative values ​​to describe spatial offset in any direction; the output value of the modulation coefficient prediction map is processed by the Sigmoid activation function, mapping it to between 0 and 1, used to adjust the feature weights of different sampling points. The output feature map of the deformable convolutional layer undergoes channel transformation through a 1×1×1 3D convolutional layer, adjusting the number of output channels to 3, corresponding to... axis, shaft and The displacement components in the three axes generate the initial deformation field. Each spatial location in the initial deformation field The corresponding three-dimensional spatial displacement vector is Describe the location in axis, shaft and Spatial displacement in the axial direction.

[0097] Please refer to the attached document. Figure 3 Furthermore, the initial deformation field is extracted in axis, shaft and The displacement component maps are generated along the three axes. For each displacement component map, the displacement gradient values ​​between adjacent pixels are calculated, and the absolute values ​​of these gradient values ​​are summed to generate a spatial smoothing penalty term. The absolute value of the determinant of the Jacobian matrix corresponding to the initial deformation field is calculated, and the number of pixels whose absolute value is less than a preset constant is used as a deformation folding penalty term. The spatial smoothing penalty term and the deformation folding penalty term are added according to preset weights to generate a regularization loss term, which is then added to the total loss function of the deformable convolutional layer.

[0098] Initial deformation field The three displacement component diagrams are as follows: , , Spatial smoothing penalty term The formula for constraining the spatial continuity of the displacement component diagram and preventing abrupt displacement changes is as follows:

[0099]

[0100] in, , , They are respectively axis, shaft and The first-order difference operator in the axial direction is used to calculate the displacement gradient between adjacent pixels. This is for absolute value operations.

[0101] For the initial deformation field any spatial location Its corresponding deformation mapping Jacobian matrix The calculation formula is:

[0102]

[0103] The partial derivatives are calculated using the first-order difference operator. Deformation-folding penalty term. It is used to suppress spatial folding in the deformation field and avoid anatomical misalignment in the registered image. The calculation formula is as follows:

[0104]

[0105] in, For matrix determinant operations, This is a preset constant with a value of 0.001. For the maximum value operation, when the absolute value of the determinant of the Jacobian matrix is ​​less than 100%, the maximum value is calculated as follows: At that time, a folding penalty value is generated for the corresponding pixel.

[0106] Regularization loss term The calculation formula is:

[0107]

[0108] in, The preset weight for the spatial smoothing penalty term is 0.5; The preset weight for the deformation folding penalty term is 0.2. Regularization loss term. The total loss function superimposed on deformable convolutional layers In addition, the total loss function also includes a registration similarity loss term. The registration similarity loss term is calculated using normalized mutual information to measure the structural similarity between the registered image and the reference image. The formula for the total loss function is as follows: .

[0109] Table 3. Mapping rules between offset vectors and modulation weights of deformable convolution sampling points.

[0110] (-1,-1,-1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (-1,-1,0) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (-1,-1,1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (-1,0,-1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (-1,0,0) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (-1,0,1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (-1,1,-1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (-1,1,0) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (-1,1,1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (0,-1,-1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (0,-1,0) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (0,-1,1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (0,0,-1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (0,0,0) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (0,0,1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (0,1,-1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (0,1,0) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (0,1,1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (1,-1,-1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (1,-1,0) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (1,-1,1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (1,0,-1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (1,0,0) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (1,0,1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (1,1,-1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (1,1,0) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation. (1,1,1) [-2.0,2.0] [0.0,1.0] The sampled feature values ​​are multiplied by the modulation weights and then included in the summation operation.

[0111] The mapping rules specified in Table 3 are uniformly executed globally during the sampling and weighting calculation of deformable convolutional layers. All 27 sampling points corresponding to the 3×3×3 convolutional kernels use the same offset vector value range and modulation weight value range to ensure consistency in the sampling process. The value range of the three-dimensional offset vector is constrained between [-2.0, 2.0] to avoid sampling points exceeding the boundary range of the feature map, while also adapting to the reasonable deformation range of the anatomical structure. The value range of the scalar modulation weights is constrained between [0.0, 1.0], achieving adaptive filtering of features at different sampling points through weight adjustment, thereby improving the ability of deformation field prediction to perceive the boundaries of the anatomical structure. The feature weighting calculation rule uniformly adopts the method of multiplying the feature value by the modulation weight and then summing the results, ensuring the consistency of the calculation logic of the deformable convolution output features.

[0112] In this embodiment, a deformable convolutional layer learns the spatial deformation mapping relationship between the image to be registered and the reference image. Adaptive sampling and feature weighting are achieved based on the offset prediction map and the modulation coefficient prediction map to generate the initial deformation field. A regularization loss term, including a spatial smoothing penalty term and a deformation folding penalty term, is introduced to constrain the spatial continuity of the initial deformation field, suppress deformation folding, and improve the prediction accuracy and stability of the initial deformation field, providing a reliable foundation for subsequent adaptive contrast correction.

[0113] In a preferred embodiment, local gray-level histogram features of the image to be registered and the reference image at corresponding spatial locations are extracted. Based on these local gray-level histogram features, a gray-level distribution offset between the image to be registered and the reference image is calculated. Specifically, this includes: defining cubic windows with side length N pixels, centered on the same preset coordinate point in both the image to be registered and the reference image; counting the pixel frequency of the image to be registered in each gray-level interval within the cubic window to generate a first local gray-level histogram feature vector; counting the pixel frequency of the reference image in each gray-level interval within the cubic window to generate a second local gray-level histogram feature vector; calculating the Wasserstein distance between the first and second local gray-level histogram feature vectors; and using the Wasserstein distance as the gray-level distribution offset at the preset coordinate point.

[0114] For the preset coordinate points The images to be registered are in The first local gray-level histogram feature vector within the centered cube window is The feature vector of the second local grayscale histogram of the reference image within the same window is: ,in The number of grayscale intervals is preset to 9, consistent with the number of grayscale intervals in Table 1. Both feature vectors have been normalized to satisfy... , ,in This is an index for the grayscale range.

[0115] The formula for calculating the first-order Wasserstein distance between two eigenvectors is:

[0116]

[0117] in, Let be the cumulative distribution function of the feature vector of the first local gray-level histogram, calculated using the following formula: ; The cumulative distribution function of the feature vector of the second local gray-level histogram is calculated using the following formula: ; This is an absolute value operation. The calculated Wasserstein distance is... That is, the preset coordinate point The gray-level distribution offset at a given location is positively correlated with the degree of difference in the distribution between the two local gray-level histograms.

[0118] The side length N of the cube window is preset to 7 pixels, that is, the size of the cube window is 7×7×7 pixels. The preset coordinate points are divided into grids in the image space according to a fixed step size of 3 pixels to ensure that there is a spatial overlap between adjacent cube windows.

[0119] Further, converting the grayscale distribution offset into a contrast compensation coefficient includes: constructing a multilayer perceptron containing two fully connected layers. The grayscale distribution offset is input into the first fully connected layer of the multilayer perceptron, which maps the input grayscale distribution offset to the hidden layer feature space. The output vector of the hidden layer feature space is input into the second fully connected layer of the multilayer perceptron, which maps the output vector to a real number range between 0 and 1 using a sigmoid activation function. The real number output by the sigmoid activation function is used as the contrast compensation coefficient corresponding to the grayscale distribution offset, wherein the value of the contrast compensation coefficient monotonically increases as the grayscale distribution offset increases.

[0120] The input to the multilayer perceptron is the grayscale distribution offset. The first fully connected layer will input a scalar Mapped to dimension The hidden layer feature space, The default value is 16, which represents the hidden layer feature vector. The calculation formula is:

[0121]

[0122] in, This is the weight matrix of the first fully connected layer. For the bias term of the first fully connected layer, It is a linear rectification activation function.

[0123] The second fully connected layer will store the hidden layer feature vectors The output is mapped to a scalar value, processed by the Sigmoid activation function, and then contrast compensation coefficients are generated. The calculation formula is as follows:

[0124]

[0125] in, This is the weight matrix for the second fully connected layer. For the bias term of the second fully connected layer, The sigmoid activation function has an output range between 0 and 1. The training process of the multilayer perceptron is completed using paired medical image data. The training objective is to minimize the registration error between the registered image and the reference image, ensuring that the generated contrast compensation coefficient is monotonically increasing with the grayscale distribution offset.

[0126] Please refer to the attached document. Figure 4 Furthermore, after using the Wasserstein distance as the grayscale distribution offset at the preset coordinate points, it is determined whether there is a spatial overlap region between the cube windows corresponding to adjacent preset coordinate points. If the spatial overlap region exists, the grayscale values ​​of the pixels located within the spatial overlap region in the image to be registered are weighted and allocated according to the inverse spatial distance from the adjacent preset coordinate points to the pixels, generating updated first local grayscale histogram feature vectors and updated second local grayscale histogram feature vectors. The grayscale distribution offset is recalculated based on the updated feature vectors, and the recalculated grayscale distribution offset is input into the multilayer perceptron to generate target contrast compensation coefficients. A three-dimensional Gaussian filtering operation is performed on the target contrast compensation coefficients to generate filtered target contrast compensation coefficients.

[0127] For any pixel within the spatially overlapping region The set of its adjacent preset coordinate points is ,in The number of adjacent preset coordinate points, in pixels. to the preset coordinate point The Euclidean distance is Then the pixel The grayscale value for the preset coordinate point Weights of local gray-level histogram statistics The calculation formula is:

[0128]

[0129] in, For L2 norm operations, weights satisfy .

[0130] pixel The grayscale values ​​are determined by weight. In the pixel frequency statistics of the cube windows allocated to each adjacent preset coordinate point, updated first and second local gray-level histogram feature vectors are generated. Based on the updated feature vectors, the gray-level distribution offset is recalculated and input into a multilayer perceptron to generate the target contrast compensation coefficient. ,in These are preset coordinate points.

[0131] Please refer to the attached document. Figure 5 A three-dimensional Gaussian filter operation is performed on the target contrast compensation coefficients across the entire space. The filtered contrast compensation coefficients are then obtained. The calculation formula is:

[0132]

[0133] in, For coordinate points The Gaussian filter window centered on the target is a 3×3×3 window by default. It is a three-dimensional Gaussian kernel function. The standard deviation of the Gaussian kernel is set to 1.0. The 3D Gaussian filtering operation smooths the contrast compensation coefficients of adjacent coordinate points, avoiding abrupt changes in local compensation coefficients and improving the stability of the gradient correction process.

[0134] Please refer to the attached document. Figure 6Further, the contrast compensation coefficient is applied to the initial deformation field gradient calculated by the deformable convolutional layer during the backpropagation stage to adaptively correct the contrast of the spatial displacement vectors contained in the initial deformation field, generating the target deformation field. This includes: during the backpropagation process of the deformable convolutional layer, obtaining the partial derivative matrix of the loss function with respect to each of the spatial displacement vectors in the initial deformation field; expanding the contrast compensation coefficient into a diagonal weight matrix with the same dimension as the partial derivative matrix; calculating the product of the diagonal weight matrix and the partial derivative matrix to generate a corrected partial derivative matrix; replacing the partial derivative matrix with the corrected partial derivative matrix, continuing the gradient descent update process of the network weights in the deformable convolutional layer, and re-performing forward propagation with the updated network weights to generate the target deformation field.

[0135] During backpropagation of deformable convolutional layers, the total loss function For the initial deformation field The partial derivative matrix is Among them, spatial location The partial derivative vector at point is .

[0136] For spatial location Its corresponding contrast compensation coefficient is Expand it into a 3×3 diagonal weight matrix The calculation formula is:

[0137]

[0138] The corrected partial derivative vector at this spatial location The calculation formula is:

[0139]

[0140] The modified partial derivative matrix is ​​composed of the modified partial derivative vectors of the entire space. Its dimension is the same as the original partial derivative matrix. Completely identical. Using the corrected partial derivative matrix. Replace the original partial derivative matrix The backpropagation process involves chained differentiation to calculate the gradients of the network weights in the deformable convolutional layer, completing the gradient descent update process. The updated network weights then undergo a re-execution of the forward propagation process to generate the target deformation field, which has been adaptively corrected for contrast.

[0141] Further, the partial derivative matrix is ​​replaced with the corrected partial derivative matrix, and the gradient descent update process of the network weights in the deformable convolutional layer is continued. This includes: calculating the L2 norm of each element in the corrected partial derivative matrix, scaling elements with L2 norms greater than a preset clipping threshold proportionally to the preset clipping threshold, and generating a clipping corrected partial derivative matrix. The gradient descent update process of the network weights is then performed using the clipping corrected partial derivative matrix. The step of inputting the target image sequence into a temporal difference network to extract pathological evolution features and output disease progression assessment results includes: inputting the reference image and the target image after spatial transformation using the target deformation field into the feature alignment layer of the temporal difference network, calculating the absolute difference feature map between the reference image features and the target image features, and inputting the absolute difference feature map into a three-dimensional convolutional layer to extract the pathological evolution features.

[0142] For the corrected partial derivative matrix any spatial location Partial derivative vector at point The formula for calculating the second norm is as follows: The preset cropping threshold is The value is 1.0. The cropped partial derivative vector. The calculation formula is:

[0143]

[0144] in, Let be the local constant, and let its value be . This is used to avoid cases where the denominator is 0. The partial derivative vectors after clipping the entire space form the clipped corrected partial derivative matrix. The gradient descent update process of the deformable convolutional layer network weights is performed based on the cropped and corrected partial derivative matrix, which avoids gradient explosion during backpropagation and maintains the stability of network training.

[0145] The feature alignment layer of the temporal difference network receives the reference image. Registered target image First, the features of the reference image are extracted through a feature extraction branch with shared weights. With target image features The feature extraction branch uses the same network structure as the aforementioned feature encoder to ensure consistency in feature extraction logic. Absolute difference feature map. The calculation formula is:

[0146]

[0147] in, For feature channel index, , , For spatial coordinate index, This is for absolute value operations. Absolute difference feature map. The feature differences between the reference and target images were encoded, corresponding to changes in the subject's anatomical structure and lesion area between the two image acquisition time points. The absolute difference feature map was input into the three-dimensional convolutional layer of the temporal difference network to extract high-dimensional pathological evolution features. The pathological evolution features were then input into the fully connected layer and the classification / regression head to output the disease progression assessment results, including quantitative values ​​of lesion volume changes, predicted results of pathological grade changes, and disease progression risk level.

[0148] Table 4 Mapping between grayscale distribution offset and contrast compensation coefficient

[0149] [0.00,0.10) 16 [0.0,0.2) [0.0,0.2) [0.10,0.25) 16 [0.2,0.4) [0.2,0.4) [0.25,0.40) 16 [0.4,0.6) [0.4,0.6) [0.40,0.60) 16 [0.6,0.8) [0.6,0.8) [0.60,1.00] 16 [0.8,1.0] [0.8,1.0]

[0150] The mapping relationship specified in Table 4 is executed during the transformation of grayscale distribution offset to contrast compensation coefficient. The multilayer perceptron uses a fixed hidden layer dimension to ensure the consistency of feature mapping. The division of grayscale distribution offset intervals corresponds linearly to the value interval of contrast compensation coefficients, ensuring that the contrast compensation coefficients monotonically increase with the increase of grayscale distribution offset. Low grayscale distribution offset intervals correspond to low contrast compensation coefficients, with smaller adjustments to the initial deformation field gradient, preserving the original gradient update direction; high grayscale distribution offset intervals correspond to high contrast compensation coefficients, with larger adjustments to the initial deformation field gradient, amplifying the gradient update weights in grayscale difference regions and correcting deformation field prediction bias caused by inconsistent grayscale distribution.

[0151] In this embodiment, the local gray-level distribution difference between the image to be registered and the reference image is quantified using Wasserstein distance to generate a gray-level distribution offset. A corresponding contrast compensation coefficient is then generated based on a multilayer perceptron. During the backpropagation stage, the initial deformation field gradient is adaptively corrected using the contrast compensation coefficient, eliminating the interference of gray-level distribution differences on deformation field prediction. Spatial inverse distance weighting is applied to the gray-level values ​​in overlapping regions, combined with 3D Gaussian filtering to smooth the contrast compensation coefficient, avoiding abrupt changes in local compensation. Gradient clipping is performed on the corrected partial derivative matrix to maintain the stability of network training. Based on the registered target image sequence, pathological evolution features are extracted using a temporal difference network, outputting disease progression assessment results, providing a reliable quantitative basis for clinical auxiliary diagnosis.

Claims

1. A method for assisting diagnosis through health image data analysis and processing, characterized in that, include: Acquire a sequence of health images of the same subject at different time points, and divide the sequence of health images into images to be registered and reference images; The image to be registered and the reference image are input into a preset feature encoder to extract the first multimodal feature map of the image to be registered and the second multimodal feature map of the reference image; The first multimodal feature map and the second multimodal feature map are input into a deformable convolutional layer to predict the initial deformation field of the image to be registered relative to the reference image. Extracting local grayscale histogram features of the image to be registered and the reference image at corresponding spatial locations, calculating the grayscale distribution offset between the image to be registered and the reference image based on the local grayscale histogram features, and converting the grayscale distribution offset into a contrast compensation coefficient, including: constructing a multilayer perceptron containing two fully connected layers; The grayscale distribution offset is input into the first fully connected layer of the multilayer perceptron, and the first fully connected layer maps the input grayscale distribution offset to the hidden layer feature space. The output vector of the hidden layer feature space is input into the second fully connected layer of the multilayer perceptron, and the second fully connected layer maps the output vector to the real number interval between 0 and 1 through the Sigmoid activation function; The real number output by the Sigmoid activation function is used as the contrast compensation coefficient corresponding to the gray-scale distribution offset, wherein the value of the contrast compensation coefficient increases monotonically as the gray-scale distribution offset increases. Applying the contrast compensation coefficient to the initial deformation field gradient calculated by the deformable convolutional layer during the backpropagation stage, and performing contrast adaptive correction on the spatial displacement vectors contained in the initial deformation field to generate the target deformation field, includes: obtaining the partial derivative matrix of the loss function with respect to each of the spatial displacement vectors in the initial deformation field during the backpropagation process of the deformable convolutional layer. The contrast compensation coefficient is extended into a diagonal weight matrix with the same dimension as the partial derivative matrix; Calculate the product of the diagonal weight matrix and the partial derivative matrix to generate the modified partial derivative matrix; The partial derivative matrix is ​​replaced by the corrected partial derivative matrix, and the gradient descent update process of the network weights in the deformable convolutional layer is continued. The forward propagation is re-executed with the updated network weights to generate the target deformation field. The target deformation field is used to perform spatial transformation on the image to be registered, and the registered target image sequence is output. The target image sequence is input into a temporal difference network to extract pathological evolution features and output disease progression assessment results.

2. The method for assisting diagnosis by analyzing and processing health image data according to claim 1, characterized in that, The step of inputting the image to be registered and the reference image into a preset feature encoder to extract the first multimodal feature map of the image to be registered and the second multimodal feature map of the reference image includes: the feature encoder includes three convolutional blocks connected in sequence, each of the convolutional blocks containing cascaded dilated convolutional layers and residual connection branches; Regarding the first Each convolutional block takes the input feature map as input, and the hole rate is... The hollow convolutional layer is connected to the residual connection branch, wherein The value of varies The increase of is exponential; The output feature map of the dilated convolutional layer and the output feature map of the residual connection branch are concatenated along the channel dimension. The concatenated feature map is then input into the first convolutional kernel for dimensionality reduction to generate the first convolutional layer. Output feature maps of each convolutional block; The output feature map of the third convolutional block is used as the first multimodal feature map and the second multimodal feature map, respectively.

3. The method for assisting diagnosis by analyzing and processing health image data according to claim 2, characterized in that, The step of inputting the first multimodal feature map and the second multimodal feature map into a deformable convolutional layer to predict the initial deformation field of the image to be registered relative to the reference image includes: concatenating the first multimodal feature map and the second multimodal feature map in the channel dimension to generate a fused feature map; The fused feature map is input into the second convolutional kernel to generate an offset prediction map and a modulation coefficient prediction map; For each sampling point in the fused feature map, the three-dimensional offset vector corresponding to the sampling point is extracted from the offset prediction map, and the scalar modulation weight corresponding to the sampling point is extracted from the modulation coefficient prediction map. Based on the three-dimensional offset vector, the neighborhood offset sampling points of the sampling point are determined in the fused feature map. The feature vector of the sampling point and the feature vector of the neighborhood offset sampling points are weighted and summed according to the scalar modulation weight. The weighted summation result is recombined into the initial deformation field containing the set of three-dimensional spatial displacement vectors.

4. The method for assisting diagnosis by analyzing and processing health image data according to claim 3, characterized in that, The step of extracting the local grayscale histogram features of the image to be registered and the reference image at corresponding spatial positions, and calculating the grayscale distribution offset between the image to be registered and the reference image based on the local grayscale histogram features, includes: delineating a cube window with a side length of N pixels, centered on the same preset coordinate point in the image to be registered and the reference image. The pixel frequency of the image to be registered in each gray-level interval within the cube window is counted to generate a first local gray-level histogram feature vector. The pixel frequency of the reference image in each gray-level interval within the cube window is counted to generate a second local gray-level histogram feature vector. Calculate the Wasserstein distance between the first local gray-level histogram feature vector and the second local gray-level histogram feature vector, and use the Wasserstein distance as the gray-level distribution offset at the preset coordinate point.

5. The method for assisting diagnosis by analyzing and processing health image data according to claim 4, characterized in that, Before performing the channel-dimensional concatenation operation between the output feature map of the dilated convolutional layer and the output feature map of the residual connection branch, the method further includes: inputting the output feature map of the dilated convolutional layer into the channel attention calculation branch, and the channel attention calculation branch performing a global average pooling operation on the output feature map of the dilated convolutional layer to generate a one-dimensional channel descriptor. The one-dimensional channel descriptor is sequentially input into a dimensionality-reducing fully connected layer and an dimensionality-increasing fully connected layer. The output vector of the dimensionality-increasing fully connected layer is then input into the Sigmoid function to generate a channel attention weight vector. The channel attention weight vector is multiplied channel by channel by channel by the output feature map of the residual connection branch, and the feature map after multiplication is used as the output feature map of the residual connection branch participating in the splicing operation.

6. The method for assisting diagnosis by analyzing and processing health image data according to claim 5, characterized in that, After recombining the weighted summation result into the initial deformation field containing a set of three-dimensional spatial displacement vectors, the method further includes: extracting the displacement component maps of the initial deformation field in the three directions of the X-axis, Y-axis and Z-axis; For each direction of the displacement component map, calculate the displacement gradient value between adjacent pixels in the displacement component map, sum the absolute values ​​of the displacement gradient values, and generate a spatial smoothing penalty term. Calculate the absolute value of the determinant of the Jacobian matrix corresponding to the initial deformation field, and use the number of pixels whose absolute value is less than a preset constant as the deformation folding penalty term. The spatial smoothing penalty term and the deformation folding penalty term are added together according to a preset weight to generate a regularization loss term, which is then superimposed on the total loss function of the deformable convolutional layer.

7. The method for assisting diagnosis by analyzing and processing health image data according to claim 6, characterized in that, After using the Wasserstein distance as the grayscale distribution offset at the preset coordinate point, the method further includes: determining whether there is a spatial overlap region between the cube windows corresponding to adjacent preset coordinate points. If the spatial overlap region exists, the gray values ​​of the pixels located in the spatial overlap region in the image to be registered are weighted according to the spatial inverse distance from the adjacent preset coordinate points to the pixels, and an updated first local gray-level histogram feature vector and an updated second local gray-level histogram feature vector are generated. The grayscale distribution offset is recalculated based on the updated feature vector, and the recalculated grayscale distribution offset is input into the multilayer perceptron to generate the target contrast compensation coefficient. A three-dimensional Gaussian filtering operation is performed on the target contrast compensation coefficient to generate the filtered target contrast compensation coefficient.

8. The method for assisting diagnosis by analyzing and processing health image data according to claim 7, characterized in that, The step of replacing the partial derivative matrix with the modified partial derivative matrix and continuing to execute the gradient descent update process of the network weights in the deformable convolutional layer includes: calculating the L2 norm of each element in the modified partial derivative matrix, scaling the elements whose L2 norm is greater than the preset pruning threshold proportionally to the preset pruning threshold, and generating a pruning modified partial derivative matrix. The gradient descent update process of the network weights is performed using the cropped partial derivative matrix. The step of inputting the target image sequence into a temporal difference network to extract pathological evolution features and output disease progression assessment results includes: inputting the reference image and the target image after spatial transformation using the target deformation field into the feature alignment layer of the temporal difference network; calculating the absolute difference feature map between the features of the reference image and the features of the target image; and inputting the absolute difference feature map into a three-dimensional convolutional layer to extract the pathological evolution features.

Images