A neural image spatio-temporal coding method based on dynamic course learning

Abstract

This invention belongs to the field of medical image processing technology and discloses a neural image spatiotemporal coding method based on dynamic curriculum learning, including the following steps: S1, inputting a three-dimensional slice image and preprocessing it; S2, performing approximate sorted pooling spatiotemporal coding on the preprocessed slice image to output a reconstructed two-dimensional dynamic image; S3, performing hierarchical and intra-stage iterative training based on a dynamic curriculum learning strategy to achieve adaptive stage switching; S4, performing adaptive dynamic channel relabeling and spatial attention based on a dynamic grouping mechanism to obtain the final output of the corresponding stage; S5, after all stages are completed, sending the processing results into a task head for classification, segmentation, or regression. This invention is suitable for computationally limited environments, clearly demonstrates the evolution process of features from macro to micro, conforms to clinical diagnostic logic, and significantly improves classification accuracy.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically relating to a spatiotemporal coding method for neuroimaging based on dynamic curriculum learning. Background Technology

[0002] With the increasing incidence of neurodegenerative diseases (such as Alzheimer's disease) and brain tumors, early and accurate imaging diagnosis has become a pressing clinical need. However, existing technologies have the following shortcomings: While existing 3D convolutional neural networks (3DCNNs) can capture complete spatial structures, they have a large number of model parameters, high GPU memory requirements, and large sample size requirements, which are not conducive to resource-constrained clinical environments. Existing technologies use 2D CNNs after slicing 3D sequences into 2D images, which, although computationally efficient, lose the spatial continuity between slices and struggle to capture global structures and subtle lesions. Meanwhile, traditional pooling (Mean / Max) methods ignore the slice order and cannot reflect the progression of the disease. On the other hand, existing curriculum learning (CL) relies heavily on static, manually segmented data and cannot adapt to the complexity of images; large-scale general-purpose pre-trained models (such as MedSAM and BioGPT) struggle to achieve optimal performance in zero-shot inference due to data scarcity, computational costs, and privacy requirements. Summary of the Invention

[0003] The purpose of this invention is to provide a spatiotemporal coding method for neural images based on dynamic curriculum learning, in order to solve the technical problems of existing technologies that are difficult to reduce computational resource consumption while ensuring sufficient performance, and are difficult to adapt to the complexity of medical images in terms of capturing global structures and subtle lesions.

[0004] The method for preparing a neural image spatiotemporal coding method based on dynamic curriculum learning includes the following steps:

[0005] S1. Input the 3D slice image and perform preprocessing;

[0006] S2. Perform approximate sorted pooling spatiotemporal coding on the preprocessed slice image to output a reconstructed two-dimensional dynamic image;

[0007] S3. Implement layered and phased iterative training based on dynamic course learning strategies to achieve adaptive phase switching;

[0008] S4. Adaptive dynamic relabeling of channels and spatial attention based on dynamic grouping mechanism to obtain the final output of the corresponding stage;

[0009] S5. After all stages are completed, the processing results are sent to the task header for classification, segmentation, or regression.

[0010] Preferably, step S2 includes:

[0011] S2.1 Slice feature extraction: Feature vectors are extracted from the ordered three-dimensional slice sequence through a shallow coding network;

[0012] S2.2 Harmonic number and weight calculation: The harmonic number is set to non-linearly characterize the position, and the weight is calculated by combining the first term emphasizing the early slice and the second term emphasizing the later slice to balance the global temporal information.

[0013] S2.3 Feature aggregation: After obtaining the weights, the feature vectors are aggregated to generate a two-dimensional dynamic image descriptor, resulting in a reconstructed two-dimensional dynamic image.

[0014] Preferred, For an ordered 3D slice sequence, the formula for extracting the feature vector is: , Indicates shallow coding network f enc The d-dimensional feature vector of the extracted t-th slice; harmonic number H t and weight The calculation formulas are as follows: ,

[0015] in, T The total number of slices in the 3D volume data determines the temporal length and affects the upper bound of the harmonic number. t This represents the index of the current slice in the sequence, and 1 ≤ t ≤ T ; Ht For the first t Harmonic number; H T -H t-1 The obtained value follows t Increases monotonically and decreases; α t Indicates the first t The weight of each slice, the first term in the weight calculation formula is 2( Tt +1), the second item is ( T +1)( H T -H t-1 Two-dimensional dynamic image descriptor D * The calculation formula is as follows: In the formula, This represents the weighted aggregated dynamic image descriptor, which is a two-dimensional dynamic image. X The flattening characteristic of 0.

[0016] Preferably, step S3 includes:

[0017] S3.1, Set the number of stages. Using the reconstructed two-dimensional dynamic graph as input, set... N Each stage;

[0018] S3.2 Calculate the complexity of the current iteration input in this stage;

[0019] S3.3 Perform threshold judgment and set the complexity threshold corresponding to this stage. If the complexity of the current iteration input is greater than the complexity threshold, execute step S4; otherwise, continue iterative training of dynamic course learning in this stage.

[0020] Preferably, for the first i Phase input X i-1 computational complexity l ( X i-1 The complexity is calculated as follows:

[0021] ,

[0022] In the formula, , X i-1 Let C be the input feature map of the i-th stage, and C be the number of channels. Indicates the first i Phase input X i-1 For the first c Spatial gradient of the channel; It represents the sum of absolute values ​​at the element level, measuring the total magnitude of the gradient.

[0023] Preferably, step S4 includes: for the first i Input feature map of the stage X i-1 First, perform grouped convolution, then ReLU activation, then grouped convolution again, and finally Sigmoid activation to calculate the weights for each stage. W Next, the weighted feature map is calculated, and finally, group point convolution is performed to obtain the output of the corresponding stage.

[0024] Preferably, the expression for step S4 is:

[0025] W=σ ( Conv G ( ReLU ( Conv G ( X i-1 ))))

[0026] X ′=X i-1 ⊙ W ,

[0027] X i =GpConv( X ′);

[0028] In the formula, Conv G This is a grouped convolution, where the number of groups G can be adjusted according to C; ReLU(·) is the linear rectified activation function ReLU, which introduces non-linearity; s (·) indicates the attention weights of the output channels and space after activation via the Sigmoid function. W ∈(0 , 1) H×W×C ;⊙ indicates element-wise multiplication. X ′ represents a weighted feature map used to highlight important regions; GpConv represents group point convolution used to further fuse the relabeled features.

[0029] Preferably, in step S5, the output of step S4 is obtained and used as the input for the next stage. After iterative training of each stage is completed in this way, the output of the final stage is the result. The task head classifies, segments, or regresses the result, including:

[0030] S5.1 Classification: The AdamW optimizer is used in conjunction with the cosine annealing learning rate scheduling strategy to achieve rapid convergence in the early stage of training and to refine and adjust the weights in the later stage to effectively prevent overfitting.

[0031] S5.2 Segmentation: Lovász series loss is used to improve segmentation quality by directly optimizing the IoU index; and Boundary Loss is used to further refine the edges.

[0032] S5.3 Regression: The OneCycleLR strategy is adopted to accelerate the model out of saddle points and improve regression stability by periodically changing the learning rate.

[0033] Preferably, in step S1, T slices are extracted sequentially from the data source file, uniformly cropped to the same size, and normalized.

[0034] The technical advantages of this invention are as follows: On the one hand, this invention employs Approximate Rank Pooling (ARP) spatiotemporal coding technology to reconstruct ordered 3D slices into a 2D dynamic image; on the other hand, it achieves adaptive grouping training based on image complexity by combining dynamic grouping mechanism with dynamic course learning technology. DGM uses grouped convolution and depthwise separable convolution, requiring only... ～With 8M parameters and approximately 1.1 GFLOPS, it is suitable for computing-constrained environments. The dynamic graphs of ARP output and the visualization of t-SNE and Grad-CAM at each stage clearly demonstrate the evolution of features from macro to micro, which aligns with clinical diagnostic logic.

[0035] Compared to existing technologies (such as Mean / Max Pooling), this invention significantly improves classification accuracy in spatiotemporal coding. Its weight calculation method during reconstruction balances global temporal information, and the dynamic course learning process enables adaptive stage switching, improving training stability and convergence speed. Dynamic relabeling using DGM significantly enhances sensitivity to minute lesions. The overall framework (DCL-SE framework) is highly versatile; simply replacing the task head supports various applications such as 2D / 3D classification, segmentation, and brain age regression. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the overall process of a neural image spatiotemporal coding method based on dynamic curriculum learning according to the present invention.

[0037] Figure 2 This is a flowchart of step 2 of the present invention, which involves ARP spatiotemporal coding.

[0038] Figure 3 This is a flowchart of step 4 of the present invention, which performs adaptive dynamic recalibration based on DGM. Detailed Implementation

[0039] The following detailed description of the embodiments, with reference to the accompanying drawings, will further illustrate the specific implementation of the present invention, in order to help those skilled in the art to have a more complete, accurate, and in-depth understanding of the inventive concept and technical solution of the present invention.

[0040] like Figure 1-Figure 3 As shown, this invention provides a neural image spatiotemporal encoding method based on dynamic curriculum learning. This method constructs the Dynamic Curriculum Learning for Spatiotemporal Encoding (DCL-SE) framework and includes the following steps.

[0041] S1. Input the 3D slice image and perform preprocessing.

[0042] In this embodiment, the three-dimensional slice image is a clinical brain MRI / CT three-dimensional slice image. For example, T slices are extracted sequentially from the NIfTI / DICOM file, uniformly cropped to H×W (height×width) size, and normalized to [0, 1] (height and width).

[0043] S2. Perform Approximate Rank Pooling (ARP) spatiotemporal coding on the preprocessed slice image to output the reconstructed two-dimensional dynamic image. t

[0044] Specifically, this step includes the following sub-steps.

[0045] S2.1, Slice Feature Extraction. Ordered 3D Slice Sequence Through shallow coding network f enc Extracting feature vectors The calculation formula is: , Let represent the d-dimensional feature vector of the t-th slice extracted by the shallow coding network.

[0046] S2.2, Harmonic Number and Weight Calculation. Corresponding Harmonic Number H t and weight The calculation formulas are as follows:

[0047] ,

[0048] in, T The total number of slices in the 3D volume data determines the temporal length and affects the upper bound of the harmonic number. t This represents the index of the current slice in the sequence, and 1 ≤ t ≤ T ; H t For the first t The order harmonic number is used to characterize the nonlinearity of position; H T - H t-1 The obtained value follows t Increases monotonically and decreases; α t Indicates the first t The weight of each slice, the first term in the weight calculation formula is 2( Tt +1) Emphasizes early sections, the second item ( T +1)( H T -H t-1 It emphasizes late-stage slicing, and the combination of the two is used to balance global temporal information.

[0049] S2.3 Feature Aggregation. After obtaining the weights, the feature vectors are aggregated to generate a two-dimensional dynamic image descriptor. D * The calculation formula is as follows:

[0050] ,

[0051] In the formula, The weighted aggregated dynamic image descriptor can be viewed as a two-dimensional dynamic graph. X The flattening feature of 0. This results in a reconstructed two-dimensional dynamic graph. X 0 preserves spatial process information across slices.

[0052] S3. Based on the Dynamic Curriculum Learning (DCL) strategy, perform layered and phase-based iterative training to achieve adaptive phase switching.

[0053] Specifically, this step includes the following sub-steps.

[0054] S3.1, Set the number of stages. Using the reconstructed two-dimensional dynamic graph as input, set... N There are several stages. For example, if the number of stages N=4, the module order is S1→S4→S6→C1, where S1, S4, S6, and C1 are all set stage names, and the complexity of the input processing is arranged from low to high.

[0055] S3.2 Calculate the complexity of the current iteration input in this stage. For the ... i Phase input X i-1 computational complexity l ( X i-1 The complexity is calculated as follows:

[0056] ,

[0057] In the formula, , X i-1 Let C be the input feature map of the i-th stage, and C be the number of channels. Indicates the first i Phase input X i-1 For the first c The spatial gradient of a channel is often calculated using Sobel or differential operators to capture local texture changes; This represents the sum of absolute values ​​at the element-wise level, measuring the total magnitude of the gradient, and is derived from it. l ( X i-1 The larger the value, the more complex the characteristics.

[0058] S3.3, Perform threshold judgment. Set the corresponding threshold. i Stage complexity threshold t i ,when l ( Xi-1 > t i If the condition is met, proceed to step S4; otherwise, continue iterative training for dynamic course learning in this stage. t i It can be set adaptively through cross-validation or based on the complexity distribution in the training set.

[0059] S4. Adaptive dynamic relabeling of channels and spatial attention based on the Dynamic Group Mechanism (DGM) is performed to obtain the final output of the corresponding stage, thereby significantly enhancing the sensitivity to small lesions.

[0060] In this step, for the first i Input feature map of the stage X i-1 First, perform grouped convolution, then ReLU activation, then grouped convolution again, and finally Sigmoid activation to calculate the weights for each stage. W Next, the weighted feature map is calculated, and finally, grouped point convolution is performed to obtain the output of the corresponding stage. The expression for this step is:

[0061] W=σ ( Conv G ( ReLU ( Conv G ( X i-1 ))))

[0062] X ′= X i-1 ⊙ W ,

[0063] X i =GpConv( X ′);

[0064] In the formula, Conv G This technique, known as group convolution, reduces the number of parameters, where the number of groups G can be adjusted according to C; ReLU(·) is the linear rectified activation function ReLU, which introduces non-linearity. s (·) indicates the attention weights of the output channels and space after activation via the Sigmoid function. W ∈(0 , 1) H×W×C ;⊙ indicates element-wise multiplication. X' represents the weighted feature map, used to highlight important regions; GpConv represents group pointwise convolution, used to further fuse the re-labeled features. The output of this step corresponds to the first step. i Stage output X i This is also the input for the next stage.

[0065] S5. After all stages are completed, the processing results are sent to the task header for classification, segmentation, or regression.

[0066] Step S4 obtains the output of this stage and uses it as the input for the next stage. This iterative training is repeated for each stage. The output of the final stage is the result. The task head then classifies, segments, or regresses the result to obtain the prediction result. The training details for the prediction results are as follows.

[0067] S5.1 Classification:

[0068] – Optimizer: AdamW, learning rate lr =1×10 -3 Weight decay 0 . 01.

[0069] –Learning rate scheduling: Cosine Annealing, which uses cosine annealing to smooth the decrease of the learning rate and avoid oscillations.

[0070] – Training cycle: 300 epochs to ensure the model fully converges.

[0071] In this step, the AdamW optimizer is used in conjunction with the learning rate scheduling strategy Cosine Annealing to achieve rapid convergence in the early stages of training and to refine and adjust the weights in the later stages, effectively preventing overfitting.

[0072] S5.2. Segmentation:

[0073] –Loss function:

[0074] *Lovász-Softmax: Optimizes IoU for multi-class classification.

[0075] *Lovász-Hinge: Optimized for binary classification interface.

[0076] *Boundary Loss: Enhances boundary accuracy.

[0077] - Weighting ratio: The weighting ratio of the loss function is 0.5:0.3:0.2, balancing the two types of targets: the region and the boundary.

[0078] In this step, the Lovász series loss is used to improve the segmentation quality by directly optimizing the IoU index; and the edges are further refined by Boundary Loss.

[0079] S5.3 Regression:

[0080] –Loss function: L1 loss, which robustly penalizes prediction bias.

[0081] –Learning rate scheduling: OneCycleLR improves generalization and convergence speed by rapidly increasing the learning rate and then decreasing it.

[0082] This step employs the OneCycleLR strategy to accelerate the model's escape from saddle points and improve regression stability by periodically varying the learning rate.

[0083] This approach was compared with other existing technologies through experiments. The specific results for the 3D Alzheimer's image classification task are shown in Table 1 below. Here, Fold represents the x-th fold of cross-validation; the experiment used three-fold cross-validation, and BE represents the best round result.

[0084] Table 1: Performance comparison on the Alzheimer's Disease (AD) 3D dataset

[0085]

[0086] Table 2 shows the performance comparison results of various models for cerebral artery segmentation on the DCL-SE and CAS2023 datasets. The dataset type is 3D MRA (.ni.gz), derived from the CodaLab competition (https: / / codalab.lisn.upsaclay.fr / competitions / 9804). Results are provided in the form of DSC values. Note that DSC0 represents the DSC value of the background, while DSC1 represents the DSC value of the vascular tissue.

[0087] Table 2: Performance comparison of DCL-SE and different models in cerebral artery segmentation

[0088]

[0089] Table 3 summarizes the brain age prediction results under full supervision (validation MAE).

[0090] Table 3: Model Performance Comparison on the SPR Head CT Age Prediction Challenge Dataset

[0091]

[0092] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A spatiotemporal coding method for neural images based on dynamic curriculum learning, characterized in that: Includes the following steps: S1. Input the 3D slice image and perform preprocessing; S2. Perform approximate sorted pooling spatiotemporal coding on the preprocessed slice image to output a reconstructed two-dimensional dynamic image; S3. Implement layered and phased iterative training based on dynamic course learning strategies to achieve adaptive phase switching; S4. Adaptive dynamic relabeling of channels and spatial attention based on dynamic grouping mechanism to obtain the final output of the corresponding stage; S5. After all stages are completed, the processing results are sent to the task header for classification, segmentation, or regression.

2. The neural image spatiotemporal coding method based on dynamic curriculum learning according to claim 1, characterized in that: Step S2 includes: S2.1 Slice feature extraction: Feature vectors are extracted from the ordered three-dimensional slice sequence through a shallow coding network; S2.2 Harmonic number and weight calculation: The harmonic number is set to non-linearly characterize the position, and the weight is calculated by combining the first term emphasizing the early slice and the second term emphasizing the later slice to balance the global temporal information. S2.3 Feature aggregation: After obtaining the weights, the feature vectors are aggregated to generate a two-dimensional dynamic image descriptor, resulting in a reconstructed two-dimensional dynamic image.

3. The neural image spatiotemporal coding method based on dynamic curriculum learning according to claim 2, characterized in that: For an ordered 3D slice sequence, the formula for extracting the feature vector is: , Indicates shallow coding network f enc The d-dimensional feature vector of the extracted t-th slice; harmonic number H t and weight The calculation formulas are as follows: , in, T The total number of slices in the 3D volume data determines the temporal length and affects the upper bound of the harmonic number. t This represents the index of the current slice in the sequence, and 1 ≤ 0. t ≤ T ; Ht For the first t Harmonic number; H T -H t-1 The obtained value follows t Increases monotonically and decreases; α t Indicates the first t The weights of the slices are calculated using the formula where the first term is 2. Tt +1), the second item is ( T +1)( H T -H t-1 Two-dimensional dynamic image descriptor D * The calculation formula is as follows: In the formula, This represents the weighted aggregated dynamic image descriptor, which is a two-dimensional dynamic image. X The flattening characteristic of 0.

4. The neural image spatiotemporal coding method based on dynamic curriculum learning according to claim 1, characterized in that: Step S3 includes: S3.1, Set the number of stages; using the reconstructed two-dimensional dynamic graph as input, set... N Each stage; S3.2 Calculate the complexity of the current iteration input in this stage; S3.3 Perform threshold judgment and set the complexity threshold corresponding to this stage. If the complexity of the current iteration input is greater than the complexity threshold, execute step S4; otherwise, continue iterative training of dynamic course learning in this stage.

5. A neural image spatiotemporal coding method based on dynamic curriculum learning according to claim 4, characterized in that: For the i Phase input X i-1 computational complexity λ ( X i-1 The complexity is calculated as follows: ， In the formula, , X i-1 It is the input feature map of the i-th stage, and C is the number of channels; Indicates the first i Phase input X i-1 For the c Spatial gradient of the channel; It represents the sum of absolute values ​​at the element level, measuring the total magnitude of the gradient.

6. The neural image spatiotemporal coding method based on dynamic curriculum learning according to claim 4, characterized in that: Step S4 includes: for the first i Input feature map of the stage X i-1 First, perform grouped convolution, then ReLU activation, then grouped convolution again, and finally Sigmoid activation to calculate the weights for each stage. W Next, the weighted feature map is calculated, and finally, group point convolution is performed to obtain the output of the corresponding stage.

7. The neural image spatiotemporal coding method based on dynamic curriculum learning according to claim 6, characterized in that: The expression for step S4 is: W=σ ( Conv G ( ReLU ( Conv G ( X i-1 )))), X ′= X i-1 ⊙ W , X i = GpConv( X ′); In the formula, Conv G This is a grouped convolution, where the number of groups G can be adjusted according to C; ReLU(·) is the linear rectified activation function ReLU, which introduces non-linearity; σ (·) indicates that the attention weights of the output channel and space are calculated after activation using the Sigmoid function. W ∈(0 , 1) H×W×C ;⊙ indicates element-wise multiplication. X ′ represents a weighted feature map used to highlight important regions; GpConv represents group point convolution used to further fuse the relabeled features.

8. The neural image spatiotemporal coding method based on dynamic curriculum learning according to claim 1, characterized in that: In step S5, the output of step S4 is obtained and used as the input for the next stage. This iterative training is repeated for each stage. The output of the final stage is the result. The task head then classifies, segments, or regresses the result, including: S5.1 Classification: The AdamW optimizer is used in conjunction with the cosine annealing learning rate scheduling strategy to achieve rapid convergence in the early stage of training and to refine and adjust the weights in the later stage to effectively prevent overfitting. S5.2 Segmentation: Lovász series loss is used to improve segmentation quality by directly optimizing the IoU index; and Boundary Loss is used to further refine the edges. S5.3 Regression: The OneCycleLR strategy is adopted to accelerate the model out of saddle points and improve regression stability by periodically changing the learning rate.

9. The neural image spatiotemporal coding method based on dynamic curriculum learning according to claim 1, characterized in that: In step S1, T slices are extracted sequentially from the data source file, uniformly cropped to the same size, and normalized.

Images