Pituitary adenoma invasion image data analysis method and system

Abstract

The invention discloses a pituitary adenoma invasion image data analysis method and system, and relates to the technical field of medical prediction. The pituitary adenoma invasion image data analysis system comprises an image feature extraction module, an auxiliary factor analysis module and an invasion prediction and judgment module. According to the method, image data and auxiliary biomarker data of a patient are combined, a cross attention mechanism and a feature fusion strategy are adopted for analysis, information from different data sources can be fully utilized, and the comprehensiveness and precision of pituitary adenoma invasion prediction are improved; the pituitary adenoma image is processed by using a bidirectional propagation network, so that more detailed and accurate image features can be extracted; through the pituitary adenoma invasion prediction model, in combination with the invasion data auxiliary prediction factor and the patient image feature image, high-precision invasion prediction can be performed, key features of different modalities are fused, and the accuracy and reliability of invasion prediction can be improved.

Description

Technical Field

[0001] The present invention relates to the field of medical prediction technology, and in particular to a method and system for analyzing pituitary adenoma invasion image data. Background Art

[0002] The invasiveness of pituitary adenomas has a huge impact on the health of patients, especially when the tumor invades adjacent structures such as the cavernous sinus, optic chiasm and brainstem, which may lead to severe neurological dysfunction, vision loss, headache and hormone secretion disorders. Therefore, early and accurate prediction of the invasiveness of pituitary adenomas is crucial to optimize treatment plans, reduce complications and improve the quality of life of patients. Currently, conventional methods for imaging analysis of pituitary adenoma invasion mainly rely on MRI and CT images to evaluate the invasiveness of the tumor by observing its morphology, size, boundary and relationship with surrounding structures. However, traditional imaging technology is often unable to clearly show the subtle invasion of tumors and surrounding tissues due to resolution limitations, thus affecting the accuracy of prediction.

[0003] Therefore, it is necessary to improve the resolution of images and perform high-dimensional feature processing. At the same time, the fusion of imaging and biomarker data can provide more levels of information and further improve the accuracy and comprehensiveness of the prediction of pituitary adenoma invasion of the cavernous sinus, suprasellar region or other sites. Summary of the invention

[0004] The present invention aims to provide a method and system for analyzing pituitary adenoma invasion image data, and to predict invasion using high-dimensional feature pituitary adenoma image data.

[0005] A method for analyzing pituitary adenoma invasion image data comprises the following steps: Obtain a patient pituitary adenoma image set; the patient pituitary adenoma image set contains N consecutive patient pituitary adenoma images P with interval time windows. n , n=1, 2, ..., N, the length of the time window is L; based on the patient's pituitary adenoma image set and the pituitary adenoma image feature extraction model, the patient's pituitary adenoma invasion feature image is obtained; the pituitary adenoma image feature extraction model uses a bidirectional propagation network to process the patient's pituitary adenoma image set to obtain high-quality patient pituitary adenoma invasion feature images for subsequent analysis; Obtaining patient auxiliary biomarker data; performing analysis based on the patient auxiliary biomarker data and the pituitary adenoma biomarker prediction analysis model to obtain invasion data auxiliary prediction factors; Predictive analysis is performed based on invasion data auxiliary prediction factors, patient pituitary adenoma invasion characteristic images and pituitary adenoma invasion prediction models to obtain patient pituitary adenoma prediction analysis results; the pituitary adenoma invasion prediction model uses a cross-attention mechanism and a feature fusion strategy to perform predictive analysis for high-precision invasion prediction based on invasion data auxiliary prediction factors and patient pituitary adenoma invasion characteristic images; follow-up operations are performed on patients based on the patient pituitary adenoma prediction analysis results.

[0006] As a preferred technical solution of the present invention, the pituitary adenoma image feature extraction model includes an image preprocessing layer, an image feature extraction layer and an image output layer; The image preprocessing layer is used to perform image super-resolution processing on the patient pituitary adenoma image set to obtain a preprocessed patient pituitary adenoma image set; wherein the preprocessed patient pituitary adenoma image set includes N preprocessed patient pituitary adenoma images C n ; The image feature extraction layer is used to extract features from the preprocessed patient pituitary adenoma image set to obtain the patient's pituitary adenoma invasion feature image; The image output layer is used to output the patient's pituitary adenoma invasion feature image.

[0007] As a preferred technical solution of the present invention, the specific steps of performing image super-resolution processing in the image preprocessing layer include: The image preprocessing layer includes an image alignment layer and an image reconstruction layer; the image alignment layer includes a forward propagation network and a back propagation network; the image reconstruction layer includes a feature extraction network, a residual refinement network and an image reconstruction network; In the back propagation network, the patient’s pituitary adenoma image P is registered using the image registration method. n+1 , patient pituitary adenoma image P n+2 and patient pituitary adenoma images P n Align and obtain the reverse patient pituitary adenoma image set F n ; Reverse the patient's pituitary adenoma image set F n The image F is sent to the deformable convolution block for further feature alignment to obtain the reverse patient pituitary adenoma image F n '; In the forward propagation network, the patient’s pituitary adenoma image P is registered using the image registration method. n-1 , patient pituitary adenoma image P n-2 and patient pituitary adenoma images P n Align and obtain the positive patient pituitary adenoma image set Z n ; The positive patient pituitary adenoma image set Z n The image Z of the patient’s pituitary adenoma is sent to the deformable convolution block for further feature alignment. n '; In the image reconstruction layer, the feature extraction network is used to extract the reverse patient pituitary adenoma image F n ', positive patient pituitary adenoma image Z n ' and patient pituitary adenoma images P n Feature extraction and splicing are performed to obtain the patient's pituitary adenoma feature image T n ; Using residual refinement network to refine the patient's pituitary adenoma feature image T n Through multiple residual blocks, feature refinement is performed to obtain the refined patient pituitary adenoma feature image T n '; In the image reconstruction network, the patient's pituitary adenoma feature image T is refined n 'Use the pixel rearrangement method to expand four times and obtain a high-resolution patient pituitary adenoma feature image G n ; The patient's pituitary adenoma image P n The low-resolution patient pituitary adenoma feature image D is obtained by four times magnification using bilinear interpolation. n ; High-resolution patient pituitary adenoma feature image G n and low-resolution patient pituitary adenoma characteristic image D n Add them together to obtain the pre-processed patient pituitary adenoma image C n .

[0008] As a preferred technical solution of the present invention, the specific steps of performing feature extraction in the image feature extraction layer include: The image feature extraction layer includes a global feature extraction layer, a local feature division layer, and a feature scoring layer; In the global feature extraction layer, the preprocessed patient pituitary adenoma image C n Perform global feature extraction to obtain the patient's pituitary adenoma feature global feature vector Q n ; In the local feature division layer, a sliding window is used to pre-process the patient's pituitary adenoma image C n Perform feature image block division to obtain the patient's pituitary adenoma feature image block set J n ; Patient pituitary adenoma feature image block set J n Contains I patient pituitary adenoma feature image blocks; Based on the patient pituitary adenoma feature image block set J n Perform feature extraction to obtain the local feature vector J of the patient's pituitary adenoma ni , i=1,2,…,I; Specific steps for extracting feature image blocks: Using the formula I = (X + Y - X') / Y, where I is the pre-processed patient pituitary adenoma image C n The total number of feature image blocks extracted, X is the pre-processed patient pituitary adenoma image C nThe height of X' is the horizontal height of the patient's pituitary adenoma feature image block after division; Y is the step length of the sliding window; In the feature scoring layer, the global feature vector Q of the patient's pituitary adenoma is n and the local feature vector J of the patient's pituitary adenoma ni Perform feature scoring to obtain the patient's pituitary adenoma feature vector score set U n ; Based on the pituitary adenoma feature vector score set U of all patients n Screening is performed, and feature fusion is performed on the feature vectors corresponding to the screening conditions to obtain the patient's pituitary adenoma invasion feature image; The feature scoring layer is trained based on the attention mechanism.

[0009] As a preferred technical solution of the present invention, the pituitary adenoma invasion prediction model includes an invasion prediction layer and a result output layer; The invasion prediction layer is used to perform prediction analysis based on invasion data auxiliary prediction factors and the patient's pituitary adenoma invasion characteristic images to obtain the patient's pituitary adenoma prediction analysis results; The invasion prediction layer is constructed based on the CNN model and the cross-attention mechanism; The result output layer is used to output the patient's pituitary adenoma prediction analysis results.

[0010] As a preferred technical solution of the present invention, the specific steps of performing prediction analysis in the invasion prediction layer include: The invasion prediction layer includes M convolution kernels of increasing sizes, a parallel cross-attention fusion layer and a fully connected prediction layer; The high-dimensional features of the patient's pituitary adenoma invasion feature image are extracted in M ​​convolution kernels, and the invasion data auxiliary prediction factors are fused by parallel cross-attention fusion layers to obtain the patient's pituitary adenoma invasion feature prediction vector; In the fully connected prediction layer, invasion prediction analysis is performed based on the patient's pituitary adenoma invasion feature prediction vector to obtain the patient's pituitary adenoma prediction analysis result.

[0011] As a preferred technical solution of the present invention, the pituitary adenoma biomarker prediction and analysis model includes a data preprocessing layer, a feature judgment layer and a feature output layer; The data preprocessing layer is used to preprocess the patient auxiliary biomarker data to obtain preprocessed patient auxiliary biomarker data; The feature judgment layer is used to perform feature judgment on the auxiliary biomarker data of pre-treated patients to obtain auxiliary prediction factors for invasion data; Specific steps for training the feature judgment layer: Collecting several groups of biomarker level training samples; each group of biomarker level training samples contains verified characteristic factors and corresponding biomarker data; combining several groups of biomarker level training samples to obtain a biomarker level training set; The biomarker level training set is input into the BP neural network for model training to obtain an initial feature judgment layer; the initial feature judgment layer is evaluated for the model, and if the initial feature judgment layer passes the model evaluation, the initial feature judgment layer is used as the feature judgment layer in the pituitary adenoma biomarker prediction analysis model; otherwise, the model training is continued using the biomarker level training set; The feature output layer is used to output invasion data auxiliary prediction factors.

[0012] A pituitary adenoma invasion image data analysis system, comprising: The image feature extraction module includes an image acquisition unit and a feature extraction unit; the image acquisition unit is used to acquire a patient pituitary adenoma image set; the patient pituitary adenoma image set includes N consecutive patient pituitary adenoma images P at interval time windows. n , n=1, 2, ..., N, the length of the time window is L; the feature extraction unit is used to process based on the patient's pituitary adenoma image set and the pituitary adenoma image feature extraction model to obtain the patient's pituitary adenoma invasion feature image; the pituitary adenoma image feature extraction model uses a bidirectional propagation network to process the patient's pituitary adenoma image set to obtain high-quality patient pituitary adenoma invasion feature images for subsequent analysis; The auxiliary factor analysis module includes a data acquisition unit and an auxiliary analysis unit; the data acquisition unit is used to acquire the auxiliary biomarker data of the patient; the auxiliary analysis unit is used to analyze the auxiliary biomarker data of the patient and the pituitary adenoma biomarker prediction analysis model to obtain the invasion data auxiliary prediction factor; The invasion prediction and judgment module includes a prediction and judgment unit and a result output unit; the prediction and judgment unit is used to perform prediction analysis based on invasion data auxiliary prediction factors, patient pituitary adenoma invasion characteristic images and pituitary adenoma invasion prediction models to obtain patient pituitary adenoma prediction analysis results; the pituitary adenoma invasion prediction model uses a cross-attention mechanism and a feature fusion strategy to perform prediction analysis, and is used to perform high-precision invasion prediction based on invasion data auxiliary prediction factors and patient pituitary adenoma invasion characteristic images; the result output unit is used to perform subsequent operations on the patient according to the patient's pituitary adenoma prediction analysis results.

[0013] The present invention has the following advantages: 1. The present invention combines the patient's imaging data and auxiliary biomarker data, adopts a cross-attention mechanism and a feature fusion strategy for analysis, and can make full use of information from different data sources to improve the comprehensiveness and accuracy of pituitary adenoma invasion prediction; uses a bidirectional propagation network to process pituitary adenoma images to extract more detailed and accurate imaging features; through the pituitary adenoma invasion prediction model, combined with the invasion data auxiliary prediction factor and the patient's imaging feature image, it can perform high-precision invasion prediction, and fuse the key features of different modalities, which helps to improve the accuracy and reliability of invasion prediction.

[0014] 2. The present invention performs image registration through forward propagation and back propagation networks to ensure the consistency of the patient's pituitary adenoma images in the time series, align images at different time points, and eliminate the impact caused by shooting angles, position differences or equipment changes; ensure that the time series images are spatially aligned, so that subsequent feature extraction and comparison are more accurate, and the accuracy of model analysis is improved; by aligning images in multiple time periods, the changes and progression of pituitary adenomas can be better captured, helping to analyze the growth pattern and invasion characteristics of the tumor. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of the structure of a pituitary adenoma invasion image data analysis system used in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to enable persons skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0017] Embodiment 1, a method for analyzing pituitary adenoma invasion image data, comprising the following steps: Obtain a patient pituitary adenoma image set; the patient pituitary adenoma image set contains N consecutive patient pituitary adenoma images P with interval time windows. n , n=1, 2, ..., N, the length of the time window is L; the length of the time window is set by professional technicians according to actual conditions, L can be set to a few weeks, a few months or a year, and the image acquisition interval within each time window is selected; this interval can be fixed, such as once a month, or it can be flexible, determined by the patient's treatment plan and the availability of medical images; when obtaining the patient's pituitary adenoma image set, ensure that the patient's informed consent is allowed to allow their image data to be used for research, and use medical imaging equipment to obtain it. For each patient, multiple images are collected according to the selected time window L and image interval to ensure that the shooting method of each image is consistent; Based on the patient's pituitary adenoma image set and the pituitary adenoma image feature extraction model, the patient's pituitary adenoma invasion feature image is obtained; the pituitary adenoma image feature extraction model uses a bidirectional propagation network to process the patient's pituitary adenoma image set to obtain high-quality patient pituitary adenoma invasion feature images for subsequent analysis; The pituitary adenoma image feature extraction model includes an image preprocessing layer, an image feature extraction layer and an image output layer; The image preprocessing layer is used to perform image super-resolution processing on the patient pituitary adenoma image set to obtain a preprocessed patient pituitary adenoma image set; wherein the preprocessed patient pituitary adenoma image set includes N preprocessed patient pituitary adenoma images C n The image preprocessing layer uses super-resolution processing to enhance the patient's pituitary adenoma images, which helps to improve the spatial resolution of the image and make the lesion area clearer, especially in small-sized, blurred or low-resolution images. Through super-resolution processing, the details of the image are enhanced and the edges of the lesions are more defined, which helps to more accurately identify the tumor and its invasion area, especially when the tumor invades the surrounding brain tissue. The image feature extraction layer is used to extract features from the preprocessed patient pituitary adenoma image set to obtain the patient pituitary adenoma invasion feature image; the image feature extraction layer can automatically extract biologically significant features from the preprocessed pituitary adenoma image, such as the morphology, texture, edge, density, etc. of the tumor. These features not only include the size and shape of the tumor, but also reveal the invasion characteristics of the tumor to the adjacent tissues, such as the extensibility of the tumor, the fuzziness of the boundary, the invasive characteristics, etc. The image output layer is used to output the patient's pituitary adenoma invasion characteristic image; through image processing and feature extraction, the generated patient's pituitary adenoma invasion characteristic image can provide doctors with more detailed image information about the tumor's aggressiveness; The specific steps of performing image super-resolution processing in the image preprocessing layer include: The image preprocessing layer includes an image alignment layer and an image reconstruction layer; the image alignment layer includes a forward propagation network and a back propagation network; the image reconstruction layer includes a feature extraction network, a residual refinement network and an image reconstruction network; In the back propagation network, the patient’s pituitary adenoma image P is registered using the image registration method. n+1 , patient pituitary adenoma image P n+2 and patient pituitary adenoma images P n Align and obtain the reverse patient pituitary adenoma image set F n ; Reverse the patient's pituitary adenoma image set F n The image F is sent to the deformable convolution block for further feature alignment to obtain the reverse patient pituitary adenoma image F n '; In the forward propagation network, the patient’s pituitary adenoma image P is registered using the image registration method. n-1 , patient pituitary adenoma image P n-2 and patient pituitary adenoma images P n Align and obtain the positive patient pituitary adenoma image set Z n ; The positive patient pituitary adenoma image set Z n The image Z of the patient’s pituitary adenoma is sent to the deformable convolution block for further feature alignment. n '; In the image reconstruction layer, the feature extraction network is used to extract the reverse patient pituitary adenoma image F n ', positive patient pituitary adenoma image Z n ' and patient pituitary adenoma images P n Feature extraction and splicing are performed to obtain the patient's pituitary adenoma feature image T n ; Using residual refinement network to refine the patient's pituitary adenoma feature image T n Through multiple residual blocks, feature refinement is performed to obtain the refined patient pituitary adenoma feature image T n '; In the image reconstruction network, the patient's pituitary adenoma feature image T is refined n 'Use the pixel rearrangement method to expand four times and obtain a high-resolution patient pituitary adenoma feature image G n ; The patient's pituitary adenoma image P n The low-resolution patient pituitary adenoma feature image D is obtained by four times magnification using bilinear interpolation. n ; High-resolution patient pituitary adenoma feature image G n and low-resolution patient pituitary adenoma characteristic image D n Add them together to obtain the pre-processed patient pituitary adenoma image C n ; Image registration is performed through forward propagation and back propagation networks to ensure the consistency of the patient's pituitary adenoma images in the time series, align images at different time points, and eliminate the impact caused by shooting angles, position differences or equipment changes; ensure that time series images are spatially aligned, making subsequent feature extraction and comparison more accurate and improving the accuracy of model analysis; by aligning images at multiple time periods, the changes and progression of pituitary adenomas can be better captured, helping to analyze the growth pattern and invasion characteristics of tumors; In the backward and forward propagation networks, after the images are registered, they will enter the deformable convolution block for further feature alignment. The deformable convolution can be flexibly adjusted in space to handle deformation or transformation in the image, which is especially suitable for cases where the tumor boundary is blurred or the morphology is complex. The deformable convolution block can be dynamically adjusted according to the content of the image. When aligning images, it is not limited to rigid transformation and can better adapt to the diversity of tumor morphology. In the image reconstruction layer, the feature extraction network extracts key features and splices these features to generate the patient's pituitary adenoma feature image T n ; This multi-view, multi-time point feature fusion helps capture all-round information about the tumor. By fusing image features from multiple time points, the dynamic changes of the tumor can be fully captured, helping to improve the understanding of complex features such as tumor morphology and invasiveness. The spliced ​​feature image contains information from multiple time points, providing more context and improving the model's ability to recognize subtle changes. Through pixel rearrangement and bilinear interpolation, the image resolution is significantly improved, making the details of the tumor and its invasive area clearer, facilitating subsequent analysis and diagnosis. By combining the advantages of high-resolution and low-resolution images, the final pre-processed image can provide clear and detailed image data for subsequent feature extraction and analysis. The specific steps of feature extraction in the image feature extraction layer include: The image feature extraction layer includes a global feature extraction layer, a local feature division layer, and a feature scoring layer; In the global feature extraction layer, the preprocessed patient pituitary adenoma image C n Perform global feature extraction to obtain the patient's pituitary adenoma feature global feature vector Q n ; In the local feature division layer, a sliding window is used to pre-process the patient's pituitary adenoma image C n Perform feature image block division to obtain the patient's pituitary adenoma feature image block set J n ; Patient pituitary adenoma feature image block set J n Contains I patient pituitary adenoma feature image blocks; Based on the patient pituitary adenoma feature image block set J n Perform feature extraction to obtain the local feature vector J of the patient's pituitary adenoma ni , i=1,2,…,I; Specific steps for extracting feature image blocks: Using the formula I = (X + Y - X') / Y, where I is the pre-processed patient pituitary adenoma image C n The total number of feature image blocks extracted, X is the pre-processed patient pituitary adenoma image C n The height of X' is the horizontal height of the characteristic image block of the patient's pituitary adenoma after division; Y is the step length of the sliding window; the specific values ​​of I and Y are set by professional and technical personnel according to actual conditions; In the feature scoring layer, the global feature vector Q of the patient's pituitary adenoma is n and the local feature vector J of the patient's pituitary adenoma ni Perform feature scoring to obtain the patient's pituitary adenoma feature vector score set U n; Based on the pituitary adenoma feature vector score set U of all patients n Screening is performed, and feature fusion is performed on the feature vectors corresponding to the screening conditions to obtain the patient's pituitary adenoma invasion feature image; The feature scoring layer is trained based on the attention mechanism; By pre-processing the patient's pituitary adenoma images C n Perform global feature extraction to obtain a global feature vector; global features usually include information such as the overall morphology, structure, and size of the image, and can describe the macroscopic features of the image; use a sliding window to locally divide the pre-processed patient pituitary adenoma image to obtain multiple small blocks of images; after feature extraction of each image block, a local feature vector is generated; local features are usually used to describe the details of the image, such as the local area of ​​the tumor, shape irregularity, etc.; through local feature extraction, local changes and details of the tumor can be captured, such as tumor edges, shape changes, invasion of blood vessels or surrounding tissues, etc.; local features can reveal changes in the tumor at the microscopic level, which is of great significance for analyzing the invasiveness, extensibility, and boundary irregularity of the tumor; the sliding window method ensures that every part of the image can be fully analyzed, avoiding the situation where local features of the image are ignored; The scoring mechanism can screen out the most helpful features for diagnosis, avoiding the interference of redundant or irrelevant features, thereby improving the efficiency and accuracy of analysis. The feature scoring layer uses the attention mechanism to perform weighted training on each feature. The attention mechanism can adaptively adjust the weight of the feature according to its importance, so that important features receive more attention during the training process. Obtaining patient auxiliary biomarker data; performing analysis based on the patient auxiliary biomarker data and the pituitary adenoma biomarker prediction analysis model to obtain invasion data auxiliary prediction factors; The pituitary adenoma biomarker prediction and analysis model includes a data preprocessing layer, a feature judgment layer, and a feature output layer; The data preprocessing layer is used to preprocess the patient auxiliary biomarker data to obtain preprocessed patient auxiliary biomarker data; The feature judgment layer is used to perform feature judgment on the auxiliary biomarker data of pre-treated patients to obtain auxiliary prediction factors for invasion data; Specific steps for training the feature judgment layer: Collecting several groups of biomarker level training samples; each group of biomarker level training samples contains verified characteristic factors and corresponding biomarker data; combining several groups of biomarker level training samples to obtain a biomarker level training set; The biomarker level training set is input into the BP neural network for model training to obtain an initial feature judgment layer; the initial feature judgment layer is evaluated for the model, and if the initial feature judgment layer passes the model evaluation, the initial feature judgment layer is used as the feature judgment layer in the pituitary adenoma biomarker prediction analysis model; otherwise, the model training is continued using the biomarker level training set; The feature output layer is used to output invasion data auxiliary prediction factors; The feature judgment layer is mainly used to determine which features are related to the patient's aggressiveness from the pre-processed biomarker data and extract effective predictive factors. This process uses the various features in the biomarker data and the verified invasive factors for analysis to determine which biomarkers can best reflect the aggressiveness of the tumor; through the feature judgment layer, the biomarkers most related to the aggressiveness of pituitary adenomas can be screened out, providing more accurate input for subsequent analysis and improving the accuracy of prediction; the feature judgment layer accurately screens out the biomarkers most related to aggressiveness and provides efficient input features; through the training and optimization of the BP neural network, the accuracy of the prediction model is continuously improved, providing accurate predictive factors for clinical diagnosis; Based on the invasion data auxiliary prediction factor, the patient's pituitary adenoma invasion characteristic image and the pituitary adenoma invasion prediction model, the prediction analysis is performed to obtain the patient's pituitary adenoma prediction analysis results; the pituitary adenoma invasion prediction model uses the cross-attention mechanism and feature fusion strategy to perform prediction analysis for high-precision invasion prediction based on the invasion data auxiliary prediction factor and the patient's pituitary adenoma invasion characteristic image; follow-up operations are performed on the patient according to the patient's pituitary adenoma prediction analysis results; The pituitary adenoma invasion prediction model includes an invasion prediction layer and a result output layer; The invasion prediction layer is used to perform prediction analysis based on invasion data auxiliary prediction factors and the patient's pituitary adenoma invasion characteristic images to obtain the patient's pituitary adenoma prediction analysis results; The invasion prediction layer is constructed based on the CNN model and the cross-attention mechanism; The result output layer is used to output the patient's pituitary adenoma prediction analysis results; In the invasion prediction layer, the model uses convolution kernels to process the patient's pituitary adenoma invasion characteristic images, which can extract rich spatial features from the images and identify the tumor's morphology, edge, structure and other information; combined with the pituitary adenoma invasion characteristic images, it can help the model capture the important characteristics of tumor invasiveness; the cross-attention mechanism can dynamically adjust the weights of different input data by jointly modeling the invasion data auxiliary prediction factors and the patient's pituitary adenoma invasion characteristic images; the cross-attention mechanism can enhance the information fusion effect, allowing the model to better integrate information from different modalities and improve prediction accuracy; the invasion prediction layer not only relies on the pituitary adenoma invasion characteristic images, but also combines the invasion data auxiliary prediction factors to perform multi-modal joint prediction. This method combines the morphological information of the image and the numerical information of the biomarker data, which can more comprehensively evaluate the invasiveness of the tumor; The specific steps of predictive analysis in the invasion prediction layer include: The invasion prediction layer includes M convolution kernels of increasing sizes, a parallel cross-attention fusion layer and a fully connected prediction layer; The high-dimensional features of the patient's pituitary adenoma invasion feature image are extracted in M ​​convolution kernels, and the invasion data auxiliary prediction factors are fused by parallel cross-attention fusion layers to obtain the patient's pituitary adenoma invasion feature prediction vector; In the fully connected prediction layer, an invasion prediction analysis is performed based on the patient's pituitary adenoma invasion feature prediction vector to obtain the patient's pituitary adenoma prediction analysis result; In the invasion prediction layer, M convolution kernels with increasing sizes are used to perform multi-level and multi-scale high-dimensional feature extraction on the patient's pituitary adenoma invasion feature image. Each convolution kernel will process image features of different scales and capture detailed information in the tumor image, such as morphology, edge, texture, etc.; convolution kernels of different sizes can extract features of different scales, and features from details to global features can be effectively captured, thereby improving the model's meticulousness in judging the tumor invasiveness; on the basis of convolution feature extraction, a parallel cross-attention fusion layer is used to fuse the invasion data auxiliary prediction factor and the features of the patient's pituitary adenoma invasion feature image; the cross-attention mechanism assigns different weights to image features and data features by calculating the correlation between different modalities, thereby focusing on important information and enhancing the comprehensive ability of the model; after feature fusion, the final patient pituitary adenoma invasion feature prediction vector will be sent to the fully connected prediction layer, which maps the high-dimensional feature vector to a low-dimensional space, performs the final prediction analysis, and outputs the invasive prediction result of the patient's pituitary adenoma; Pituitary adenoma is a common benign brain tumor that usually originates from the anterior pituitary gland. Depending on the type, size, and growth direction of the tumor, common invasion sites of pituitary adenoma include the optic chiasm, which may cause visual impairment or ipsilateral hemi-blindness; adjacent cerebral blood vessels, such as the internal carotid artery, which may cause hemodynamic problems; brain structures around the sella turcica, such as the hippocampus, which may cause memory impairment; the brainstem, which may cause changes in vital signs in extreme cases; the sphenoid sinuses and paranasal sinuses, which may cause headaches and nasal congestion; and cranial nerves, especially the trigeminal nerve and optic nerve, which may cause facial numbness or vision problems. Invasion prediction based on pituitary adenoma images can accurately identify the compression or invasion of the surrounding structures by the tumor. If the patient's pituitary adenoma is predicted to be of low invasiveness, the tumor has not invaded important surrounding structures such as the optic chiasm, brain stem, etc., and the growth rate is slow, conservative treatment is generally chosen to observe whether the tumor is stable or shrinks on its own; if the patient's pituitary adenoma is predicted to be moderately invasive, it may have begun to invade adjacent tissues, and the tumor is growing rapidly, then surgical removal of the tumor may be recommended, especially when the patient has symptoms such as visual impairment and headache; actual specific analysis is performed based on the predicted analysis results of the patient's pituitary adenoma.

[0018] Example 2, a pituitary adenoma invasion image data analysis system, see Figure 1 As shown, including: The image feature extraction module includes an image acquisition unit and a feature extraction unit; the image acquisition unit is used to acquire a patient pituitary adenoma image set; the patient pituitary adenoma image set includes N consecutive patient pituitary adenoma images P at interval time windows. n , n=1, 2, ..., N, the length of the time window is L; the feature extraction unit is used to process based on the patient's pituitary adenoma image set and the pituitary adenoma image feature extraction model to obtain the patient's pituitary adenoma invasion feature image; the pituitary adenoma image feature extraction model uses a bidirectional propagation network to process the patient's pituitary adenoma image set to obtain high-quality patient pituitary adenoma invasion feature images for subsequent analysis; The auxiliary factor analysis module includes a data acquisition unit and an auxiliary analysis unit; the data acquisition unit is used to acquire the auxiliary biomarker data of the patient; the auxiliary analysis unit is used to analyze the auxiliary biomarker data of the patient and the pituitary adenoma biomarker prediction analysis model to obtain the invasion data auxiliary prediction factor; The invasion prediction and judgment module includes a prediction and judgment unit and a result output unit; the prediction and judgment unit is used to perform prediction analysis based on invasion data auxiliary prediction factors, patient pituitary adenoma invasion characteristic images and pituitary adenoma invasion prediction models to obtain patient pituitary adenoma prediction analysis results; the pituitary adenoma invasion prediction model uses a cross-attention mechanism and a feature fusion strategy to perform prediction analysis, and is used to perform high-precision invasion prediction based on invasion data auxiliary prediction factors and patient pituitary adenoma invasion characteristic images; the result output unit is used to perform subsequent operations on the patient according to the patient's pituitary adenoma prediction analysis results.

[0019] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A method for analyzing pituitary adenoma invasion image data, characterized in that: The following steps are involved: Obtain a patient pituitary adenoma image set; the patient pituitary adenoma image set contains N consecutive patient pituitary adenoma images P with interval time windows. n , n=1, 2, ..., N, the length of the time window is L; based on the patient's pituitary adenoma image set and the pituitary adenoma image feature extraction model, the patient's pituitary adenoma invasion feature image is obtained; the pituitary adenoma image feature extraction model uses a bidirectional propagation network to process the patient's pituitary adenoma image set to obtain high-quality patient pituitary adenoma invasion feature images for subsequent analysis; Obtaining patient auxiliary biomarker data; performing analysis based on the patient auxiliary biomarker data and the pituitary adenoma biomarker prediction analysis model to obtain invasion data auxiliary prediction factors; Predictive analysis is performed based on invasion data auxiliary prediction factors, patient pituitary adenoma invasion characteristic images and pituitary adenoma invasion prediction models to obtain patient pituitary adenoma prediction analysis results; the pituitary adenoma invasion prediction model uses a cross-attention mechanism and a feature fusion strategy to perform predictive analysis for high-precision invasion prediction based on invasion data auxiliary prediction factors and patient pituitary adenoma invasion characteristic images; follow-up operations are performed on patients based on the patient pituitary adenoma prediction analysis results.

2. A method for analyzing pituitary adenoma invasion image data according to claim 1, characterized in that: The pituitary adenoma image feature extraction model includes an image preprocessing layer, an image feature extraction layer and an image output layer; The image preprocessing layer is used to perform image super-resolution processing on the patient pituitary adenoma image set to obtain a preprocessed patient pituitary adenoma image set; wherein the preprocessed patient pituitary adenoma image set includes N preprocessed patient pituitary adenoma images C n ; The image feature extraction layer is used to extract features from the preprocessed patient pituitary adenoma image set to obtain the patient's pituitary adenoma invasion feature image; The image output layer is used to output the patient's pituitary adenoma invasion feature image.

3. A method for analyzing pituitary adenoma invasion image data according to claim 2, characterized in that: The specific steps of image super-resolution processing in the image preprocessing layer include: The image preprocessing layer includes an image alignment layer and an image reconstruction layer; the image alignment layer includes a forward propagation network and a back propagation network; the image reconstruction layer includes a feature extraction network, a residual refinement network and an image reconstruction network; In the back propagation network, the patient’s pituitary adenoma image P is registered using the image registration method. n+1 , patient pituitary adenoma image P n+2 and patient pituitary adenoma images P n Align and obtain the reverse patient pituitary adenoma image set F n ; Reverse the patient's pituitary adenoma image set F n The image F is sent to the deformable convolution block for further feature alignment to obtain the reverse patient pituitary adenoma image F n '; In the forward propagation network, the patient’s pituitary adenoma image P is registered using the image registration method. n-1 , patient pituitary adenoma image P n-2 and patient pituitary adenoma images P n Align and obtain the positive patient pituitary adenoma image set Z n ; The positive patient pituitary adenoma image set Z n The image Z of the patient’s pituitary adenoma is sent to the deformable convolution block for further feature alignment. n '; In the image reconstruction layer, the feature extraction network is used to extract the reverse patient pituitary adenoma image F n ', positive patient pituitary adenoma image Z n ' and patient pituitary adenoma images P n Feature extraction and splicing are performed to obtain the patient's pituitary adenoma feature image T n ; Using residual refinement network to refine the patient's pituitary adenoma feature image T n Through multiple residual blocks, feature refinement is performed to obtain the refined patient pituitary adenoma feature image T n '; In the image reconstruction network, the patient's pituitary adenoma feature image T is refined n 'Use the pixel rearrangement method to expand four times and obtain a high-resolution patient pituitary adenoma feature image G n ; The patient's pituitary adenoma image P n The low-resolution patient pituitary adenoma feature image D is obtained by four times magnification using bilinear interpolation. n ; High-resolution patient pituitary adenoma feature image G n and low-resolution patient pituitary adenoma characteristic image D n Add them together to obtain the pre-processed patient pituitary adenoma image C n .

4. A method for analyzing pituitary adenoma invasion image data according to claim 3, characterized in that: The specific steps of feature extraction in the image feature extraction layer include: The image feature extraction layer includes a global feature extraction layer, a local feature division layer, and a feature scoring layer; In the global feature extraction layer, the preprocessed patient pituitary adenoma image C n Perform global feature extraction to obtain the patient's pituitary adenoma feature global feature vector Q n ; In the local feature division layer, a sliding window is used to pre-process the patient's pituitary adenoma image C n Perform feature image block division to obtain the patient's pituitary adenoma feature image block set J n ; Patient pituitary adenoma feature image block set J n Contains I patient pituitary adenoma feature image blocks; Based on the patient pituitary adenoma feature image block set J n Perform feature extraction to obtain the local feature vector J of the patient's pituitary adenoma ni , i=1,2,…,I; Specific steps for extracting feature image blocks: Using the formula I = (X + Y - X') / Y, where I is the pre-processed patient pituitary adenoma image C n The total number of feature image blocks extracted, X is the pre-processed patient pituitary adenoma image C n The height of X' is the horizontal height of the patient's pituitary adenoma feature image block after division; Y is the step length of the sliding window; In the feature scoring layer, the global feature vector Q of the patient's pituitary adenoma is n and the local feature vector J of the patient's pituitary adenoma ni Perform feature scoring to obtain the patient's pituitary adenoma feature vector score set U n ; Based on the pituitary adenoma feature vector score set U of all patients n Screening is performed, and feature fusion is performed on the feature vectors corresponding to the screening conditions to obtain the patient's pituitary adenoma invasion feature image; The feature scoring layer is trained based on the attention mechanism.

5. A method for analyzing pituitary adenoma invasion image data according to claim 4, characterized in that: The pituitary adenoma invasion prediction model includes an invasion prediction layer and a result output layer; The invasion prediction layer is used to perform prediction analysis based on invasion data auxiliary prediction factors and the patient's pituitary adenoma invasion characteristic images to obtain the patient's pituitary adenoma prediction analysis results; The invasion prediction layer is constructed based on the CNN model and the cross-attention mechanism; The result output layer is used to output the patient's pituitary adenoma prediction analysis results.

6. A method for analyzing pituitary adenoma invasion image data according to claim 5, characterized in that: The specific steps of predictive analysis in the invasion prediction layer include: The invasion prediction layer includes M convolution kernels of increasing sizes, a parallel cross-attention fusion layer and a fully connected prediction layer; The high-dimensional features of the patient's pituitary adenoma invasion feature image are extracted in M ​​convolution kernels, and the invasion data auxiliary prediction factors are fused by parallel cross-attention fusion layers to obtain the patient's pituitary adenoma invasion feature prediction vector; In the fully connected prediction layer, invasion prediction analysis is performed based on the patient's pituitary adenoma invasion feature prediction vector to obtain the patient's pituitary adenoma prediction analysis result.

7. A method for analyzing pituitary adenoma invasion image data according to claim 6, characterized in that: The pituitary adenoma biomarker prediction and analysis model includes a data preprocessing layer, a feature judgment layer, and a feature output layer; The data preprocessing layer is used to preprocess the patient auxiliary biomarker data to obtain preprocessed patient auxiliary biomarker data; The feature judgment layer is used to perform feature judgment on the auxiliary biomarker data of pre-treated patients to obtain auxiliary prediction factors for invasion data; Specific steps for training the feature judgment layer: Collecting several groups of biomarker level training samples; each group of biomarker level training samples contains verified characteristic factors and corresponding biomarker data; combining several groups of biomarker level training samples to obtain a biomarker level training set; The biomarker level training set is input into the BP neural network for model training to obtain the initial feature judgment layer; Performing model evaluation on the initial feature judgment layer. If the initial feature judgment layer passes the model evaluation, the initial feature judgment layer is used as the feature judgment layer in the pituitary adenoma biomarker prediction analysis model; otherwise, the model training is continued using the biomarker level training set; The feature output layer is used to output invasion data auxiliary prediction factors.

8. A pituitary adenoma invasion image data analysis system, characterized in that: The system uses a method for analyzing pituitary adenoma invasion image data according to any one of claims 1 to 7, comprising: The image feature extraction module includes an image acquisition unit and a feature extraction unit; the image acquisition unit is used to acquire a patient pituitary adenoma image set; the patient pituitary adenoma image set includes N consecutive patient pituitary adenoma images P at interval time windows. n , n=1, 2, ..., N, the length of the time window is L; the feature extraction unit is used to process based on the patient's pituitary adenoma image set and the pituitary adenoma image feature extraction model to obtain the patient's pituitary adenoma invasion feature image; the pituitary adenoma image feature extraction model uses a bidirectional propagation network to process the patient's pituitary adenoma image set to obtain high-quality patient pituitary adenoma invasion feature images for subsequent analysis; The auxiliary factor analysis module includes a data acquisition unit and an auxiliary analysis unit; the data acquisition unit is used to acquire the auxiliary biomarker data of the patient; the auxiliary analysis unit is used to analyze the auxiliary biomarker data of the patient and the pituitary adenoma biomarker prediction analysis model to obtain the invasion data auxiliary prediction factor; The invasion prediction and judgment module includes a prediction and judgment unit and a result output unit; the prediction and judgment unit is used to perform prediction analysis based on invasion data auxiliary prediction factors, patient pituitary adenoma invasion characteristic images and pituitary adenoma invasion prediction models to obtain patient pituitary adenoma prediction analysis results; the pituitary adenoma invasion prediction model uses a cross-attention mechanism and a feature fusion strategy to perform prediction analysis, and is used to perform high-precision invasion prediction based on invasion data auxiliary prediction factors and patient pituitary adenoma invasion characteristic images; the result output unit is used to perform subsequent operations on the patient according to the patient's pituitary adenoma prediction analysis results.

Images