Breast cancer metastasis state analysis system based on image recognition

Abstract

The invention discloses a breast cancer metastasis state analysis system based on image recognition. The system comprises an image collection processing module, a feature fusion improvement module, a cell cluster space-time modeling module, a prognosis mapping module and a metastasis state analysis module. The invention belongs to the technical field of breast cancer metastasis state analysis, a tumor cell evolution process is modeled through multi-source pathological image preprocessing, adaptive feature fusion and a space-time diagram convolutional network, treatment response is predicted in combination with multi-omics data, and finally the breast cancer metastasis state is comprehensively judged. According to the system, the accuracy of focus feature extraction, the dynamic modeling capability and the comprehensiveness of prognosis prediction are improved, and efficient and intelligent technical support is provided for precise medical treatment.

Description

Technical Field

[0001] The present invention belongs to the technical field of breast cancer metastasis status analysis, and specifically relates to a breast cancer metastasis status analysis system based on image recognition. Background Art

[0002] The breast cancer metastasis status analysis system based on image recognition is an intelligent auxiliary diagnosis system that automatically analyzes pathological sections or medical images through artificial intelligence technology to determine whether breast cancer has metastasized. This system uses deep learning algorithms to identify key indicators such as the morphological characteristics of tumor cells, the pattern of stromal infiltration, and lymph node micrometastasis foci, and provides an objective metastasis risk assessment for clinicians through quantitative analysis. Its core role is to improve the efficiency of pathological diagnosis, reduce the subjective differences in manual interpretation, detect the tendency of metastasis at an early stage, provide imaging evidence for formulating personalized treatment plans, and ultimately improve the prognostic management of patients.

[0003] However, in the existing breast cancer metastasis status analysis systems, there are technical problems such as fragmented pathological image information, lack of dynamic evolution analysis, and incomplete clinical prognosis prediction; in the existing feature fusion process, there are technical problems such as insufficient information fusion and unstable analysis results due to complex image sources and large differences in tissue structure scales; in the existing spatio-temporal modeling of cell clusters, there is a technical problem of insufficient understanding of the spatial aggregation law and the trend of evolution over time of cell clusters; in the existing treatment response prediction process, there is a technical problem that it often relies on a single indicator or static data and is difficult to comprehensively reflect the multi-dimensional characteristics and treatment adaptability of patients. Summary of the Invention

[0004] To address the above technical problems, the present invention provides a breast cancer metastasis status analysis system based on image recognition. In the existing breast cancer metastasis status analysis systems, there are technical problems such as fragmented pathological image information, lack of dynamic evolution analysis, and incomplete clinical prognosis prediction. This solution creatively adopts an integrated breast cancer metastasis status intelligent analysis system that combines three intelligent functions: a feature fusion improvement module, a cell cluster spatio-temporal modeling module, and a prognosis mapping module. It can conduct a more systematic and in-depth analysis of the breast cancer metastasis process, thereby providing doctors with more scientific judgment bases and demonstrating significant advantages in disease management and precision medicine. In the existing feature fusion process, due to complex image sources and large differences in tissue structure scales, there are technical problems such as insufficient information fusion and unstable analysis results. This solution creatively adopts a variable convolutional neural network that combines a quantization attention mechanism and a cross-scale feature pyramid for adaptive feature fusion. By simulating the "first global, then focused" observation idea when doctors read images, it reasonably integrates tissue structure features at different levels, effectively improving the extraction effect of key lesion information and laying a reliable foundation for subsequent lesion behavior recognition. In the existing cell cluster spatio-temporal modeling process, there is a technical problem of insufficient understanding of the spatial aggregation law and the temporal evolution trend of cell clusters. This solution creatively adopts a spatio-temporal graph convolutional network improved by combining graph topology construction for cell cluster spatio-temporal modeling, which can more accurately reproduce the potential diffusion paths of tumors at different stages, help predict the possible migration directions and development trends of tumors, and is of great significance for grasping the timing of tumor intervention. In the existing treatment response prediction process, it often relies on a single indicator or static data and is difficult to comprehensively reflect the multi-dimensional characteristics and treatment adaptability of patients. This solution creatively adopts an adversarial interpretation network for multi-omics data fusion to conduct treatment response prediction and obtain reference data for treatment effect prediction, achieving the ability to provide more personalized and highly credible treatment effect prediction bases, assisting doctors in formulating treatment plans more scientifically, and improving the survival rate and quality of life of patients.

[0005] The technical solution adopted by the present invention is as follows: A breast cancer metastasis status analysis system based on image recognition provided by the present invention includes an image collection and processing module, a feature fusion improvement module, a cell cluster spatio-temporal modeling module, a prognosis mapping module, and a metastasis status analysis module;

[0006] The image collection and processing module is used for image collection and processing. Through image collection and processing, a breast cancer metastasis status analysis data set is obtained, and the breast cancer metastasis status analysis data set is sent to the feature fusion improvement module;

[0007] The feature fusion improvement module is used for adaptive feature fusion. Through adaptive feature fusion, fused tumor feature data is obtained, and the fused tumor feature data is sent to the cell cluster spatio-temporal modeling module;

[0008] The cell cluster spatio-temporal modeling module is used for cell cluster spatio-temporal modeling. Through cell cluster spatio-temporal modeling, cell cluster metastasis prediction reference data is obtained, and the cell cluster metastasis prediction reference data is sent to the prognosis mapping module and the metastasis status analysis module;

[0009] The prognosis mapping module is used for treatment response prediction. Through treatment response prediction, treatment effect prediction reference data is obtained, and the treatment effect prediction reference data is sent to the metastasis status analysis module;

[0010] The metastasis status analysis module is used for metastasis status analysis. Through metastasis status analysis, comprehensive reference data for breast cancer metastasis status analysis is obtained.

[0011] Furthermore, the image collection and processing is used for collecting pathological image data and performing preprocessing and enhancement. Specifically, by standardizing, multi-source breast cancer pathological image data is collected to obtain original case image data, and through preprocessing and image enhancement, a breast cancer metastasis status analysis data set is obtained;

[0012] The original case image data includes stained section images, immunohistochemical staining graphics, and tissue section graphics; the breast cancer subtypes collected by the original case image data include ductal carcinoma, lobular carcinoma, and triple-negative breast cancer; the data types of the original case image data include images, diagnostic information, and pathological reports;

[0013] The steps of the preprocessing and image enhancement include: image denoising processing, image slice cropping, image normalization, color normalization, and image enhancement.

[0014] Furthermore, the adaptive feature fusion is used for extracting multi-scale pathological features. Specifically, based on the breast cancer metastasis status analysis data set, an adaptable convolutional neural network combining a quantization attention mechanism and a cross-institutional feature pyramid is used for adaptive feature fusion to obtain fused tumor feature data, including the following steps: multi-scale feature extraction, improvement of the adaptable convolutional network, quantization attention fusion, and adaptive feature fusion;

[0015] The multi-scale feature extraction is specifically to construct a cross-institutional feature pyramid network. Based on the breast cancer metastasis status analysis data set, preliminary feature extraction is performed, and by introducing a normalization module, feature normalization of the multi-source data information in the breast cancer metastasis status analysis data set is carried out to obtain normalized features, and the normalized features are fused to obtain multi-scale fused feature data;

[0016] The improvement of the deformable convolutional network is specifically as follows: based on the multi-scale fusion feature data, a convolutional neural network improved by deformable convolution is adopted, and the convolutional neural network improved by deformable convolution is applied to the cross-scale feature pyramid network in the multi-scale feature extraction process. Deformable convolution is introduced in each layer of the feature pyramid network to improve the deformable convolutional network, and deformable convolutional fusion feature data is obtained;

[0017] The quantization attention fusion is specifically as follows: based on the deformable convolutional fusion feature data, through quantum state modeling and feature cross-channel correlation modeling, attention heat map calculation is performed, and through discrete grading, a quantization level vector is constructed, and the attention weight is optimized according to the quantization level vector to obtain quantization attention weight data;

[0018] The quantum state modeling is used to simulate the superposition of multi-cell states and optimize the processing of complex case images. Specifically, by constructing a quantum state mapping, quantum state modeling is performed to obtain quantum state composite feature data;

[0019] The feature cross-channel correlation modeling is used to enhance the capture of the feature interaction correlation between lesion regions. Specifically, by introducing a Hamiltonian and an adaptive Bayesian adjustment coefficient, cross-channel correlation modeling is performed to obtain weighted attention feature data;

[0020] The adaptive feature fusion is specifically as follows: based on the quantization attention weight data, adaptive feature fusion is performed on the deformable convolutional fusion feature data to obtain fused tumor feature data;

[0021] The fused tumor feature data includes time frames, cell cluster node features, and cell cluster feature dimension information.

[0022] Furthermore, the cell cluster spatio-temporal modeling is used to analyze the spatial distribution pattern and temporal evolution law of tumor cell clusters. Specifically, based on the fused tumor feature data, an improved spatio-temporal graph convolutional network combined with graph topology construction is adopted to perform cell cluster spatio-temporal modeling to obtain cell cluster metastasis prediction reference data, including the following steps: constructing a spatial graph structure, constructing a temporal graph structure, constructing an improved spatio-temporal graph convolutional network, constructing an output prediction layer, and cell cluster spatio-temporal modeling;

[0023] The construction of the spatial graph structure is specifically as follows: based on each time frame image data in the fused tumor feature data, a spatial graph structure is constructed. Specifically, the vertex parameters of the spatial graph structure are used to represent the cell cluster body, and the edge parameters of the spatial graph structure are used to represent the spatial relationship between cell clusters to obtain spatial graph structure data;

[0024] The construction of the time graph structure specifically involves making time connections between different time frames based on the time frames in the fused tumor feature data, constructing the time graph structure through the nearest neighbor feature matching algorithm to obtain time graph structure data, and combining the spatial graph structure data to construct graph topology improvement data;

[0025] The graph topology improvement data includes spatial graph and time graph data;

[0026] The construction of the improved spatio-temporal graph convolutional network specifically involves constructing an alternately sequential structure of spatial graph convolution and time graph convolution, using a graph neural network with topological attention correction as the spatial graph convolution, and using a one-dimensional convolution combined with a gating mechanism as the time graph convolution to construct the improved spatio-temporal graph convolutional network and extract cell cluster spatio-temporal structure feature data;

[0027] The construction of the output prediction layer specifically involves calculating the predicted cell cluster metastasis risk data based on the cell cluster spatio-temporal structure feature data through the transfer risk scores of each cell cluster node by a multi-layer perceptron;

[0028] The cell cluster spatio-temporal modeling specifically involves performing cell cluster spatio-temporal modeling through the construction of the spatial graph structure, the construction of the time graph structure, the construction of the improved spatio-temporal graph convolutional network, and the construction of the output prediction layer, obtaining a cell cluster spatio-temporal prediction model through model training, and using the cell cluster spatio-temporal prediction model to obtain cell cluster metastasis prediction reference data;

[0029] The cell cluster metastasis prediction reference data includes the metastasis potential heat map of each cell cluster in each frame and the overall metastasis prediction vector.

[0030] Furthermore, the treatment response prediction is used to combine the state transition probability for patient treatment and survival analysis. Specifically, based on the cell cluster metastasis prediction reference data, an adversarial interpretation network for multi-omics data fusion is used to perform treatment response prediction to obtain treatment effect prediction reference data, including the following steps: multi-omics feature fusion, adversarial discriminator training, feature extraction generator optimization, construction of an interpretable constraint prediction subnet, construction of an improved loss function, and treatment response prediction;

[0031] The multi-omics feature fusion specifically involves introducing a cross-modal gating fusion mechanism, dynamically adjusting the feature fusion weights according to the data correlation of different omics, and performing feature fusion based on the cell cluster metastasis prediction reference data and the breast cancer metastasis state analysis dataset to obtain multi-omics fusion feature data;

[0032] The adversarial discriminator training specifically involves constructing a standard multi-omics discriminator and discriminating the source omics of the multi-omics fusion feature data to obtain omics discrimination data;

[0033] The optimization of the feature extraction generator specifically uses the model of multi-omics feature fusion as the generator model to generate multi-omics feature data, and through combining the omics discrimination data, adversarial training is performed to obtain adversarial discrimination optimized data;

[0034] The construction of the interpretable constraint prediction subnet specifically constructs a treatment response prediction subnet based on the adversarial discrimination optimized data, and introduces an attention mechanism to enhance the interpretability of the prediction, obtaining an interpretable optimized prediction output subnet;

[0035] The construction of the improved loss function specifically constructs a cross-entropy loss function, a generative adversarial loss, a discriminative adversarial loss, and a treatment effect prediction loss, performs loss integration to obtain the improved loss function, and uses it for model training optimization;

[0036] The treatment response prediction specifically trains a treatment response prediction model through the multi-omics feature fusion, the adversarial discriminator training, the optimization of the feature extraction generator, the construction of the interpretable constraint prediction subnet, and the construction of the improved loss function, obtains the treatment response prediction model, and uses the treatment response prediction model to perform treatment response prediction to obtain treatment effect prediction reference data.

[0037] Furthermore, the metastasis state analysis is used to comprehensively determine the breast cancer metastasis stage and provide a graded treatment recommendation. Specifically, based on the cell cluster metastasis prediction reference data and the treatment effect prediction reference data, a comprehensive metastasis state analysis is performed to obtain comprehensive reference data for breast cancer metastasis state analysis.

[0038] The beneficial effects achieved by the present invention using the above solution are as follows:

[0039] (1) Aiming at the technical problems in the existing breast cancer metastasis state analysis system, such as fragmented pathological image information, lack of dynamic evolution analysis, and incomplete clinical prognosis prediction, this solution creatively adopts a comprehensive breast cancer metastasis state intelligent analysis system that combines three intelligent functions: a feature fusion improvement module, a cell cluster spatio-temporal modeling module, and a prognosis mapping module, which can analyze the breast cancer metastasis process more systematically and deeply, thereby providing a more scientific judgment basis for doctors and showing significant advantages in disease management and precision medicine;

[0040] (2)In view of the technical problems existing in the existing feature fusion process, such as insufficient information fusion and unstable analysis results due to complex image sources and large differences in organizational structure scales, this solution creatively adopts a variable convolutional neural network that combines a quantization attention mechanism and a cross-institutional feature pyramid for adaptive feature fusion. By simulating the observation idea of "first global, then focused" when doctors read images, it reasonably integrates the organizational structure features at different levels, effectively improving the extraction effect of key lesion information and laying a reliable foundation for subsequent lesion behavior recognition;

[0041] (3)In view of the technical problems existing in the existing spatio-temporal modeling of cell clusters, such as insufficient understanding of the spatial aggregation law and the temporal evolution trend of cell clusters, this solution creatively adopts an improved spatio-temporal graph convolutional network combined with graph topology construction for spatio-temporal modeling of cell clusters, which can relatively accurately reproduce the potential diffusion paths of tumors at different stages, help predict the possible migration directions and development trends of tumors, and is of great significance for grasping the timing of tumor intervention;

[0042] (4)In view of the technical problems existing in the existing treatment response prediction process, such as often relying on a single indicator or static data and being difficult to comprehensively reflect the multi-dimensional characteristics and treatment adaptability of patients, this solution creatively adopts an adversarial interpretation network for multi-omics data fusion to predict treatment responses, obtains reference data for treatment effect prediction, realizes the ability to provide more personalized and more reliable treatment effect prediction basis, assists doctors in formulating treatment plans more scientifically, and improves the survival rate and quality of life of patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic structural diagram of a breast cancer metastasis status analysis system provided by the present invention;

[0044] Figure 2 It is a schematic flow diagram of the steps executed in the preprocessing and image enhancement in the image collection and processing module;

[0045] Figure 3 It is a schematic flow diagram of the steps executed by the feature fusion improvement module;

[0046] Figure 4 It is a schematic flow diagram of the steps executed by the cell cluster spatio-temporal modeling module;

[0047] Figure 5 It is a schematic flow diagram of the steps executed by the prognosis mapping module.

[0048] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

[0050] Example 1. Refer to Figure 1 , a breast cancer metastasis status analysis system based on image recognition provided by the present invention includes an image collection and processing module, a feature fusion and improvement module, a cell cluster spatio-temporal modeling module, a prognosis mapping module, and a metastasis status analysis module;

[0051] The image collection and processing module is used for image collection and processing. Through image collection and processing, a breast cancer metastasis status analysis data set is obtained, and the breast cancer metastasis status analysis data set is sent to the feature fusion and improvement module;

[0052] The feature fusion and improvement module is used for adaptive feature fusion. Through adaptive feature fusion, fused tumor feature data is obtained, and the fused tumor feature data is sent to the cell cluster spatio-temporal modeling module;

[0053] The cell cluster spatio-temporal modeling module is used for cell cluster spatio-temporal modeling. Through cell cluster spatio-temporal modeling, cell cluster metastasis prediction reference data is obtained, and the cell cluster metastasis prediction reference data is sent to the prognosis mapping module and the metastasis status analysis module;

[0054] The prognosis mapping module is used for treatment response prediction. Through treatment response prediction, treatment effect prediction reference data is obtained, and the treatment effect prediction reference data is sent to the metastasis status analysis module;

[0055] The metastasis status analysis module is used for metastasis status analysis. Through metastasis status analysis, comprehensive reference data for breast cancer metastasis status analysis is obtained.

[0056] By performing the above operations, aiming at the technical problems existing in the existing breast cancer metastasis status analysis system, such as fragmented pathological image information, lack of dynamic evolution analysis, and incomplete clinical prognosis prediction, this solution creatively adopts a comprehensive breast cancer metastasis status intelligent analysis system combining three intelligent functions of a feature fusion and improvement module, a cell cluster spatio-temporal modeling module, and a prognosis mapping module, which can analyze the breast cancer metastasis process more systematically and deeply, thereby providing a more scientific judgment basis for doctors and showing significant advantages in disease management and precision medicine.

[0057] Example 2. This example is based on the above example. Refer to Figure 2, the image collection and processing is used to collect pathological image data and perform preprocessing and enhancement. Specifically, by standardizing the collection of multi-source breast cancer pathological image data, the original data of case images is obtained, and through preprocessing and image enhancement, a breast cancer metastasis status analysis dataset is obtained;

[0058] The original data of the case images includes stained section images, immunohistochemical staining patterns, and tissue section images; the breast cancer subtypes collected in the original data of the case images include ductal carcinoma, lobular carcinoma, and triple-negative breast cancer; the data types of the original data of the case images include images, diagnostic information, and pathological reports;

[0059] The steps of the preprocessing and image enhancement include: image denoising processing, image slice cropping, image normalization, color normalization, and image enhancement.

[0060] Example 3, this example is based on the above example, refer to Figure 3 , the adaptive feature fusion is used to extract multi-scale pathological features. Specifically, based on the breast cancer metastasis status analysis dataset, a variable convolutional neural network combining a quantization attention mechanism and a cross-institution feature pyramid is used to perform adaptive feature fusion to obtain fused tumor feature data, including the following steps: multi-scale feature extraction, variable convolutional network improvement, quantization attention fusion, and adaptive feature fusion;

[0061] The multi-scale feature extraction is specifically to construct a cross-institution feature pyramid network. Based on the breast cancer metastasis status analysis dataset, preliminary feature extraction is performed, and by introducing a normalization module, feature normalization is performed on the multi-source data information in the breast cancer metastasis status analysis dataset to obtain normalized features, and the normalized features are fused to obtain multi-scale fused feature data. The calculation formula is:

[0062] ;

[0063] In the formula, P l is the multi-scale fused feature data, is the Swish activation function, k is the input feature scale index, l is the extracted feature scale index, N(l) is the input feature of the upper and lower domains corresponding to the lth layer of the extracted feature scale, Resample(·) is the upsampling and downsampling operation, F k is the kth input feature data, which is used to represent the breast cancer metastasis status analysis dataset, is the channel weighted fusion operation operator, SA(·) is the self-attention module operation, which is used to capture cross-institution features, F l is the feature data corresponding to the lth layer of the feature scale;

[0064] The improvement of the deformable convolutional network is specifically as follows: based on the multi-scale fusion feature data, a convolutional neural network improved by deformable convolution is adopted, and the convolutional neural network improved by deformable convolution is applied to the cross-scale feature pyramid network in the multi-scale feature extraction process. Deformable convolution is introduced in each layer of the feature pyramid network to improve the deformable convolutional network, and deformable convolution fusion feature data is obtained;

[0065] The convolutional neural network improved by deformable convolution specifically improves the feature adaptability of deformable convolution by introducing a variance regulation gating mechanism to obtain dynamically regulated improved variable feature data. The calculation formula is:

[0066] ;

[0067] In the formula, is the dynamically regulated improved variable feature data, l is the feature extraction scale index, C is the total number of feature channels, j is the feature channel index, Conv var is the deformable convolution operation introducing feature variance perception, is the feature data of the j-th feature channel corresponding to the l-th layer of feature extraction scale, G j is the variance normalization weight corresponding to the j-th feature channel, where Var(·) is the variance calculation function, J is the channel traversal index, is the two-dimensional feature data after channel traversal of the J-th channel corresponding to the l-th layer of feature extraction scale;

[0068] The quantization attention fusion is specifically as follows: based on the deformable convolution fusion feature data, through quantum state modeling and feature cross-channel correlation modeling, the attention heat map is calculated, and through discrete grading, the quantization level vector is constructed, and the attention weight is optimized according to the quantization level vector to obtain quantization attention weight data;

[0069] The quantum state modeling is used to simulate the superposition of multi-cell states and optimize the processing of complex case images. Specifically, by constructing a quantum state mapping, quantum state modeling is carried out to obtain quantum state composite feature data. The calculation formula is:

[0070] ;

[0071] In the formula, is the quantum state composite feature data, which is used to represent the composite state feature data obtained by mapping the deformable convolution fusion feature data through the quantum state mapping. Z is the normalization factor, C is the total number of feature channels, c is the feature channel index, is the learnable attenuation coefficient, is the deformable convolution fusion feature data of the c-th feature channel, is the base vector of the feature channel index, is the vector representation of the variable convolution fusion feature data of the c-th feature channel, is the tensor product operator;

[0072] The feature cross-channel correlation modeling is used to enhance the capture of the feature interaction correlation between lesion regions. Specifically, by introducing the Hamiltonian and the adaptive Bayesian adjustment coefficient, the cross-channel correlation modeling is carried out to obtain the weighted attention feature data. The calculation formula is:

[0073] ;

[0074] In the formula, is the weighted attention feature data, Softmax(·) is the softmax classifier function, T is the adaptive Bayesian adjustment coefficient, and R[·] is the real part taking function, The whole is the quantum state expectation value, which is used to represent the measured value of the quantum state composite feature data under the feature correlation structure, is the Hamiltonian matrix, is the quantum state identifier, F is the original feature identifier, which is used to represent the variable convolution fusion feature data;

[0075] Preferably, the calculation formula of the adaptive Bayesian adjustment coefficient T is:

[0076] ;

[0077] In the formula, T is the adaptive Bayesian adjustment coefficient, a is the learnable parameter, and Var(·) is the variance calculation function;

[0078] The adaptive feature fusion is specifically to perform adaptive feature fusion on the variable convolution fusion feature data according to the quantized attention weight data to obtain the fused tumor feature data;

[0079] The fused tumor feature data includes the time frame, the cell cluster node features, and the cell cluster feature dimension information.

[0080] By performing the above operations, in the existing churn risk prediction process and in the existing feature fusion process, there are technical problems such as insufficient information fusion and unstable analysis results due to the complex image sources and large differences in the organizational structure scales. This solution creatively uses a variable convolutional neural network combined with a quantized attention mechanism and a cross-system feature pyramid to perform adaptive feature fusion. By simulating the observation idea of "first global, then focused" when doctors read films, the organizational structure features at different levels are reasonably integrated, effectively improving the extraction effect of key lesion information and laying a reliable foundation for subsequent lesion behavior recognition.

[0081] Example 4. This example is based on the above example. Refer to Figure 4 , the spatio-temporal modeling of cell clusters is used to analyze the spatial distribution pattern and temporal evolution law of tumor cell clusters. Specifically, based on the fused tumor feature data, an improved spatio-temporal graph convolutional network is constructed by combining graph topology to perform spatio-temporal modeling of cell clusters, and cell cluster metastasis prediction reference data is obtained, including the following steps: constructing a spatial graph structure, constructing a temporal graph structure, constructing an improved spatio-temporal graph convolutional network, constructing an output prediction layer, and spatio-temporal modeling of cell clusters;

[0082] The construction of the spatial graph structure is specifically to construct a spatial graph structure based on each time frame image data in the fused tumor feature data. Specifically, the vertex parameters of the spatial graph structure are used to represent the cell cluster body, and the edge parameters of the spatial graph structure are used to represent the spatial relationship between cell clusters, and spatial graph structure data is obtained;

[0083] The construction of the temporal graph structure is specifically to perform temporal connection between different time frames based on the time frames in the fused tumor feature data, and construct a temporal graph structure through the nearest neighbor feature matching algorithm to obtain temporal graph structure data, and combine the spatial graph structure data to construct graph topology improved data;

[0084] The graph topology improved data includes spatial graph and temporal graph data;

[0085] The construction of the improved spatio-temporal graph convolutional network is specifically to construct an alternately sequential spatial graph convolution and temporal graph convolution structure, and introduce a graph neural network corrected by topological attention as the spatial graph convolution, and use a one-dimensional convolution combined with a gating mechanism as the temporal graph convolution to construct an improved spatio-temporal graph convolutional network, and extract cell cluster spatio-temporal structure feature data. The calculation formula is:

[0086] ;

[0087] In the formula, K st is the cell cluster spatio-temporal structure feature data, K t is the output of the temporal graph convolution, specifically using a one-dimensional convolution combined with a gating mechanism. Among them, Conv1D(·) is the one-dimensional convolution operation operator, t is the time index, is the continuous feature sequence of feature node i from time to time t, is the time window parameter, i is the feature node index, SIG(·) is the S-shaped activation function, Gate1D(·) is the gating network operation operator, K s is the spatial graph convolution, specifically using a graph neural network corrected by topological attention. Among them, is the adjacent node index of feature node i, and N(i) is the total number of adjacent nodes of feature node i. is the topological attention correction weight, W L is the weight of the spatial graph convolutional network at the L-th layer, and L is the layer index of the spatial graph convolutional network. corresponds to the spatial feature output of the j-th adjacent node of the spatial graph convolutional network at the L-th layer;

[0088] The construction of the output prediction layer is specifically to calculate the predicted cell cluster metastasis risk data based on the spatio-temporal structure feature data of the cell clusters and the metastasis risk scores of each cell cluster node through a multi-layer perceptron.

[0089] The spatio-temporal modeling of the cell clusters is specifically to perform spatio-temporal modeling of the cell clusters through the construction of the spatial graph structure, the construction of the temporal graph structure, the construction of the improved spatio-temporal graph convolutional network, and the construction of the output prediction layer. A spatio-temporal prediction model of the cell clusters is obtained through model training, and the spatio-temporal prediction model of the cell clusters is used to obtain the reference data for cell cluster metastasis prediction.

[0090] The reference data for cell cluster metastasis prediction includes the metastasis potential heat map of each cell cluster in each frame and the overall metastasis prediction vector.

[0091] By performing the above operations, in view of the technical problem of insufficient understanding of the spatial aggregation law and the temporal evolution trend of cell clusters in the existing spatio-temporal modeling process of cell clusters, this solution creatively uses an improved spatio-temporal graph convolutional network combined with graph topology construction to perform spatio-temporal modeling of cell clusters, which can more accurately reproduce the potential diffusion paths of tumors at different stages, helps predict the possible migration directions and development trends of tumors, and is of great significance for grasping the timing of tumor intervention.

[0092] Example 5, this example is based on the above example, refer to Figure 5 The treatment response prediction is used to combine the state transition probability for patient treatment and survival analysis. Specifically, based on the reference data for cell cluster metastasis prediction, a multi-omics data fusion adversarial interpretation network is used to perform treatment response prediction to obtain the reference data for treatment effect prediction, including the following steps: multi-omics feature fusion, adversarial discriminator training, feature extraction generator optimization, construction of an interpretable constraint prediction subnet, construction of an improved loss function, and treatment response prediction.

[0093] The multi-omics feature fusion is specifically to introduce a cross-modal gating fusion mechanism, dynamically adjust the feature fusion weight according to the data correlation of different omics, and perform feature fusion based on the reference data for cell cluster metastasis prediction and the breast cancer metastasis state analysis data set to obtain multi-omics fusion feature data.

[0094] The calculation formula for the multi-omics feature fusion is as follows:

[0095] ;

[0096] In the formula, Z0 is the multi-omics fusion feature data, GELU(·) is the Gaussian error linear activation function, X p is the case image feature vector, which is used to represent the metastasis potential heat map of the cell clusters in the cell cluster metastasis prediction reference data, X g is the genomics feature vector, which is used to represent the overall metastasis prediction vector of the cell clusters in the cell cluster metastasis prediction reference data, [·||·] is the feature concatenation operator, W f is the linear transformation weight, b f is the linear transformation bias term, is the fusion adjustment factor, and CrossAttention(·) is the cross-modal attention operation function;

[0097] The training of the adversarial discriminator is specifically to construct a standard multi-omics discriminator and discriminate the source omics of the multi-omics fusion feature data to obtain omics discrimination data;

[0098] The optimization of the feature extraction generator is specifically to use the model of the multi-omics feature fusion as the generator model to generate multi-omics feature data, and perform generative adversarial training by combining the omics discrimination data to obtain adversarial discrimination optimization data;

[0099] The construction of the interpretable constraint prediction subnet is specifically to construct a treatment response prediction subnet based on the adversarial discrimination optimization data and introduce an attention mechanism to enhance the predictability of interpretation to obtain an interpretable optimization prediction output subnet;

[0100] The construction of the improved loss function is specifically to construct a cross-entropy loss function, a generative adversarial loss, a discriminative adversarial loss, and a treatment effect prediction loss, perform loss integration to obtain an improved loss function, and use it for model training optimization;

[0101] The treatment response prediction is specifically to train a treatment response prediction model through the multi-omics feature fusion, the training of the adversarial discriminator, the optimization of the feature extraction generator, the construction of the interpretable constraint prediction subnet, and the construction of the improved loss function, and use the treatment response prediction model to perform treatment response prediction to obtain treatment effect prediction reference data.

[0102] By performing the above operations, aiming at the technical problem that in the existing treatment response prediction process, it often relies on a single index or static data and is difficult to comprehensively reflect the multi-dimensional characteristics and treatment adaptability of patients, this solution creatively adopts an adversarial interpretation network for multi-omics data fusion to perform treatment response prediction, obtains reference data for treatment effect prediction, realizes the ability to provide a more personalized and more reliable basis for treatment effect prediction, assists doctors in formulating treatment plans more scientifically, and improves the survival rate and quality of life of patients.

[0103] Example VI. This example is based on the above example and refers to Figure 1 , the metastasis state analysis is used to comprehensively determine the breast cancer metastasis stage and provide graded treatment suggestions. Specifically, based on the cell cluster metastasis prediction reference data and the treatment effect prediction reference data, a comprehensive metastasis state analysis is performed to obtain comprehensive reference data for breast cancer metastasis state analysis.

[0104] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process and method including a series of elements not only include those elements, but also include other elements not explicitly listed, or also include elements inherent to this process and method.

[0105] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention.

[0106] The above describes the present invention and its embodiments. This description is not restrictive. What is shown in the drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. All in all, if those of ordinary skill in the art are inspired by it and design similar structural modes and embodiments without creative efforts without departing from the purpose of the present invention, they shall fall within the protection scope of the present invention.

Claims

1. A breast cancer metastasis status analysis system based on image recognition, characterized in that: It includes an image collection and processing module, a feature fusion and improvement module, a cell cluster spatio-temporal modeling module, a prognosis mapping module, and a metastasis status analysis module; The image collection and processing module is used for image collection and processing. Through image collection and processing, a breast cancer metastasis status analysis data set is obtained, and the breast cancer metastasis status analysis data set is sent to the feature fusion and improvement module; The feature fusion and improvement module is used for adaptive feature fusion. Through adaptive feature fusion, fused tumor feature data is obtained, and the fused tumor feature data is sent to the cell cluster spatio-temporal modeling module; The adaptive feature fusion includes the following steps: multi-scale feature extraction, deformable convolutional network improvement, quantization attention fusion, and adaptive feature fusion; The quantization attention fusion is specifically to calculate the attention heat map based on the deformable convolution fused feature data through quantum state modeling and feature cross-channel correlation modeling, and construct a quantization level vector through discrete grading, and optimize the attention weight according to the quantization level vector to obtain quantization attention weight data; The quantum state modeling is used to process complex case images by simulating multi-cell state superposition and optimization. Specifically, by constructing a quantum state mapping, quantum state modeling is performed to obtain quantum state composite feature data; The feature cross-channel correlation modeling is used to enhance the capture of feature interaction correlations between lesion regions. Specifically, by introducing a Hamiltonian and an adaptive Bayesian adjustment coefficient, cross-channel correlation modeling is performed to obtain weighted attention feature data; The cell cluster spatio-temporal modeling module is used for cell cluster spatio-temporal modeling. Through cell cluster spatio-temporal modeling, cell cluster metastasis prediction reference data is obtained, and the cell cluster metastasis prediction reference data is sent to the prognosis mapping module and the metastasis status analysis module; The prognosis mapping module is used for treatment response prediction. Through treatment response prediction, treatment effect prediction reference data is obtained, and the treatment effect prediction reference data is sent to the metastasis status analysis module; The metastasis status analysis module is used for metastasis status analysis. Through metastasis status analysis, comprehensive reference data for breast cancer metastasis status analysis is obtained.

2. The breast cancer metastasis status analysis system based on image recognition according to claim 1, characterized in that: The image collection and processing is used to collect pathological image data and perform preprocessing and enhancement. Specifically, by standardizing the collection of multi-source breast cancer pathological image data, original case image data is obtained, and through preprocessing and image enhancement, a breast cancer metastasis status analysis data set is obtained; The original case image data includes stained section images, immunohistochemical staining patterns, and tissue section images.

3. The breast cancer metastasis status analysis system based on image recognition according to claim 2, characterized in that: The steps of the preprocessing and image enhancement include: image denoising processing, image slicing and cropping, image normalization, color normalization, and image enhancement.

4. The breast cancer metastasis status analysis system based on image recognition according to claim 3, wherein: The adaptive feature fusion is used to extract multi-scale pathological features. Specifically, based on the breast cancer metastasis status analysis data set, a deformable convolutional neural network combining a quantization attention mechanism and a cross-institutional feature pyramid is used for adaptive feature fusion to obtain fused tumor feature data, including the following steps: multi-scale feature extraction, deformable convolutional network improvement, quantization attention fusion, and adaptive feature fusion; The multi-scale feature extraction is specifically to construct a cross-regime feature pyramid network. Based on the breast cancer metastasis state analysis dataset, preliminary feature extraction is performed. By introducing a normalization module, feature normalization is carried out on the multi-source data information in the breast cancer metastasis state analysis dataset to obtain normalized features. Then, the normalized features are fused to obtain multi-scale fusion feature data; The improvement of the deformable convolutional network is specifically based on the multi-scale fusion feature data. A convolutional neural network improved by deformable convolution is adopted and applied to the cross-regime feature pyramid network in the multi-scale feature extraction process. Deformable convolution is introduced in each layer of the feature pyramid network to improve the deformable convolutional network and obtain deformable convolution fusion feature data; The quantization attention fusion is specifically based on the deformable convolution fusion feature data. Through quantum state modeling and feature cross-channel correlation modeling, attention heat maps are calculated. By discrete grading, a quantization level vector is constructed, and attention weight optimization is carried out according to the quantization level vector to obtain quantization attention weight data; The adaptive feature fusion is specifically based on the quantization attention weight data to perform adaptive feature fusion on the deformable convolution fusion feature data to obtain fused tumor feature data.

5. The breast cancer metastasis status analysis system based on image recognition according to claim 4, characterized in that: The fused tumor feature data includes time frames, cell cluster node features, and cell cluster feature dimension information.

6. The breast cancer metastasis status analysis system based on image recognition according to claim 5, characterized in that: The cell cluster spatio-temporal modeling is used to analyze the spatial distribution pattern and temporal evolution law of tumor cell clusters. Specifically, based on the fused tumor feature data, a spatio-temporal graph convolutional network improved by combining graph topology is adopted to perform cell cluster spatio-temporal modeling to obtain cell cluster metastasis prediction reference data, including the following steps: constructing a spatial graph structure, constructing a temporal graph structure, constructing an improved spatio-temporal graph convolutional network, constructing an output prediction layer, and cell cluster spatio-temporal modeling; The construction of the spatial graph structure is specifically based on each time frame image data in the fused tumor feature data to construct a spatial graph structure. Specifically, the vertex parameters of the spatial graph structure are used to represent the cell cluster body, and the edge parameters of the spatial graph structure are used to represent the spatial relationship between cell clusters to obtain spatial graph structure data; The construction of the temporal graph structure is specifically based on the time frames in the fused tumor feature data to perform temporal connection between different time frames. Through the nearest neighbor feature matching algorithm, a temporal graph structure is constructed to obtain temporal graph structure data, and combined with the spatial graph structure data, graph topology improved data is constructed; The graph topology improved data includes spatial graph and temporal graph data; The construction of the improved spatio-temporal graph convolutional network is specifically to construct an alternately arranged structure of spatial graph convolution and temporal graph convolution. By introducing a graph neural network corrected by topological attention as spatial graph convolution and adopting a one-dimensional convolution combined with a gating mechanism as temporal graph convolution, an improved spatio-temporal graph convolutional network is constructed to extract cell cluster spatio-temporal structure feature data; The construction of the output prediction layer specifically calculates the predicted cell cluster metastasis risk data based on the spatio-temporal structural feature data of the cell clusters and the metastasis risk scores of each cell cluster node through a multi-layer perceptron. The spatio-temporal modeling of the cell clusters specifically performs spatio-temporal modeling of the cell clusters through the construction of the spatial graph structure, the construction of the temporal graph structure, the construction of the improved spatio-temporal graph convolutional network, and the construction of the output prediction layer. A spatio-temporal prediction model of the cell clusters is obtained through model training, and the spatio-temporal prediction model of the cell clusters is used to obtain the reference data for cell cluster metastasis prediction. The reference data for cell cluster metastasis prediction includes the metastasis potential heat map of each cell cluster in each frame and the overall metastasis prediction vector.

7. The breast cancer metastasis status analysis system based on image recognition according to claim 6, wherein: The treatment response prediction is used to combine the state transition probability for patient treatment and survival analysis. Specifically, based on the reference data for cell cluster metastasis prediction, a multi-omics data fusion adversarial interpretation network is used to perform treatment response prediction to obtain the reference data for treatment effect prediction. It includes the following steps: multi-omics feature fusion, adversarial discriminator training, feature extraction generator optimization, construction of an interpretable constraint prediction subnet, construction of an improved loss function, and treatment response prediction. The multi-omics feature fusion specifically introduces a cross-modal gating fusion mechanism, dynamically adjusts the feature fusion weights according to the data correlation of different omics, and performs feature fusion based on the reference data for cell cluster metastasis prediction and the breast cancer metastasis status analysis dataset to obtain multi-omics fusion feature data. The adversarial discriminator training specifically constructs a standard multi-omics discriminator and discriminates the omics from which the multi-omics fusion feature data is derived to obtain omics discrimination data. The feature extraction generator optimization specifically uses the model of the multi-omics feature fusion as a generator model to generate multi-omics feature data, and performs generative adversarial training by combining the omics discrimination data to obtain adversarial discrimination optimization data. The construction of the interpretable constraint prediction subnet specifically constructs a treatment response prediction subnet based on the adversarial discrimination optimization data and introduces an attention mechanism to enhance the interpretability of the prediction to obtain an interpretable optimization prediction output subnet. The construction of the improved loss function specifically constructs a cross-entropy loss function, a generative adversarial loss, a discriminative adversarial loss, and a treatment effect prediction loss, performs loss integration to obtain an improved loss function, and uses it for model training optimization. The treatment response prediction specifically performs treatment response prediction model training through the multi-omics feature fusion, the adversarial discriminator training, the feature extraction generator optimization, the construction of the interpretable constraint prediction subnet, and the construction of the improved loss function to obtain a treatment response prediction model, and uses the treatment response prediction model to perform treatment response prediction to obtain the reference data for treatment effect prediction.

8. The breast cancer metastasis status analysis system based on image recognition according to claim 7, wherein: The metastasis status analysis is used to comprehensively determine the breast cancer metastasis stage and provide grading treatment suggestions. Specifically, based on the reference data for cell cluster metastasis prediction and the reference data for treatment effect prediction, a comprehensive metastasis status analysis is performed to obtain the comprehensive reference data for breast cancer metastasis status analysis.

Images