Training method and device of early postoperative recurrence identification model of hepatocellular carcinoma, equipment and medium

Abstract

The application relates to a training method and device of a hepatocellular carcinoma postoperative early recurrence identification model, equipment and a medium. The method comprises the following steps: performing end-to-end joint optimization training based on a recurrence prediction loss, a morphological response consistency loss and a causal feature orthogonalization loss to obtain a hepatocellular carcinoma postoperative early recurrence identification model; the morphological response consistency loss is obtained by calculating the distribution difference between an attention morphological prototype response spectrum and a global morphological prototype response spectrum; the causal feature orthogonalization loss is obtained by calculating the orthogonality between a whole slice morphological feature vector and an average morphological feature vector; the hepatocellular carcinoma postoperative early recurrence identification model is an end-to-end neural network model comprising a feature extractor, a gene encoding network, a context decoupling module, a weighted pooling module and a fusion prediction module. The hepatocellular carcinoma postoperative early recurrence identification model trained by the method can output a recurrence risk score and generate a gene mutation and image region causal relationship explanation chain.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence-assisted diagnostic technology, and in particular relates to a training method, device, equipment and medium for identifying early recurrence of hepatocellular carcinoma after surgery. Background Technology

[0002] With the development of digital pathology whole-slice imaging technology and high-throughput genome sequencing technology, an analysis technique has emerged that integrates whole-slice pathology images with genomic mutation profiles in a multimodal manner. This technique can achieve more precise individualized recurrence risk stratification by jointly mining biomarkers closely related to early recurrence of hepatocellular carcinoma at both the molecular and tissue morphology levels.

[0003] In traditional techniques, the fusion of whole-section pathological images and genomic mutation profiles mainly employs late-stage feature concatenation. This involves first extracting global image feature vectors from the whole-section pathological image using a pre-trained deep convolutional neural network or multi-instance learning framework, while simultaneously encoding the genomic mutation profile into gene feature vectors. The feature vectors from the two modalities are then directly concatenated and fed into a classifier for recurrence risk prediction. Another mainstream paradigm utilizes a data-driven attention mechanism to automatically learn the importance weights of each image patch in the whole-section pathological image. After aggregating the images through attention pooling, the global image representation is obtained and then fused with genomic features.

[0004] However, the aforementioned late-stage feature stitching methods and multimodal fusion methods based on purely data-driven attention mechanisms suffer from fundamental cross-modal semantic gaps and alignment challenges. Gene mutations are global molecular events affecting the entire tumor, while whole-slice pathological images are collections of local spatial morphological information at the level of hundreds of millions of pixels. These two are inherently mismatched in scale and nature. Late-stage stitching methods treat them as independent features, directly superimposing them, completely ignoring the biological logic that gene mutations, as a cause, may manifest as the local spatial morphology of the tumor as an effect. Simultaneously, while data-driven attention mechanisms can statistically locate regions associated with recurrence, this learning process lacks the guidance of prior biological knowledge. It cannot explain the molecular mechanisms underlying the importance of a particular image region, nor can it reveal the causal relationship between that region and a specific gene mutation pathway, making it difficult for the model's high-risk predictions to form verifiable logical links. Furthermore, the lack of large-scale ground-value datasets containing paired whole-slice pathological images, spatial transcriptome data, and genomic mutation data to directly supervise the mapping relationship between global molecular events and local morphological features further exacerbates the aforementioned alignment and interpretability challenges. Summary of the Invention

[0005] Based on this, it is necessary to provide a training method, device, equipment, and medium for a hepatocellular carcinoma postoperative early recurrence identification model that can achieve accurate semantic alignment between global molecular events and local spatial phenotypes to mine multimodal features strongly correlated with early recurrence under the coordinated effect of the two.

[0006] Firstly, this application provides a training method for a model to identify early recurrence of hepatocellular carcinoma after surgery, including:

[0007] The first whole-section pathological image and the first genomic mutation spectrum of a hepatocellular carcinoma patient were obtained, and the first whole-section pathological image was segmented into image blocks to obtain the first tissue image block;

[0008] Feature extraction is performed on each first tissue image block using a feature extractor to obtain a first initial morphological feature vector for each first tissue image block. The first morphological prototype soft assignment vector corresponding to each first tissue image block is obtained by calculating the similarity between each first initial morphological feature vector and each pathological prototype in the pathological prototype library. The feature extractor is a deep convolutional neural network pre-trained on natural images and completed contrastive self-supervised learning on a large-scale pathological image dataset. The pathological prototype library is obtained by clustering the second initial morphological feature vectors of second tissue image blocks extracted from second whole-section pathological images in a training set of known hepatocellular carcinoma patients.

[0009] The first genome mutation spectrum is input into the gene coding network to generate the expected first global morphological prototype response spectrum. The first global morphological prototype response spectrum is then decoupled from the context by a context decoupling module to obtain the first gene-guided attention vector. The gene coding network consists of a genome mutation spectrum embedding layer and a multilayer perceptron pre-trained with prior knowledge. The multilayer perceptron is trained by using the first genome mutation spectrum as input and the third global morphological prototype response spectrum obtained by statistical analysis of the third whole-slice pathological images based on the public cancer genome atlas database as the regression target. The context decoupling module is an encoder composed of a multi-head self-attention mechanism and a feedforward neural network, used to model the synergistic regulation or antagonistic relationship between the morphological prototype dimensions in the first global morphological prototype response spectrum.

[0010] Based on the weighted pooling module, the first morphological prototype soft allocation vector of each first tissue image block is modulated by the first gene-guided attention vector to obtain the first gene-guided attention raw score of each first tissue image block, and the first attention weight is obtained by normalizing the first gene-guided attention raw score. The first initial morphological feature vector is then weighted and pooled to obtain the first gene-guided full slice morphological feature vector.

[0011] The fusion prediction module integrates the average morphological feature vector, the first full-slice morphological feature vector, and the global recurrence risk value for risk prediction, generating an early postoperative recurrence risk prediction score. Based on the first attention weight and the first morphological prototype soft assignment vector, morphological prototype response distribution information is generated to explain the causal relationship between gene mutations and image regions. The average morphological feature vector is obtained by global average pooling of each first initial morphological feature vector. The global recurrence risk value is extracted by the gene coding network based on the first genome mutation spectrum. The fusion prediction module is a neural network consisting of a feature splicing layer and a fully connected classification layer.

[0012] An end-to-end joint optimization training was performed on the early postoperative recurrence identification model for hepatocellular carcinoma based on recurrence prediction loss, morphological response consistency loss, and causal feature orthogonalization loss to obtain the trained early postoperative recurrence identification model for hepatocellular carcinoma. The recurrence prediction loss is the binary cross-entropy loss between the early postoperative recurrence risk prediction score and the actual recurrence status label of the hepatocellular carcinoma patient. The morphological response consistency loss is obtained by calculating the distribution difference between the response spectrum of the attention morphological prototype obtained by weighting the soft assignment vector of each first morphological prototype based on the first attention weight and the response spectrum of the first global morphological prototype. The causal feature orthogonalization loss is obtained by calculating the orthogonality between the first full-slice morphological feature vector and the average morphological feature vector. The early postoperative recurrence identification model for hepatocellular carcinoma is an end-to-end neural network model including a feature extractor, a gene encoding network, a context decoupling module, a weighted pooling module, and a fusion prediction module.

[0013] Secondly, this application also provides a training device for a model for identifying early recurrence of hepatocellular carcinoma after surgery, comprising:

[0014] The image preprocessing module is used to acquire the first whole-section pathological image and the first genomic mutation spectrum of a hepatocellular carcinoma patient, and to segment the first whole-section pathological image into image blocks to obtain the first tissue image block;

[0015] The morphological prototype mapping module is used to extract features from each first tissue image block using a feature extractor to obtain a first initial morphological feature vector for each first tissue image block. It then calculates the similarity between each first initial morphological feature vector and each pathological prototype in the pathological prototype library to obtain a first morphological prototype soft-assignment vector corresponding to each first tissue image block. The feature extractor is a deep convolutional neural network pre-trained on natural images and subjected to contrastive self-supervised learning on a large-scale pathological image dataset. The pathological prototype library is obtained by clustering the second initial morphological feature vectors of second tissue image blocks extracted from second whole-section pathological images in a training set of known hepatocellular carcinoma patients.

[0016] The causal guidance signal generation module is used to input the first genome mutation spectrum into the gene coding network to generate the expected first global morphological prototype response spectrum of the gene. The first global morphological prototype response spectrum is then decoupled from the context by the context decoupling module to obtain the first gene guidance attention vector. The gene coding network consists of a genome mutation spectrum embedding layer and a multilayer perceptron pre-trained with prior knowledge. The multilayer perceptron is trained by using the first genome mutation spectrum as input and the third global morphological prototype response spectrum obtained by statistical analysis of the third whole-slice pathological images based on the public cancer genome map database as the regression target. The context decoupling module is an encoder composed of a multi-head self-attention mechanism and a feedforward neural network, used to model the synergistic regulation or antagonistic relationship between the morphological prototype dimensions in the first global morphological prototype response spectrum.

[0017] The causal alignment feature aggregation module is used to modulate the first morphological prototype soft assignment vector of each first tissue image block through the first gene-guided attention vector based on the weighted pooling module, to obtain the first gene-guided attention raw score of each first tissue image block, and to obtain the first attention weight based on the normalization of the first gene-guided attention raw score, and to perform weighted pooling on the first initial morphological feature vector to obtain the gene-guided first full slice morphological feature vector.

[0018] The multimodal fusion prediction module is used to perform risk prediction by fusing the average morphological feature vector, the first full-slice morphological feature vector, and the global recurrence risk value, generating an early postoperative recurrence risk prediction score. Based on the first attention weight and the first morphological prototype soft assignment vector, it generates morphological prototype response distribution information to explain the causal relationship between gene mutations and image regions. The average morphological feature vector is obtained by global average pooling of each first initial morphological feature vector. The global recurrence risk value is extracted by the gene coding network based on the first genome mutation spectrum. The fusion prediction module is a neural network composed of a feature splicing layer and a fully connected classification layer.

[0019] An end-to-end joint training module is used to perform end-to-end joint optimization training on the early postoperative recurrence identification model for hepatocellular carcinoma based on recurrence prediction loss, morphological response consistency loss, and causal feature orthogonalization loss, resulting in a trained early postoperative recurrence identification model for hepatocellular carcinoma. The recurrence prediction loss is the binary cross-entropy loss between the early postoperative recurrence risk prediction score and the actual recurrence status label of the hepatocellular carcinoma patient. The morphological response consistency loss is obtained by calculating the distribution difference between the response spectrum of the attention morphological prototype obtained by weighting the soft assignment vector of each first morphological prototype based on the first attention weight and the response spectrum of the first global morphological prototype. The causal feature orthogonalization loss is obtained by calculating the orthogonality between the first full-slice morphological feature vector and the average morphological feature vector. The early postoperative recurrence identification model for hepatocellular carcinoma is an end-to-end neural network model including a feature extractor, a gene encoding network, a context decoupling module, a weighted pooling module, and a fusion prediction module.

[0020] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the training method for any of the above-mentioned early recurrence identification models for hepatocellular carcinoma after surgery.

[0021] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the training method for any of the above-mentioned early recurrence identification models for hepatocellular carcinoma after surgery.

[0022] The training method, apparatus, equipment, and media of the aforementioned early recurrence identification model for hepatocellular carcinoma postoperatively construct a discretized morphological semantic space based on a pathological morphological prototype library. By transforming the initial morphological features of tissue image blocks extracted by the feature extractor into interpretable morphological prototype soft-assignment vectors, local regions of whole-slice pathological images obtain clear morphological semantic representations. Through a gene coding network, the probabilistic association prior between gene mutations and global morphological patterns learned from a public cancer genome atlas database is injected into the model to generate a global morphological prototype response spectrum of genes. Then, through a context decoupling module, the gene synergistic regulation or antagonistic relationship between prototype dimensions is modeled, thereby... This approach transforms static gene mutation information into dynamic, context-aware gene-guided attention vectors, providing a clear molecular-level basis for image feature selection. In the weighted pooling module, the gene-guided attention vector is element-wise modulated with the morphological prototype soft-assignment vector of each image patch to obtain the original gene-guided attention score, which is then normalized to an attention weight. This weighted pooling of the initial morphological feature vector achieves precise causal alignment from global molecular events to local spatial morphology through soft addressing. This allows the model to selectively focus on the core morphological regions in the whole-slice image that are driven by specific gene mutations and possess the most molecular specificity and prognostic significance. This approach resolves the unexplainable problem caused by the blindness of traditional data-driven attention mechanisms. Simultaneously, by fusing the gene-guided full-slice morphological feature vector, the average morphological feature vector obtained by global average pooling of all initial morphological feature vectors, and the global recurrence risk value independently output by the gene coding network, the model can fully utilize the molecular-specific morphological information guided by genes while preserving the global morphological background and independent risk assessment at the pure molecular level, achieving complementarity and synergy of multi-dimensional information. During training, the attention-weighted prototype response spectrum of the morphology being studied and the gene-predicted global prototype response spectrum are calculated by introducing morphological response consistency loss. The distribution differences between them force the model to focus on morphological distributions that are consistent with its morphological expectations based on molecular information reasoning. This internalizes biological causal logic as a training objective in the form of soft constraints. At the same time, the introduction of causal feature orthogonalization loss forces the gene-guided full-slice morphological feature vector to be orthogonal to the average morphological feature vector in the projection subspace. This prompts the model to learn key discriminative features that go beyond the conventional global average appearance and are driven by gene mutation specificity. Finally, under the condition of only recurrence label supervision, an end-to-end training is conducted to obtain an early recurrence identification model for hepatocellular carcinoma after surgery that can output accurate recurrence risk scores and automatically generate an explanatory chain of causal relationship between gene mutations and image regions. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the training method of the early recurrence identification model for hepatocellular carcinoma after surgery according to the present invention.

[0025] Figure 2 This is a structural diagram of the training system for the early recurrence identification model of hepatocellular carcinoma after surgery according to the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0027] In one embodiment, such as Figure 1 As shown, a training method for a model to identify early recurrence after hepatocellular carcinoma surgery is provided. This embodiment illustrates the application of this method to a model training terminal. It is understood that this method can also be applied to a model training server, and further to a model training system including both a model training terminal and a model training server, and is implemented through the interaction between the model training terminal and the model training server. In this embodiment, the method includes the following steps:

[0028] S101. Obtain the first whole-section pathological image and the first genomic mutation spectrum of a hepatocellular carcinoma patient, and divide the first whole-section pathological image into image blocks to obtain the first tissue image block.

[0029] In illustrative terms, a whole-section pathological image refers to a scan generated by a digital pathology scanner at a preset magnification of a postoperative paraffin-embedded tissue section from a hepatocellular carcinoma patient. The objective magnification can be set to 40x. A genomic mutation profile refers to a matrix of mutation states of core genes related to the occurrence and development of hepatocellular carcinoma, obtained by target region capture sequencing or whole-exome sequencing of tumor tissue samples from the same patient, followed by bioinformatics analysis. This mutation state matrix can be a binary vector with a length equal to the number of pre-selected genes, where each dimension takes the value 1 or 0, representing whether the corresponding gene has functional mutations such as non-synonymous mutations, frameshift mutations, or splice site mutations.

[0030] Specifically, the model training terminal can use the OTSU threshold segmentation algorithm or a deep learning-based tissue segmentation model to exclude obvious background blank areas and fat vacuoles in the whole-section pathological image under low magnification in the masked area of ​​the tissue. Furthermore, at the highest resolution level corresponding to the 40x objective magnification, the tissue region is divided into a large number of tissue image blocks with a preset physical size and a certain overlap rate or no overlap. Each tissue image block corresponds to a local tissue microscopic image with a clear spatial coordinate position.

[0031] S102. Feature extraction is performed on each first tissue image block using a feature extractor to obtain the first initial morphological feature vector of each first tissue image block. The first morphological prototype soft assignment vector corresponding to each first tissue image block is obtained by calculating the similarity between each first initial morphological feature vector and each pathological prototype in the pathological prototype library. The feature extractor is a deep convolutional neural network pre-trained on natural images and completed contrastive self-supervised learning on a large-scale pathological image dataset. The pathological prototype library is obtained by clustering the second initial morphological feature vectors of the second tissue image blocks extracted from the second whole slice pathological images of known hepatocellular carcinoma patients in the training set.

[0032] Optionally, the backbone network of the feature extractor can be a residual network, whose initial parameters are pre-trained on natural image datasets such as ImageNet for supervised classification tasks to obtain general low-level visual feature extraction capabilities. Furthermore, the model training terminal performs secondary pre-training on a large dataset of unlabeled pathological tissue image patches using a contrastive self-supervised learning framework. Specifically, two random data augmentations are applied to each image patch to generate positive sample pairs. The two augmented versions of the same image patch are considered positive samples, and augmented versions of different image patches are considered negative samples. The backbone network is trained by minimizing the contrastive loss function, which brings positive sample pairs closer together in the feature space and pushes negative sample pairs further apart, allowing the network to learn feature representations with high discriminative power for pathological tissue morphology. After completing the contrastive self-supervised learning, the model training terminal removes the last fully connected classification layer of the backbone network, retaining only its convolutional part and global average pooling layer as a fixed-weight feature extractor. Its input is a tissue image patch that has been color-normalized, and its output is a high-dimensional initial morphological feature vector. This initial morphological feature vector represents the compact embedding representation of the input tissue image patch in a general and robust pathological morphological feature space.

[0033] Optionally, the pathological morphology prototype library is pre-constructed by the model training terminal through clustering the initial morphological feature vectors of tissue image blocks extracted from whole-slice pathological images of known hepatocellular carcinoma patients in the training set. Specifically, the model training terminal randomly extracts a large number of tissue image blocks from the whole-slice pathological images of all hepatocellular carcinoma patients in the training set, after excluding non-tumor areas such as background, necrosis, and hemorrhage. The initial morphological feature vectors are extracted using the same feature extractor described above. Furthermore, a K-means clustering algorithm is performed on the initial morphological feature vectors. The number of clusters K in K-means clustering is a hyperparameter much larger than the number of categories in conventional classification, and can be set to 5000. After clustering convergence, the model training terminal obtains K cluster center vectors, and the set of these K cluster center vectors constitutes the pathological morphology prototype library. , of which each It is a vector with the same dimension as the initial morphological feature vector, representing a fine-grained pathological morphological pattern that repeatedly appears in the training data.

[0034] Specifically, for a tissue image patch from a current patient, the model training terminal calculates its initial morphological feature vector. With each of the pathological morphology prototypes in the library The cosine similarity is used to obtain a similarity score vector containing K elements; further, this similarity score vector is normalized by introducing a temperature parameter using a Softmax function to generate a morphological prototype soft-assignment vector for the image patch. , its first The formula for calculating each component can be expressed as follows: ,in, This is a temperature coefficient used to control the sharpness of soft distribution. The smaller the value, the more the distribution of the soft-assigned vector tends to be a one-hot vector.

[0035] S103. Input the first genome mutation spectrum into the gene coding network to generate the expected first global morphological prototype response spectrum of the gene. Then, perform context decoupling processing on the first global morphological prototype response spectrum through the context decoupling module to obtain the first gene-guided attention vector. The gene coding network consists of a genome mutation spectrum embedding layer and a multilayer perceptron cascaded with prior knowledge. The multilayer perceptron is trained by taking the first genome mutation spectrum as input and the third global morphological prototype response spectrum obtained by statistical analysis of the third whole slice pathological image based on the public cancer genome map database as the regression target. The context decoupling module is an encoder composed of a multi-head self-attention mechanism and a feedforward neural network, which is used to model the synergistic regulation or antagonistic relationship between the morphological prototype dimensions in the first global morphological prototype response spectrum.

[0036] Specifically, the genome mutation spectrum embedding layer maps the discrete binary genome mutation spectrum to a continuous low-dimensional dense vector space, obtaining the initial gene feature vector. Its specific structure can be a trainable embedding matrix, with the number of rows equal to the number of genes and the number of columns equal to the embedding dimension. The multilayer perceptron consists of alternating stacks of fully connected layers and activation function layers. Its input is the initial gene feature vector, and its output is a K-dimensional vector, representing the expected global morphological prototype response spectrum of the gene, where K equals the number of prototypes in the pathological morphological prototype library. Optionally, the model training terminal utilizes whole-section pathological images and genomic mutation spectra of hepatocellular carcinoma patients from a public cancer genome atlas database. For each patient, it calculates the morphological prototype soft-assignment vectors of all tissue image blocks in the whole-section pathological image. Further, it performs summation pooling on all morphological prototype soft-assignment vectors of the whole-section pathological image, i.e., summing element-wise to obtain a global K-dimensional vector, i.e., the global morphological prototype response spectrum. The global morphological prototype response spectrum reflects the overall response intensity distribution of the patient's tumor on K pathological morphological prototypes. Optionally, the model training terminal uses the patient's genomic mutation spectrum as the input to a multilayer perceptron and its corresponding global morphological prototype response spectrum as the regression target. By minimizing the mean squared error loss function between the predicted response spectrum and the true response spectrum, it trains all weight parameters of the multilayer perceptron, thereby encoding the probabilistic association prior knowledge between gene mutation combinations and the overall pathological morphological pattern of the tumor into the weights of the network.

[0037] Furthermore, the context decoupling module is an encoder composed of a multi-head self-attention mechanism and a feedforward neural network. Its working principle is to treat a global morphological prototype response spectrum of length K as a sequence signal, where each position in the sequence corresponds to a pathological morphological prototype. Specifically, the model training terminal generates a positional code for each morphological prototype dimension. This positional code can be a sine / cosine positional code or a learnable embedding vector to represent the relative relationship information of different prototypes in the morphological semantic space. The positional code is added to the corresponding element of the global morphological prototype response spectrum to obtain an input sequence with positional codes. This input sequence is fed into an attention sublayer composed of multi-head self-attention layers. This multi-head self-attention mechanism, by calculating the attention weight between any two prototype dimensions in the sequence, can explicitly model the synergistic regulatory relationship of different gene mutation pathways on multiple morphological prototypes. For example, when multiple genes are co-mutated, they will jointly upregulate or downregulate the expected response values ​​of several specific morphological prototypes, demonstrating a synergistic regulatory relationship; or the mutation effect of one gene may be offset by the mutation effect of another gene, demonstrating an antagonistic relationship. After residual connection and layer normalization are performed on the output and input of the attention sub-layer, they are fed into the feedforward neural network sub-layer composed of fully connected layers and nonlinear activation functions. Residual connection and layer normalization are performed again, and finally, a K-dimensional vector with the same dimension as the input is output. This vector is the gene-guided attention vector.

[0038] S104. Based on the weighted pooling module, the first morphological prototype soft allocation vector of each first tissue image block is modulated by the first gene-guided attention vector to obtain the first gene-guided attention raw score of each first tissue image block, and the first attention weight is obtained by normalizing the first gene-guided attention raw score. The first initial morphological feature vector is then weighted and pooled to obtain the first gene-guided full slice morphological feature vector.

[0039] Optionally, the weighted pooling module is a functional computational module that performs modulation, normalization, and weighted pooling operations. The model training terminal achieves cross-modal alignment between global molecular events and local spatial phenotypes by injecting molecular-level causal guidance signals into the attention evaluation process of the image region. Specifically, for the first... Original score of gene-guided attention for each tissue image patch The calculation formula can be expressed as: In the formula, the morphological prototype soft assignment vector Each dimension component The gene-guided attention vector represents the degree of similarity between the tissue image patch and the k-th morphological prototype. Each dimension component This indicates how much weight the model should give to the region that presents the k-th morphological prototype due to the specific gene mutation carried by the patient.

[0040] Furthermore, after obtaining the raw gene-guided attention scores for all N tissue image patches, the model training terminal normalizes all scores using the Softmax function to obtain the final attention weights for each image patch. Its calculation formula is Optionally, the model training terminal uses attention weights. As coefficients, the initial morphological feature vectors of all tissue image patches Weighted summation is performed to obtain the gene-guided whole-slice morphological feature vector. Its calculation process can be expressed as follows: , To selectively extract the core morphological features most relevant to molecularly driven events from a whole-section pathological image under the causal guidance of a specific gene mutation spectrum in a specific patient, it discards a large amount of irrelevant background morphological information and focuses on spatial morphological patterns with prognostic significance driven by key gene mutations.

[0041] S105. Through the fusion prediction module, the average morphological feature vector, the first full-slice morphological feature vector, and the global recurrence risk value are fused together for risk prediction, generating an early postoperative recurrence risk prediction score. Based on the first attention weight and the first morphological prototype soft assignment vector, morphological prototype response distribution information is generated to explain the causal relationship between gene mutation and image region. The average morphological feature vector is obtained by global average pooling of each first initial morphological feature vector. The global recurrence risk value is extracted by the gene coding network based on the first genome mutation spectrum. The fusion prediction module is a neural network composed of a feature splicing layer and a fully connected classification layer.

[0042] Optionally, due to the full slice morphological feature vector The extraction process is highly selective, potentially losing some global morphological background information that is not directly explained by known driver gene mutations but still has prognostic value, such as the overall tumor differentiation degree, the average proportion of tumor stroma, and the overall abundance of immune cell infiltration. It also requires an independent risk assessment from the molecular modality itself as a baseline reference for fusion to achieve complementary and synergistic multi-dimensional information. Specifically, the average morphological feature vector is derived from the initial morphological feature vectors of all tissue image patches. It is obtained by performing global average pooling, and its calculation method is as follows: In the formula, This represents the global average representation of the whole-slice image in the morphological feature space, without any attention weighting, preserving the most complete but also the coarsest overall morphological profile. Global recurrence risk value. Extracted from the gene coding network based on the genome mutation spectrum. This represents an independent prediction of the patient's early relapse risk based solely on the patient's molecular mutation profile without observing any pathological morphological information, providing a purely molecular-level baseline reference for fusion decision-making.

[0043] Specifically, the feature concatenation layer combines the above three feature vectors , and The features are concatenated to form an expanded feature vector. Further, this feature vector is fed into a fully connected classification layer consisting of multiple fully connected layers. Each fully connected layer is followed by a non-linear activation function and a dropout regularization layer. The final output layer is a single-node fully connected layer activated by the sigmoid function, generating a postoperative early recurrence risk prediction score between 0 and 1. , This indicates the estimated probability that the model predicts the patient will experience an early recurrence within the predetermined postoperative time window.

[0044] Optionally, the model training terminal can also generate morphological prototype response distribution information to explain the causal relationship between gene mutations and image regions, based on attention weights and morphological prototype soft-assignment vectors. Specifically, the model training terminal can generate morphological prototype response distribution information based on attention weights. From highest to lowest weight, select the highest-weighted tissue image patches as key image patches. For each key image patch, a soft-assigned vector is generated based on its morphological prototype. Identify the morphological prototype with the strongest response, and simultaneously guide the attention vector from the genes. The model extracts the morphological prototypes corresponding to the dimensions with the largest response values, and generates an interpretable visualization output that maps the original image region of the key image block, its corresponding morphological prototype identifier, and the gene-driven prototype identifier that drives the focus. This allows clinicians to intuitively see which image region the model focuses on due to which gene mutation and what pathological morphological features that region has.

[0045] S106. An end-to-end joint optimization training of the early postoperative recurrence identification model for hepatocellular carcinoma is performed based on recurrence prediction loss, morphological response consistency loss, and causal feature orthogonalization loss to obtain the trained early postoperative recurrence identification model for hepatocellular carcinoma. The recurrence prediction loss is the binary cross-entropy loss between the early postoperative recurrence risk prediction score and the actual recurrence status label of the hepatocellular carcinoma patient. The morphological response consistency loss is obtained by calculating the distribution difference between the response spectrum of the attention morphological prototype obtained by weighting the soft assignment vector of each first morphological prototype based on the first attention weight and the response spectrum of the first global morphological prototype. The causal feature orthogonalization loss is obtained by calculating the orthogonality between the first full slice morphological feature vector and the average morphological feature vector. The early postoperative recurrence identification model for hepatocellular carcinoma is an end-to-end neural network model including a feature extractor, a gene encoding network, a context decoupling module, a weighted pooling module, and a fusion prediction module.

[0046] Before end-to-end joint optimization training, the model training terminal had completed the construction of the pathological morphology prototype library and the prior knowledge pre-training of the multilayer perceptron part of the gene coding network. At this time, the feature extractor parameters in the model remained frozen, the weights of the gene coding network were initialized by the pre-training parameters, and the weight parameters of the other modules were randomly initialized. The model training terminal used a dataset of hepatocellular carcinoma patients with paired complete whole-slice pathological images and genomic mutation profiles collected from this medical center or public databases to perform end-to-end training on the model. Each sample in this dataset was labeled with a true recurrence status label indicating whether recurrence occurred within a specific time window after surgery. The overall loss function of the training... It consists of a weighted sum of three losses, and its expression can be represented as follows: ,in , and This is a hyperparameter used to balance the contributions of various losses.

[0047] The first loss is the recurrence prediction loss. Specifically, the model training terminal uses the binary cross-entropy loss function for calculation, and its calculation formula can be expressed as follows: ,in, This is a label representing the patient's true relapse status, with a value of 0 or 1. It serves as a score for predicting the risk of early postoperative recurrence. It is used to measure the consistency between the model's predicted recurrence probability and the true label, and is the main supervisory signal that drives the model to learn to have the ability to detect recurrence.

[0048] The second loss is the loss of morphological response consistency. Specifically, the model training terminal calculates the response spectrum of the prototype of the form of interest, and the formula for calculating the response spectrum of the prototype of the form of interest is as follows: In the formula, For the first Attention weights for each tissue image patch, The soft-assigned vector is the morphological prototype. The response spectrum of the prototype of the desired form. Further, the model training terminal computes... Global morphological prototype response spectrum The distributional differences between them result in morphological response consistency loss. The difference in this distribution can be measured using the two-way symmetric Kullback-Leibler divergence. The visual attention behavior generated by the forced model based on gene guidance must maintain statistical consistency with the global morphological expectations inferred from the gene information itself, thereby internalizing the biological causal logic as part of the training objective in the form of soft constraints.

[0049] The third loss is the orthogonalization loss of causal characteristics. Specifically, the model training terminal constructs a trainable projection matrix. , the full slice morphological feature vector and average morphological eigenvector Each through the projection matrix Perform a linear transformation to obtain the projection vector. Second projection vector Furthermore, the model training terminal computes... and Cosine similarity between Its calculation formula is ; Orthogonalization loss of causal features By minimizing the causal feature orthogonalization loss, the model training terminal encourages the gene-guided full-slice morphological feature vector and the globally average morphological feature vector to tend to be orthogonal or at least uncorrelated in the projection subspace for predicted recurrence. The forced We must capture beyond The key discriminative information represented by the simple average appearance and molecular specificity further strengthens the causal pathway from gene mutation to specific morphological phenotype.

[0050] The model training terminal uses the backpropagation algorithm with the aforementioned total loss function. With the goal of optimization, the parameters of all trainable modules other than the feature extractor are iteratively optimized and updated until the model's performance converges on the validation set. After training, the model training terminal saves the complete model structure and trained parameter weights, resulting in a trained early recurrence recognition model for hepatocellular carcinoma after surgery.

[0051] In the training method of the aforementioned early recurrence identification model for hepatocellular carcinoma, the feature extractor discretizes the continuous pathological image feature space into standardized prototype representations with pathological semantics, making the internal representations of the model more transparent. The pathological morphology prototype library can assist in comparing with existing pathological knowledge and discovering new gene-morphology association patterns. The gene coding network and context decoupling module realize the biological causal chain modeling from gene mutation to pathological morphology, breaking the black box defect of traditional models and enabling the model prediction results to have molecular-level causal attribution logic. The weighted pooling module accurately filters and purifies the core pathological morphological features determined by gene mutation and possessing prognostic value, improving the effectiveness of feature prediction. Noise ratio; the fusion prediction module enables causal interpretability of model prediction results; by combining the recurrence prediction loss corresponding to the real recurrence label, the morphological response consistency loss that measures the difference in prototype response spectrum distribution, and the causal feature orthogonalization loss that constrains the orthogonality of two types of morphological features, the overall end-to-end neural network, including modules such as feature extractors and gene coding networks, is jointly optimized and trained. It does not require scarce and expensive spatial omics ground truth data, and can achieve cross-modal accurate weakly supervised alignment by relying only on routine clinical global labels, which greatly reduces data dependence. At the same time, it effectively captures the molecular-morphological synergistic prognostic effect and comprehensively improves the overall performance of early recurrence prediction after hepatocellular carcinoma surgery.

[0052] In one embodiment, the pathological morphology prototype library is constructed through the following steps:

[0053] S21. Extract the tumor region from each second whole slice pathological image in the training set of known hepatocellular carcinoma patients and divide it into second tissue image blocks according to preset physical dimensions.

[0054] Indicatively, a training set of known hepatocellular carcinoma patients serves as the foundational dataset for constructing a pathological morphology prototype library. This dataset can originate from a multi-center clinical cohort or public database that has undergone patient privacy-sensitive data protection. Each patient's whole-slice pathological image is a hematoxylin-eosin stained whole-slice scan image of a hepatocellular carcinoma radical surgical resection specimen. Exemplarily, the model training terminal downsamples the whole-slice pathological images at a low-resolution level. Further, the model training terminal extracts the hematoxylin staining intensity map using a color deconvolution-based hematoxylin and eosin channel separation algorithm. Then, it binarizes the hematoxylin intensity map using the OTSU automatic thresholding segmentation algorithm, while simultaneously using morphological closing operations and area filtering to remove small fragments and luminal blank areas, thereby generating a tumor tissue foreground mask. After determining the foreground mask of the tumor region, the model training terminal performs sliding window segmentation on the region within the mask at the highest resolution level with a preset physical size, such as a sliding window of 256 pixels by 256 pixels at a 40x objective lens magnification. The model training terminal can slide the window in a non-overlapping manner and save the image region corresponding to each sliding window as an independent tissue image block. Each tissue image block carries its spatial coordinate information in the original whole slice image.

[0055] S22. Perform color normalization processing on each second tissue image block to obtain each standard tissue image block.

[0056] Optionally, the model training terminal can use a color normalization method based on staining separation and template matching to perform color normalization processing on each tissue image block. Specifically, the model training terminal uses a preset color deconvolution matrix or one estimated from the target image to decompose the tissue image block in the RGB color space into hematoxylin staining channels and eosin staining channels, where the column vectors of the color deconvolution matrix correspond to the projection directions of the characteristic absorption spectra of hematoxylin and eosin dyes in the RGB space, respectively. Further, the mean and standard deviation of the pixel intensity of each staining channel are calculated as a statistical description of the staining distribution of the tissue image block. Optionally, a template image block with good staining quality is selected, and the mean and standard deviation of its hematoxylin and eosin channels are extracted as parameters of the target distribution. By performing a linear transformation on the values ​​of each staining channel of the original tissue image block, the mean and standard deviation of the original distribution are mapped to the mean and standard deviation of the target distribution. Then, the transformed staining channels are resynthesized into an RGB image through the inverse operation of the color deconvolution matrix, thus obtaining the standard tissue image block after color normalization. Optionally, the model training terminal can also adopt a domain transfer method based on generative adversarial networks. By training an image style transfer network, the coloring style of any input image patch can be transformed into a fixed reference coloring style, thereby unifying the color appearance while maintaining the organizational structure.

[0057] S23. Input each standard tissue image block into the feature extractor for dimensionality reduction feature extraction to obtain the second initial morphological feature vector of each standard tissue image block.

[0058] For example, the backbone network of the feature extractor can adopt a residual network structure, specifically configured with 50 layers, comprising four stages consisting of multiple stacked bottleneck residual blocks. Each bottleneck residual block consists of one 1×1 convolution, one 3×3 convolution, and one 1×1 convolution, with a batch normalization layer and a rectified linear unit activation function appended after each convolution. This residual network is pre-trained on a natural image dataset for a supervised classification task to learn common low-level visual features such as edges, textures, and shapes. Furthermore, the model is trained on a large-scale unlabeled pathological image dataset, using momentum contrastive learning or a simple contrastive learning framework for contrastive self-supervised training of the backbone network. The loss function for this contrastive self-supervised training is the information-noise contrastive estimation loss, which can be expressed as: In the formula, To query the feature vector of a sample, The feature vector of the key vector of positive samples belonging to the same image patch as the query sample. The feature vector of the key vector of the negative sample that does not belong to the same image patch as the query sample. Temperature parameters are used to control the concentration of the distribution. The feature extractor is used to map the input tissue image patch, which is 256×256×3 channels in size, into an initial morphological feature vector of length 2048 dimensions.

[0059] S24. Based on the preset major category hyperparameters, k-means clustering is performed on each second initial morphological feature vector to obtain multiple cluster center vectors.

[0060] Specifically, through clustering The initial morphological feature vectors are divided into Clusters are defined as disjoint subsets, such that the distance between vectors within each subset is minimized in the morphological feature space, while the distance between cluster centers of different subsets is maximized. Large-class hyperparameters. The preset value should be significantly larger than the number of categories in a conventional classification task. Specifically, it should be based on the complex combination of different cell differentiation degrees, different growth patterns, different types of interstitial reactions, and different degrees of immune infiltration involved in the pathological heterogeneity of hepatocellular carcinoma. For example, it can be set to... Optionally, the model training terminal can perform K-means clustering using the standard Lloyd algorithm. Specifically, the model training terminal randomly selects... The initial morphological feature vectors are used as cluster center vectors. The initial value; for each initial morphological feature vector Calculate its relationship with indivual The distance between the cluster centers is calculated and assigned to the clusters with the smallest distances. This distance metric can be Euclidean distance. Or use cosine distance For each cluster Calculate the arithmetic mean of all initial morphological feature vectors within a cluster, and use this as the updated cluster center vector. The model training terminal repeatedly performs the initial morphological feature vector allocation and cluster center vector update until the change in the cluster center vector is less than the preset convergence threshold. Alternatively, the clustering process may converge once the preset maximum number of iterations is reached. After convergence, the model training terminal retains the final result. Cluster center vectors.

[0061] S25. A pathological morphology prototype library is constructed based on the cluster center vectors. The cluster center vectors in the pathological morphology prototype library are used to characterize the fine-grained pathological morphological feature patterns that repeatedly appear in the training set of known hepatocellular carcinoma patients.

[0062] Furthermore, the pathological morphology prototype library is A sorted set of cluster center vectors, the model training terminal can store this sorted set as a file of size ... A floating-point matrix, where, Let be the dimension of the initial morphological feature vector. Each row of this floating-point matrix corresponds to a pathological morphological prototype, denoted as . ,in, The value is from arrive Integers. Optionally, the model training terminal can backsample the tissue image patch clusters corresponding to each cluster center vector, that is, randomly extract several original images from all standard tissue image patches belonging to the cluster, and summarize the potential pathological morphological semantics corresponding to the prototype through manual or automated visual inspection. For example, a certain pathological morphological prototype may correspond to a highly aggressive pattern of poorly differentiated tumor cells arranged in solid sheets and with abundant nuclear mitotic figures.

[0063] In one embodiment, the gene-coding network is constructed through the following steps:

[0064] S31. Obtain the third whole slide pathological images and third genome mutation profiles of each hepatocellular carcinoma patient from the public cancer genome map database.

[0065] As an illustration, a public cancer genome atlas database could be an open dataset maintained by the Cancer Genome Atlas Project, which contains multi-omics data from a large cohort of hepatocellular carcinoma patients from multiple clinical centers around the world, including paired digitized whole-slice pathological images and somatic mutation information, i.e., genomic mutation profiles, detected through standardized analysis procedures.

[0066] S32. Input each third tissue image block in the third whole slice pathological image into the feature extractor to obtain the third initial morphological feature vector of each third tissue image block.

[0067] Specifically, the model training terminal sequentially performs the same preprocessing steps on each whole-slice pathological image: extracting the effective region of tumor tissue, dividing it into tissue image blocks according to a preset physical size using a sliding window, and performing color normalization on each tissue image block. Further, the model training terminal calls the feature extractor, taking each color-normalized tissue image block as input. After multiple convolutional operations and global average pooling by the feature extractor, it outputs a dimensionless vector image. The initial morphological feature vector. For each hepatocellular carcinoma patient, the whole-section pathological image is usually divided into tens of thousands to hundreds of thousands of tissue image blocks, and each patient will correspond to a vector containing tens of thousands to hundreds of thousands of features. A set of dimensional feature vectors that fully encodes all local morphological information of the patient's tumor slice in a general pathological morphological feature space.

[0068] S33. Calculate the cosine similarity between each third initial morphological feature vector and each cluster center vector in the pathological morphological prototype library, and perform nonlinear activation and probability distribution transformation on the cosine similarity through the softmax function to obtain the third morphological prototype soft assignment vector.

[0069] Furthermore, regarding the first Initial morphological feature vector of each tissue image patch The model training terminal first calculates the vector and its relationship with each pathological morphology prototype in the pathological morphology prototype library. Cosine similarity between Its calculation formula can be expressed as: ,in, It is the L2 norm. Similarly, the model training terminal introduces a temperature parameter. The softmax function is used to calculate the similarity vector. Transform into a morphological prototype soft-assignment vector Optional, temperature parameter It is a positive real hyperparameter used to adjust the information entropy of the softmax output distribution. As the softmax expression approaches zero, the output tends towards a one-hot vector, meaning each image patch is forced to be assigned to a unique morphological prototype that is most similar to it. This corresponds to the extreme case of hard assignment. As the softmax value approaches infinity, the output tends towards a uniform distribution, meaning that each image patch responds consistently to all morphological prototypes, losing its discriminative power for morphological identification. For example, Possible values arrive The constants between these values ​​make the soft assignment vector of each image patch typically exhibit a sparse distribution with a few dominant prototypes having significant responses and most prototypes having responses close to zero. This soft assignment strategy not only preserves the biological rationale that an image patch may simultaneously contain a mixture of features from multiple prototypes, but also provides numerical stability for subsequent pooling statistics.

[0070] S34. Perform summation pooling aggregation on the soft-assigned vectors of the third morphological prototypes corresponding to each third tissue image block in the third whole-section pathological image to obtain the third global morphological prototype response spectrum of hepatocellular carcinoma patients.

[0071] Specifically, for a patient with hepatocellular carcinoma, the model training terminal collects all the pathological images from the whole slide. The corresponding tissue image patch Each morphological prototype soft assignment vector ,in The value is from arrive Furthermore, an element-wise summation operation is performed on all morphological prototype soft-assignment vectors to obtain a vector of length [missing information]. vector , its first Dimensional components The calculation formula can be expressed as: That is, for the first A pathological morphological prototype, It is proportional to the cumulative count of the total number of image patches in the patient's whole-section image that are similar to the prototype morphology in the sense of soft allocation. Vector quantification describes the overall tumor size in this patient. The total response intensity distribution across fine-grained pathological morphological patterns. Optionally, this is determined by the number of effective tissue image blocks contained in different whole-section pathological images. The differences are significant, and the model training terminal can further refine the model. L1 normalization of the vector is performed by dividing each component by the sum of all components, so that the L1 norm of the global morphological prototype response spectrum is . This eliminates the influence of image area differences on the magnitude of the response spectrum, making the response spectra comparable between different patients, and ultimately yielding a global morphological prototype response spectrum. The global morphological prototype response spectrum represents a high-order statistical summary of the global morphological features of the whole-section pathological image of the hepatocellular carcinoma patient in the morphological prototype semantic space.

[0072] S35. Using the third genome mutation spectrum as input and the third global morphological prototype response spectrum as the regression target, the gene coding network is pre-trained by minimizing the mean squared error loss between the predicted global morphological prototype response spectrum and the third global morphological prototype response spectrum. This yields the weight parameters of the multilayer perceptron and the probabilistic prior association between gene mutation combinations and the overall pathological morphological pattern of the tumor.

[0073] For example, the specific structure of a multilayer perceptron may include an input layer, several hidden layers, and an output layer. The number of nodes in the input layer is equal to the number of genes in the genomic mutation spectrum. Each hidden layer consists of a fully connected layer, a batch normalization layer, and an activation function layer cascaded together. The number of hidden layers can be three to five, and the number of hidden units in each layer can be gradually reduced from the number of input genes to... One; the output layer is a collection A fully connected layer with n nodes, without using an activation function, directly outputs a single value. A real-valued vector of dimension, denoted as the predicted global morphological prototype response spectrum. .

[0074] During pre-training, the multilayer perceptron portion of the gene-coding network has not yet been assigned meaningful weights, and its parameters can be initialized using a uniform distribution initialization method. The model training terminal inputs the genomic mutation spectrum into this multilayer perceptron, and the results are obtained through layer-by-layer forward propagation. Furthermore, the model training terminal computes... Global morphological prototype response spectrum The mean squared error loss function between them can be expressed as follows: ,in, and The predicted and actual global morphological prototype response spectra are respectively at the 1st... The values ​​can be taken in each prototype dimension. Optionally, the model training terminal can use an adaptive moment estimation optimizer to iteratively update all trainable parameters of the multilayer perceptron, and training continues until the mean squared error loss on the validation set no longer decreases. After pre-training convergence, the weight parameters of the multilayer perceptron encode the probabilistic prior association between gene mutation combinations and the overall pathological morphological pattern of tumors, which is statistically valid at the level of large samples in public databases.

[0075] S36. Using the weight parameters of the multilayer perceptron as the initial weights of the multilayer perceptron in the gene coding network, a gene coding network is constructed by combining the genome mutation spectrum embedding layer and the probabilistic association prior. The gene coding network is used to transform the genome mutation spectrum into an initial gene feature vector through the genome mutation spectrum embedding layer, and based on the probabilistic association prior, the initial gene feature vector is mapped through the multilayer perceptron to obtain the predicted global morphological prototype response spectrum. The first path of the multilayer perceptron is used to output the global relapse risk value based on molecular information, and the second path of the multilayer perceptron is used to output the gene-guided global morphological prototype response spectrum.

[0076] Optionally, the specific structure of the genome mutation spectrum embedding layer can be a trainable embedding matrix, where the embedding dimension can be set to map each gene to a 64-dimensional embedding vector. Each row of this embedding matrix corresponds to a learnable distributed representation vector of a gene, used to capture the semantic similarity of different genes in functional pathways. When the input genome mutation spectrum is a binary vector consisting of 0s and 1s, the model training terminal obtains the embedding vector corresponding to the gene with a mutation state of 1 through a lookup table operation, concatenates or weights the embedding vectors of all mutated genes, and then further compresses them through a fully connected layer to finally obtain an initial gene feature vector with a fixed dimension.

[0077] Optionally, the output layer of the multilayer perceptron can be a branched structure with dual outputs, splitting into a first output and a second output at the end of the network. The first output, through an additional fully connected layer and a sigmoid activation function, produces a scalar value between 0 and 1, representing a global relapse risk value based on molecular information. The second output is the predicted global morphological prototype response spectrum, which is a K-dimensional vector. In the same forward propagation, the gene-encoding network can output both molecular guidance signals for image morphological attention and an independent baseline assessment value of relapse risk at the pure molecular level, providing complementary molecular-side information for the multimodal fusion module.

[0078] In one embodiment, the first global morphological prototype response spectrum is decoupled from the context using a context decoupling module to obtain a first gene-guided attention vector, including:

[0079] S41. Generate positional codes for each morphological prototype dimension of the first global morphological prototype response spectrum to obtain a global morphological prototype response spectrum with positional codes.

[0080] The global morphological prototype response spectrum is an unordered vector, and the arrangement of its elements does not include the relative positional relationships between different morphological prototypes in the semantic space. If this unordered vector is directly fed into the subsequent self-attention mechanism, the multi-head self-attention mechanism will be unable to distinguish the structural differences between different morphological prototypes in the pathological morphological semantic space, and thus will be unable to utilize the latent semantic topology between morphological prototypes to model the complex regulatory effects of gene mutations. To endow each morphological prototype dimension in the sequence with identifiable positional information, optionally, the model training terminal generates a unique positional encoding vector for each morphological prototype dimension. The dimension of this positional encoding vector is the same as the embedding dimension of a single element in the global morphological prototype response spectrum. Positional encoding can adopt a sine-cosine positional encoding scheme. Specifically, for the first element in the sequence... The position and the first Each dimension, if If the number is even, the position code takes the value. ;like If the number is odd, the position code takes the value. ,in, This represents the index position of the morphological prototype in the sequence. The dimension of the encoding vector. This is the dimension index within the encoded vector. Optionally, the model training terminal can also employ a learnable positional encoding scheme, that is, initializing the positional encoding to a vector of size [size missing]. The trainable parameter matrix automatically adjusts the encoded value at each position through backpropagation during end-to-end training, enabling it to adaptively capture the optimal topological relationship between morphological prototypes in the training data.

[0081] S42. Input the position-encoded global morphological prototype response spectrum into the attention sub-layer based on the multi-head self-attention mechanism. The attention sub-layer calculates the attention weights between the dimensions of each morphological prototype to obtain the influence relationship of different gene mutation pathways on multiple morphological prototypes. The influence relationship is either a synergistic regulation or an antagonistic relationship.

[0082] Specifically, the model training terminal denotes the input positionally encoded global morphological prototype response spectrum as follows: Its size is ,in, This represents the feature dimension of the model. The model training terminal uses three different linear transformation matrices. , and ,Will Mapped to query matrices respectively Key matrix Sum matrix Its calculation process can be expressed as follows: , , ,in, , and All are of size or The trainable weight matrix, and These are the dimensions of the query vector and the value vector, respectively. Furthermore, the model training terminal calculates the scaled dot product attention, the formula of which can be expressed as... The multi-head self-attention mechanism executes the attention calculation process in parallel. Each time, a set of independent linear transformation matrices is used. , and ,get The outputs of each head are then concatenated along the feature dimension and passed through an additional output linear transformation matrix. Map back The model training terminal, through the aforementioned operations of the multi-head self-attention sublayer, transforms the input positionally encoded global morphological prototype response spectrum into a feature representation matrix that comprehensively encodes the interaction relationships between morphological prototypes by the attention mechanism, denoted as... .

[0083] S43. Perform residual connection and layer normalization processing on the global morphological prototype response spectrum and influence relationship with positional encoding to obtain the normalization result. Then, input the normalization result into the feedforward neural network sub-layer based on the fully connected layer to obtain the context-aware result.

[0084] Furthermore, residual connection is an identity mapping technique that directly sums the sublayer inputs to the sublayer outputs, and can be represented as: For example, the model training terminal outputs the attention sublayer. With positional encoding global morphological prototype response spectrum Perform element-wise addition to obtain the vector after residual concatenation. , This effectively alleviates the vanishing gradient problem in deep network training, while ensuring that even in the early stages of training when the attention sublayer has not yet learned meaningful morphological prototype interaction relationships, the original morphological expectation information in the positionally encoded global morphological prototype response spectrum is not lost or distorted. Furthermore, the model training terminal... Execution layer normalization operation, that is, for For each sample vector, calculate ,in, The mean, For variance, For trainable scaling parameters, These are trainable translation parameters.

[0085] Optionally, the feedforward neural network sublayer consists of two fully connected layers and a non-linear activation function, and its calculation process can be expressed as follows: ,in, and These are the weight matrices for the first and second fully connected layers, respectively. and For the corresponding bias vector, is the rectified linear unit activation function. The first fully connected layer reduces the feature dimension from Expand to , It can be Two to four times larger to provide greater intermediate representation capacity; the second fully connected layer further increases the dimension from Compress back To obtain context-aware results The feedforward neural network sublayer is used to independently perform further deep nonlinear transformations on the attention-aggregated contextual features at each morphological prototype location to extract higher-level, location-specific feature representations.

[0086] S44. Perform residual connection and layer normalization processing on the normalization result and the context-aware result to obtain the first gene-guided attention vector.

[0087] Specifically, the output of the second residual connection is Similarly, the model training terminal... Perform layer normalization again to obtain the gene-guided attention vector. The vector is a vector of length . The vector has the same dimensions as the input global morphological prototype response spectrum, but the values ​​of each dimension are no longer isolated global morphological predictions based solely on linear inference from the gene mutation spectrum. Instead, they comprehensively consider all... Following the modeling of synergistic regulation and antagonistic relationships among morphological prototypes via a multi-head self-attention mechanism, a recalibrated molecular guidance signal with global context awareness is generated. This signal represents the first gene in the gene-guided attention vector. Dimensional components This represents the attention modulation intensity assigned to the dimension of the morphological prototype by the model training terminal after fully considering the complex interaction effects caused by gene mutations between the morphological prototype and all other morphological prototypes. The attention modulation intensity is used to modulate the soft assignment vector of the morphological prototype element by element, thereby realizing the precise causal guidance of global molecular events on local spatial morphology.

[0088] In one embodiment, based on a weighted pooling module, the first morphological prototype soft-assignment vector of each first tissue image block is modulated using a first gene-guided attention vector to obtain a first gene-guided attention raw score for each first tissue image block. Then, a first attention weight is obtained by normalizing the first gene-guided attention raw score. Finally, a weighted pooling is performed on the first initial morphological feature vector to obtain a gene-guided first full-slice morphological feature vector, including:

[0089] S51. For each first tissue image block, calculate the sum of the element-wise products of the first gene-guided attention vector and the first morphological prototype soft assignment vector to obtain the original score of the first gene-guided attention for each first tissue image block.

[0090] Specifically, gene-guided attention raw score The numerical value also depends on the soft assignment vector of the morphological prototype. The sharpness of the distribution, gene-guided attention vector The intensity of each component, and and exist Directional consistency in the dimensional prototype semantic space. When The most responsive prototype dimensions and When the prototype dimensions with the highest modulation intensity highly overlap, the dot product value is... When the value is maximized, the tissue image patch is assigned a high raw attention score; conversely, if... The prototype dimension with the strongest response is precisely... In dimensions with very low or even near-zero modulation intensity, the dot product value approaches zero, and the original attention score of that tissue image patch will be significantly suppressed. The model training terminal applies this calculation process to all tissue image patches in the whole slice image, obtaining a set of patches equal in number to the total number of image patches. Gene-guided attention original score.

[0091] S52. The original attention scores of each first gene are normalized by the softmax function to obtain the first attention weights of each first tissue image block.

[0092] Furthermore, regarding the first Attention weights for each tissue image patch That is, for any ,have ,and Gene-guided raw attention score The weight of slightly higher-scoring tissue image patches is further amplified after exponential transformation by the softmax function, while the weight of lower-scoring image patches is exponentially compressed to near zero. This results in a sparse distribution of attention, meaning that only a few image patches that are highly causally matched with gene guidance signals receive significant attention weights, while the weights of the vast majority of background or irrelevant image patches are close to zero.

[0093] S53. Based on the first attention weights, the first initial morphological feature vectors corresponding to each first tissue image block are weighted and summed to obtain the first whole slice morphological feature vector. The first whole slice morphological feature vector is used to characterize the morphological information with the most molecular specificity and prognostic significance in the first whole slice pathological image under the causal guidance of the first genome mutation spectrum.

[0094] Optionally, the final result obtained by the model training terminal It is a with A high-dimensional vector of the same dimension is used to characterize the compact representation of the core morphological features that are most molecularly specific and prognostic, selectively extracted and aggregated from tens of thousands to hundreds of thousands of local tissue images in a whole-section pathological image under the causal guidance of the specific genomic mutation spectrum carried by the hepatocellular carcinoma patient. Instead of passively reflecting the overall morphological profile of the whole slice image, it actively focuses on key spatial phenotypic patterns caused by or highly related to known driver gene mutations, thus laying the information foundation for achieving accurate causal alignment from molecular mechanisms to tissue morphology and subsequent high-performance recurrence risk prediction.

[0095] In one embodiment, the morphological response consistency loss is calculated through the following steps:

[0096] S61. Based on the first attention weight, the first morphological prototype soft assignment vectors corresponding to each first tissue image block are weighted and summed to obtain the response spectrum of the attention morphological prototype.

[0097] Schematic, prototype response spectrum of the form of interest The Dimensional components If a certain form of prototype corresponds to A higher value indicates that in the patient's whole-section image, the model training terminal, guided by genes, focused on and adopted a large number of tissue regions presenting the prototype features of this morphology. Conversely, if... A value close to zero indicates that the model training terminal hardly included the region presenting the morphological prototype features in the high-weight range during the visual attention process, regardless of the absolute abundance of the morphological prototype in the whole slice image.

[0098] S62. By using the temperature coefficient, the response spectrum of the prototype of the morphology of interest and the response spectrum of the first global morphology prototype are softened respectively to obtain the first smooth probability distribution and the second smooth probability distribution.

[0099] The expressions for the first smooth probability distribution and the second smooth probability distribution are as follows:

[0100] ;

[0101] ;

[0102] In the formula, For the first smooth probability distribution at the th The probability value of the morphological prototype dimension. For the second smooth probability distribution in the th... The probability value of the morphological prototype dimension. For temperature coefficient, The first in the prototype response spectrum of the concerned form The components of the morphological prototype dimension, The first global form prototype response spectrum The components of the morphological prototype dimension are called the morphological prototype dimension.

[0103] For example, the response spectrum of the prototype of the form of interest The model training terminal calculates the first smooth probability distribution. , its first Each component The calculation formula is: For the first global form prototype response spectrum The model training terminal calculates the second smooth probability distribution in the same way. , its first Each component Temperature coefficient It is a positive real hyperparameter whose physical function is to regulate the smoothness of the softmax function's output distribution. When When the value of is large, the input of the exponential function is compressed, the relative differences between the components decrease, and the resulting probability distribution approaches a uniform distribution. In this case, the model training terminal has a higher tolerance for subtle differences between the two response spectra; when When the value of is small, the differences between the original components are amplified by the exponential function, and the probability distribution tends towards a one-hot distribution, that is, only a few prototype dimensions with the strongest response are retained. In this case, the model training terminal is extremely sensitive to the differences between the response spectra. Optionally, Possible values arrive A certain constant between these two values ​​ensures that the output distribution retains the significant peak and valley information in the original response spectrum while avoiding excessive sharpness that could lead to gradient vanishing.

[0104] S63. Calculate the two-way symmetric KL divergence between the first smooth probability distribution and the second smooth probability distribution to obtain the morphological response consistency loss.

[0105] The expression for the morphological response consistency loss is:

[0106] ;

[0107] ;

[0108] ;

[0109] In the formula, For the loss of morphological response consistency, For the first smooth probability distribution, For the second smooth probability distribution, Let KL divergence be denoted as KL divergence.

[0110] Furthermore, the KL divergence, or Kullback-Leibler divergence, is an asymmetric measure in information theory used to measure the difference between two probability distributions. For example, for two discrete probability distributions... and , Compared to The KL divergence is defined as That is, given the true distribution is Under the premise of using distribution If we approximate the encoding, the average additional encoding length for each event is then... To obtain a symmetric metric that comprehensively measures the bidirectional difference between the two distributions, the model training terminal employs bidirectional symmetric KL divergence, i.e., morphological response consistency loss. The loss of this symmetric form simultaneously penalizes approximate and approximate Information loss in both directions causes the optimization process to impose equal penalty pressure on the two divergences distributed in any direction. Measured the expected distribution using genes To approximate the actual distribution of concern The additional information cost incurred at that time Measured by actual attention distribution To approximate the expected distribution of genes The additional information cost incurred at that time.

[0111] In one embodiment, the causal feature orthogonalization loss is calculated through the following steps:

[0112] S71. Perform global average pooling on the first morphological prototype soft allocation vector corresponding to each first tissue image block to obtain the average morphological feature vector.

[0113] Schematic, average morphological feature vector The geometric center or first-order statistical moment of the entire slice image in the morphological feature space encodes the global average morphological profile of the tumor tissue, including the average degree of differentiation of tumor cells, the mean overall tissue staining intensity, and macroscopic trends in cell density and stroma ratio. If a certain morphological feature with prognostic significance is uniformly distributed across all image patches, then... This is sufficient to capture the feature; conversely, if a key morphological feature is concentrated only in a local region composed of a few image patches, then... The signal of this feature will become weak due to dilution by the features of a large number of irrelevant image patches. This represents the upper limit of morphological information that a model training terminal can passively obtain from a whole-slice image without any prior molecular knowledge.

[0114] S72. The first full slice morphological feature vector and the average morphological feature vector are linearly transformed by the projection matrix to obtain the first projection vector and the second projection vector.

[0115] Optionally, the model training terminal constructs a trainable projection matrix. Its size is ,in, The dimension of the initial morphological feature vector. Let be the dimension of the projected subspace. It can take a value that is much smaller than integers, such as Wei or Dimension. Projection matrix Each row represents a basis vector in the projection subspace. These basis vectors are optimized and updated through backpropagation during end-to-end training. Part of the optimization objective is to maximize the discriminative power of recurrence prediction, while the other part comes from the backpropagation drive of the causal feature orthogonalization loss. This allows the projection subspace to simultaneously serve two parallel learning tasks: recurrence risk prediction and decoupling between gene-specific features and general features.

[0116] Specifically, the model training terminal will Through projection matrix Perform a linear transformation to obtain the projection vector. Its calculation process can be expressed as follows: ,Will Vie Mapping from the original high-dimensional morphological feature space to a A compact subspace of dimension, Each component is The weighted linear combination of each dimensional component along its corresponding projection direction. The model training terminal uses the exact same projection matrix. For the average morphological feature vector Perform a linear transformation to obtain the second projection vector. Its calculation process can be expressed as follows: .

[0117] S73. Calculate the absolute value of the cosine similarity between the first projection vector and the second projection vector to obtain the causal feature orthogonalization loss.

[0118] Specifically, the model training terminal first calculates and The dot product between them, i.e. ,in, and They are respectively and The Dimensional components. The L2 norm is calculated as follows: , The L2 norm is .

[0119] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0120] Based on the same inventive concept, this application also provides a training device for a hepatocellular carcinoma early recurrence identification model, which is used to implement the training method for the hepatocellular carcinoma early recurrence identification model described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more training device embodiments for the hepatocellular carcinoma early recurrence identification model provided below can be found in the limitations of the training method for the hepatocellular carcinoma early recurrence identification model described above, and will not be repeated here.

[0121] In one exemplary embodiment, such as Figure 2 As shown, a training device 200 for identifying early recurrence of hepatocellular carcinoma after surgery is provided, comprising:

[0122] Image preprocessing module 201 is used to acquire the first whole-section pathological image and the first genomic mutation spectrum of a hepatocellular carcinoma patient, and to segment the first whole-section pathological image into image blocks to obtain the first tissue image block;

[0123] The morphological prototype mapping module 202 is used to extract features from each first tissue image block using a feature extractor to obtain a first initial morphological feature vector for each first tissue image block, and to obtain a first morphological prototype soft assignment vector corresponding to each first tissue image block by calculating the similarity between each first initial morphological feature vector and each pathological prototype in the pathological morphological prototype library; the feature extractor is a deep convolutional neural network pre-trained on natural images and completed contrastive self-supervised learning on a large-scale pathological image dataset; the pathological morphological prototype library is obtained by clustering the second initial morphological feature vectors of second tissue image blocks extracted from second whole-section pathological images in a training set of known hepatocellular carcinoma patients.

[0124] The causal guidance signal generation module 203 is used to input the first genome mutation spectrum into the gene coding network, generate the expected first global morphological prototype response spectrum of the gene, and perform context decoupling processing on the first global morphological prototype response spectrum through the context decoupling module to obtain the first gene guidance attention vector. The gene coding network consists of a genome mutation spectrum embedding layer and a multilayer perceptron cascaded with prior knowledge. The multilayer perceptron is trained by taking the first genome mutation spectrum as input and the third global morphological prototype response spectrum obtained by statistical analysis of the third whole slice pathological image based on the public cancer genome atlas database as the regression target. The context decoupling module is an encoder composed of a multi-head self-attention mechanism and a feedforward neural network, used to model the synergistic regulation or antagonistic relationship between the morphological prototype dimensions in the first global morphological prototype response spectrum.

[0125] The causal alignment feature aggregation module 204 is used to modulate the first morphological prototype soft allocation vector of each first tissue image block through the first gene-guided attention vector based on the weighted pooling module, to obtain the first gene-guided attention raw score of each first tissue image block, and to obtain the first attention weight based on the normalization of the first gene-guided attention raw score, and to perform weighted pooling on the first initial morphological feature vector to obtain the gene-guided first full slice morphological feature vector.

[0126] The multimodal fusion prediction module 205 is used to perform risk prediction by fusing the average morphological feature vector, the first full-slice morphological feature vector, and the global recurrence risk value through the fusion prediction module, generating an early postoperative recurrence risk prediction score, and generating morphological prototype response distribution information to explain the causal relationship between gene mutations and image regions based on the first attention weight and the first morphological prototype soft assignment vector; the average morphological feature vector is obtained by global average pooling of each first initial morphological feature vector; the global recurrence risk value is extracted by the gene coding network based on the first genome mutation spectrum; the fusion prediction module is a neural network composed of a feature splicing layer and a fully connected classification layer;

[0127] The end-to-end joint training module 206 is used to perform end-to-end joint optimization training on the early recurrence identification model after hepatocellular carcinoma surgery based on recurrence prediction loss, morphological response consistency loss, and causal feature orthogonalization loss, to obtain the trained early recurrence identification model after hepatocellular carcinoma surgery. The recurrence prediction loss is the binary cross-entropy loss between the early recurrence risk prediction score after surgery and the actual recurrence status label of the hepatocellular carcinoma patient. The morphological response consistency loss is obtained by calculating the distribution difference between the response spectrum of the attention morphological prototype obtained by weighting the soft assignment vector of each first morphological prototype based on the first attention weight and the response spectrum of the first global morphological prototype. The causal feature orthogonalization loss is obtained by calculating the orthogonality between the first full slice morphological feature vector and the average morphological feature vector. The early recurrence identification model after hepatocellular carcinoma surgery is an end-to-end neural network model including a feature extractor, a gene encoding network, a context decoupling module, a weighted pooling module, and a fusion prediction module.

[0128] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps in the above method embodiments.

[0129] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0130] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0131] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A training method for a model to identify early recurrence of hepatocellular carcinoma after surgery, characterized in that, The method includes: The first whole-section pathological image and the first genomic mutation spectrum of a hepatocellular carcinoma patient were obtained, and the first whole-section pathological image was segmented into image blocks to obtain the first tissue image block; Feature extraction is performed on each of the first tissue image blocks using a feature extractor to obtain a first initial morphological feature vector for each of the first tissue image blocks. The similarity between each first initial morphological feature vector and each pathological morphological prototype in the pathological morphology prototype library is calculated to obtain a first morphological prototype soft-assignment vector corresponding to each of the first tissue image blocks. The feature extractor is a deep convolutional neural network pre-trained on natural images and subjected to contrastive self-supervised learning on a large-scale pathological image dataset. The pathological morphology prototype library is obtained by clustering the second initial morphological feature vectors of second tissue image blocks extracted from second whole-section pathological images in a training set of known hepatocellular carcinoma patients. The first genomic mutation spectrum is input into the gene coding network to generate the expected first global morphological prototype response spectrum. The first global morphological prototype response spectrum is then decoupled from the context by a context decoupling module to obtain the first gene-guided attention vector. The gene coding network consists of a cascaded genomic mutation spectrum embedding layer and a multilayer perceptron pre-trained with prior knowledge. The multilayer perceptron is trained by using the first genomic mutation spectrum as input and the third global morphological prototype response spectrum obtained by statistical analysis of the third whole-slice pathological images based on a public cancer genome atlas database as the regression target. The context decoupling module is an encoder composed of a multi-head self-attention mechanism and a feedforward neural network, used to model the synergistic regulation or antagonistic relationship between the morphological prototype dimensions in the first global morphological prototype response spectrum. Based on the weighted pooling module, the first morphological prototype soft allocation vector of each first tissue image block is modulated by the first gene-guided attention vector to obtain the first gene-guided attention raw score of each first tissue image block, and the first attention weight is obtained by normalizing the first gene-guided attention raw score. The first initial morphological feature vector is then weighted and pooled to obtain the first gene-guided full slice morphological feature vector. The fusion prediction module fuses the average morphological feature vector, the first full-slice morphological feature vector, and the global recurrence risk value for risk prediction, generating an early postoperative recurrence risk prediction score. Based on the first attention weight and the first morphological prototype soft assignment vector, morphological prototype response distribution information is generated to explain the causal relationship between gene mutations and image regions. The average morphological feature vector is obtained by global average pooling of each of the first initial morphological feature vectors. The global recurrence risk value is extracted by the gene coding network based on the first genome mutation spectrum. The fusion prediction module is a neural network composed of a feature splicing layer and a fully connected classification layer. An end-to-end joint optimization training of the early postoperative recurrence identification model for hepatocellular carcinoma is performed based on recurrence prediction loss, morphological response consistency loss, and causal feature orthogonalization loss to obtain the trained early postoperative recurrence identification model for hepatocellular carcinoma. The recurrence prediction loss is the binary cross-entropy loss between the early postoperative recurrence risk prediction score and the actual recurrence status label of the hepatocellular carcinoma patient. The morphological response consistency loss is obtained by calculating the distribution difference between the response spectrum of the attention morphological prototype obtained by weighting the soft assignment vector of each first morphological prototype based on the first attention weight and the response spectrum of the first global morphological prototype. The causal feature orthogonalization loss is obtained by calculating the orthogonality between the first full-slice morphological feature vector and the average morphological feature vector. The early postoperative recurrence identification model for hepatocellular carcinoma is an end-to-end neural network model including the feature extractor, the gene encoding network, the context decoupling module, the weighted pooling module, and the fusion prediction module.

2. The method according to claim 1, characterized in that, The pathological morphology prototype library was constructed through the following steps: The tumor region was extracted from each of the second whole-slice pathological images in the training set of known hepatocellular carcinoma patients and segmented according to a preset physical size to obtain a second tissue image block; Color normalization is performed on each of the second tissue image blocks to obtain each standard tissue image block; Each of the standard tissue image blocks is input into the feature extractor for dimensionality reduction feature extraction to obtain the second initial morphological feature vector of each of the standard tissue image blocks; Based on the preset major category hyperparameters, k-means clustering is performed on each of the second initial morphological feature vectors to obtain multiple cluster center vectors; The pathological morphology prototype library is constructed based on the cluster center vectors of each cluster; the cluster center vectors of each cluster center vector in the pathological morphology prototype library are used to characterize the fine-grained pathological morphological feature patterns that repeatedly appear in the training set of known hepatocellular carcinoma patients.

3. The method according to claim 2, characterized in that, The gene coding network was constructed through the following steps: Obtain the third whole-section pathological images and third genome mutation profiles of each hepatocellular carcinoma patient from the public cancer genome atlas database; Each third tissue image block in the third whole-section pathological image is input into the feature extractor to obtain the third initial morphological feature vector of each third tissue image block; Calculate the cosine similarity between each of the third initial morphological feature vectors and each of the cluster center vectors in the pathological morphological prototype library, and perform nonlinear activation and probability distribution transformation on the cosine similarity using the softmax function to obtain the third morphological prototype soft assignment vector. The third global morphological prototype response spectrum of the hepatocellular carcinoma patient is obtained by summing and pooling the soft-assigned vectors of the third tissue image blocks corresponding to each of the third whole-slice pathological images and performing a pooling and aggregation process on the soft-assigned vectors of the third morphological prototypes in each of the third tissue image blocks in the third whole-slice pathological image. Using the third genomic mutation spectrum as input and the third global morphological prototype response spectrum as the regression target, the gene coding network is pre-trained by minimizing the mean squared error loss between the predicted global morphological prototype response spectrum and the third global morphological prototype response spectrum, thereby obtaining the weight parameters of the multilayer perceptron and the probabilistic association prior between gene mutation combinations and the overall pathological morphological pattern of the tumor. The gene coding network is constructed by using the weight parameters of the multilayer perceptron as the initial weights of the multilayer perceptron in the gene coding network, combined with the genome mutation spectrum embedding layer and the probabilistic association prior. The gene coding network is used to convert the genome mutation spectrum into an initial gene feature vector through the genome mutation spectrum embedding layer, and based on the probabilistic association prior, the multilayer perceptron maps the initial gene feature vector to obtain the predicted global morphological prototype response spectrum. The first path of the multilayer perceptron is used to output the global relapse risk value based on molecular information, and the second path of the multilayer perceptron is used to output the gene-guided global morphological prototype response spectrum.

4. The method according to claim 1, characterized in that, The step of performing context decoupling processing on the first global morphological prototype response spectrum through the context decoupling module to obtain the first gene-guided attention vector includes: A positional code is generated for each of the morphological prototype dimensions of the first global morphological prototype response spectrum to obtain a global morphological prototype response spectrum with positional codes. The position-encoded global morphological prototype response spectrum is input into an attention sub-layer based on the multi-head self-attention mechanism. The attention sub-layer calculates the attention weights between each morphological prototype dimension to obtain the influence relationship of different gene mutation pathways on multiple morphological prototypes. The influence relationship is either the synergistic regulation or the antagonistic relationship. The position-encoded global morphological prototype response spectrum and the influence relationship are subjected to residual connection and layer normalization to obtain a normalization result. The normalization result is then input into a feedforward neural network sub-layer based on a fully connected layer to obtain a context-aware result. The normalized result and the context-aware result are subjected to residual connection and layer normalization to obtain the first gene-guided attention vector.

5. The method according to claim 1, characterized in that, The weighted pooling module modulates the first morphological prototype soft-assignment vector of each of the first tissue image blocks using the first gene-guided attention vector to obtain the first gene-guided attention raw score for each of the first tissue image blocks. Based on the normalization of the first gene-guided attention raw score, a first attention weight is obtained. The first initial morphological feature vector is then weighted and pooled to obtain the gene-guided first full-slice morphological feature vector, including: For each of the first tissue image blocks, the sum of the element-wise products of the first gene-guided attention vector and the first morphological prototype soft assignment vector is calculated to obtain the first gene-guided attention raw score for each of the first tissue image blocks. The original attention scores of each first gene are normalized using the softmax function to obtain the first attention weights of each first tissue image block. Based on each of the first attention weights, the first initial morphological feature vectors corresponding to each of the first tissue image blocks are weighted and summed to obtain the first whole-slice morphological feature vector; the first whole-slice morphological feature vector is used to characterize the morphological information with the most molecular specificity and prognostic significance in the first whole-slice pathological image under the causal guidance of the first genomic mutation spectrum.

6. The method according to claim 1, characterized in that, The morphological response consistency loss is calculated through the following steps: Based on the first attention weight, the first morphological prototype soft assignment vector corresponding to each first tissue image block is weighted and summed to obtain the attention morphological prototype response spectrum. By using a temperature coefficient, the response spectrum of the prototype of the morphology of interest and the response spectrum of the first global morphology prototype are softened respectively to obtain a first smooth probability distribution and a second smooth probability distribution. Calculate the bidirectional symmetric KL divergence between the first smooth probability distribution and the second smooth probability distribution to obtain the morphological response consistency loss; The expressions for the first smooth probability distribution and the second smooth probability distribution are as follows: ； ； In the formula, For the first smooth probability distribution at the th The probability value of the morphological prototype dimension. For the second smooth probability distribution in the th... The probability value of the morphological prototype dimension. For temperature coefficient, The first in the prototype response spectrum of the concerned form The components of the morphological prototype dimension, The first global form prototype response spectrum The components of the morphological prototype dimension are called the morphological prototype dimension. The expression for the morphological response consistency loss is: ； ； ； In the formula, For the loss of morphological response consistency, For the first smooth probability distribution, For the second smooth probability distribution, Let KL divergence be denoted as KL divergence.

7. The method according to claim 1, characterized in that, The orthogonalization loss of the causal features is calculated through the following steps: Global average pooling is performed on the first morphological prototype soft assignment vector corresponding to each first tissue image block to obtain the average morphological feature vector. The first projection vector and the second projection vector are obtained by linearly transforming the first full slice morphological feature vector and the average morphological feature vector respectively using the projection matrix. The absolute value of the cosine similarity between the first projection vector and the second projection vector is calculated to obtain the causal feature orthogonalization loss.

8. A training device for a model to identify early recurrence of hepatocellular carcinoma after surgery, characterized in that, The device includes: The image preprocessing module is used to acquire the first whole-section pathological image and the first genomic mutation spectrum of a hepatocellular carcinoma patient, and to segment the first whole-section pathological image into image blocks to obtain the first tissue image block; The morphological prototype mapping module is used to extract features from each of the first tissue image blocks using a feature extractor to obtain a first initial morphological feature vector for each of the first tissue image blocks, and to obtain a first morphological prototype soft assignment vector corresponding to each of the first tissue image blocks by calculating the similarity between each of the first initial morphological feature vectors and each pathological morphological prototype in the pathological morphological prototype library; the feature extractor is a deep convolutional neural network pre-trained on natural images and performing contrastive self-supervised learning on a large-scale pathological image dataset; the pathological morphological prototype library is obtained by clustering the second initial morphological feature vectors of the second tissue image blocks extracted from the second whole-section pathological images of known hepatocellular carcinoma patients in the training set. A causal guidance signal generation module is used to input the first genomic mutation spectrum into the gene coding network to generate the expected first global morphological prototype response spectrum of the gene, and to perform context decoupling processing on the first global morphological prototype response spectrum through a context decoupling module to obtain a first gene guidance attention vector. The gene coding network consists of a cascaded genomic mutation spectrum embedding layer and a multilayer perceptron pre-trained with prior knowledge. The multilayer perceptron is trained by using the first genomic mutation spectrum as input and the third global morphological prototype response spectrum obtained by statistical analysis of the third whole-slice pathological image based on the public cancer genome atlas database as the regression target. The context decoupling module is an encoder composed of a multi-head self-attention mechanism and a feedforward neural network, used to model the synergistic regulation or antagonistic relationship between the morphological prototype dimensions in the first global morphological prototype response spectrum. The causal alignment feature aggregation module is used to modulate the first morphological prototype soft allocation vector of each first tissue image block through the first gene-guided attention vector based on the weighted pooling module, to obtain the first gene-guided attention raw score of each first tissue image block, and to obtain the first attention weight based on the normalization of the first gene-guided attention raw score, and to perform weighted pooling on the first initial morphological feature vector to obtain the first gene-guided full slice morphological feature vector. The multimodal fusion prediction module is used to perform risk prediction by fusing the average morphological feature vector, the first full-slice morphological feature vector, and the global recurrence risk value through the fusion prediction module, generating an early postoperative recurrence risk prediction score, and generating morphological prototype response distribution information to explain the causal relationship between gene mutations and image regions based on the first attention weight and the first morphological prototype soft assignment vector; the average morphological feature vector is obtained by global average pooling of each of the first initial morphological feature vectors; the global recurrence risk value is extracted by the gene coding network based on the first genome mutation spectrum; the fusion prediction module is a neural network composed of a feature splicing layer and a fully connected classification layer; An end-to-end joint training module is used to perform end-to-end joint optimization training on the early postoperative recurrence identification model for hepatocellular carcinoma based on recurrence prediction loss, morphological response consistency loss, and causal feature orthogonalization loss, to obtain the trained early postoperative recurrence identification model for hepatocellular carcinoma. The recurrence prediction loss is the binary cross-entropy loss between the early postoperative recurrence risk prediction score and the actual recurrence status label of the hepatocellular carcinoma patient. The morphological response consistency loss is obtained by calculating the distribution difference between the response spectrum of the attention morphological prototype obtained by weighting the soft assignment vector of each first morphological prototype based on the first attention weight and the response spectrum of the first global morphological prototype. The causal feature orthogonalization loss is obtained by calculating the orthogonality between the first full-slice morphological feature vector and the average morphological feature vector. The early postoperative recurrence identification model for hepatocellular carcinoma is an end-to-end neural network model including the feature extractor, the gene encoding network, the context decoupling module, the weighted pooling module, and the fusion prediction module.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images