An information processing method for traditional Chinese medicine auxiliary diagnosis and treatment of endometrial lesions

Abstract

This invention discloses an information processing method for TCM-assisted diagnosis and treatment of endometrial lesions. The method first fuses time-series hysteroscopic images, TCM four diagnostic methods, and clinical indicators to construct a dynamic graph sequence. By using phantoms as analysis units, multi-dimensional features encoding local interactions, lesion propagation, internal co-evolution, and sudden events are generated. Learnable weights are used to fuse these features, and intra-group and inter-group attention calculations are performed using a Token-Group Transformer to capture long-term temporal dependencies. Finally, anomaly scores are obtained by calculating the Mahalanobis distance between the phantom representation and the normal distribution. These scores are compared with clinical thresholds to output endometrial lesion classification information as auxiliary diagnostic and treatment information. This invention effectively integrates multimodal heterogeneous data, reduces dependence on labeled data, and improves the processing capability and efficiency of medical and health data for atypical lesions.

Description

Technical Field

[0001] This invention relates to the field of medical data processing and endometrial disease prediction technology, and in particular to an information processing method for the auxiliary diagnosis and treatment of endometrial lesions using traditional Chinese medicine. Background Technology

[0002] Hysteroscopy, as a routine diagnostic tool for endometrial lesions, faces several areas for improvement in its technological development. Current diagnostic methods have room for improvement in the following aspects: Regarding diagnostic modalities, existing methods rely heavily on physician experience. This reliance is particularly evident in scenarios such as early lesion identification, and related data processing tools have room for improvement in providing quantitative analysis support. At the data processing level, existing systems primarily employ a single-frame image analysis architecture. This architecture has limitations in characterizing the dynamic development of lesions, making it difficult to fully present the temporal evolution characteristics of lesions. Furthermore, current systems have relatively limited capabilities in multimodal data integration, especially in the insufficient integration and application of non-imaging data such as the four diagnostic methods of traditional Chinese medicine. From a technical implementation perspective, graph neural network-based methods currently mostly employ static graph processing, which may have insufficient adaptability in dynamic modeling applications. Simultaneously, existing learning methods rely heavily on labeled data, while labeled data resources in the medical field are relatively limited, which affects the scope of application of these methods. Additionally, medical data typically has diverse sources, and there are certain differences between data from different institutions. This difference places special demands on the applicability of the method and needs to be considered during the technical design phase.

[0003] Based on the above, the industry is exploring solutions that can better handle multimodal data and adapt to dynamic analysis needs, in order to better meet the actual needs of clinical diagnosis and treatment. Therefore, there is an urgent need in this field for a medical data processing method that can effectively integrate multimodal information and support dynamic lesion analysis. Summary of the Invention

[0004] The purpose of this invention is to provide an information processing method for the auxiliary diagnosis and treatment of endometrial lesions using traditional Chinese medicine. This method integrates hysteroscopic images, information from the four diagnostic methods of traditional Chinese medicine, and clinical indicator data using a multimodal fusion mechanism to generate multi-dimensional phantom codes. Based on a group attention mechanism and distribution deviation calculation, it performs auxiliary detection of abnormalities, thereby achieving auxiliary detection and risk assessment of endometrial lesions.

[0005] The technical solution adopted in this invention is:

[0006] A method for information processing in the auxiliary diagnosis and treatment of endometrial lesions using traditional Chinese medicine includes the following steps:

[0007] Step S1: Acquire time-series hysteroscopic image data, TCM four diagnostic methods information, and clinical indicator data to construct a dynamic image sequence of the endometrium. Where V represents the set of fixed uterine region image nodes obtained by segmenting hysteroscopic images. This represents the dynamic relationship edges between image nodes at time t;

[0008] Step S2: Obtain the mesoscale substructures in the dynamic graph sequence as a motif. This is used to capture multi-regional collaborative lesion patterns. The mesoscale substructure is composed of several image nodes with spatiotemporal correlation and their dynamic relationship edges, and its scale is between that of a single image node and a complete dynamic graph. Indicates the first There are 10 modalities. Among them, the i-th modality... Image features of image node x of the lesion area extracted from hysteroscopic image data at time point t. Traditional Chinese Medicine diagnostic information for image node x Clinical indicator data features after normalization of image node x fusion generation The initial feature representation of any image node x at time point t ; Let k be the i-th modifier, and k be the number of modifiers.

[0009] Step S3: Sample naturally co-occurring image node triples from historical clinical data as normal phantoms. It generates abnormal phantoms by injecting preset cancerous noise into image node features. This forms a binary training label set. ;

[0010] Step S4, calculate the phantom The average features were extracted from the phantom. The neighborhood set;

[0011] Step S5, for each phantom Generate multidimensional motif codes, which include neighbor relationship codes representing local interaction patterns, propagation path codes representing lesion spread trends, evolutionary synchronization codes representing internal co-evolution, and standardized event burst codes.

[0012] Step S6: The multi-dimensional phantom encoding is concatenated into vectors and linearly projected and fused using a learnable fusion weight matrix to obtain a preliminary phantom representation; the preliminary fused representation is then subjected to layer normalization and superimposed with sine-cosine position embedding enhancement based on phantom spatial coordinates to output a standardized phantom representation with unified spatiotemporal embedding.

[0013] Step S7: Divide the standardized motif representations into image motif groups, traditional Chinese medicine four diagnostic methods motif groups, and clinical indicator motif groups according to the data source; using the Token-Group Transformer architecture, perform intra-group self-attention calculation on each motif group in turn to capture intra-group coordination patterns, and perform cross-attention calculation between different modal groups to model multimodal dependencies and output the final motif representation with long temporal dependencies.

[0014] Step S8: The trained anomaly detection model is used to predict the reconstructed adjacency matrix of the final motif representation. The difference between the reconstructed adjacency matrix of the final motif representation and the true adjacency matrix of the normal motif feature distribution is calculated as the anomaly score. The normal motif feature distribution is determined by the final motif representation set obtained from naturally co-occurring normal motifs in historical clinical data after multi-dimensional motif encoding, feature fusion, and attention calculation. The mean value is calculated based on this set. Covariance By comparing abnormal scores with preset clinical judgment thresholds, the system outputs classification prediction results on whether the endometrial region corresponding to the phantom is a lesion region, serving as auxiliary diagnostic information.

[0015] Specifically, the abnormal score is compared with a preset clinical judgment threshold. If the abnormal score is greater than the clinical judgment threshold, the endometrial area corresponding to the current phantom is determined to be a lesion area and marked as a predicted lesion. Otherwise, it is a normal area and marked as a predicted normal area, thus generating auxiliary diagnostic and treatment information output.

[0016] Furthermore, in step S1, the dynamic relationship edge The establishment of satisfies any of the following conditions:

[0017] (a) The cosine similarity of the image features of the hysteroscopic image regions corresponding to the two image nodes exceeds a preset threshold. ;

[0018] (b) The TCM diagnostic information of the corresponding regions of the two image nodes belong to the same TCM pathological pattern after being mapped by predefined rules;

[0019] (c) The clinical indicator values ​​of the corresponding regions of the two image nodes are in the same preset risk range or the values ​​are closer than the preset threshold.

[0020] Each dynamic relation edge is assigned an initial weight when it is established. Initial weights The calculation formula is:

[0021] ,

[0022] in, The image feature similarity between two image nodes. The strength of the TCM four diagnostic methods information of two image nodes belonging to the same TCM pathological pattern. The normalized proximity of clinical indicators between two image nodes. and Image nodes and image nodes The clinical indicator feature values ​​for the corresponding region are obtained by normalizing the clinical indicator data for the corresponding region. and These are the maximum and minimum values ​​of the clinical indicator features of each image node involved in the construction of the current dynamic graph.

[0023] Furthermore, the implementation of step S5 includes the following steps:

[0024] S5.1, based on phantom The spatiotemporal co-occurrence frequency, temporal decay function, and location encoding of neighboring image nodes n are used to generate a neighbor relationship code representing the local interaction pattern. ;

[0025] S5.2, sampling the propagation paths between motifs, integrating the weights of traditional Chinese medicine meridian theory with historical path frequencies, to generate propagation path codes that characterize the spread trend of lesions. ;

[0026] S5.3 utilizes recurrent neural networks to analyze multimodal temporal data of image nodes within a phantom, and generates evolutionary synchronization codes representing internal co-evolution through correlation measurement. ;

[0027] S5.4 Detects abrupt changes in phantom features, calculates activation intensity based on abrupt change intervals, and generates standardized event burst codes by combining neighborhood information. .

[0028] Furthermore, neighbor relationship coding The calculation includes the following steps:

[0029] Computational phantom Co-occurrence frequency with external neighbor image node n at time t ,in As an indicator function, when the neighboring image node n exists in the image node neighborhood Returns 1 if the condition is met, otherwise returns 0.

[0030] Computational phantom Time of last interaction with external neighbor image node n decay function ,in, Represented by natural constant An exponential function with base 0. motif Image nodes with external neighbors The time of the most recent interaction This represents the decay rate, used to control how quickly the interaction between phantoms decays as the time interval increases;

[0031] To characterize the relative positional relationship between the phantom and its neighborhood in the spatiotemporal dimension, the model introduces sine-cosine position encoding, thereby enhancing the modeling capability for the spatiotemporal dynamic evolution of the phantom, specifically:

[0032] , ,

[0033] in, For time step The corresponding positional encoding vector, and These represent the positions in the encoded vector at that location. peacekeeping The components of the dimension, Indexed by feature dimensions, For the embedded dimension;

[0034] Obtain the initial feature vector of the external neighbor image node n at time t. Enhanced neighbor feature representation combining time decay and spatiotemporal location encoding The enhanced features are used to better represent the relationship between the motif and its neighborhood.

[0035] The neighbor relationship code is obtained by weighted summation. The calculation formula is as follows:

[0036] ;

[0037] in, For the model The neighborhood set.

[0038] Furthermore, propagation path coding The calculation includes the following steps:

[0039] Based on dynamic graph Using a graph neural network to sample from the phantom To other phantoms Path set The sampling probability is positively correlated with the historical frequency of path P; wherein, the path length l is limited to not exceeding a preset threshold. The sampling probability is , For historical frequencies, Temperature factor controls the sharpness of sampling. Indicates a proportional relationship;

[0040] The path is obtained through weighted calculation. The similarity of TCM features to the weight of TCM meridians ,in The edge weight; This is a cosine similarity calculation function used to measure the similarity between two image nodes along a path. and Corresponding characteristics of the four diagnostic methods in Traditional Chinese Medicine and The strength of the correlation between them;

[0041] The path is obtained by merging the weights of all paths. Interaction intensity , ,in, Representing a path The number of image nodes or edges involved in the calculation is used to normalize the length of the path interaction strength. and Representing the phantom and The phantom feature representation obtained after the aforementioned feature fusion and dynamic phantom attention network encoding It is a multilayer perceptron; through weighted aggregation phantoms To other phantoms Information from all sampling paths is used to obtain the propagation path code. The calculation formula is:

[0042] ,

[0043] in, For path Sampling weights, measuring path Its importance in path propagation; To the phantom To other phantoms The set of paths Representing a path The feature representation of the starting phantom or starting image node; Representing a path The most recent point in time when a propagation interaction occurred in the historical dynamic graph sequence. Representing a path The time decay function; Represents a set of paths Any candidate path participating in the normalization calculation; Representing a path Frequency of occurrence within a historical dynamic graph sequence or a preset time window.

[0044] Furthermore, evolutionary synchronization coding The calculation includes the following steps:

[0045] Extraction phantom Internal image node time series ;

[0046] Initialize LSTM hidden state Long Short-Term Memory (LSTM) networks are used to process the time series of internal image nodes to obtain the hidden state at each time step t. ; , The hidden state at time step t-1;

[0047] The mean Pearson correlation coefficient between the hidden states at all time steps and the average hidden state is calculated as a measure of intramotif synchronization. ; ,in The Pearson correlation coefficient is... In the mean-hidden state, For time steps, For the model At time step The hidden state;

[0048] Synchronization measurement Hidden state at the current time step Input multilayer perceptron generates evolutionary synchronization code .

[0049] Furthermore, event outbreak coding The calculation includes the following steps:

[0050] Computational phantom The magnitude of the evolution of features at the current time step compared to features at the previous time step , ,in, For the model Features at time step t For the model Features at time step t-1; when the clinical significance threshold is exceeded. If the change in the region's characteristics exceeds the threshold and the time interval between the change and the last significant evolution is recorded, it is considered a significant evolution; that is, the change in the region's characteristics exceeds the threshold and the time interval between the last significant evolution and the threshold is recorded. A shorter duration indicates acute progression in the region;

[0051] Calculate the current treatment time Compared with the most recent significant evolution time point time interval , , For the current time, The most recent significant evolution point, motif At historical junctures The magnitude of the evolution is used to determine whether a significant change has occurred at that historical time point; and the activation intensity of acute progression is calculated using the Sigmoid function. ; ,in The steepness parameter controls the degree of influence of the time interval on the intensity of acute progression.

[0052] In addition to changes in the phantom itself, it is also necessary to consider the co-occurrence evolution information of the phantom's neighborhood region to comprehensively assess the risk of acute progression in that region. By combining the co-occurrence frequency and enhancement features of image nodes in the phantom's neighborhood, a weighted calculation is performed to obtain the event burst code for the corresponding region. , ;in, Indicates time step lower body The set of neighboring image nodes; Indicates time step Next Neighbor Image Nodes Relative to the phantom The enhanced neighbor feature representation is obtained by fusing the initial features of the neighboring image nodes, the time decay function, and the spatiotemporal location encoding; The number of co-occurrences indicates the number of times a neighboring image node co-occurs with a motif. The strength of the association.

[0053] To ensure that the intensity of an emergency is within a reasonable range, emergency coding is performed. By performing time-based normalization, standardized event burst codes are obtained. , ,in, motif The maximum value of the event burst encoding within all time steps or a preset time window. Through normalization, the burst event intensity value will be limited to a standard range, enabling accurate comparison and evaluation of burst changes in the phantom.

[0054] Further, step S6 includes the following steps:

[0055] Preliminary phantom representation The calculation formula is:

[0056] ,

[0057] in, This represents a vector concatenation operation; This is a learnable fusion weight matrix used to weight and fuse multiple features into a unified representation; Encode neighbor relationships; Encoding the propagation path; For evolutionary synchronization coding; Encode the sudden occurrence of the event;

[0058] Enhancing the spatial perception and clinical interpretability of phantom representations, and improving the initial phantom representation... Perform layer normalization processing to output the final normalized motif representation. The calculation formula is:

[0059] ,

[0060] in, For the embedding of partition coordinate positions, the same sine-cosine function calculation method as the temporal position encoding is used, but its input is the spatial coordinates of the model. Instead of time step This achieves a unified representation of spatiotemporal location coding; For layer normalization, Spatial coordinates of the model .

[0061] Further, step S7 includes the following steps:

[0062] The standardized motif representations are grouped into image motif groups according to their source modality. Traditional Chinese Medicine Four Diagnostic Methods Text Group and clinical indicator group , For image motif representation, This is a text template representation of the four diagnostic methods in Traditional Chinese Medicine. The clinical indicator model is represented; queries for each group are generated separately. ,key Value vector ; , , ,in respectively query ,key Value vector The weight, For head dimension, Indicates the first Modal group The matrix representing the in-group motif.

[0063] Calculate the self-attention within each motif group to capture the coordination patterns of the motif sequences within the group: ; For the self-attention head dimension;

[0064] Calculate the cross-attention between the query vector of the current motif group and the key-value vectors of other motif groups to model the dependencies between multimodal groups:

[0065] ,

[0066] in, and These are the keys and values ​​from other modalities, respectively.

[0067] By stacking multiple layers of Token-Group Transformer modules with a feedforward neural network, and employing residual connections and layer normalization for deep feature extraction, complex spatiotemporal dependencies are captured; Layer output The calculation formula is obtained by combining the core computational residual connections of multi-head attention and feedforward networks with layer normalization.

[0068] ,

[0069] in, For the first Layer output; This is a multi-head attention module used to fuse information from different representation subspaces. The computation process is as follows: , For the number of heads, To output the projection matrix, For the self-attention head dimension; This represents the feature dimension of the motif representation, which is also the unified hidden dimension output by the multi-head attention module; This is a feedforward network module used to introduce nonlinear transformations. The calculation process is as follows: , For activation function, For learnable weights and biases; For layer normalization.

[0070] Further, step S8 includes the following steps:

[0071] Abnormal scores For the final motif representation The Mahalanobis distance from the characteristic distribution of the normal phantom is calculated as follows:

[0072] ,

[0073] in The mean and covariance of the normal motif distribution;

[0074] mold abnormal scores Compared with the preset judgment threshold Compare and thus output the model This information serves as supplementary diagnostic information regarding whether the corresponding endometrial regions are abnormal.

[0075] The anomaly detection model is trained and optimized using a hybrid loss function, L, which is a weighted sum of reconstruction loss and anomaly ranking loss.

[0076] ;

[0077] in, Let Frobenius norm reconstruction loss be the adjacency matrix A. Reconstructing the adjacency matrix of the model Weights for learnable attention decoding; and Representing the phantom and phantom Final motif representation; outlier scores The ranking loss, of which, This represents the index of the normal-abnormal sample pairs that participate in the ranking loss calculation. Indicates the first The anomalous score of each anomalous motif sample. λ represents the abnormality score of the normal motif sample paired with it; λ is the balance coefficient.

[0078] The present invention adopts the above technical solution and has the following beneficial effects compared with the prior art:

[0079] (1) By combining hysteroscopic images with information from the four diagnostic methods of traditional Chinese medicine and clinical indicator data, a dynamic graph sequence reflecting the characteristics of endometrial lesions is constructed, which effectively solves the problem of strong heterogeneity of medical data and enhances the adaptability of the model in different clinical environments. (2) The introduction of "dynamic phantoms" as meso-level analysis units can identify complex pathological patterns such as multi-regional collaborative lesions and synchronous proliferation. By encoding the phantoms in multiple dimensions, a comprehensive analysis of the group behavior and individual abnormalities of the lesion area is realized, which enhances the sensitivity of identifying atypical lesions. (3) The graph self-supervised learning method is introduced. Through dynamic phantom modeling and anomaly detection mechanism, the model can automatically learn the lesion feature representation from unlabeled medical data, which significantly reduces the dependence on a large amount of labeled data and improves the practicality of the model in data-scarce scenarios. (4) By constructing an end-to-end anomaly detection framework, automatic detection and risk assessment of endometrial lesions are realized, providing clinicians with reliable auxiliary diagnostic and treatment information data, which helps to improve the efficiency of diagnosis and treatment. Attached Figure Description

[0080] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments;

[0081] Figure 1 This is a flowchart illustrating the information processing method for traditional Chinese medicine-assisted diagnosis and treatment of endometrial lesions according to the present invention.

[0082] Figure 2 This is a schematic diagram of the system architecture of an information processing method for traditional Chinese medicine-assisted diagnosis and treatment of endometrial lesions according to the present invention. Detailed Implementation

[0083] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0084] With the continuous development of hysteroscopic imaging technology, its application in the clinical diagnosis of endometrial lesions is becoming increasingly widespread. Current diagnostic methods still have room for improvement in handling the integration of multi-source heterogeneous data and modeling the dynamic process of lesions. In particular, the differences in data structure and feature representation between hysteroscopic image features, information from the four diagnostic methods of traditional Chinese medicine, and clinical quantitative indicators pose technical challenges to achieving effective collaborative analysis of multimodal data.

[0085] like Figure 1As shown in Figure 2, this invention discloses an information processing method for TCM-assisted diagnosis and treatment of endometrial lesions. First, time-series acquired hysteroscopic examination data is constructed into a dynamic graph sequence, where the set of image nodes corresponds to the anatomical regions of the uterus determined by semantic segmentation, and the set of edges is dynamically generated based on the correlation relationships of multimodal data. The features of each image node are jointly constructed by fusing visual features extracted from a convolutional neural network, TCM four diagnostic information vectors generated based on a pre-trained language model, and standardized clinical indicator data. The core of this architecture is the phantom analysis stage, which defines a mesoscale dynamic phantom as the analysis unit. Each phantom consists of a set of image nodes with spatiotemporal correlation, used to represent multi-regional collaborative lesion patterns. Feature encoding is generated through four independent computation modules: neighbor relationship encoding based on spatiotemporal co-occurrence statistics and location encoding; propagation path encoding fusing TCM meridian theory and historical propagation frequency; internal evolution encoding using recurrent neural networks to analyze temporal dependencies; and burst event encoding combining feature mutation detection and a time decay model. In the feature integration stage, a learnable parameter matrix is ​​used to weight and fuse the above multi-dimensional encodings to generate a preliminary phantom representation. Subsequently, by superimposing layer normalization operations with spatial coordinate position embedding, the spatial awareness capability of the motif representation is enhanced, and a standardized motif representation is output. In the anomaly detection stage, the standardized motif representations are modally grouped according to their data source, and a Token-Group Transformer architecture is used to calculate intra-group self-attention and inter-group cross-attention respectively. Finally, a motif anomaly score is calculated based on a distribution distance metric algorithm, and by comparing it with a threshold determined through clinical experience, auxiliary diagnostic information is generated at the clinical output layer.

[0086] This invention constructs a spatiotemporal framework for analyzing endometrial lesions by employing dynamic graph sequence modeling and multimodal feature fusion, providing a new analytical approach for clinical diagnosis and treatment decisions. The modules work collaboratively to achieve an end-to-end processing flow from multimodal data input to auxiliary diagnostic information output. This implementation fully presents a complete technical solution from data preprocessing to anomaly detection, with each step closely integrated to form a systematic solution. The specific implementation steps of this invention include:

[0087] Step S1: Model the dynamic endometrial map into a dynamic map sequence. ,in This represents the set of fixed uterine region image nodes obtained from hysteroscopic image segmentation. Indicates time The dynamic relationship edges between the image nodes are based on hysteroscopic image data, information from the four diagnostic methods of traditional Chinese medicine, and clinical indicator data.

[0088] Step S2, Define the model This refers to a mesoscale substructure in a dynamic endometrial graph. The mesoscale substructure is composed of several spatiotemporally related image nodes and their dynamic relationship edges, and its scale lies between that of a single image node and a complete dynamic graph. Indicates the first There are 100 phantoms. Each phantom... Composed of small-scale evolutionary subgraphs, used to capture coordinated patterns of lesions such as multi-regional synchronous proliferation, image nodes. Fusion of image features extracted by CNN ( The image is a hysteroscopic image at time t. (representing image nodes in the lesion area) and BERT-embedded information from the four diagnostic methods of traditional Chinese medicine. and normalized clinical phenotype data Initial image node features , Image nodes representing lesion areas At the point of time The characteristics of time.

[0089] Step S3: Sample normal and abnormal phantoms using the dynamic endometrial image. (Normal phantom) Extracting naturally occurring co-occurring triplets from historical clinical data Abnormal motif Generate by injecting cancerous noise , ( (Using texture perturbation vectors to simulate mutations in lesion regions caused by cancer) to form a binary tag set. ;

[0090] Step S4: Calculate the average features of the phantom. As the initial embedding Indicates the number of image nodes within the phantom; synchronously extracts the phantom neighborhood set. ,in Represents image nodes In time The neighborhood region at that time is used to capture information about image nodes and their surroundings;

[0091] Step S5: Calculate the co-occurrence frequency of the phantom and its surrounding area, and combine temporal decay and spatial location to generate a neighbor relationship code representing the local interaction pattern. .

[0092] Step S6: Sample the propagation paths between phantoms, integrate the weights of TCM meridians with historical frequencies, and generate propagation path codes that characterize the spread trend of lesions. .

[0093] Step S7: Analyze the image and evidence time-series data of the image nodes inside the phantom, and generate evolutionary synchronization codes representing the internal co-evolution using recurrent neural networks and correlation metrics. .

[0094] Step S8: Detect abrupt changes in phantom features, calculate activation intensity based on abrupt change time intervals, and generate standardized burst event detection codes by combining neighborhood information. .

[0095] Step S9 involves concatenating and linearly fusing the four types of encoding: neighborhood, path, internal synchronization, and burst, to generate a preliminary motif representation. This representation is then enhanced by layer normalization and spatial location embedding to obtain a standardized motif representation.

[0096] Step S10: The standardized multimodal motif sequence is grouped, and the Token-Group Transformer is used to perform intra-group self-attention and inter-group cross-attention calculations in sequence to fuse intra-modal features and capture inter-modal correlations, and output the final motif representation with long temporal dependencies.

[0097] Step S11: Based on the final phantom representation, calculate the reconstruction loss and Mahalanobis distance anomaly score, optimize the model through a hybrid loss function, compare the anomaly score with the clinical threshold, and output the diagnostic conclusion of whether or not a lesion exists.

[0098] Furthermore, the specific steps of step S1 are as follows:

[0099] Step S1-1: For any two image nodes, the cosine similarity of the hysteroscopic image features between the image nodes exceeds a preset threshold if and only if the cosine similarity exceeds a preset threshold. When the TCM diagnostic information of the corresponding region of an image node is determined to belong to the same TCM pathological pattern by a predefined mapping rule, or when the clinical indicators of the corresponding region of an image node are close in value or even in the same preset risk range, an untyped connection edge is established between the two image nodes.

[0100] Step S1-2: Assign an initial weight to each established connection edge. This weight is calculated by fusing multimodal correlation strength, and its calculation formula is as follows:

[0101] ,

[0102] in The image feature similarity between two image nodes. The strength of the TCM four diagnostic methods information of two image nodes belonging to the same TCM pathological pattern. Normalized proximity of clinical indicators between two image nodes;

[0103] Furthermore, the specific steps of step S5 are as follows:

[0104] Step S5-1: By fusing image features extracted by a convolutional neural network (CNN), TCM diagnostic information based on BERT embedding, and normalized clinical indicator data, an initial feature vector of the external neighbor image nodes at time t is constructed. ;

[0105] Step S5-2, Calculate the phantom with external neighbors In time The co-occurrence frequency at time. The formula for calculating the co-occurrence frequency is:

[0106] ,

[0107] in For indicator functions, when Existing in image nodes neighborhood Returns 1 if the condition is met, otherwise returns 0.

[0108] Step S5-3, Calculate the phantom Its neighboring regions Last interaction time Through the time decay function The relationship between the simulated phantom and its neighborhood decays over time, specifically as follows:

[0109] ,

[0110] in, Indicates the decay rate, controlling the decay speed of interactions between phantoms;

[0111] Step S5-4: To characterize the relative positional relationship between the phantom and its neighborhood in the spatiotemporal dimension, the model introduces sine-cosine positional encoding, thereby enhancing the modeling capability for the spatiotemporal dynamic evolution of the phantom. Specifically:

[0112] , ,

[0113] in, For dimensional indexing, For the embedded dimension;

[0114] Step S5-5: By combining temporal decay and location encoding with neighbor features, an enhanced neighbor feature representation is generated.

[0115] ,

[0116] in, It is an image node In time The initial features are used to enhance the features to better represent the relationship between the motif and its neighborhood;

[0117] Steps S5-6: Encoding neighbor relationships To capture the interaction information between the motif and its neighborhood:

[0118] ,

[0119] This step helps the system understand the collective interaction patterns between the phantom and its neighboring regions, and is used for subsequent anomaly detection.

[0120] Furthermore, the specific steps of step S6 are as follows:

[0121] Step S6-1, based on the dynamic graph defined in claim 1 Using GNN sampling path , where the path From the phantom To the phantom Path length The sampling probability of the path is ,in For historical frequencies, Temperature factor controls the sharpness of sampling. Indicates proportional to;

[0122] Step S6-2: Obtain the path through weighted calculation. The weight of meridians in traditional Chinese medicine ,in For edge weights, This is a cosine similarity calculation function used to measure path edges. The two image nodes Characteristics of the four diagnostic methods in Traditional Chinese Medicine The strength of the correlation between them;

[0123] Step S6-3 calculates the path by merging the weights of all paths. The interaction strength is expressed by the following formula:

[0124] ,

[0125] in It is a multilayer perceptron, used for phantom features. and Calculations were performed to obtain the interaction strength between the phantoms;

[0126] Step S6-4, Phantom In time The path interaction encoding at that time aggregates the propagation path information from this motif to other motifs in the graph. The calculation formula is:

[0127] ,

[0128] in , for path Sampling weights, measuring path Its importance in path propagation From The set of sampling paths.

[0129] Furthermore, the specific steps of step S7 are as follows:

[0130] Step S7-1: Extract the image node sequence inside the phantom. ;

[0131] Step S7-2, Initialize the LSTM hidden state And update the hidden state through the LSTM network. :

[0132] ,

[0133] The LSTM gate functions include:

[0134] Forgotten Gate ,

[0135] Input gate ,

[0136] Output gate ,

[0137] Candidate values ,

[0138] Cell state ,

[0139] Output , For element-wise multiplication, for , and These are the forget gate, input gate, output gate, and the learnable weight matrix and bias term corresponding to the candidate state;

[0140] Step S7-3 uses the Pearson correlation coefficient to quantify the synchronicity of temporal evolution within the phantom. ,in The Pearson correlation coefficient is... In the mean-hidden state, For time steps;

[0141] Step S7-4: Fuse the synchronization information with the hidden state to generate a phantom. The final evolutionary synchronization encoding representation:

[0142] .

[0143] Furthermore, the specific steps of step S8 are as follows:

[0144] Step S8-1: First, calculate the current treatment time. With the phantom The time point at which the corresponding region last underwent a significant change The time interval between ,in For the range of evolution of phantom features, This is the threshold for clinically significant evolution. If the change in the region's characteristics exceeds the threshold and the time interval since the last significant evolution... A shorter duration indicates acute progression in the region;

[0145] Step S8-2: Calculate the activation intensity of acute progression. ,in The steepness parameter controls the degree of influence of the time interval on the intensity of acute progression.

[0146] Step S8-3, in addition to the changes in the phantom itself, also requires combining the co-occurrence evolution information of the phantom's neighborhood region to comprehensively assess the risk of acute progression in that region. The specific formula is as follows:

[0147] ,

[0148] Step S8-4: To ensure that the intensity of the sudden event is within a reasonable range, the calculated intensity of the sudden event is normalized. Through normalization, the value of the sudden event intensity will be limited to a standard range, which can accurately compare and evaluate the sudden changes of the phantom.

[0149] Furthermore, the specific steps of step S9 are as follows:

[0150] Step S9-1, the encoding , , , Vector concatenation is performed to generate a preliminary fusion modality representation. The calculation formula is as follows:

[0151] ,

[0152] in The learnable fusion weight matrix is ​​used to weight and fuse features from multiple modalities into a unified representation.

[0153] Step S9-2, to enhance the spatial awareness and clinical interpretability of the phantom representation, the fused phantom representation is... Perform layer normalization processing to output the final normalized motif representation. The calculation formula is as follows:

[0154] ,

[0155] Furthermore, the specific steps of step S10 are as follows:

[0156] Step S10-1: Input the phantom sequence Based on their data sources, they are divided into three Token-Groups: Image Phantom Group, ... Traditional Chinese Medicine Four Diagnostic Methods Text Group and clinical indicator group ;

[0157] Step S10-2, for each mode group Generate its query, key, and value vectors respectively:

[0158] , , ,

[0159] in , For head dimension;

[0160] Step S10-3: Execute a self-attention mechanism within each modality group to capture the coordination patterns of the motif sequences within the group. ;

[0161] Step S10-4: Perform a cross-attention mechanism between different modal groups to model the dependencies between multimodal groups. ,

[0162] in, Queries representing the current modality and These are the keys and values ​​from other modalities, respectively;

[0163] Step S10-5, by stacking The Layer Token-Group Transformer module performs deep feature processing to capture complex spatiotemporal dependencies. Each layer... Output The core computational residual connections, which include multi-head attention and feedforward networks, are obtained after layer normalization. The calculation formula is as follows:

[0164] ,

[0165] Among them, multi-head attention module The calculation process for fusing information from different representation subspaces is as follows: ,in For the number of heads, For output projection matrix; feedforward network module It is used to introduce nonlinear transformations, and its calculation process is as follows: ,in These are learnable weights and biases.

[0166] Furthermore, the specific steps of step S11 are as follows:

[0167] Step S11-1, the anomaly detection module adopts a hybrid optimization objective function, which is composed of a weighted sum of reconstruction loss and anomaly ranking loss, and its expression is:

[0168] ,

[0169] in, Adjacency matrix Frobenius norm reconstruction loss, For model reconstruction output, It is sigmoid; outlier scores To mitigate ranking loss, use contrastive learning to rank normal / abnormal pairs. This is the balance coefficient;

[0170] Step S11-2, based on the final schema representation output by the Token-Group Transformer module. Calculate the Mahalanobis distance between the motif and the normal motif characteristic distribution, and use this distance as the anomaly score of the motif:

[0171] ,

[0172] in The mean and covariance of the normal phantom distribution are given by the phantom. abnormal scores Compared with the preset judgment threshold The comparison is performed to output auxiliary diagnostic information on whether the corresponding endometrial region of the phantom is diseased.

[0173] This invention employs the above technical solution, combining hysteroscopic image data with information from the four diagnostic methods of Traditional Chinese Medicine and clinical indicator data to construct a dynamic image sequence reflecting the characteristics of endometrial lesions. Based on this, the invention uses self-supervised learning to identify abnormal phantoms in the generated dynamic image sequence. The differences between phantom features and normal patterns are revealed by calculating reconstruction loss and distribution distance. This invention constructs a comprehensive abnormal feature representation by generating feature codes in four dimensions: neighborhood relationships, lesion propagation, internal evolution, and sudden events. Based on this, a unified abnormality score is calculated using Mahalanobis distance to achieve a comprehensive evaluation of the lesion phantom. This method effectively transforms the classification problem into an abnormality score comparison process, facilitating rapid identification of potential lesion areas and providing auxiliary reference information for subsequent clinical diagnosis, thus offering a new technical approach for the diagnosis of endometrial lesions.

[0174] Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

Claims

1. An information processing method for the auxiliary diagnosis and treatment of endometrial lesions using traditional Chinese medicine, characterized in that, Includes the following steps: Step S1: Acquire time-series hysteroscopic image data, TCM four diagnostic methods information, and clinical indicator data to construct a dynamic image sequence of the endometrium. Where V represents the set of fixed uterine region image nodes obtained by segmenting hysteroscopic images. This represents the dynamic relationship edges between image nodes at time t; Step S2: Obtain the mesoscale substructures in the dynamic graph sequence as a motif. This is used to capture multi-regional collaborative lesion patterns. The mesoscale substructure consists of several spatiotemporally related image nodes and corresponding dynamic relationship edges, and its scale is between that of a single image node and a complete dynamic graph. k is the number of phantoms; where the i-th phantom is... Image features of image node x of the lesion area extracted from hysteroscopic image data at time point t. Traditional Chinese Medicine diagnostic information for image node x Clinical indicator data features after normalization of image node x Perform fusion to generate a phantom The initial feature representation of any image node x at time point t ; Step S3: Sample naturally co-occurring image node triples from historical clinical data as normal phantoms. Abnormal phantoms are generated by injecting pre-defined cancerous noise into the image node features of normal phantoms. This forms a binary training label set. ; Step S4, calculate the phantom The average features were extracted from the phantom. The neighborhood set; Step S5, for each phantom Generate multidimensional motif codes, which include neighbor relationship codes representing local interaction patterns, propagation path codes representing lesion spread trends, evolutionary synchronization codes representing internal co-evolution, and standardized event burst codes. Step S6: The multi-dimensional phantom encoding is concatenated into vectors and linearly projected and fused using a learnable fusion weight matrix to obtain a preliminary phantom representation; the preliminary fused representation is then subjected to layer normalization and superimposed with sine-cosine position embedding enhancement based on phantom spatial coordinates to output a standardized phantom representation with unified spatiotemporal embedding. Step S7: Divide the standardized motif representations into image motif groups, traditional Chinese medicine four diagnostic methods motif groups, and clinical indicator motif groups according to the data source; using the Token-Group Transformer architecture, perform intra-group self-attention calculation on each motif group in turn to capture intra-group coordination patterns, and perform cross-attention calculation between different modal groups to model multimodal dependencies and output the final motif representation with long temporal dependencies. Step S8: The trained anomaly detection model is used to predict the reconstructed adjacency matrix of the final motif representation. The difference between the reconstructed adjacency matrix of the final motif representation and the true adjacency matrix of the normal motif feature distribution is calculated as the anomaly score. The normal motif feature distribution is determined by the final motif representation set obtained from naturally co-occurring normal motifs in historical clinical data after multi-dimensional motif encoding, feature fusion, and attention calculation. The mean value is calculated based on the final motif representation set. Covariance By comparing abnormal scores with preset clinical judgment thresholds, the system outputs classification prediction results on whether the endometrial region corresponding to the phantom is a lesion region, serving as auxiliary diagnostic information.

2. The information processing method for traditional Chinese medicine-assisted diagnosis and treatment of endometrial lesions according to claim 1, characterized in that, In step S1, dynamic relation edges The establishment of satisfies any of the following conditions: (a) The cosine similarity of the image features of the hysteroscopic image regions corresponding to the two image nodes exceeds a preset threshold. ; (b) The TCM diagnostic information of the corresponding regions of the two image nodes belong to the same TCM pathological pattern after being mapped by predefined rules; (c) The clinical indicator values ​​of the corresponding regions of the two image nodes are in the same preset risk range or the values ​​are closer than the preset threshold. Each dynamic relation edge is assigned an initial weight when it is established. Initial weights The calculation formula is: ， In the formula The image feature similarity between two image nodes. The strength of the TCM four diagnostic methods information of two image nodes belonging to the same TCM pathological pattern. The normalized proximity of clinical indicators between two image nodes. and Image nodes and image nodes The clinical indicator characteristic values ​​for the corresponding region are obtained by normalizing the clinical indicator data for the corresponding region. and These represent the maximum and minimum values ​​of the clinical indicator features of each image node involved in the construction of the current dynamic graph.

3. The information processing method for traditional Chinese medicine-assisted diagnosis and treatment of endometrial lesions according to claim 1, characterized in that, Step S5 includes the following steps: S5.1, based on phantom The spatiotemporal co-occurrence frequency, temporal decay function, and location encoding of neighboring image nodes n are used to generate a neighbor relationship code representing the local interaction pattern. ; S5.2, sampling the propagation paths between motifs, integrating the weights of traditional Chinese medicine meridian theory with historical path frequencies, to generate propagation path codes that characterize the spread trend of lesions. ; S5.3 utilizes recurrent neural networks to analyze multimodal temporal data of image nodes within a phantom, and generates evolutionary synchronization codes representing internal co-evolution through correlation measurement. ; S5.4 Detects abrupt changes in phantom features, calculates activation intensity based on abrupt change intervals, and generates standardized event burst codes by combining neighborhood information. .

4. The information processing method for traditional Chinese medicine-assisted diagnosis and treatment of endometrial lesions according to claim 3, characterized in that, Neighbor Relationship Coding The calculation includes the following steps: Computational phantom Co-occurrence frequency with external neighbor image node n at time t ,in As an indicator function, when the neighboring image node n exists in the image node neighborhood Returns 1 if the condition is met, otherwise returns 0. Computational phantom Time of last interaction with external neighbor image node n decay function ,in, Represented by natural constant An exponential function with base 0. motif Image nodes with external neighbors The time of the most recent interaction This represents the decay rate, used to control how quickly the interaction between phantoms decays as the time interval increases; The model introduces sine-cosine position encoding to enhance its ability to model the spatiotemporal dynamic evolution of the phantom, specifically: ， ， in, For time steps The corresponding positional encoding vector, and These represent the positions in the encoding vector. peacekeeping The components of the dimension, Indexed by feature dimensions, For the embedded dimension; Obtain the initial feature vector of the external neighbor image node n at time t. Enhanced neighbor feature representation combining time decay and spatiotemporal location encoding ; The neighbor relationship code is obtained by weighted summation. The calculation formula is as follows: ； in, For the model The neighborhood set.

5. The information processing method for traditional Chinese medicine-assisted diagnosis and treatment of endometrial lesions according to claim 3, characterized in that, Propagation path coding The calculation includes the following steps: Based on dynamic graph Using a graph neural network to sample from the phantom To other phantoms Path set The sampling probability is positively correlated with the historical frequency of path P; wherein, the path length is limited to not exceeding a preset threshold. The sampling probability is , For historical frequencies, Temperature factor controls the sharpness of sampling. Indicates a proportional relationship; The path is obtained through weighted calculation. The similarity of TCM features to the weight of TCM meridians ,in The edge weight; This is a cosine similarity calculation function used to measure the similarity between two image nodes along a path. and Corresponding characteristics of the four diagnostic methods in Traditional Chinese Medicine and The strength of the correlation between them; The path is obtained by merging the weights of all paths. Interaction intensity , ,in, Representing a path The number of image nodes or edges involved in the calculation is used to normalize the length of the path interaction strength. and Representing the phantom and The phantom feature representation obtained after feature fusion and dynamic phantom attention network encoding. It is a multilayer perceptron; Through weighted aggregation modulus To other phantoms Information from all sampling paths is used to obtain the propagation path code. The calculation formula is: ， in, For path Sampling weights, measuring path Its importance in path propagation; To the phantom To other phantoms The set of paths Representing a path The characteristic representation of the initial motif; Representing a path The most recent point in time when a propagation interaction occurred in the historical dynamic graph sequence. Representing a path The time decay function; Represents a set of paths Any candidate path participating in the normalization calculation; Representing a path Frequency of occurrence within a historical dynamic graph sequence or a preset time window.

6. The information processing method for traditional Chinese medicine-assisted diagnosis and treatment of endometrial lesions according to claim 3, characterized in that, Evolutionary Synchronization Coding The calculation includes the following steps: Extraction phantom Internal image node time series; The hidden state at each time step t is obtained by processing the time series of internal image nodes using a Long Short-Term Memory (LSTM) network. ; The mean Pearson correlation coefficient between the hidden states at all time steps and the average hidden state is calculated as a measure of intramotif synchronization. ; Synchronization measurement Hidden state at the current time step Input multilayer perceptron generates evolutionary synchronization code .

7. The information processing method for traditional Chinese medicine-assisted diagnosis and treatment of endometrial lesions according to claim 3, characterized in that, Incident Encounter Coding The calculation includes the following steps: Computational phantom The magnitude of the evolution of features at the current time step compared to features at the previous time step When the clinical significance threshold is exceeded If , then it is recorded as a significant evolution; Calculate the time interval between the current time and the most recent significant evolution. The activation intensity of acute progression was calculated using the Sigmoid function. ; By combining the co-occurrence frequency and enhancement features of the phantom's neighborhood image nodes, a weighted calculation is performed to obtain the event burst code for the corresponding region. ; Encoding of sudden events By performing time-based normalization, standardized event burst codes are obtained. .

8. The information processing method for traditional Chinese medicine-assisted diagnosis and treatment of endometrial lesions according to claim 1, characterized in that, Step S6 includes the following steps: Preliminary phantom representation The calculation formula is: ， in, This represents a vector concatenation operation; It is a learnable fusion weight matrix used to weight and fuse multiple features into a unified representation; Encode neighbor relationships; Encoding the propagation path; For evolutionary synchronization coding; Encode the sudden occurrence of the event; Preliminary phantom representation Perform layer normalization processing to output the final normalized motif representation. The calculation formula is: ， in, Embedding for partition coordinate positions, For layer normalization, Spatial coordinates of the model .

9. The information processing method for traditional Chinese medicine-assisted diagnosis and treatment of endometrial lesions according to claim 1 or 8, characterized in that, Step S7 includes the following steps: The standardized phantom representations are grouped by source modality into image phantom group, traditional Chinese medicine four diagnostic methods phantom group, and clinical indicator phantom group, and queries are generated for each group respectively. ,key Value vector ; Calculate the self-attention within each motif group. ; For the self-attention head dimension; Calculate the cross-attention between the query vector of the current motif group and the key-value vectors of other motif groups. , in, and These are the keys and values ​​from other modalities, respectively. Deep feature extraction is achieved by stacking multiple layers of Token-Group Transformer modules with a feedforward neural network, and employing residual connections and layer normalization; Layer output The calculation formula is obtained by combining the core computational residual connections of multi-head attention and feedforward networks with layer normalization: ， in, For the first Layer output; This is a multi-head attention module used to fuse information from different representation subspaces. The computation process is as follows: , For the number of heads, To output the projection matrix, For the self-attention head dimension; The feature dimension of the motif representation is the unified hidden dimension output by the multi-head attention module. This is a feedforward network module used to introduce nonlinear transformations. The calculation process is as follows: , For activation function, For learnable weights and biases; For layer normalization.

10. The information processing method for traditional Chinese medicine-assisted diagnosis and treatment of endometrial lesions according to claim 8, characterized in that, Step S8 includes the following steps: Abnormal scores For the final motif representation The Mahalanobis distance from the characteristic distribution of the normal phantom is calculated as follows: ， in The mean and covariance of the normal motif distribution; mold abnormal scores Compared with the preset judgment threshold Compare and thus output the model This information serves as supplementary diagnostic information regarding whether the corresponding endometrial regions are abnormal. The anomaly detection model is trained and optimized using a hybrid loss function, L, which is a weighted sum of the reconstruction loss and the anomaly ranking loss. ； in, Let Frobenius norm reconstruction loss be the adjacency matrix A. To reconstruct the adjacency matrix of the model, For learnable attention decoding weights; and Representing the phantom and phantom The final motif representation; outlier scores The ranking loss This represents the index of the normal-abnormal sample pairs that participate in the ranking loss calculation. Indicates the first The anomalous score of each anomalous motif sample. Indicates the relationship with the first The anomalous score of a normal motif paired with an anomalous motif; λ is the balance coefficient.

Images