Identity recommendation method based on multi-view association fusion, medium and equipment

Abstract

The invention discloses a syndrome element recommendation method based on multi-view association fusion, a medium and equipment. The method comprises the following steps: extracting potential embedded representation from original text data by using a pre-trained convolutional neural network; respectively processing the potential embedding representation by adopting three modes of full-connection composition, random edge composition and Euclidean distance composition to obtain three types of graph structure data; respectively processing the three kinds of graph structure data by using a graph attention neural network to obtain potential feature representation corresponding to each view; mapping the potential feature representations of different views to the same feature space through a multi-layer perceptron mapping function sharing the weight, and ensuring that the potential feature representation of each view does not deviate from the center of the view by adopting norm constraint; and gradually fusing the potential feature representations of the multiple views through a splicing operation and a full-connection neural network layer, and outputting discriminative representations for syndrome element classification. Through multi-view collaborative learning and a dynamic fusion mechanism, the accuracy and robustness of traditional Chinese medicine syndrome element recommendation are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of smart healthcare, specifically to a method, medium, and device for recommending evidence elements based on multi-view association fusion. Background Technology

[0002] With the rapid development of deep learning technology, intelligent TCM recommendation systems have demonstrated significant value in assisting clinical decision-making. However, existing TCM syndrome element recommendation methods still face significant technical bottlenecks when processing complex and heterogeneous medical data, mainly in the following aspects:

[0003] (1) Insufficient fusion of multi-view information: Traditional methods usually input symptom data from different sources or types (such as text descriptions, clinical indicators, tongue / pulse features, etc.) into the model after simple feature splicing, while ignoring the potential correlation and complementarity between different views. This coarse-grained fusion method is difficult to effectively explore the deep interaction relationship between multi-view features, resulting in the model's limited reasoning ability for complex symptom combinations.

[0004] (2) Limitations of a single graph composition perspective: Existing technologies are mostly based on a single graph composition strategy (such as fully connected graphs or Euclidean distance graphs), which cannot comprehensively model the complex structure between symptoms from multiple perspectives (such as functional association, spatial distribution, semantic similarity, etc.). For example, the co-occurrence relationship (functional association) and semantic similarity (implicit association in descriptive text) between symptoms may have different graph topological characteristics. A single graph composition method cannot take into account these differences, resulting in low information utilization.

[0005] (3) Cross-view Figure One Lack of consistency: Most methods do not impose consistency constraints on the feature representation of multiple views, which may lead to biases in the features extracted from different views. For example, fully connected graphs may overemphasize global relationships, while Euclidean distance graphs focus more on local similarities. Without an alignment mechanism, the fused global representation may introduce noise, reducing the robustness and interpretability of the model. Summary of the Invention

[0006] In view of the above problems, this application provides a technical solution for syndrome element recommendation based on multi-view association fusion, so as to solve the technical problems of insufficient multi-view information fusion and low information utilization in existing TCM syndrome element recommendation methods, which leads to a decrease in the accuracy of model feature extraction.

[0007] To achieve the above objectives, in a first aspect, this application provides a method for recommending evidence elements based on multi-view association fusion, the method comprising:

[0008] S1: using a pre-trained convolutional neural network to extract a latent embedding representation from original text data, the original text data including symptom text information, the symptom text information being adjusted into a fixed-size two-dimensional matrix input into the convolutional neural network through word vector conversion and zero padding;

[0009] S2: processing the latent embedding representation in a full connection graph, a random edge graph and a Euclidean distance graph Figure Three in a manner respectively to obtain three kinds of graph structure data;

[0010] S3: using an independent graph attention neural network to process the three kinds of graph structure data respectively to obtain latent feature representations corresponding to each view;

[0011] S4: mapping the latent feature representations of different views to the same feature space through a multi-layer perception mapping function with shared weights, and adopting a norm constraint to ensure that the latent feature representations of each view do not deviate from the view center;

[0012] S5: gradually fusing the latent feature representations of multiple views through a splicing operation and a fully connected neural network layer to output a discriminative representation for evidence classification.

[0013] Further, the pre-trained convolutional neural network is a ResNet50 network, and step S1 is calculated by formula (1), which is expressed as follows:

[0014] X = ResNet50(X raw );

[0015] wherein, X raw represents the original text data, and X represents the extracted latent embedding representation.

[0016] Further, the graph structure data includes full connection graph structure data;

[0017] In step S2, the latent embedding representation is processed by a full connection graph to obtain full connection graph structure data, and the calculation manner is shown in formula (2), which is as follows:

[0018] G F = Full-Graph(X);

[0019]

[0020] wherein, G F represents the full connection graph structure data, all positions in the adjacency matrix are 1, and Full-Graph(X) represents a full connection graph function;

[0021] The graph attention neural network includes a first graph attention neural network, and step S3 includes:

[0022] The full connection graph structure data is processed using the first graph attention neural network to obtain a latent feature representation corresponding to the full connection graph, and the calculation formula is as follows:

[0023] Z F =GAT1(G F ,X);

[0024] Wherein, GAT1 represents the first graph attention neural network, X represents the latent embedding representation extracted in step S1, Z F represents the latent feature representation corresponding to the full connection graph.

[0025] Further, the graph structure data includes Euclidean distance graph structure data;

[0026] In step S2, the latent embedding representation is processed using Euclidean distance graphing to obtain Euclidean distance graph structure data, and the calculation method is shown in formula (3), and formula (3) is as follows:

[0027]

[0028] G D =Dist-Graph(X);

[0029] Wherein, a ij represents the value of the corresponding position of the adjacency matrix, G D represents the Euclidean distance graph structure data, Dist-Graph(X) represents the Euclidean distance graph function, ||·|| 2 represents the L2 regularization norm function, ζ represents the scaling constant, ν i , ν j respectively represent the feature of adjacent nodes, k is a self-defined threshold, the value range is (0, 1), OW represents the value of a i -ν j ||≤k case, a ij represents the value of node feature ν ij , ν i in the corresponding position of the adjacency matrix; j

[0030] The graph attention neural network includes a second graph attention neural network, and step S3 includes:

[0031] The Euclidean distance graph structure data is processed using the second graph attention neural network to obtain a latent feature representation corresponding to the Euclidean distance graph, and the calculation formula is as follows:

[0032] Z D =GAT2(G D ,X);

[0033] ​GAT2(X), wherein GAT2 represents the second graph attention neural network, X represents the latent embedding representation extracted in step S1, Z D represents the latent feature representation corresponding to the random edge graph.

[0034] Further, the graph structure data comprises random edge graph structure data;

[0035] In step S2, the latent embedding representation is processed by using random edge graph processing to obtain random edge graph structure data, and the calculation manner is shown in formula (4), and formula (4) is as follows:

[0036] G R = Rand-Graph (X) ;

[0037] wherein Rand-Graph (X) represents a random edge graph function, G R represents the random edge graph structure data, and is obtained by the following manner: randomly selecting n nodes, and randomly assigning k nodes except the current node as the first-order neighbor of the current node, and n and k are both self-defined parameters;

[0038] The graph attention neural network comprises a third graph attention neural network, and step S3 comprises:

[0039] The random edge graph structure data is processed by using the third graph attention neural network to obtain the latent feature representation corresponding to the random edge graph, and the calculation formula is as follows:

[0040] Z R = GAT3 (G R , X) ;

[0041] wherein GAT3 represents the third graph attention neural network, X represents the latent embedding representation extracted in step S1, Z R represents the latent feature representation corresponding to the random edge graph.

[0042] Further, the calculation formula of step S4 is as follows:

[0043] Z Rm = MAP (Z R ) ;

[0044] Z Dm = MAP (Z D ) ;

[0045] Z Fm = MAP (Z F ) ;

[0046] L re = ||Z Fm -Z Dm ||1+||Z Dm -ZRm ||1;

[0047] wherein, Z R is the latent feature representation obtained by processing the random edge graph structure data based on the graph attention neural network, denoted as the first latent feature representation, Z D is the latent feature representation obtained by processing the Euclidean distance graph structure data based on the graph attention neural network, denoted as the second latent feature representation, Z F is the latent feature representation obtained by processing the full connection graph structure data based on the graph attention neural network, denoted as the third latent feature representation, Z Rm , Z Dm , Z Fm respectively represent the representations of the first latent feature representation, the second latent feature representation, and the third latent feature representation after being mapped to the same feature space, MAP(·) is a multi-layer perceptron mapping function with shared weights, ||·||1 represents the one-norm, and L re is a consistency regularization loss function used to ensure that the latent feature representations of each view do not deviate from the view center.

[0048] Further, in step S5, the latent feature representations of multiple views are gradually fused by concatenation operation and fully connected neural network layer, and the calculation formula is as follows:

[0049] Z1=FC(CONCAT(Z Fm ,Z Dm ));

[0050] Z2=FC(CONCAT(Z Dm ,Z Rm ));

[0051] Z=Z1·α1+Z2·α2;

[0052] wherein, CONCAT(·) represents a concatenation operation function, FC(·) represents a fully connected neural network layer, and α1 and α2 represent learnable fusion coefficients, and α2 represents a learnable fusion coefficient, and Z represents the fused latent feature representation of multiple views.

[0053] Further, before outputting the discriminative representation for evidence classification, the method further comprises:

[0054] The loss function is calculated by the following formula, and the model is optimized by the loss function:

[0055]

[0056] wherein, N represents the number of samples, f(·) represents a fully connected classifier, y represents the true evidence category information, and L cls represents the error sum of classification.

[0057] The total loss function is as follows:

[0058] L final = L cls + L re ;

[0059] wherein L final represents the final loss function.

[0060] In a second aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the syndrome recommendation method based on multi-view association fusion as described in the first aspect of the present application.

[0061] In a third aspect, the present application provides an electronic device having stored thereon a computer program, comprising a processor and a storage medium having stored thereon a computer program which, when executed by the processor, implements the syndrome recommendation method based on multi-view association fusion as described in the first aspect of the present application.

[0062] Distinguished from the prior art, the above technical solution relates to a syndrome recommendation method based on multi-view association fusion, medium and device, which comprises: using a pre-trained convolutional neural network to extract latent embedding representation from original text data; using full connection graph, random edge graph and Euclidean distance graph Figure Three to process the latent embedding representation in three ways, respectively, to obtain three kinds of graph structure data; using an independent graph attention neural network to process the three kinds of graph structure data, respectively, to obtain latent feature representation corresponding to each view; mapping the latent feature representation of different views to the same feature space through a multi-layer perceptron mapping function with shared weights, and using a one-norm constraint to ensure that the latent feature representation of each view does not deviate from the view center; gradually fusing the latent feature representation of multiple views through a splicing operation and a fully connected neural network layer, and outputting discriminative representation for syndrome classification. Through multi-view collaborative learning and dynamic fusion mechanism, the present application significantly improves the accuracy and robustness of TCM syndrome recommendation, and provides more reliable intelligent support for clinical decision-making.

[0063] The above summary of the invention is only a summary of the technical solutions of the present application. In order to enable those skilled in the art to more clearly understand the technical solutions of the present application, and then implement the content recorded in the specification and drawings, and in order to enable the above and other purposes, characteristics and advantages of the present application to be more easily understood, the following will be described in combination with the specific embodiments of the present application and the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0064] The drawings are only used to show the principles, implementation methods, applications, characteristics and effects of the specific embodiments of the present application and other related contents, and cannot be considered as limiting the present application.

[0065] In the drawings:

[0066] Figure 1 Flow chart of the syndrome element recommendation method based on multi-view correlation fusion according to an example embodiment of the present application;

[0067] Figure 2 Flow chart of the syndrome element recommendation method based on multi-view correlation fusion according to an example embodiment of the present application;

[0068] Figure 3 Module schematic diagram of the traditional Chinese medicine syndrome element recommendation system according to an example embodiment of the present application;

[0069] Figure 4 Flow chart of the traditional Chinese medicine syndrome element recommendation method according to an example embodiment of the present application;

[0070] Figure 5 Schematic diagram of the electronic device according to an example embodiment of the present application;

[0071] The reference signs in the above-mentioned drawings are explained as follows:

[0072] 10. Electronic device;

[0073] 101. Processor;

[0074] 102. Storage medium;

[0075] 20. Traditional Chinese medicine syndrome element recommendation system;

[0076] 201. Text preprocessing module;

[0077] 202. Multi-view graph generation module;

[0078] 203. Visualization module. DETAILED DESCRIPTION

[0079] To explain the possible application scenarios, technical principles, specific schemes that can be implemented, purposes and effects that can be achieved of the present application in detail, the specific embodiments listed below are combined with the drawings for detailed description. The embodiments described in this paper are only used to more clearly explain the technical solutions of the present application, and therefore cannot be used to limit the protection scope of the present application.

[0080] The term "embodiment" is mentioned in this document means that the specific features, structures or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The term "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or association with other embodiments. In principle, in this application, as long as there is no technical contradiction or conflict, the technical features mentioned in each embodiment can be combined in any way to form a corresponding implementable technical solution.

[0081] Unless otherwise defined, the meaning of technical terms used in this document is the same as that generally understood by those skilled in the art to which the present application belongs; the use of related terms in this document is only for the purpose of describing specific embodiments, and is not intended to limit the present application.

[0082] In the description of the present application, the phrase "and / or" is a description of the logical relationship between the objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases: A exists, B exists, and A and B exist at the same time. In addition, the character " / " in this document generally represents that the associated objects before and after are a "or" logical relationship.

[0083] In this application, such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantity, primary and secondary or order relationship between them.

[0084] In this application, without more limitation, the "includes", "contains", "has" or other similar open expressions used in the statement are intended to cover non-exclusive inclusion, and these expressions do not exclude the presence of other elements in the process, method or product including the described elements, so that the process, method or product including a series of elements can not only include those limited elements, but also include other elements not explicitly listed, or also include the elements inherent in such process, method or product.

[0085] As the same understanding as in the "Guidelines for Examination", in this application, "greater than", "less than", "exceed" and other expressions are understood as not including the number; "above", "below", "within" and other expressions are understood as including the number. In addition, the meaning of "multiple" in the description of the embodiments of the present application is more than two (including two), and similar expressions related to "multiple" are also understood in this way, for example, "multiple groups", "multiple times" and the like, unless otherwise specifically limited.

[0086] In the description of the embodiments of the present application, the spatially-related expressions used, such as "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "perpendicular", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", and the like, indicate the orientation or positional relationship shown in the specific embodiments or the drawings, and are only for the convenience of describing the specific embodiments of the present application or for the reader to understand, and do not indicate or imply that the indicated device or component must have a particular position, a particular orientation, or be constructed or operated in a particular orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0087] Unless otherwise explicitly specified or limited, in the description of the embodiments of the present application, the terms "mount", "connect", "connect", "fix", "set", and the like should be understood broadly. For example, the "connection" can be fixed connection, or detachable connection, or integral setting; it can be mechanical connection, or electrical connection, or communication connection; it can be direct connection, or indirect connection through intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art to which the present application belongs, the specific meaning of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0088] In the first aspect, as shown in the first aspect, Figure 1 The present application provides a feature recommendation method based on multi-view association fusion, which comprises:

[0089] S1: using a pre-trained convolutional neural network to extract latent embedding representation from original text data, the original text data including symptom text information.

[0090] In step S1, the symptom text information is adjusted to a fixed-size two-dimensional matrix input into the convolutional neural network through word vector conversion and zero padding. Word vector conversion can be achieved by a word vector model, which can be Word2Vec, BERT, etc. The pre-trained convolutional neural network can be ResNet, TextCNN, etc. The two-dimensional matrix is processed by the pre-trained convolutional neural network to output a low-dimensional dense latent embedding representation (such as an N×128 matrix), which captures the semantic features of the symptoms.

[0091] S2: using full connection graph, random edge graph and Euclidean distance graph Figure Three to process the latent embedding representation respectively, to obtain three kinds of graph structure data.

[0092] In step S2, the full connection graph method includes: constructing a global association graph between symptoms, all nodes in the adjacency matrix are connected to each other, and the weight is 1.

[0093] The Euclidean distance configuration mode includes: calculating the Euclidean distance of symptom embedding, retaining only the k nearest neighbor nodes (i.e., the k nodes with the smallest Euclidean distance) of each symptom node, retaining only the Top-k connected edges (non-zero elements) in the adjacency matrix, and setting the remaining edges to zero to form a sparse structure and capture local similarity. The random edge configuration mode includes: on the basis of a global fully connected graph, randomly discarding part of the edges to generate a sparse and randomly structured graph, and introducing randomness to enhance the robustness of the model and prevent overfitting to noise or false associations in the training data.

[0094] S3: Use independent graph attention neural networks to process the three types of graph structure data respectively to obtain latent feature representations corresponding to each view.

[0095] In step S3, three independent graph attention networks (GAT) are used to process the three types of graph structures respectively. Each GAT aggregates neighbor node information through a multi-head attention mechanism and outputs view-specific feature representations. The attention weights can reflect the importance differences between symptoms, such as the contribution of "white and greasy tongue" to "fine and slow pulse".

[0096] S4: Map the latent feature representations of different views to the same feature space through a multi-layer perceptron mapping function with shared weights, and use a norm constraint to ensure that the latent feature representations of each view do not deviate from the view center.

[0097] Through the processing of step S4, the semantic consistency of features of different views is ensured.

[0098] S5: Gradually fuse the latent feature representations of multiple views through concatenation operation and fully connected neural network layers to output discriminative representations for syndrome classification.

[0099] The above scheme uses multi-view collaborative learning and dynamic fusion mechanism to comprehensively improve the integrity of symptom representation by utilizing the complementarity of multiple views. Specifically, it captures global associations through a fully connected graph, focuses on local similarity through an Euclidean distance graph, and enhances robustness through a random edge graph. The random discarding mechanism of the random edge graph and the dynamic k-nearest neighbor selection of the Euclidean distance graph effectively alleviate overfitting. At the same time, the attention weights of GAT show the strength of the association between symptoms, which can highlight the features that need to be focused on, assist in diagnosis and decision-making, solve the problem of single feature and poor robustness in traditional TCM syndrome recommendation methods, and provide an interpretable decision basis, significantly improving the practicality and reliability of the auxiliary diagnosis system.

[0100] In some embodiments, the pre-trained convolutional neural network is a ResNet50 network, and step S1 is calculated by formula (1), which is expressed as follows:

[0101] X = ResNet50(X raw );

[0102] Among them, X raw X represents the original text data, and X represents the extracted latent embedding representation.

[0103] In some embodiments, the graph structure data includes fully connected graph structure data;

[0104] In step S2, the latent embedding representation is processed using a fully connected graph to obtain fully connected graph structure data. The calculation method is shown in formula (2), which is as follows:

[0105] G F =Full-Graph(X);

[0106]

[0107] Among them, G F This represents a fully connected graph structure where all positions in the adjacency matrix are 1.

[0108] The graph attention neural network includes a first graph attention neural network, and step S3 includes:

[0109] The first graph attention neural network is used to process the fully connected graph structure data to obtain the latent feature representations corresponding to the fully connected graphs. The calculation formula is as follows:

[0110] Z F =GAT1(G F ,X);

[0111] Where GAT1 represents the first graph attention neural network, X represents the latent embedding representation extracted in step S1, and Z represents the latent embedding representation extracted in step S1. F This represents the latent feature representation corresponding to the fully connected graph.

[0112] In some embodiments, the graph structure data includes Euclidean distance graph structure data;

[0113] In step S2, the latent embedding representation is processed using Euclidean distance graphing to obtain Euclidean distance graph structure data. The calculation method is shown in formula (3), which is as follows:

[0114]

[0115] G D =Dist-Graph(X);

[0116] Among them, G D Represents Euclidean distance plot structure data, ||·|| 2 Let ζ denote the L2 regularization normal form function, ν denote the scaling constant, and ν denote the scaling constant.i , v j respectively represent adjacent node features, k is a self-defined threshold, and the value range is (0, 1), and OW represents the value of a i -v j except for the case of ||v ij a ij represents the node feature v i , v j corresponding to the position in the adjacency matrix;

[0117] The graph attention neural network includes a second graph attention neural network, and step S3 includes:

[0118] The Euclidean distance graph structure data is processed using the second graph attention neural network to obtain a latent feature representation corresponding to the Euclidean distance graph, and the calculation formula is as follows:

[0119] Z D =GAT2(G D , X);

[0120] Wherein, GAT2 represents the second graph attention neural network, X represents the latent embedding representation extracted in step S1, and Z D represents the latent feature representation corresponding to the Euclidean distance graph.

[0121] In some embodiments, the graph structure data includes random edge graph structure data;

[0122] In step S2, the latent embedding representation is processed using a random edge graph to obtain random edge graph structure data, and the calculation method is shown in formula (4), and formula (4) is as follows:

[0123] G R = Rand-Graph (X) ;

[0124] Wherein, G R represents random edge graph structure data, and is obtained by the following method: randomly selecting n nodes, and randomly assigning k nodes except the current node as the first-order neighbor of the current node, and n and k are both self-defined parameters;

[0125] The graph attention neural network includes a third graph attention neural network, and step S3 includes:

[0126] The random edge graph structure data is processed using the third graph attention neural network to obtain a latent feature representation corresponding to the random edge graph, and the calculation formula is as follows:

[0127] Z R =GAT3(G R , X);

[0128] wherein GAT3 represents the third graph attention neural network, X represents the latent embedding representation extracted in step S1, Z R represents the latent feature representation corresponding to the random edge graph.

[0129] In some embodiments, step S4 is calculated as follows:

[0130] Z Rm = MAP(Z R );

[0131] Z Dm = MAP(Z D );

[0132] Z Fm = MAP(Z F );

[0133] L re = ||Z Fm - Z Dm ||1+ ||Z Dm - Z Rm ||1;

[0134] wherein Z R is a latent feature representation obtained by processing the random edge graph structure data based on the graph attention neural network, denoted as the first latent feature representation, Z D represents a latent feature representation obtained by processing the Euclidean distance graph structure data based on the graph attention neural network, denoted as the second latent feature representation, Z F represents a latent feature representation obtained by processing the fully connected graph structure data based on the graph attention neural network, denoted as the third latent feature representation, Z Rm , Z Dm , Z Fm respectively represent the first latent feature representation, the second latent feature representation, and the third latent feature representation after being mapped to the same feature space, MAP(·) is a multi-layer perceptron mapping function with shared weights, ||·||1 represents the one-norm, L re represents a consistency regularization loss function, which is used to ensure that the latent feature representations of each view do not deviate from the view center.

[0135] In some embodiments, in step S5, the latent feature representations of multiple views are gradually fused by a concatenation operation and a fully connected neural network layer, and the calculation formula is as follows:

[0136] Z1= FC(CONCAT(Z Fm , Z Dm ));

[0137] Z2= FC(CONCAT(Z Dm , Z Rm ));

[0138] Z = Z1 · a1 + Z2 · a2;

[0139] wherein, CONCAT(·) represents a concatenation operation function, FC(·) represents a fully connected neural network layer, a1 and a2 represent learnable fusion coefficients, a2 represents a learnable fusion coefficient, and Z represents a fused multi-view latent feature representation.

[0140] In some embodiments, before outputting the discriminative representation for syndrome classification, the method further comprises:

[0141] The loss function is calculated by the following formula, and the model is optimized by the loss function:

[0142]

[0143] wherein, N represents the number of samples, f(·) represents a fully connected classifier, y represents the true syndrome class information, L cls represents the error sum of classification;

[0144] The total loss function is as follows:

[0145] L final = L cls + L re ;

[0146] wherein, L final represents the final loss function.

[0147] As Figure 2 shown, the model involved in the present application is processed by the following steps: extracting symptom information features, constructing multi-view graph relationships, extracting view latent feature representations, establishing view Figure One consistency constraints and dynamically fusing multi-view features, and then judging whether the model is optimized based on the loss function, if not, performing the above steps until the optimization is completed, and then obtaining the syndrome item suitable for the current symptom information through the recommendation system.

[0148] The method model training process involved in the present application is divided into two stages of view information extraction and view information fusion. In the view information extraction, the feature extraction network corresponding to different views encodes the view features, and obtains respective latent embedding representations at the last network layer, and then the latent embedding representations are fused to make the model obtain a more discriminative syndrome classification representation, which effectively assists medical diagnosis.

[0149] The method has the following beneficial effects:

[0150] First, by modeling different types of symptom data using various mapping methods, we can fully explore the local correlation characteristics within the data from multiple perspectives, effectively alleviating the problem of insufficient information in single-perspective modeling.

[0151] Secondly, using multi-view graph neural networks for joint modeling can achieve deep fusion of heterogeneous symptom information, improving the integrity and robustness of feature representation.

[0152] Third, the cross-view feature interaction and consistency alignment mechanism further strengthens the collaborative expression between features of different views, and improves the discriminative and generalization capabilities of fused features.

[0153] Finally, the system performs refined syndrome element reasoning based on the fused global features, which can more accurately capture the complex inference relationships between TCM syndrome elements, thereby significantly improving the intelligence level and clinical application value of the TCM auxiliary decision-making system.

[0154] like Figure 3 As shown, in a second aspect, this application also provides a traditional Chinese medicine syndrome element recommendation system 20, comprising:

[0155] The text preprocessing module 201 is used to convert symptom text information into a two-dimensional matrix input format that can be processed by a pre-trained convolutional neural network;

[0156] The multi-view graph generation module 202 is used to process the converted symptom text information according to the method of S1-85 of the first aspect of this application, and output a discriminative representation for syndrome element classification.

[0157] The visualization module 203 is used to visualize the heatmap of the probability distribution of evidence elements and the interpretability analysis results of the mapping process based on the discriminative representation of evidence element classification output.

[0158] like Figure 4 As shown, in a third aspect, this application also provides a method for recommending TCM syndrome elements, applicable to the system described in the second aspect of this application. This method includes the following steps:

[0159] Step S401: Convert the symptom text information into a two-dimensional matrix input format that can be processed by a pre-trained convolutional neural network;

[0160] Step S402: Process the converted symptom text information according to the method of S1-85 of the first aspect of this application, and output a discriminative representation for syndrome element classification.

[0161] Step S403: The discriminative representation of the evidence element classification is visualized, and the results of the interpretability analysis of the evidence element probability distribution heatmap and the mapping process are presented.

[0162] In step S403, the key symptoms can be located using the Grad-CAM technology, a heat map is generated by calculating the gradient response of the syndrome element classification layer to the CNN feature map, for example, the contribution of "red tongue" and "rapid pulse" to "liver stagnation syndrome" is highlighted in red, indicating a high contribution, the attention scores between symptoms can also be displayed in suspension, and a structured conclusion can be automatically generated.

[0163] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the syndrome recommendation method based on multi-view association fusion according to the first aspect of the present application.

[0164] The computer readable storage medium can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories.

[0165] The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD ROM).

[0166] The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synclink dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The computer-readable storage media described in the embodiments of the present invention are intended to include these and any other suitable types of memory.

[0167] like Figure 5 As shown, in a fifth aspect, the present invention provides an electronic device 10, including a processor 101 and a storage medium 102, wherein a computer program is stored on the storage medium, and the computer program, when executed by the processor, implements the evidence recommendation method based on multi-view association fusion as described in the first aspect of the present invention.

[0168] In some embodiments, the processor can be implemented by software, hardware, firmware or a combination thereof, and can use at least one of circuit, single or multiple Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), controller, microcontroller, microprocessor, so that the processor can execute part or all of the steps or any combination of steps in the evidence recommendation method based on multi-view association fusion in various embodiments of the present application.

[0169] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of the present application, the patent protection scope of the present application should not be limited. Any technical solutions obtained by replacing or modifying the equivalent structure or equivalent process based on the essential concept of the present application, using the content described in the specification and drawings of the present application, and directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are all included in the patent protection scope of the present application.

Claims

1. A method for evidence recommendation based on multi-view association fusion, characterized in that, The method comprises: S1: using a pre-trained convolutional neural network to extract a latent embedding representation from original text data, the original text data comprising symptom text information, the symptom text information being adjusted into a fixed-size two-dimensional matrix input into the convolutional neural network through word vector conversion and zero padding; S2: processing the latent embedding representation in three ways of full connection mapping, random edge mapping and Euclidean distance mapping to obtain three kinds of graph structure data; S3: using an independent graph attention neural network to process the three kinds of graph structure data to obtain latent feature representations corresponding to each view; S4: mapping the latent feature representations of different views to the same feature space through a multi-layer perceptron mapping function with shared weights, and using a one-norm constraint to ensure that the latent feature representations of each view do not deviate from the view center; S5: gradually fusing the latent feature representations of multiple views through a concatenation operation and a fully connected neural network layer to output a discriminative representation for syndrome classification. 2.The method of claim 1, wherein, The pre-trained convolutional neural network is a ResNet50 network, and step S1 is calculated by formula (1), which is expressed as follows: X = ResNet50(X raw ); where X raw represents the original textual data, X represents the extracted latent embedding representation. 3.The method of claim 1, wherein, The graph structure data comprises full connection graph structure data; In step S2, the latent embedding representation is processed by full connection mapping to obtain full connection graph structure data, and the calculation method is shown in formula (2), which is as follows: G F = Full-Graph(X); wherein G F represents a full connection graph structure data, all positions in the adjacency matrix are 1, and Full-Graph(X) represents a full connection graph function; The graph attention neural network comprises a first graph attention neural network, and step S3 comprises: The full connection graph structure data is processed by the first graph attention neural network to obtain the latent feature representation corresponding to the full connection mapping, and the calculation formula is as follows: Z F = GAT1(G F , X); wherein GAT1 represents a first graph attention neural network, X represents the latent embedding representation extracted in step S1, Z F represents a latent feature representation corresponding to the fully connected composition. 4.The method of claim 1, wherein, The graph structure data comprises Euclidean distance graph structure data; In step S2, the latent embedding representation is processed by Euclidean distance mapping to obtain Euclidean distance graph structure data, and the calculation method is shown in formula (3), which is as follows: G D = Dist-Graph(X); wherein a ij represents the value of the corresponding position of the adjacency matrix, G D represents the Euclidean distance graph structure data, Dist-Graph(X) represents the Euclidean distance graph function, ||·|| 2 represents the L2 regularization norm function, ζ represents the scaling constant, ν i , ν j respectively represent the feature of the adjacent node, k is a self-defined threshold value, the value range is (0, 1), and OW represents the value of a i -ν j except that a ij is the value of a ij under the condition that ||ν i , ν j is the value of the corresponding position in the adjacency matrix; The graph attention neural network comprises a second graph attention neural network, and step S3 comprises: The Euclidean distance graph structure data is processed by the second graph attention neural network to obtain the latent feature representation corresponding to the Euclidean distance mapping, and the calculation formula is as follows: Z D = GAT2(G D , X); wherein GAT2 denotes a second graph attention neural network, X denotes the latent embedding representation extracted in step S1, Z D denotes the latent feature representation corresponding to the Euclidean distance configuration. 5.The method of claim 1, wherein, The graph structure data comprises random edge graph structure data; In step S2, the latent embedding representation is processed by random edge mapping to obtain random edge graph structure data, and the calculation method is shown in formula (4), which is as follows: G R = Rand-Graph(X); Wherein, Rand-Graph (X) represents a random edge graph function, G R represents random edge graph structure data, and is obtained by the following manner: randomly selecting n nodes, and randomly allocating k nodes other than the current node as the first-order neighbor of the current node, wherein n and k are both self-defined parameters. The graph attention neural network comprises a third graph attention neural network, and step S3 comprises: The random edge graph structure data is processed by the third graph attention neural network to obtain the latent feature representation corresponding to the random edge mapping, and the calculation formula is as follows: Z R = GAT3(G R , X); wherein GAT3 represents a third graph attention neural network, X represents the latent embedding representation extracted in step S1, Z R represents the latent feature representation corresponding to the random edge configuration. 6.The method of claim 1, wherein, The calculation formula of step S4 is as follows: Z Rm = MAP(Z R ); Z Dm = MAP(Z D ); Z Fm = MAP(Z F ); L re =||Z Fm -Z Dm ||1+||Z Dm -Z Rm ||1; wherein Z R Z denotes the latent feature representation obtained by processing the random edge graph structure data based on the graph attention neural network, and is referred to as a first latent feature representation D Z denotes the latent feature representation obtained by processing the Euclidean distance graph structure data based on the graph attention neural network, and is referred to as a second latent feature representation F Z denotes the latent feature representation obtained by processing the fully connected graph structure data based on the graph attention neural network, and is referred to as a third latent feature representation Rm , Z Dm , Z Fm , Z re denote the representations of the first latent feature representation, the second latent feature representation and the third latent feature representation after being mapped to the same feature space, respectively, MAP(·) is a multi-layer perceptron mapping function with shared weights, ||·||1 denotes one-norm, and L re denotes a consistency regularization loss function, which is used to ensure that the latent feature representations of the views do not deviate from the view centers.

7. The method of claim 6, wherein the method further comprises: In step S5, the latent feature representations of multiple views are gradually fused through a concatenation operation and a fully connected neural network layer, and the calculation formula is as follows: Z1 = FC(CONCAT(Z Fm , Z Dm )); Z2 = FC(CONCAT(Z Dm , Z Rm )); Z=Z1·α1+Z2·α2; Wherein, CONCAT(·) represents a concatenation operation function, FC(·) represents a fully connected neural network layer, α1 and α2 represent learnable fusion coefficients, and α2 represents a learnable fusion coefficient, and Z represents the fused latent feature representation of multiple views. 8.The method of claim 7, wherein, Before outputting the discriminative representation for syndrome classification, the method further comprises: The loss function is calculated by the following formula, and the model is optimized by the loss function: where N denotes the number of samples, f (·) denotes the fully connected classifier, y denotes the real evidence class information, L cls denotes the error and of the classification; The total loss function is as follows: L final = L cls + L re ; where L final represents the final loss function.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the evidence recommendation method based on multi-view association fusion as claimed in any one of claims 1 to 8.

10. An electronic device on which a computer program is stored, characterized by The computer program is stored on the storage medium and executed by the processor to realize the evidence recommendation method based on multi-view association fusion as claimed in any one of claims 1 to 8.

Images