Computer-implemented method, apparatus and computer program product

Abstract

A computer-implemented method is provided. The computer-implemented method includes constructing a feature hypergraph, performing feature selection and combination, and performing predictive analysis. Performing feature selection and combination includes updating a representation of a hyperedge in the feature hypergraph by aggregating information from vertices connected to the hyperedge and selected by a gate function, and updating a representation of a vertex in the feature hypergraph by aggregating information from hyperedges to which the vertex belongs and selected by a gate function.

Description

Technical Field

[0001] This invention relates to predictive analytics technology, and more specifically, to a computer-implemented method, apparatus, and computer program product. Background Technology

[0002] Predictive analytics is a fundamental task in the field of artificial intelligence. It involves using given features as input to infer predictive targets (such as regression values ​​or classification labels). Predictive analytics plays a crucial role in medical research, for example, predicting the efficacy and side effects of drugs for diseases, assessing the impact of a treatment plan on patients, and estimating patient survival time. Summary of the Invention

[0003] In one aspect, this disclosure provides a computer-implemented method comprising: constructing a feature hypergraph; performing feature selection and combination; and performing predictive analysis; wherein performing feature selection and combination comprises: updating the representation of the hyperedge in the feature hypergraph by aggregating information from vertices connected to the hyperedge and selected by a gate function; and updating the representation of the vertex in the feature hypergraph by aggregating information from the hyperedge to which the vertex belongs and selected by a gate function.

[0004] Optionally, updating the representation of the hyperedge in the feature hypergraph includes performing information aggregation according to the following formula:

[0005]

[0006] Among them, W (l) f and W (l) x These are model parameters; Represents the hyperedge x i The vector in layer l; N x (x i ) indicates connection to the hyperedge x i A set of vertices; σ represents the activation function; and δ represents the gate function.

[0007] Optionally, updating the representation of the vertices in the feature hypergraph includes performing information aggregation according to the following formula:

[0008]

[0009] Among them, W (l) f fj represents the model parameters; (l) Represents vertex f j The vector in layer l; N f (fj) represents the vertex f j The set of hyperedges to which it belongs; σ represents the activation function; and δ represents the gate function.

[0010] Optionally, the computer-implemented method further includes repeating the following operations: updating the representation of the hyperedge in the feature hypergraph and updating the representation of the vertex in the feature hypergraph, thereby enabling the combination of features through successive layers.

[0011] Optionally, constructing the feature hypergraph includes: providing a plurality of instances and a plurality of features; defining each of the plurality of features as a vertex in the hypergraph; and defining each of the plurality of instances as a hyperedge in the hypergraph, the hyperedge connecting at least two vertices in the respective instances.

[0012] Optionally, constructing the feature hypergraph includes: providing a plurality of instances and a plurality of features; representing the plurality of features as a plurality of vectors, each of the plurality of vectors having a corresponding value across the plurality of instances; calculating the similarity between the plurality of vectors; identifying the nearest neighbor of each of the plurality of features based on the similarity; and forming each hyperedge by grouping the respective features with the nearest neighbor.

[0013] Optionally, a Gaussian kernel function can be used to calculate the similarity between feature vectors:

[0014]

[0015] Among them, f k f j The nearest neighbor of a feature.

[0016] Alternatively, a cosine similarity function can be used to calculate the similarity between feature vectors:

[0017]

[0018] Among them, f k f j The nearest neighbor of a feature.

[0019] Optionally, performing predictive analysis includes: obtaining updated representations of the hyperedges and the vertices in the feature hypergraph; passing the final layer representation of the hyperedge through a first fully connected layer and then through an activation function; and passing the result through a second fully connected layer to produce a prediction result, wherein the result comes from passing the final layer representation of the hyperedge through the first fully connected layer and then through the activation function.

[0020] Optionally, the prediction result is represented by the following formula:

[0021] Layer; σ represents the ReLU activation function; and Represents the hyperedge x iThe vector in layer l.

[0022] Optionally, the computer-implemented method further includes optimizing model parameters by minimizing a loss function:

[0023] loss = ∑ i ||y' i -y i || 2 ;

[0024] Among them, y' i This represents the prediction result for the instance, and y i This represents the target result of the instance.

[0025] Optionally, the computer-implemented method further includes preprocessing patient data by at least one of the following operations: extracting medical codes from the patient's clinical records; or obtaining information about differentially expressed biomarkers from the patient's omics data.

[0026] Optionally, the patient is the corresponding hyperedge among a plurality of hyperedges in the feature hypergraph; and the plurality of features include the medical code and the markers of the differential expression.

[0027] Optionally, the feature hypergraph can be represented as G = (V, E); where G represents the feature hypergraph; V represents the plurality of vertices; and E represents the plurality of hyperedges; wherein a neural network is used to learn the feature representation; and the representation of the vertices in the feature hypergraph is expressed as:

[0028]

[0029] in, f represents the representation vector of vertex v in layer l; E→V This represents the message passing function from the hyperedge to the vertex; and This represents the representation of the hyperedge containing vertex v.

[0030] Optionally, the representation of hyperedges in the feature hypergraph is expressed as:

[0031]

[0032] in, f represents the representation vector of hyperedge e in layer l; V→E This represents the message passing function from a vertex to a hyperedge; and This represents the vertices contained in the hyperedge e.

[0033] Optionally, performing predictive analysis includes obtaining the final representation vector after L iterations. Make the final representation vector The result is passed through a first multilayer perceptron, then through an activation function; and then through a second multilayer perceptron to obtain a prediction result, wherein the result comes from the final representation vector. It passes through the first multilayer perceptron, and then through the activation function.

[0034] Optionally, the computer-implemented method further includes: mapping text data to a first vector representation; mapping image data to a second vector representation; and fusing the first vector representation and the second vector representation to form a combined vector representation.

[0035] Optionally, the computer-implemented method further includes: using a training dataset D = {d1, ...,d...} N Train the model, where d i Let n represent the nth patient, the features associated with the nth patient, the survival time of the nth patient, and the maximum tumor diameter of the nth patient; n and N are positive integers, 1≤n≤N; minimize the loss function to learn the parameters of the model, wherein the loss function is expressed as: in, This represents the survival time of the nth patient. y represents the maximum tumor diameter of the nth patient. 1,i y represents the predicted survival time of the nth patient. 2,i The predicted maximum tumor diameter of the nth patient is represented by β, and the harmonic parameter is represented by β. The second image is generated by inputting the prediction result and the first image into the model. The second image is generated based on the prediction result from the model and the first image.

[0036] In another aspect, this disclosure provides an apparatus comprising: a memory; and one or more processors; wherein the memory and the one or more processors are interconnected; and the memory stores computer-executable instructions for controlling the one or more processors to perform the following operations: constructing a feature hypergraph; performing feature selection and combination; and performing predictive analysis; wherein, for performing feature selection and combination, the memory stores computer-executable instructions for controlling the one or more processors to perform the following operations: updating the representation of the hyperedge in the feature hypergraph by aggregating information from vertices connected to the hyperedge and selected by a gate function; and updating the representation of the vertex in the feature hypergraph by aggregating information from the hyperedges to which the vertex belongs and selected by a gate function.

[0037] In another aspect, this disclosure provides a computer program product comprising a non-transitory tangible computer-readable medium having computer-readable instructions thereon, the computer-readable instructions being executable by a processor to cause the processor to perform: constructing a feature hypergraph; performing feature selection and combination; and performing predictive analysis; wherein performing feature selection and combination includes: updating the representation of the hyperedge in the feature hypergraph by aggregating information from vertices connected to the hyperedge and selected by a gate function; and updating the representation of the vertex in the feature hypergraph by aggregating information from the hyperedges to which the vertex belongs and selected by a gate function. Attached Figure Description

[0038] The following figures are merely illustrative examples based on various disclosed embodiments and are not intended to limit the scope of the invention.

[0039] Figure 1 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure.

[0040] Figure 2 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure.

[0041] Figure 3 A hypergraph constructed using methods according to some embodiments of this disclosure is shown.

[0042] Figure 4 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure.

[0043] Figure 5 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure.

[0044] Figure 6 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure.

[0045] Figure 7 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure.

[0046] Figure 8 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure.

[0047] Figure 9 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure.

[0048] Figure 10 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure. Detailed Implementation

[0049] This disclosure will now be described in more detail with reference to the following embodiments. It should be noted that the following description of some embodiments presented herein is for illustrative and descriptive purposes only. It is not exhaustive or limited to the precise forms disclosed.

[0050] In predictive analytics, a large number of discrete features are frequently used. For example, when predicting patient survival, factors such as the type of gene mutation, the presence of specific methylation, family history, and smoking habits are often considered. These features are mostly sparse, and a single feature alone may not provide effective predictive information. Instead, they need to be combined with other features to provide valuable information. Therefore, when using these features for predictive analytics, it is essential to consider their interactions.

[0051] In predictive analytics, feature combination is an effective way to improve accuracy. However, when the number of features is large, feature combinations face the problem of combinatorial explosion. For example, if there are N features, then (2... N -1) latent feature combinations. Related feature combination methods have limitations. They only consider low-order feature interactions (e.g., Factorization Machines only consider single features and combinations of two features), or they use deep learning methods to represent N features as N vectors, then concatenate these vectors and use a multilayer perceptron for feature combination. These methods have certain drawbacks. The first method only considers low-order feature combinations, ignoring higher-order combinations. The second method implicitly combines all features without considering their relevance to the final task, which may introduce noise.

[0052] Therefore, this disclosure particularly provides a computer-implemented method, apparatus, and computer program product that substantially eliminates one or more problems caused by the limitations and disadvantages of related technologies. In one aspect, this disclosure provides a predictive analytics method. In some embodiments, the computer-implemented method includes constructing a feature hypergraph; performing feature selection and combination; and performing predictive analytics. Optionally, performing feature selection and combination includes: updating the representation of hyperedges in the feature hypergraph by aggregating information from vertices connected to hyperedges and selected by a gate function; and updating the representation of vertices in the feature hypergraph by aggregating information from hyperedges to which vertices belong and selected by a gate function.

[0053] Figure 1 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure. (Reference) Figure 1 In some embodiments, the predictive analytics method includes: constructing a feature hypergraph; performing feature selection and combination; and performing predictive analytics.

[0054] Figure 2 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure. (Reference) Figure 2 In some embodiments, constructing a feature hypergraph includes: providing a plurality of instances and a plurality of features; defining each feature among the plurality of features as a vertex in the hypergraph; and defining each instance among the plurality of instances as a hyperedge in the hypergraph, the hyperedge connecting at least two vertices in each instance. Optionally, at least two vertices in each instance have non-zero values.

[0055] In a specific example, constructing a feature hypermap involves providing a dataset {(x1,y1),…,(x... M ,y M )}, where x i = <f1,…,f N >(i=1,…,M); x i This represents the i-th instance (e.g., the i-th patient); f j This indicates that the corresponding x i The j-th feature of y; i It corresponds to x i The prediction target is defined as follows: M, N, i, j are positive integers; 1 ≤ i ≤ M; and 1 ≤ j ≤ N. In another specific example, the instance can be a patient; features can be, for example, smoking status or tumor grade; and the prediction target can be the patient's survival time. This disclosure defines features f1,…,f N Consider them as vertices of a hypergraph; let x1,…,x M Consider it as a hyperedge. Specifically, hyperedge x i include <f1,…,f N Features with non-zero values ​​in >. Table 1 shows an exemplary dataset that includes multiple instances and multiple features.

[0056] Table 1: The exemplary dataset includes multiple instances and multiple features.

[0057] <![CDATA[f1]]> <![CDATA[f2]]> <![CDATA[f3]]> <![CDATA[f4]]> <![CDATA[f5]]> <![CDATA[x1]]> 1 0 1 0 0 <![CDATA[x2]]> 1 1 0 0 0 <![CDATA[x3]]> 0 0 1 1 1

[0058] As shown in Table 1, x1, x2, and x3 are three instances; f1, f2, f3, f4, and f5 are five features. For instance x1, features f1 and f3 have non-zero values, while features f2, f4, and f5 have zero values. For instance x2, features f1 and f2 have non-zero values, while features f3, f4, and f5 have zero values. For instance x3, features f3, f4, and f5 have non-zero values, while features f1 and f2 have zero values.

[0059] Figure 3 A hypergraph constructed using methods according to some embodiments of this disclosure is shown. References Figure 3The vertices of the hypergraph are features f1, f2, f3, f4, and f5; the hyperedges are instances x1, x2, and x3. Each hyperedge includes features that have non-zero values ​​in the corresponding instance. For example, hyperedge x1 includes vertices f1 and f3; hyperedge x2 includes vertices f1 and f2; and hyperedge x3 includes vertices f3, f4, and f5.

[0060] Figure 4 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure. (Reference) Figure 4 In an alternative embodiment, constructing a feature hypergraph includes: providing a plurality of instances and a plurality of features; representing the plurality of features as a plurality of vectors, each of the plurality of vectors having a corresponding value across the plurality of instances; calculating the similarity between the plurality of vectors; identifying the nearest neighbors of each of the plurality of features based on the similarity; and forming each hyperedge by grouping each feature with its nearest neighbor.

[0061] In a specific example, constructing a feature hypermap involves providing a dataset {(x1,y1),…,(x... M ,y M )}, where x i = <f1,…,f N >(i=1,…,M); x i This represents the i-th instance (e.g., the i-th patient); f j This indicates that the corresponding x i The j-th feature of y; i It corresponds to x i The target result; M, N, i, j are positive integers; 1≤i≤M; and 1≤j≤N. Constructing a feature hypergraph also includes representing multiple features as multiple vectors, where each vector f is a vector within the multiple vectors. j It has corresponding values ​​across multiple instances. For example, f1 is represented as <1,1,0>; f2 is represented as <0,1,0>, etc. Constructing a feature hypergraph also includes using a similarity function (such as a Gaussian kernel, cosine similarity, or a neural network-based function) to calculate the similarity between feature vectors. Constructing a feature hypergraph also includes, for each feature f j The K nearest neighbors are identified based on the calculated similarity. Constructing the feature hypergraph also includes using each feature f... j Grouping with K nearest neighbors to form a superedge e j .

[0062] In one example, the Gaussian kernel function is used to calculate the similarity between feature vectors: Where f k f j The nearest neighbor of a feature.

[0063] In another example, the cosine similarity function is used to calculate the similarity between feature vectors: Where f k f j The nearest neighbor of a feature.

[0064] Figure 5 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure. (Reference) Figure 5 In some embodiments, performing feature selection and combination includes: updating the representation of a hyperedge in the feature hypergraph by aggregating information from vertices connected to the hyperedge; and updating the representation of a vertex in the feature hypergraph by aggregating information from the hyperedge to which the vertex belongs.

[0065] In some embodiments, updating the representation of hyperedges in the feature hypergraph includes performing aggregation of information from vertices connected to the hyperedges according to formula (1):

[0066]

[0067] Among them, W (l) f and W (l) x These are model parameters; Represents the hyperedge x i The vector in layer l; N x (x i ) indicates connection to the hyperedge x i A set of vertices; and σ represents the ReLU activation function.

[0068] In some embodiments, updating the representation of vertices in the feature hypergraph includes performing aggregation information from the hyperedges to which the vertex belongs, according to formula (2):

[0069]

[0070] Among them, W (l) f fj represents the model parameters; (l) Represents vertex f j The vector in layer l; N f (fj) represents vertex f j The set of hyperedges to which it belongs; and σ represents the ReLU activation function.

[0071] In some embodiments, performing feature selection and combination includes repeatedly updating the representations of hyperedges in the feature hypergraph and updating the representations of vertices in the feature hypergraph, thereby enabling the combination of features through successive layers.

[0072] In one example, refer to Figure 3The method involves updating the representation of hyperedge x1 by aggregating information from vertices f1 and f3 connected to hyperedge x1. The updated vector representation of x1 now includes information from f3. Referring to formula (1), vertex f1 then indirectly obtains information from f2 and f3 by aggregating information from x1 and x2. At this point, the representation of f1 contains combined information from f1, f2, and f3.

[0073] The inventors of this disclosure have discovered that the methods of formulas (1) and (2) above can achieve feature combination through multiple iterations; however, it cannot distinguish between effective and ineffective feature combinations, which may introduce a lot of noise. Manually determining which features to combine requires domain knowledge and a lot of resources. The inventors of this disclosure have discovered an automatic feature selection method that automatically selects and combines features beneficial to the target task and determines which hyperedges and vertices to aggregate during processing.

[0074] In some embodiments, updating the representation of hyperedges in the feature hypergraph includes performing aggregation of information from vertices connected to the hyperedges according to formula (3):

[0075]

[0076] Among them, W (l) f and W (l) x These are model parameters; Represents the hyperedge x i The vector in layer l; N x (x i ) indicates connection to the hyperedge x i A set of vertices; σ represents the activation function (e.g., ReLU activation function); and δ represents the gate function. Optionally, the gate function δ determines the relevance of each vertex, where δ returns 1 if a vertex is selected and 0 if no vertex is selected. In one example, a sigmoid function is used to implement the gate function δ to facilitate backpropagation and automatic parameter updates.

[0077] In some embodiments, updating the representation of vertices in the feature hypergraph includes performing aggregation information from the hyperedges to which the vertex belongs, according to formula (4):

[0078]

[0079] Among them, W (l) f The model parameters are represented by fj(l); fj(l) represents the vertex f. j The vector in layer l; N f (fj) represents vertex f jThe set of hyperedges to which the hyperedge belongs; σ represents the activation function (e.g., ReLU activation function); and δ represents the gate function. Optionally, the gate function δ determines the relevance of each hyperedge, where δ returns 1 if a hyperedge is selected and 0 if no hyperedge is selected. In one example, a sigmoid function is used to implement the gate function δ to facilitate backpropagation and automatic parameter updates.

[0080] In some embodiments, performing feature selection and combination includes repeatedly updating the representations of hyperedges in the feature hypergraph and updating the representations of vertices in the feature hypergraph, thereby enabling the combination of features through successive layers.

[0081] Figure 6 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure. (Reference) Figure 6 In some embodiments, performing predictive analysis includes: obtaining updated representations of hyperedges in a feature hypergraph and updated representations of vertices in the feature hypergraph; passing the final layer representation of the hyperedge through a first fully connected layer and then through an activation function; and passing the result through a second fully connected layer to produce a prediction result, wherein the result comes from passing the final layer representation of the hyperedge through the first fully connected layer and then through an activation function.

[0082] In some embodiments, performing predictive analysis includes obtaining updated representations of hyperedges in the feature hypergraph and updated representations of vertices in the feature hypergraph, wherein the representation of each hyperedge contains information about the associated vertices connected to the hyperedge selected by a gating function, and the representation of each vertex contains information about the associated hyperedge to which the vertex belongs, selected by a gating function; and making the hyperedge x i The final layer representation The result is passed through a first fully connected layer FC1, followed by a ReLU activation function σ; and then through a second fully connected layer FC2 to produce a prediction result, wherein the result comes from making the hyperedge x i The final layer representation The first fully connected layer FC1 is followed by the ReLU activation function σ.

[0083] In some embodiments, the prediction result is represented by the following formula:

[0084]

[0085] Among them, y' i The prediction result is represented by FC1; FC2 represents the second fully connected layer; σ represents the ReLU activation function; and... Represents the hyperedge x i The vector in layer l.

[0086] In one specific example, the predicted outcome is the patient's survival time.

[0087] In some embodiments, the method further includes using a loss function to optimize model parameters. Optionally, the loss function is represented by equation (6):

[0088] loss = ∑ i ||y' i -y i || 2 (6);

[0089] Among them, y' i This represents the prediction result for the instance, and y i This represents the target result of the instance.

[0090] In some embodiments, a stochastic gradient descent process is used to optimize the model parameters to minimize the loss function according to Equation (6).

[0091] In some embodiments, the dataset {(x1,y1),…,(x M ,y M To perform the optimization of model parameters, x) i = <f1,…,f N >(i=1,…,M); x i This represents the i-th instance (e.g., the i-th patient); f j This indicates that the corresponding x i The j-th feature of y; i It corresponds to x i The target result; M, N, i, j are positive integers; 1≤i≤M; 1≤j≤N.

[0092] Predictive analytics methods, devices for predictive analytics, and computer program products can be implemented in a variety of applications. In one example, the predictive analytics methods, devices for predictive analytics, and computer program products can be implemented in predicting cancer progression (e.g., lung cancer progression). Lung cancer is one of the most common cancers worldwide and a leading cause of cancer-related deaths. Predicting lung cancer progression is crucial for providing personalized treatment plans and improving the prognosis of lung cancer patients. The inventors of this disclosure have found that related methods have certain limitations, such as high cost and the inability to monitor in real time. The inventors of this disclosure have discovered a method for predicting lung cancer progression by combining clinical data, omics data, and imaging data. The method of this disclosure predicts lung cancer progression and uses a diffusion model to visually display disease progression, thereby achieving a visual representation of disease development.

[0093] In some embodiments, the predictive analytics method further includes preprocessing patient data. In some embodiments, preprocessing patient data includes extracting medical codes from the patient's clinical records.

[0094] The following is an example of a patient's clinical record:

[0095] Patient Name: John Doe

[0096] Gender: Male

[0097] Age: 62

[0098] Symptoms: Cough with blood in sputum for 2 months, right-sided chest pain for 1 week.

[0099] Present illness: Two months ago, the patient developed a cough without any obvious cause, with occasional streaks of blood in the sputum, which was not taken seriously. One week ago, the patient experienced right-sided chest pain, prompting them to seek medical attention at the hospital. The amount of blood in the sputum gradually increased, and the sputum color changed from white to red.

[0100] Past medical history: 5-year history of hypertension, controlled regularly with nifedipine controlled-release tablets (10mg qd). No history of diabetes, coronary heart disease, or other chronic diseases. Smoking for over 30 years, consuming 20 cigarettes per day.

[0101] Physical examination: Temperature: 36.8℃; Pulse: 92 beats / min; Respiration: 20 breaths / min; Blood pressure: 135 / 85 mmHg; The patient was alert and cooperative during the examination. Percussion revealed dullness in the right lung, and breath sounds were diminished in the right lung.

[0102] Auxiliary examinations:

[0103] On March 1, 2023, a chest CT scan revealed a soft tissue mass measuring approximately 4.2 × 3.6 cm in the upper lobe of the right lung, with indistinct borders and surrounding spiculated features. Multiple mediastinal lymph nodes were also observed, the largest measuring 1.5 cm in diameter.

[0104] On March 3, 2023, bronchoscopy revealed congestion of the right upper lobe bronchial mucosa, luminal narrowing, and neoplasm growth. A biopsy was sent for pathological examination.

[0105] On March 5, 2023, the pathology report showed atypical acinar structures under the microscope. Immunohistochemistry indicated TTF-1 (+), Napsin A (+), and Ki-67 positivity rate of 55%, consistent with primary lung adenocarcinoma with moderate to low differentiation.

[0106] On March 6, 2023, a PET-CT scan showed a mass in the upper lobe of the right lung with an SUVmax of 7.2 and increased metabolism in the mediastinal lymph nodes with an SUVmax of 5.4, suggesting mediastinal lymph node metastasis from lung cancer.

[0107] Diagnosis: Right upper lobe adenocarcinoma, cT3N2M0, clinical stage IIIA.

[0108] Treatment plan:

[0109] 1. After multidisciplinary discussion, considering the patient's general condition is acceptable, it is planned to perform right upper lobectomy and mediastinal lymph node dissection.

[0110] 2. Postoperative pathological staging, and discussion on whether adjuvant chemotherapy is needed.

[0111] 3. Assess the gene mutation status to determine if there is an indication for targeted therapy.

[0112] 4. Regular follow-up visits to monitor tumor recurrence and metastasis.

[0113] Below is an example medical code extracted from a patient's clinical record.

[0114] ICD-10 encoding:

[0115] R05 - Cough

[0116] R04.2-Coughing up blood

[0117] R07.1 - Chest pain

[0118] I10 - Essential hypertension

[0119] Z72.0 - Smoker

[0120] C34.11 - Malignant tumor of the right bronchus or upper lobe of the lung

[0121] SNOMED CT code:

[0122] 254637007 - Non-small cell lung cancer

[0123] 53741008-Adenocarcinoma

[0124] 23583003-Spiculation

[0125] 25990002 - Lymph node metastasis

[0126] 703842006 - Lung Cancer Staging Stage IIIA

[0127] LOINC code:

[0128] 8480-6-Systolic Blood Pressure

[0129] 8462-4-Diastolic Pressure

[0130] 8867-4-Heart Rate

[0131] 9279-1-Respiratory Rate

[0132] 8310-5-Body temperature

[0133] RxNorm encoding:

[0134] 6918-Nifedipine Controlled-Release Tablets

[0135] CPT encoding:

[0136] 71250 - Chest CT scan (without contrast)

[0137] 31622 - Diagnostic bronchoscopy

[0138] 88305 - Histopathological examination, including gross and microscopic examination.

[0139] 78815-PET-CT Imaging, Whole Body

[0140] 32480 - Lobectomy, Single

[0141] In some embodiments, preprocessing patient data includes obtaining information about differentially expressed biomarkers from the patient's omics data. Omics data includes proteomics data (protein expression) and transcriptomics data (mRNA expression). Table 2 shows the expression levels of various biomarkers in patient omics data obtained from RNA sequencing and proteomics analysis of tumor tissue and adjacent normal tissue.

[0142] Table 2. Expression levels of various biomarkers in patients' omics data

[0143]

[0144]

[0145] As shown in Table 2, the differentially expressed biomarkers from the omics data of patients include EGFR (mRNA expression), KRAS (mRNA expression), TP53 (mRNA expression), STK11 (mRNA expression), EGFR (protein expression), and TP53 (protein expression).

[0146] In some embodiments, the predictive analysis method further includes constructing a feature hypergraph. (See reference...) Figure 2 In some embodiments, constructing a feature hypergraph includes providing a plurality of instances and a plurality of features; defining each of the plurality of features as a vertex in the hypergraph; and defining each of the plurality of instances as a hyperedge in the hypergraph, the hyperedge connecting at least two vertices in the respective instance. In some embodiments, a patient is a corresponding hyperedge among the plurality of hyperedges in the feature hypergraph. In some embodiments, the plurality of features includes medical codes and markers of differential expression.

[0147] In some embodiments, a feature hypergraph can be represented as G = (V, E), where G represents a feature hypergraph; V represents multiple vertices; and E represents multiple hyperedges.

[0148] In some embodiments, the neural network is used to learn feature representations. In some embodiments, the representation of vertices in the feature hypergraph is expressed as:

[0149]

[0150] in, f represents the representation vector of vertex v in layer l; E→V This represents the message passing function from the hyperedge to the vertex; and This represents the representation of the hyperedge containing vertex v. Optionally, the message passing function f from the hyperedge to the vertex. E→V Information propagation is achieved using a multi-head self-attention mechanism. In one example, the representation vector of vertex v in layer l... It has 512 dimensions. In some embodiments, the message passing function is iterated through L layers, where l and L are positive integers, 1 ≤ l ≤ L, to obtain the final representation vector.

[0151] In some embodiments, the representation of hyperedges in the feature hypergraph is expressed as:

[0152]

[0153] in, f represents the representation vector of hyperedge e in layer l; V→E This represents the message passing function from a vertex to a hyperedge; and This represents the vertices contained in the hyperedge e. Optionally, it represents the message passing function f from the vertices to the hyperedge. V→E Information propagation is achieved using a multi-head self-attention mechanism. In one example, the representation vector of hyperedge e in layer l is... It has 512 dimensions. In some embodiments, the message passing function is iterated through L layers, where l and L are positive integers, 1 ≤ l ≤ L, to obtain the final representation vector. The final representation vector obtained after L iterations Used for predictive analytics tasks.

[0154] In some embodiments, the predictive analytics method further includes fusing text data and image data. Figure 7 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure. (Reference) Figure 7 In some embodiments, fusing text data and image data includes: mapping text data to a first vector representation; mapping image data to a second vector representation; and fusing the first vector representation and the second vector representation to form a combined vector representation.

[0155] In some embodiments, mapping text data to a first vector representation includes using a vector model to map the text data to the first vector representation. In one example, the vector model is an M3E vector model. In another example, the first vector representation has 768 dimensions.

[0156] In some embodiments, mapping image data to a second vector representation includes using an image model encoder to map the image data to the second vector representation. In one example, the image model encoder is a visual transformer (ViT) model encoder. In another example, the second vector representation has 1024 dimensions.

[0157] In some embodiments, fusing the first vector representation and the second vector representation to form a combined vector representation is performed according to the following formula:

[0158] V P =αM1V T +(1-α)M2V I (9);

[0159] Among them, V P V represents a combined vector representation; T V represents the first vector representation; I This represents the second vector representation; M1 and M2 are parameter matrices; and α is calculated according to the following formula:

[0160]

[0161] Where M3 and M4 are parameter matrices; p is a parameter column vector; and T represents the transpose operation.

[0162] In one example, M1 and M2 are parameter matrices of size 512×768 and 512×1024, respectively. In another example, M3 and M4 are parameter matrices of size 512×768 and 512×1024, respectively. In yet another example, p is a parameter column vector of dimension 512.

[0163] In some embodiments, the predictive analysis method further includes using combined vector representations as initial vector representations of hyperedges in the feature hypergraph of the neural network (e.g., The predictive analysis method further includes iteratively updating the vector representations of hyperedges in the feature hypergraph up to L layers according to formulas (7) and (8) to obtain the final representation vector. The final representation vector obtained after L iterations Used for predictive analytics tasks.

[0164] Figure 8 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure. (Reference) Figure 8 In some embodiments, performing predictive analysis includes obtaining the final representation vector after L iterations. Make the final representation vector The data is passed through a first multilayer perceptron, then through an activation function; the result is then passed through a second multilayer perceptron to obtain a first prediction result, wherein the result comes from the final representation vector. The data is passed through a first multilayer perceptron, followed by an activation function. Optionally, the activation function is a sigma activation function. In a specific example, the first prediction is the predicted patient survival time.

[0165] In some embodiments, the first prediction result is obtained according to formula (11):

[0166]

[0167] Where y1 represents the first prediction result; σ represents the final representation vector; MLP1 represents the first multilayer perceptron; MLP2 represents the second multilayer perceptron; and σ represents the activation function, such as the sigma activation function.

[0168] Figure 9 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure. (Reference) Figure 9 In some embodiments, performing predictive analysis includes obtaining the final representation vector after L iterations. Make the final representation vector The result is passed through a third multilayer perceptron, followed by an activation function; the result is then passed through a fourth multilayer perceptron to obtain a second prediction result, wherein the result is derived from the final representation vector. The tumor is then passed through a third multilayer perceptron, followed by an activation function. Optionally, the activation function is a sigma activation function. In a specific example, the second prediction is the predicted maximum tumor diameter.

[0169] In some embodiments, a second prediction result is obtained according to formula (12):

[0170]

[0171] Where y2 represents the second prediction result; σ represents the final representation vector; MLP3 represents the third multilayer perceptron; MLP4 represents the fourth multilayer perceptron; and σ represents the activation function, such as the sigma activation function.

[0172] In some embodiments, the predictive analysis method further includes generating a second image (e.g., a predicted tumor image) based on a second prediction result (e.g., the predicted maximum tumor diameter) and a first image (e.g., a current tumor image). The first image is a recently or currently captured image. The first image may be a computed tomography scan image or a magnetic resonance image.

[0173] In some embodiments, generating a second image based on a second prediction result and a first image includes inputting the second prediction result and the first image into a stable diffusion model to generate the second image.

[0174] In some embodiments, the predictive analysis method further includes using a training dataset D = {d1, ...,d...} N Train a stable diffusion model, where d i This represents the nth patient, the characteristics associated with the nth patient (e.g., image data, textual data including medical codes extracted from the patient's clinical records and differentially expressed biomarkers extracted from the patient's omics data), and the survival time of the nth patient. and the maximum tumor diameter of the nth patient n and N are positive integers, where 1 ≤ n ≤ N.

[0175] In some embodiments, training a stable diffusion model includes minimizing a loss function to learn the model's parameters, wherein the loss function is expressed as:

[0176]

[0177] in, This represents the survival time of the nth patient. y represents the maximum tumor diameter of the nth patient. 1,i y represents the predicted survival time of the nth patient. 2,i Let represent the predicted maximum tumor diameter for the nth patient, and β represent the harmonic parameter. Optionally, β ∈ (0,1). In a specific example, β = 0.4.

[0178] In some embodiments, the predictive analysis method further includes collecting image data from patients with a survival time greater than one year, including image I1 at a first time point, image I2 at a second time point, and the corresponding maximum diameter. Optionally, the second time point may be later than the first time point, for example, by one year.

[0179] In some embodiments, training a stable diffusion model includes using the image I1 at the first time point and the maximum tumor diameter of the nth patient. The stable diffusion model is trained by inputting the data into it; the image I2 at the second time point is used as the target output. Optionally, a low-rank adaptive method is used to train the stable diffusion model. In a specific example, the stable diffusion model is the SDXL model.

[0180] Figure 10 This is a flowchart illustrating a predictive analysis method according to some embodiments of the present disclosure. (Reference) Figure 10In a specific example, the predictive analytics method includes: preprocessing data, such as extracting medical codes from clinical medical record data, and extracting differentially expressed biomarkers from omics data. The predictive analytics method also includes fusing textual and image data to form combined data. The predictive analytics method further includes constructing a feature hypermap based on the medical codes extracted from clinical medical record data and the differentially expressed biomarkers extracted from omics data. The predictive analytics method also includes learning feature representations based on the feature hypermap and the combined data. The predictive analytics method further includes using the learned feature representations to predict a first prediction outcome (e.g., predicted patient survival time) and a second prediction outcome (e.g., predicted maximum tumor diameter). The predictive analytics method also includes generating a second image based on the second prediction outcome and the first image.

[0181] In another aspect, this disclosure also provides an apparatus. In some embodiments, the apparatus includes a memory; and one or more processors. In some embodiments, the memory and the one or more processors are connected to each other. In some embodiments, the memory stores computer-executable instructions for controlling the one or more processors to perform the following operations: constructing a feature hypergraph; performing feature selection and combination; and performing predictive analysis. In some embodiments, for performing feature selection and combination, the memory stores computer-executable instructions for controlling the one or more processors to perform the following operations: updating the representation of hyperedges in the feature hypergraph by aggregating information from vertices connected to hyperedges and selected by a gate function; and updating the representation of vertices in the feature hypergraph by aggregating information from hyperedges to which vertices belong and selected by a gate function.

[0182] In some embodiments, in order to update the representation of hyperedges in the feature hypergraph, memory stores computer-executable instructions for controlling one or more processors to perform the following operations: performing aggregation information according to the following formula:

[0183]

[0184] Among them, W (l) f and W (l) x These are model parameters; Represents the hyperedge x i The vector in layer l; N x (x i ) indicates connection to the hyperedge x i A set of vertices; σ represents the activation function; and δ represents the gate function.

[0185] In some embodiments, in order to update the representation of vertices in the feature hypergraph, memory stores computer-executable instructions for controlling one or more processors to perform the following operations: aggregate information according to the following formula:

[0186]

[0187] Among them, W (l) f fj represents the model parameters; (l) Represents vertex f j The vector in layer l; N f (fj) represents vertex f j The set of hyperedges to which it belongs; σ represents the activation function; and δ represents the gate function.

[0188] In some embodiments, the memory stores computer-executable instructions for controlling one or more processors to perform the following operations: repeatedly updating the representations of hyperedges in the feature hypergraph and updating the representations of vertices in the feature hypergraph, thereby enabling the combination of features through successive layers.

[0189] In some embodiments, for constructing a feature hypergraph, the memory stores computer-executable instructions for controlling one or more processors to perform the following operations: providing a plurality of instances and a plurality of features; defining each of the plurality of features as a vertex in the hypergraph; and defining each of the plurality of instances as a hyperedge in the hypergraph, the hyperedge connecting at least two vertices in the respective instance.

[0190] In some embodiments, for constructing a feature hypergraph, the memory stores computer-executable instructions for controlling one or more processors to perform the following operations: providing a plurality of instances and a plurality of features; representing the plurality of features as a plurality of vectors, each of the vectors having a corresponding value across the plurality of instances; calculating the similarity between the plurality of vectors; identifying the nearest neighbors of each of the plurality of features based on the similarity; and forming each hyperedge by grouping each feature with its nearest neighbor.

[0191] In some embodiments, in order to calculate the similarity between feature vectors, memory stores computer-executable instructions for controlling one or more processors to perform the following operations: using a Gaussian kernel function to calculate the similarity between feature vectors:

[0192]

[0193] Among them, f k f j The nearest neighbor of a feature.

[0194] In some embodiments, in order to calculate the similarity between feature vectors, the memory stores computer-executable instructions for controlling one or more processors to perform the following operations: using a cosine similarity function to calculate the similarity between feature vectors:

[0195]

[0196] Among them, f k f j The nearest neighbor of a feature.

[0197] In some embodiments, for performing predictive analytics, the memory stores computer-executable instructions for controlling one or more processors to perform the following operations: obtaining updated representations of hyperedges and vertices in a feature hypergraph; passing the final layer representation of the hyperedges through a first fully connected layer, followed by an activation function; and passing the result through a second fully connected layer to produce a prediction result, wherein the prediction result comes from passing the final layer representation of the hyperedges through the first fully connected layer, followed by an activation function.

[0198] In some embodiments, the prediction result is represented by the following formula:

[0199]

[0200] Show the ReLU activation function; and Represents the hyperedge x i The vector in layer l.

[0201] In some embodiments, the memory stores computer-executable instructions for controlling one or more processors to perform the following operations: optimizing model parameters by minimizing a loss function.

[0202] loss = ∑ i ||y' i -y i || 2 ;

[0203] Among them, y' i This represents the prediction result for the instance, and y i This represents the target result of the instance.

[0204] In some embodiments, the memory stores computer-executable instructions for controlling one or more processors to perform the following operations: preprocessing patient data by at least one of the following operations: extracting medical codes from the patient's clinical records; or obtaining information about biomarkers of differential expression from the patient's omics data.

[0205] In some embodiments, the patient is the corresponding hyperedge among a plurality of hyperedges in the feature hypergraph; and the plurality of features include medical codes and markers of differential expression.

[0206] In some embodiments, a feature hypergraph can be represented as G = (V, E); where G represents the feature hypergraph; V represents multiple vertices; and E represents multiple hyperedges. In some embodiments, a neural network is used to learn the feature representation; the representation of vertices in the feature hypergraph is expressed as:

[0207]

[0208] in, f represents the representation vector of vertex v in layer l; E→V This represents the message passing function from the hyperedge to the vertex; and This represents the representation of the hyperedge containing vertex v.

[0209] In some embodiments, the representation of hyperedges in the feature hypergraph is expressed as:

[0210]

[0211] in, f represents the representation vector of hyperedge e in layer l; V→E This represents the message passing function from a vertex to a hyperedge; and This represents the vertices contained in the hyperedge e.

[0212] In some embodiments, in order to perform predictive analysis, the memory stores computer-executable instructions for controlling one or more processors to perform the following operations: obtaining the final representation vector after L iterations. Make the final representation vector The result is passed through a first multilayer perceptron, then through an activation function; and then through a second multilayer perceptron to obtain a prediction result, wherein the result comes from making the final representation vector... It passes through a first multilayer perceptron, and then through an activation function.

[0213] In some embodiments, the memory stores computer-executable instructions for controlling one or more processors to perform the following operations: mapping text data to a first vector representation; mapping image data to a second vector representation; and fusing the first and second vector representations to form a combined vector representation.

[0214] In some embodiments, the memory stores computer-executable instructions for controlling one or more processors to perform the following operations: using a training dataset D = {d1,…,d...} N Train the model, where d i Let represent the nth patient, the features associated with the nth patient, the survival time of the nth patient, and the maximum tumor diameter of the nth patient; n and N are positive integers, 1≤n≤N; minimize the loss function to learn the parameters of the model, where the loss function is expressed as: in, This represents the survival time of the nth patient. y represents the maximum tumor diameter of the nth patient. 1,iy represents the predicted survival time of the nth patient. 2,i Let β represent the predicted maximum tumor diameter for the nth patient, and let β represent the harmonic parameter; and let β represent the second image generated by inputting the prediction results and the first image into the model, based on the prediction results from the model and the first image.

[0215] In another aspect, this disclosure also provides a computer program product. In some embodiments, the computer program product includes a non-transitory tangible computer-readable medium having computer-readable instructions thereon, which are executable by a processor to cause the processor to: construct a feature hypergraph; perform feature selection and combination; and perform predictive analysis. In some embodiments, in order to perform feature selection and combination, the computer-readable instructions are executable by a processor to cause the processor to: update the representation of hyperedges in the feature hypergraph by aggregating information from vertices connected to and selected by a gate function; and update the representation of vertices in the feature hypergraph by aggregating information from hyperedges to which vertices belong and selected by a gate function.

[0216] In some embodiments, in order to update the representation of hyperedges in the feature hypergraph, computer-readable instructions may be executed by a processor to cause the processor to perform the aggregation of information according to the following formula:

[0217]

[0218] Among them, W (l) f and W (l) x These are model parameters; Represents the hyperedge x i The vector in layer l; N x (x i ) indicates connection to the hyperedge x i A set of vertices; σ represents the activation function; and δ represents the gate function.

[0219] In some embodiments, in order to update the representation of vertices in the feature hypergraph, computer-readable instructions may be executed by a processor to cause the processor to perform the following aggregation of information:

[0220]

[0221] Among them, W (l) f The model parameters are represented by fj(l); fj(l) represents the vertex f. j The vector in layer l; N f (fj) represents vertex f j The set of hyperedges to which it belongs; σ represents the activation function; and δ represents the gate function.

[0222] In some embodiments, computer-readable instructions may be executed by a processor to cause the processor to perform: repeatedly updating the representation of hyperedges in the feature hypergraph and updating the representation of vertices in the feature hypergraph, thereby enabling the combination of features through successive layers.

[0223] In some embodiments, in order to construct a feature, computer-readable instructions may be executed by a processor to cause the processor to perform: providing a plurality of instances and a plurality of features; defining each of the plurality of features as a vertex in a hypergraph; and defining each of the plurality of instances as a hyperedge in the hypergraph, the hyperedge connecting at least two vertices in the respective instance.

[0224] In some embodiments, in order to construct features, computer-readable instructions may be executed by a processor to cause the processor to perform: providing a plurality of instances and a plurality of features; representing the plurality of features as a plurality of vectors, each of the plurality of vectors having a corresponding value across the plurality of instances; calculating the similarity between the plurality of vectors; identifying the nearest neighbors of each of the plurality of features based on the similarity; and forming each hyperedge by grouping the respective features with their nearest neighbors.

[0225] In some embodiments, to calculate the similarity between feature vectors, computer-readable instructions may be executed by a processor to cause the processor to perform the following: using a Gaussian kernel function to calculate the similarity between feature vectors:

[0226]

[0227] Among them, f k f j The nearest neighbor of a feature.

[0228] In some embodiments, to calculate the similarity between feature vectors, computer-readable instructions may be executed by a processor to cause the processor to perform the following: using a cosine similarity function to calculate the similarity between feature vectors:

[0229]

[0230] Among them, f k f j The nearest neighbor of a feature.

[0231] In some embodiments, in order to perform predictive analytics, computer-readable instructions may be executed by a processor to cause the processor to: obtain updated representations of hyperedges in a feature hypergraph and updated representations of vertices in the feature hypergraph; pass the final layer representation of the hyperedge through a first fully connected layer, followed by an activation function; and pass the result through a second fully connected layer to produce a prediction result, wherein the result comes from passing the final layer representation of the hyperedge through the first fully connected layer, followed by an activation function.

[0232] In some embodiments, the prediction result is represented as follows:

[0233]

[0234] Show the ReLU activation function; and Represents the hyperedge x i The vector in layer l.

[0235] In some embodiments, computer-readable instructions may be executed by a processor to cause the processor to perform the following: optimizing model parameters by minimizing a loss function:

[0236] loss = ∑ i ||y' i -y i || 2 ;

[0237] Among them, y' i This represents the prediction result for the instance, and y i This represents the target result of the instance.

[0238] In some embodiments, computer-readable instructions may be executed by a processor to cause the processor to perform: preprocessing patient data by at least one of the following operations: extracting medical codes from the patient's clinical records; or obtaining information about biomarkers of differential expression from the patient's omics data.

[0239] In some embodiments, the patient is the corresponding hyperedge among a plurality of hyperedges in the feature hypergraph; and the plurality of features include medical codes and markers of differential expression.

[0240] In some embodiments, a feature hypergraph can be represented as G = (V, E); where G represents a feature hypergraph; V represents multiple vertices; E represents multiple hyperedges; and a neural network is used to learn the feature representations. The representation of vertices in the feature hypergraph is expressed as:

[0241]

[0242] in, f represents the representation vector of vertex v in layer l; E→V This represents the message passing function from the hyperedge to the vertex; and This represents the representation of the hyperedge containing vertex v.

[0243] In some embodiments, the representation of hyperedges in the feature hypergraph is expressed as:

[0244]

[0245] in, f represents the representation vector of hyperedge e in layer l; V→EThis represents the message passing function from a vertex to a hyperedge; and This represents the vertices contained in the hyperedge e.

[0246] In some embodiments, in order to perform predictive analysis, computer-readable instructions may be executed by a processor to cause the processor to perform: obtaining the final representation vector after L iterations. Make the final representation vector The result is passed through a first multilayer perceptron, then through an activation function; and then through a second multilayer perceptron to obtain a first prediction result, wherein the result comes from making the final representation vector... It passes through a first multilayer perceptron, and then through an activation function.

[0247] In some embodiments, computer-readable instructions may be executed by a processor to cause the processor to perform: mapping text data to a first vector representation; mapping image data to a second vector representation; and fusing the first and second vector representations to form a combined vector representation.

[0248] In some embodiments, computer-readable instructions may be executed by a processor to cause the processor to perform: using a training dataset D = {d1,…,d...} N Train the model, where d i Let represent the nth patient, the features associated with the nth patient, the survival time of the nth patient, and the maximum tumor diameter of the nth patient; n and N are positive integers, 1≤n≤N; minimize the loss function to learn the parameters of the model, where the loss function is expressed as: in, This represents the survival time of the nth patient. y represents the maximum tumor diameter of the nth patient. 1,i y represents the predicted survival time of the nth patient. 2,i Let β represent the predicted maximum tumor diameter for the nth patient, and let β represent the harmonic parameter; and let β represent the second image generated by inputting the prediction results and the first image into the model, based on the prediction results from the model and the first image.

[0249] All or some steps of the methods disclosed above, the functional modules / units in the system, and the devices can be implemented as software, firmware, hardware, or a suitable combination thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division between physical components. For example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transient media) and communication media (or transient media). The term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data, as is known to those skilled in the art. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic tape cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by a computer. Additionally, communication media are typically embodied as computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and include any information transmission medium as known to those skilled in the art.

[0250] For illustrative and descriptive purposes, the foregoing description of embodiments of the invention has been provided. It is not exhaustive, nor is it intended to limit the invention to the precise forms or exemplary embodiments disclosed. Therefore, the foregoing description should be considered illustrative rather than restrictive. Clearly, many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described to explain the principles of the invention and its best mode of practical application, thereby enabling those skilled in the art to understand the various embodiments of the invention and the various modifications suitable for the particular use or implementation contemplated. The scope of the invention is intended to be defined by the appended claims and their equivalents, wherein, unless otherwise stated, all terms are to be interpreted in their broadest reasonable sense. Therefore, the terms “the invention,” “the present invention,” etc., do not necessarily limit the scope of the claims to the specific embodiments, and references to exemplary embodiments of the invention do not imply limitation of the invention, nor should such limitation be inferred. The invention is defined only by the spirit and scope of the appended claims. Furthermore, these claims may involve the use of “first,” “second,” etc., followed by nouns or elements. These terms should be understood as nomenclature and should not be construed as limiting the number of elements modified by these nomenclatures unless a specific number has been given. Any advantages and benefits described may not apply to all embodiments of the invention. It should be understood that changes to the described embodiments can be made by those skilled in the art without departing from the scope of the invention as defined by the appended claims. Furthermore, the elements and components in this disclosure are not intended for public distribution, whether or not they are expressly recited in the appended claims.

Claims

1. A computer-implemented method, comprising: Construct a feature hypergraph; Execution feature selection and combination; as well as Perform predictive analysis; Among them, the selection and combination of execution features include: The representation of the hyperedge in the feature hypergraph is updated by aggregating information from vertices connected to the hyperedge and selected by a gate function; and The representation of the vertex in the feature hypergraph is updated by aggregating information from the hyperedges to which the vertex belongs and selected by the gate function.

2. The computer-implemented method according to claim 1, wherein, Updating the representation of the hyperedge in the feature hypergraph includes performing information aggregation according to the following formula: Among them, W (l) f and W (l) x These are model parameters; Represents the hyperedge x i The vector in layer l; N x (x i ) indicates connection to the hyperedge x i A set of vertices; σ represents the activation function; and δ represents the gate function.

3. The computer-implemented method according to claim 1, wherein, Updating the representation of the vertices in the feature hypergraph includes performing information aggregation according to the following formula: Among them, W (l) f fj represents the model parameters; (l) Represents vertex f j The vector in layer l; N f (fj) represents the vertex f j The set of hyperedges to which it belongs; σ represents the activation function; and δ represents the gate function.

4. The computer-implemented method according to any one of claims 1 to 3, further comprising repeating the following operations: updating the representation of the hyperedge in the feature hypergraph and updating the representation of the vertex in the feature hypergraph, thereby enabling the combination of features through successive layers.

5. The computer-implemented method according to any one of claims 1 to 4, wherein, Constructing the feature hypergraph includes: Provides multiple instances and multiple features; Each of the plurality of features is defined as a vertex in the hypergraph; and Each instance in the plurality of instances is defined as a hyperedge in the hypergraph, and the hyperedge connects at least two vertices in each instance.

6. The computer-implemented method according to any one of claims 1 to 4, wherein, Constructing the feature hypergraph includes: Provides multiple instances and multiple features; The multiple features are represented as multiple vectors, each of which has a corresponding value across the multiple instances; Calculate the similarity between the multiple vectors; Based on the similarity, the nearest neighbor of each of the plurality of features is identified; and Each hyperedge is formed by grouping the individual features with the nearest neighbor.

7. The computer-implemented method according to claim 6, wherein, Use the Gaussian kernel function to calculate the similarity between feature vectors: Among them, f k f j The nearest neighbor of a feature.

8. The computer-implemented method according to claim 6, wherein, The cosine similarity function is used to calculate the similarity between feature vectors: Among them, f k f j The nearest neighbor of a feature.

9. The computer-implemented method according to any one of claims 1 to 8, wherein, Performing predictive analytics includes: Obtain the updated representations of the hyperedges and the updated representations of the vertices in the feature hypergraph; The final layer representation of the hyperedge passes through the first fully connected layer, followed by an activation function; and The result is passed through a second fully connected layer to produce a prediction result, wherein the result comes from the final layer representation of the hyperedge being passed through the first fully connected layer and then through the activation function.

10. The computer-implemented method according to claim 9, wherein, The prediction result is expressed by the following formula: Layer; σ represents the ReLU activation function; and Represents the hyperedge x i The vector in layer l.

11. The computer-implemented method of claim 9 further includes optimizing model parameters by minimizing a loss function: loss=∑ i ||y’ i -y i || 2 ; in, y' i This represents the prediction result for the instance, and y i This represents the target result of the instance.

12. The computer-implemented method according to any one of claims 1 to 8, further comprising preprocessing patient data by at least one of the following operations: Extract medical codes from the patient's clinical records; or Information about differentially expressed biomarkers was obtained from the omics data of the patients.

13. The computer-implemented method according to claim 12, wherein, The patient is the corresponding hyperedge among multiple hyperedges in the feature hypergraph; and The multiple features include the medical code and the markers of differential expression.

14. The computer-implemented method according to claim 12, wherein, The feature hypergraph can be represented as G = (V, E); Wherein, G represents the feature hypergraph; V represents the plurality of vertices; and E represents the plurality of hyperedges; Among them, neural networks are used to learn feature representations; The vertices in the feature hypergraph are represented as follows: in, f represents the representation vector of vertex v in layer l; E→V This represents the message passing function from the hyperedge to the vertex; and This represents the representation of the hyperedge containing vertex v.

15. The computer-implemented method according to claim 14, wherein, The hyperedges in the feature hypergraph are represented as follows: in, f represents the representation vector of hyperedge e in layer l; V→E This represents the message passing function from a vertex to a hyperedge; and This represents the vertices contained in the hyperedge e.

16. The computer-implemented method according to claim 15, wherein, Performing predictive analytics includes: The final representation vector is obtained after L iterations. Make the final representation vector Through the first multilayer perceptron, followed by the activation function; and The result is passed through a second multilayer perceptron to obtain a prediction result, wherein the result is derived from the final representation vector. It passes through the first multilayer perceptron, and then through the activation function.

17. The computer-implemented method according to any one of claims 12 to 16, further comprising: Map the text data to a first vector representation; Map image data to a second vector representation; as well as The first vector representation and the second vector representation are combined to form a combined vector representation.

18. The computer-implemented method according to any one of claims 12 to 17, further comprising: Using the training dataset D = {d1, ...,d2} N Train the model, where d i This represents the nth patient, the characteristics associated with the nth patient, the survival time of the nth patient, and the maximum tumor diameter of the nth patient; n and N are positive integers, 1≤n≤N; Minimize the loss function to learn the parameters of the model, wherein the loss function is expressed as: in, This represents the survival time of the nth patient. y represents the maximum tumor diameter of the nth patient. 1,i y represents the predicted survival time of the nth patient. 2,i Let β represent the predicted maximum tumor diameter of the nth patient, and β represent the harmonic parameter; and The second image is generated by inputting the prediction result and the first image into the model. The second image is generated based on the prediction result from the model and the first image.

19. An apparatus comprising: Memory; as well as One or more processors; Wherein, the memory and the one or more processors are connected to each other; and The memory stores computer-executable instructions for controlling the one or more processors to perform the following operations: Construct a feature hypergraph; Execution feature selection and combination; and Perform predictive analysis; In order to perform feature selection and combination, the memory stores computer-executable instructions for controlling the one or more processors to perform the following operations: The representation of the hyperedge in the feature hypergraph is updated by aggregating information from vertices connected to the hyperedge and selected by a gate function; and The representation of the vertex in the feature hypergraph is updated by aggregating information from the hyperedges to which the vertex belongs and selected by the gate function.

20. A computer program product comprising a non-transitory tangible computer-readable medium having computer-readable instructions thereon, the computer-readable instructions being executable by a processor to cause the processor to perform: Construct a feature hypergraph; Execution feature selection and combination; as well as Perform predictive analysis; Among them, the selection and combination of execution features include: The representation of the hyperedge in the feature hypergraph is updated by aggregating information from vertices connected to the hyperedge and selected by a gate function; and The representation of the vertex in the feature hypergraph is updated by aggregating information from the hyperedges to which the vertex belongs and selected by the gate function.

Images