Biomedical data reasoning method and device based on multi-modal alignment, equipment and medium

Abstract

The application discloses a biomedical data reasoning method and device based on multi-modal alignment, equipment and medium, relates to the field of biomedicine, and comprises the following steps: acquiring sample measurement data, determining an initial data reasoning model, training the initial data reasoning model by using the sample measurement data through a KL divergence regularization mechanism to obtain a target data reasoning model; determining initial measurement data of target biomedical data, constructing a target encoder in the target biomedical data reasoning model to obtain initial embedding representation based on the initial measurement data; constraining the initial embedding representation through an initial embedding channel and performing cross-modal filling, aligning the initial embedding representation through an L2 normalization channel to generate a target embedding vector, and merging the target embedding vector into a target embedding matrix; and performing reasoning based on the target embedding matrix by using a linear classifier through a preset clustering algorithm to determine the category of the data. In this way, the application can mine the relationship between biomedical data and complement each other.

Description

Technical Field

[0001] This invention relates to the field of biomedicine, and in particular to a biomedical data inference method, apparatus, device, and medium based on multimodal alignment. Background Technology

[0002] In biomedical research, multimodal data integration is a core challenge. Modern biomedical experiments generate a large amount of heterogeneous data, such as: omics data (genomics, transcriptomics, proteomics, metabolomics, etc.), which have different characteristic dimensions and numerical distribution characteristics; clinical medical data (medical imaging data, morphological features of pathological slides, structured clinical indicators in electronic health records, etc.); and pharmaceutical data (high-throughput screening of phenotypic profiles, with different laboratories using different cell lines, staining markers, microscopy equipment, and computational analysis workflows, resulting in phenotypic profiles with completely different feature types and dimensions). Integrating these heterogeneous data into a unified analytical framework enables cross-modal sample classification, retrieval, and missing data imputation, which is of great significance for disease mechanism research, drug discovery, and precision medicine.

[0003] Contrastive learning has achieved significant success in the field of multimodal data alignment in recent years. Its core idea is to learn meaningful cross-modal embeddings by maximizing the similarity of the same sample across different modalities in the latent space while minimizing the similarity between different samples. CLIP (Contrastive Language-Image Pre-training) is a landmark work in this field, successfully achieving alignment between image and text modalities. KL divergence regularization is widely used in variational autoencoders, constraining the distribution of the latent space to approximate a standard Gaussian distribution N(0,I), thus giving the latent space favorable mathematical properties, which is beneficial for generation and interpolation tasks.

[0004] In conclusion, how to fully explore the potential connections and complements between biomedical data is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for biomedical data inference based on multimodal alignment, which can fully explore the potential connections and complementarities between biomedical data. The specific solution is as follows: In a first aspect, this application provides a biomedical data reasoning method based on multimodal alignment, comprising: Acquire sample measurement data for each sample modality of the target biomedical sample, determine an initial data inference model for biomedical data inference, and train the initial data inference model using the sample measurement data for each sample modality through the KL divergence regularization mechanism to obtain the target data inference model. New target biomedical data is identified, and initial measurement data of each data modality corresponding to the target biomedical data is obtained. Based on each data modality, a corresponding target encoder is constructed in the target biomedical data inference model. The initial measurement data is input into the target encoder to output the initial embedding representation of each data modality. Determine the target dual-channel data mechanism in the target biomedical data inference model; the target dual-channel data mechanism includes an initial embedding channel and an L2 normalized channel; The initial embedding representation is subjected to cross-modal imputation through the initial embedding channel, and the initial embedding representation of each data modality is multimodal aligned using the L2 normalization channel to generate the target embedding vector. The target embedding vectors of each data modality are merged into a target embedding matrix. A linear classifier is used with a preset clustering algorithm, and reasoning is performed based on the target embedding matrix to determine the category of each data in the target biomedical data.

[0006] Optionally, the step of training the initial data inference model using the sample measurement data of each of the sample modalities through the KL divergence regularization mechanism to obtain the target data inference model includes: Based on each of the sample modalities, a corresponding initial encoder is constructed in the initial data inference model. The corresponding sample measurement data is mapped to the latent space using the initial encoders corresponding to each of the sample modalities to generate sample embedding representations. Using the initial embedding channel of the initial data inference model, the distribution characteristics of each sample mode are determined based on the sample embedding representation; The target KL divergence between the distribution characteristics of each sample mode and the preset standard Gaussian distribution is determined by the KL divergence regularization mechanism. The target KL divergence of each of the sample modes is averaged to obtain the target distribution regularization loss; The L2 normalization channel in the initial data inference model is used to perform L2 normalization on the sample embedding representation to obtain the normalized embedding vector of each sample mode. Select any mode from each of the sample modes as the target master mode, and concatenate the normalized embedding vectors of the non-target master modes in the sample modes along the preset sample dimension to form an auxiliary embedding matrix; Determine the target similarity matrix between the normalized embedding vector of the target main modality and the auxiliary embedding matrix; Construct a target soft label matrix based on the preset relationships of each of the sample modalities; The target cross-entropy loss corresponding to the target dominant mode is determined based on the target similarity matrix and the target soft label matrix. The target contrast loss is obtained by summing the target cross-entropy loss for each of the sample modes; The total loss is obtained by weighted summing the target distribution regularization loss and the target contrast loss. The initial data inference model is optimized and adjusted based on the total loss to obtain the target data inference model.

[0007] Optionally, a cross-modal imputation operation is performed on the initial embedding representation through the initial embedding channel to generate a target embedding vector, including: Cross-modal distribution alignment constraints are applied to the initial embedding representations output by each target encoder to map the initial embedding representations of each data modality to the target latent space and satisfy a preset distribution format, thereby obtaining the corresponding constrained embedding representations. Determine the target decoder corresponding to each of the target modes; The decoder corresponding to the target mode is used to decode each of the constraint-embedded representations to generate the predicted feature value corresponding to the target mode; Based on the generated predicted feature values, the initial measurement data in the target biomedical data that meet the preset data missing conditions are filled in to obtain the filled measurement data. The infilled measurement data is encoded using the target encoder to generate an infilled target embedding representation.

[0008] Optionally, the step of using the L2 normalized channel to perform multimodal alignment on the initial embedding representations of each data modality to generate a target embedding vector includes: The L2 normalization channel in the target dual-channel data mechanism is invoked to perform L2 norm normalization processing on the initial embedding representation of each data mode, so as to generate the normalized embedding representation of each data mode. The normalized embedding representations of each of the data modalities are mapped to the target latent space to perform cross-modal semantic alignment operations and generate semantic target embedding vectors. The semantic target embedding vectors of each data modality are dimensionally concatenated or weighted and fused to generate aligned target embedding vectors.

[0009] Optionally, merging the target embedding vectors of each data modality into a target embedding matrix includes: The target embedding vector of each of the data modes is projected onto the unit hypersphere template to generate the target hypersphere; The embedding vectors in the target hypersphere are transformed and merged into a target embedding matrix.

[0010] Optionally, the step of determining the category of each data point in the target biomedical data by using a linear classifier and reasoning based on the target embedding matrix through a preset clustering algorithm includes: An unsupervised clustering analysis is performed on the target embedding matrix using a preset clustering algorithm to automatically divide the target biomedical samples into initial category clusters, thereby generating initial clustering labels for each sample in the target biomedical samples. Based on the initial clustering labels, a linear classifier is used to infer and predict each sample in the target embedding matrix, and the category probability distribution and predicted category corresponding to each sample are output. The category of each data point in the target biomedical data is determined based on the category probability distribution and the predicted category.

[0011] Optionally, after determining the category of each data point in the target biomedical data by using a linear classifier through a preset clustering algorithm and inference based on the target embedding matrix, the method further includes: Determine the target F1 score for each category of the target biomedical data, average the target F1 scores to generate the target score average, and determine the corresponding classification evaluation index based on the target score average. The target total variation distance of each sample in the target biomedical sample is obtained, and the average value of each target total variation distance is taken to generate the target distance average value. Based on the target distance average value, the corresponding cross-modal alignment evaluation index is determined. Generate clustering results and target adjusted Rand indexes for each sample in the target biomedical sample with preset labels, and use the target adjusted Rand indexes as clustering evaluation indicators. The corresponding result evaluation indicators for the clustering results are determined based on the classification evaluation indicators, cross-modal alignment evaluation indicators, and clustering evaluation indicators.

[0012] Secondly, this application provides a biomedical data inference device based on multimodal alignment, comprising: The model acquisition module is used to acquire sample measurement data of each sample modality of the target biomedical sample, determine an initial data inference model for biomedical data inference, and train the initial data inference model using the sample measurement data of each sample modality through the KL divergence regularization mechanism to obtain the target data inference model. An embedding representation output module is used to determine new target biomedical data and obtain initial measurement data of each data modality corresponding to the target biomedical data. Based on each data modality, a corresponding target encoder is constructed in the target biomedical data inference model. The initial measurement data is input into the target encoder to output the initial embedding representation of each data modality. A mechanism determination module is used to determine the target dual-channel data mechanism in the target biomedical data inference model; the target dual-channel data mechanism includes an initial embedding channel and an L2 normalized channel. An embedding representation generation module is used to perform cross-modal imputation on the initial embedding representation through the initial embedding channel, and to perform multimodal alignment on the initial embedding representation of each data modality using the L2 normalization channel to generate a target embedding vector. The category determination module is used to merge the target embedding vectors of each data modality into a target embedding matrix, and use a linear classifier with a preset clustering algorithm to perform inference based on the target embedding matrix to determine the category of each data in the target biomedical data.

[0013] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the biomedical data inference method based on multimodal alignment as described above.

[0014] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned biomedical data inference method based on multimodal alignment.

[0015] In summary, this application first obtains sample measurement data for each sample modality of the target biomedical sample, determines an initial data inference model for biomedical data inference, and trains the initial data inference model using the sample measurement data of each sample modality through the KL divergence regularization mechanism to obtain the target data inference model; then, it determines new target biomedical data and obtains the initial measurement data for each data modality corresponding to the target biomedical data, constructs a corresponding target encoder in the target biomedical data inference model based on each data modality, and inputs the initial measurement data into the target encoder to output the initial values ​​of each data modality. Embedding representation; determining the target dual-channel data mechanism in the target biomedical data inference model; the target dual-channel data mechanism includes an initial embedding channel and an L2 normalized channel; performing cross-modal imputation on the initial embedding representation through the initial embedding channel, and using the L2 normalized channel to perform multimodal alignment on the initial embedding representation of each data modality to generate a target embedding vector; merging the target embedding vectors of each data modality into a target embedding matrix, and using a preset clustering algorithm with a linear classifier and inference based on the target embedding matrix to determine the category of each data in the target biomedical data. As described above, this application first acquires measurement data of each modality of the target biomedical sample, trains an initial data inference model using the KL divergence regularization mechanism to obtain the target data inference model, then acquires initial measurement data of each modality of the new target biomedical data, outputs the initial embedding representation of each modality through the corresponding target encoder, and then constrains and imputs the initial embedding representation across modalities using the initial embedding channel of the target dual-channel data mechanism. The imputed embedding representation is then aligned across modalities using the L2 normalization channel of this mechanism, generating target embedding vectors and merging them into a target embedding matrix. Finally, a pre-defined clustering algorithm and a linear classifier are used to infer the category of each sample based on the target embedding matrix and perform evaluation to obtain the clustering results and corresponding evaluation metrics. In this way, while maintaining the ability to compare and align multiple modalities, the introduction of the KL divergence regularization mechanism explicitly constrains the distribution of the embedding space to approximate the standard Gaussian distribution N(0,I), making the latent space have regular and smooth distribution characteristics, improving embedding quality and downstream task performance. Meanwhile, by leveraging a regularized embedding space distribution, the quality and stability of cross-modal imputation / generation are improved, making it possible to infer features from one modality to another. Furthermore, by constraining the embedding distribution to a standard Gaussian distribution, embeddings generated from different training runs and different datasets have comparable probabilistic semantics, improving the interpretability and cross-experimental transferability of the embedding representation. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a flowchart of a biomedical data reasoning method based on multimodal alignment disclosed in this application; Figure 2 This is a schematic diagram of a biomedical data inference device based on multimodal alignment disclosed in this application; Figure 3 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Currently, multimodal data integration is a core challenge in biomedical research. Modern biomedical experiments generate a large amount of heterogeneous data, such as: omics data (genomics, transcriptomics, proteomics, metabolomics, etc.), which have different feature dimensions and numerical distribution characteristics; clinical medical data (medical imaging data, morphological features of pathological slides, structured clinical indicators in electronic health records, etc.); and pharmaceutical data (high-throughput screening of phenotypic profiles, with different laboratories using different cell lines, staining markers, microscopy equipment, and computational analysis workflows, resulting in phenotypic profiles with completely different feature types and dimensions). Integrating these heterogeneous data into a unified analytical framework enables cross-modal sample classification, retrieval, and missing data imputation, which is of great significance for disease mechanism research, drug discovery, and precision medicine. To address the above technical problems, this application discloses a biomedical data inference method, device, equipment, and medium based on multimodal alignment, which can fully explore the potential connections between biomedical data and complement each other.

[0020] See Figure 1 As shown, this embodiment of the invention discloses a biomedical data inference method based on multimodal alignment, including: Step S11: Obtain sample measurement data for each sample modality of the target biomedical sample, determine the initial data inference model for biomedical data inference, and train the initial data inference model using the sample measurement data of each sample modality through the KL divergence regularization mechanism to obtain the target data inference model.

[0021] In this embodiment, if researchers obtain multiple omics measurement data of the same sample, including samples from various modalities such as gene expression profiles (transcriptomics), protein abundance (proteomics), metabolite concentrations (metabolicomics), drug sensitivity (drug response profiles), DNA methylation levels (epiggenomics), gene dependence scores (CRISPR-Cas9 screening), and copy number variations (genomics), each modal data has different feature dimensions and numerical distributions. These heterogeneous data need to be mapped to a unified latent space to achieve cross-modal sample classification, retrieval, and missing data imputation. First, it is necessary to determine the corresponding target data inference model for processing biomedical data. An initial encoder can be constructed in the initial data inference model based on each of the sample modes. The initial encoder corresponding to each sample mode maps the corresponding sample measurement data to the latent space, generating a sample embedding representation. Using the initial embedding channel of the initial data inference model, the distribution characteristics of each sample mode are determined based on the sample embedding representation. A target KL divergence between the distribution characteristics of each sample mode and a preset standard Gaussian distribution is determined through a KL divergence regularization mechanism. The target KL divergence of each sample mode is averaged to obtain the target distribution regularization loss. The sample embedding representation is L2 normalized through the L2 normalization channel in the initial data inference model to obtain the normalized embedding vector of each sample mode. Any modality in the sample modalities is selected as the target primary modality. The normalized embedding vectors of the non-target primary modalities in the sample modalities are concatenated along a preset sample dimension to form an auxiliary embedding matrix. The target similarity matrix between the normalized embedding vector of the target primary modality and the auxiliary embedding matrix is ​​determined. A target soft label matrix is ​​constructed based on the preset relationship of each sample modality. The target cross-entropy loss corresponding to the target primary modality is determined according to the target similarity matrix and the target soft label matrix. The target cross-entropy loss of each sample modality is summed to obtain the target contrast loss. The target distribution regularization loss and the target contrast loss are weighted and summed to obtain the total loss. The initial data inference model is optimized and adjusted according to the total loss to obtain the target data inference model.

[0022] Specifically, it receives input data from n modalities. Where N is the number of samples for each modality. The system outputs the embedding representations of all modalities in the shared latent space. , where each Z i The shape is (N, z) dim ), z dim represents the potential spatial dimension. For each modality, an encoder is constructed, with a network structure of a linear mapping network consisting of four fully connected layers.

[0023] For each mode i, the input data x i The encoder outputs the original embedded z i Then, the data is fed into two channels: an L2 normalization channel and the original embedding channel of the data inference model. The L2 normalization channel performs L2 normalization on the original embedding output by the encoder to obtain the embedding vector z. i , z i Projected onto a unit hypersphere. In this way, the similarity calculation between different omics modalities (such as transcriptomics and proteomics) is equivalent to cosine similarity, eliminating the influence of differences in embedding amplitude between modalities and making the contrast loss more stable. Next, for Normalized embedding of each modality { , ,…, } Calculate the contrast loss: For example, when transcriptomics is selected as the primary modality, the normalized embeddings of the other six modalities—proteomics, metabolomics, drug response profiling, DNA methylation, CRISPR-Cas9 screening, and copy number variation—are concatenated along the sample dimension to form an auxiliary embedding matrix. Formally, each modality is selected sequentially. As the primary mode, the normalized embeddings of all other modes are concatenated into an auxiliary embedding matrix: ; In the above seven omics modalities, the shape of the auxiliary embedding matrix is ​​as follows: .

[0024] Next, the scaling similarity matrix between the main modality embedding and the auxiliary embedding matrix is ​​calculated: ; in Given a learnable temperature parameter (initialized to ln(1 / 0.07)≈2.659), the shape of S is... This temperature parameter automatically adjusts the scale of similarity between the embeddings of different omics modalities during training.

[0025] For the aforementioned multi-omics alignment task, a soft label matrix is ​​constructed. Understandably, It is The matrix, by indivual It is composed of horizontally concatenated identity matrices. For example, a sample of a cell line. The embedding in the dominant modality should be consistent with the same cell line sample. Embeddings are similar across all other omics modalities but dissimilar across other cell line samples. Unlike standard pairwise contrastive learning, which only considers the relationship between two modalities, this method concatenates the embeddings of all other modalities and uses repeated diagonal labels to simultaneously constrain the alignment between one modality and all other modalities in a single forward propagation. This achieves globally consistent multimodal alignment and avoids the inconsistency issues that may occur in pairwise methods.

[0026] Based on the similarity matrix and soft tag matrix mentioned above, calculate the bidirectional soft tag cross-entropy loss: ; in .

[0027] The total contrastive loss is obtained by summing the losses of each omics mode as the dominant mode in turn: ; Simultaneously, KL divergence calculation is performed on the original embedded channels, which can effectively constrain the distribution characteristics of the encoder outputs of each omics mode, making them approximate a standard Gaussian distribution. Then, for each mode... The original embedding Calculate its batch statistics: ; ; in, , where is the numerical stability constant. In the aforementioned multi-omics scenario, unlike variational autoencoders where each sample independently outputs mean and variance parameters, the encoder in this method outputs only a single embedding vector, and the KL divergence is calculated using batch-level distribution statistics. This batch-level distribution regularization is more lightweight, does not increase the number of parameters or output dimensionality of each omics modality encoder, and at the same time, constrains the embedding distribution of each omics modality to approximate a standard Gaussian distribution from a macroscopic perspective, making it naturally compatible with batch-level optimization of contrastive loss.

[0028] For each omics mode, calculate the KL divergence between its embedding distribution and the standard Gaussian distribution N(0,I): ; The total distribution regularization loss is obtained by averaging the KL divergence of each omics mode: ; Finally, the comparison alignment loss and the distribution regularization joint optimization loss are weighted and fused to calculate the total loss: ; in, This represents the KL divergence regularization weight, typically ranging from 0.001 to 0.1, controlling the strength of regularization for the embedding distribution of each omics mode. In the aforementioned multi-omics integration tasks, a larger... Making the embedding distribution closer to a standard Gaussian distribution may weaken the effect of contrast alignment; smaller It maintains the effect of contrast alignment between different omics modes, but the regularization constraints are relatively weak.

[0029] Finally, the initial data inference model is trained using samples, and the training of the model is judged based on the total loss function to determine whether the training meets the requirements, thereby obtaining the target data inference model after training.

[0030] Step S12: Determine new target biomedical data and obtain initial measurement data for each data modality corresponding to the target biomedical data. Based on each data modality, construct a corresponding target encoder in the target biomedical data inference model. Input the initial measurement data into the target encoder to output the initial embedding representation of each data modality.

[0031] In this embodiment, after model training is complete, when new biomedical data needs to be processed, the new target biomedical data is first identified, and the initial measurement data of each data modality corresponding to the target biomedical data is obtained. This measurement data may also cover various types such as transcriptomics, proteomics, metabolomics, drug response profiles, epigenomes, CRISPR screening, and genomic variations. Each modality's data retains its own feature dimensions and numerical distribution. Based on the previously trained target biomedical data inference model, a corresponding target encoder is constructed for each data modality within the model. The network structure and parameters of these target encoders are inherited from the configuration optimized during the training phase. Subsequently, the initial measurement data of each modality are input into their corresponding target encoders. Each encoder independently extracts and transforms features from its input data, thereby outputting the initial embedding representation of each data modality in a unified latent space. It is important to note that the initial embedding representation retains the key information of the original samples at different omics levels and is mapped to a dimensionally consistent and distributionally aligned common space, providing a unified feature foundation for subsequent cross-modal analysis tasks, such as sample classification, cross-modal retrieval, or missing data imputation.

[0032] Step S13: Determine the target dual-channel data mechanism in the target biomedical data inference model; the target dual-channel data mechanism includes an initial embedding channel and an L2 normalized channel.

[0033] In this embodiment, the initial embedding channel of the target biomedical data inference model, namely the original embedding channel, is first determined. This channel receives the raw biomedical data and directly maps it to a high-dimensional vector space through the embedding layer. This vector preserves the original distribution, relative magnitude, and intrinsic relationships between features of the data, providing the model with unscaled, true semantic information. Next, an L2 normalization channel is introduced based on the original embedding channel, enhancing the comparability between different samples. By processing the embedding vector simultaneously through this dual-channel mechanism, the model can utilize both the absolute differences in the original values ​​and the normalized relative relationships, thereby more comprehensively characterizing the essential features of the biomedical data.

[0034] Step S14: Perform cross-modal imputation on the initial embedding representation through the initial embedding channel, and use the L2 normalization channel to perform multimodal alignment on the initial embedding representation of each data modality to generate the target embedding vector.

[0035] In this embodiment, in the initial embedding channel, cross-modal distribution alignment constraints are applied to the initial embedding representations output by each target encoder to map the initial embedding representations of each data modality to the target latent space and satisfy a preset distribution format, resulting in a corresponding constrained embedding representation. A target decoder corresponding to each target modality is determined. The decoder corresponding to the target modality decodes each constrained embedding representation to generate a predicted feature value corresponding to the target modality. Based on the generated predicted feature values, the initial measurement data in the target biomedical data that meets the preset data missing conditions is filled in to obtain filled measurement data. The target encoder encodes the filled measurement data to generate a filled target embedding representation. Specifically, KL divergence regularization is used to constrain the initial embedding representation to approximate a standard Gaussian distribution N(0,I), making the latent space smooth and continuous, and encoders of different modalities map the same sample to similar positions in the latent space. Therefore, when measurement data for a certain omics modality is missing, the feature values ​​of the missing modality are inferred using data from existing modalities. For example, after encoding a sample using an encoder for modality A, the latent embedding can be decoded into predicted feature values ​​for modality B using a decoder for modality B, achieving cross-modal imputation from modality A to modality B. It's important to know that imputation quality can be evaluated using metrics such as per-feature Pearson correlation coefficient, per-sample R² coefficient of determination, AUROC (classification ability after feature binarization), RMSE, and MAE.

[0036] Additionally, the decoder needs to be trained. A multilayer perceptron decoder can be trained for each target modality to map the latent embeddings back to the original feature space. For decoder training, the encoder extracts the original embeddings for each modality, and then the decoder is trained to reconstruct the original features of the corresponding modality from the original embeddings. Finally, the trained decoder for multimodal imputation is obtained.

[0037] Understandably, while performing cross-modal imputation on the embedded representation using the initial embedding channel, the L2 normalization channel in the target dual-channel data mechanism is invoked to perform L2 norm normalization on the imputed embedded representations of each data modality, thereby generating normalized embedded representations for each data modality. These normalized embedded representations are then mapped to the target latent space to perform cross-modal semantic alignment, generating semantic target embedding vectors. Finally, the semantic target embedding vectors of each data modality are dimensionally concatenated or weighted to generate aligned target embedding vectors. Specifically, the pre-constructed L2 normalization channel in the target dual-channel data mechanism is invoked. For each modality's embedding vector, its L2 norm is calculated, and then each component is divided by this norm to scale the vector to unit length, thus generating the normalized embedded representations for each data modality. After normalization, these normalized embedding representations need to be further mapped to the target latent space. This mapping process simultaneously performs cross-modal semantic alignment, making samples with the same semantic information in different modalities closer together in the latent space, thus generating semantically aligned embedding vectors. Finally, for the semantic target embedding vectors obtained from each data modality, the dimensional concatenation or weighted fusion method is selected according to the specific application requirements. For example, the vectors of all modalities can be directly concatenated along the feature dimension to form a higher-dimensional joint representation, or the vectors of each modality can be linearly weighted and summed according to preset weights to finally generate aligned target embedding vectors for subsequent classification, retrieval, or analysis tasks.

[0038] Step S15: Merge the target embedding vectors of each data modality into a target embedding matrix, and use a linear classifier with a preset clustering algorithm to perform inference based on the target embedding matrix to determine the category of each data in the target biomedical data.

[0039] In this embodiment, the target embedding vector obtained at this time is the cross-modal and aligned embedding vector of the two channels. The target embedding vectors of each data modality are projected onto a unit hypersphere template to generate a target bit hypersphere. The embedding vectors in the target bit hypersphere are then transformed and merged into a target embedding matrix. Specifically, the target embedding vectors generated for each data modality are projected onto a preset unit hypersphere template to generate the target bit hypersphere. Subsequently, the embedding vectors in the spatial domain target bit hypersphere are transformed into a target embedding matrix in the digital domain. Each row of this matrix corresponds to the embedding representation of a sample or a modality, thereby providing structured input for cross-modal analysis.

[0040] It is important to understand that the target embedding matrix is ​​subjected to unsupervised clustering analysis using a preset clustering algorithm. This automatically divides the target biomedical samples into initial category clusters, generating initial cluster labels for each sample. Based on these initial cluster labels, a linear classifier is used to infer and predict the category of each sample in the target embedding matrix, outputting the category probability distribution and predicted category for each sample. The category of each data point in the target biomedical data is then determined based on these category probability distributions and predicted categories. Specifically, Logistic Regression is used as the linear classifier. The preset clustering algorithm performs unsupervised clustering analysis on the obtained target embedding matrix. This algorithm automatically divides the target biomedical samples into several initial category clusters based on the similarity between embedding vectors and assigns a corresponding initial cluster label to each sample. Based on this, these initial cluster labels are used as supervision signals, and a linear classifier is used to infer and predict the category of each sample in the target embedding matrix. The linear classifier accepts the embedding vector of each sample as input and outputs the probability distribution of the sample belonging to each category and the final predicted category. Finally, based on the category probability distribution and predicted category output by the linear classifier, the specific category to which each sample in the target biomedical sample belongs can be determined, thus completing the joint classification process from unsupervised to supervised.

[0041] Furthermore, to facilitate a more intuitive understanding of the classification results' quality by the user, the classification results are scored, i.e., corresponding result evaluation metrics are generated. This requires determining the target F1 score for each category in the target biomedical data, averaging the target F1 scores to generate a target average score, and determining the corresponding classification evaluation metric based on this target average score. It also involves obtaining the target total variation distance for each sample in the target biomedical samples, averaging the target total variation distances to generate a target average distance, and determining the corresponding cross-modal alignment evaluation metric based on this target average distance. Finally, it involves generating the clustering results and the target adjusted Rand index of the preset labels for each sample in the target biomedical samples, using the target adjusted Rand index as the clustering evaluation metric. Based on the classification evaluation metric, the cross-modal alignment evaluation metric, and the clustering evaluation metric, the corresponding result evaluation metric for the clustering results is determined. Specifically, the macro average F1 score is used to evaluate classification performance. Simultaneously, TVD (Total Variation Distance) is used to evaluate whether samples of the same category are uniformly mixed across modalities in the embedding space. For each sample, the total variation distance (TVD) between the distribution of its nearest neighbors of the same class in each modality in the embedding space and the ideal uniform distribution is calculated. The final score is obtained by averaging the TVD of all samples. A lower TVD indicates better cross-modal alignment, meaning that samples of the same class are uniformly mixed in the embedding space regardless of which modality they come from. The ARI (Adjusted Rand Index) is used to evaluate the consistency between the clustering results and the true labels. The ARI value ranges from [-1, 1], where 1 represents perfect clustering and 0 represents random clustering. The quality of data classification is verified by the classification, clustering, and cross-modal alignment evaluation metrics, and corresponding result evaluation metrics are generated and output along with the classification results.

[0042] As described above, this embodiment first acquires measurement data of each modality of the target biomedical sample, trains an initial data inference model using the KL divergence regularization mechanism to obtain the target data inference model, then acquires initial measurement data of each modality of the new target biomedical data, outputs the initial embedding representation of each modality through the corresponding target encoder, then constrains and imputs the initial embedding representation across modalities using the initial embedding channel of the target dual-channel data mechanism, performs multimodal alignment on the imputed embedding representation using the L2 normalization channel of this mechanism, generates target embedding vectors and merges them into a target embedding matrix, and finally uses a preset clustering algorithm and a linear classifier to infer the category of each sample based on the target embedding matrix and performs evaluation to obtain clustering results and corresponding evaluation indicators. In this way, while maintaining the ability of multimodal contrast alignment, by introducing the KL divergence regularization mechanism, the distribution of the embedding space is explicitly constrained to approximate the standard Gaussian distribution N(0,I), so that the latent space has regular and smooth distribution characteristics, improving the embedding quality and downstream task performance. Meanwhile, by leveraging a regularized embedding space distribution, the quality and stability of cross-modal imputation / generation are improved, making it possible to infer features from one modality to another. Furthermore, by constraining the embedding distribution to a standard Gaussian distribution, embeddings generated from different training runs and different datasets have comparable probabilistic semantics, improving the interpretability and cross-experimental transferability of the embedding representation.

[0043] As described in the previous embodiment, this application discloses a biomedical data reasoning method based on multimodal alignment, which can fully explore the potential connections and complementarities between biomedical data. This application has broad application prospects in the biomedical field. In terms of omics data integration, it can align multi-omics data such as genomics, transcriptomics, proteomics, and metabolomics into a unified latent space, enabling multi-omics joint analysis and disease subtype discovery, helping researchers understand the mechanisms of disease development at multiple molecular levels. In terms of high-throughput drug screening data integration, it can align HCS phenotypic profile data from different laboratories and under different experimental conditions, and by sharing reference compound information in the latent space, it can predict the transfer of functions of unlabeled compounds, accelerating the drug discovery process. In terms of clinical multimodal data fusion, it can integrate heterogeneous clinical data such as medical imaging features, pathological slide morphological features, and gene testing results, providing more comprehensive information support for disease diagnosis and prognosis prediction, and promoting the development of precision medicine. In single-cell multi-omics analysis, this technology can align single-cell data generated by different technical platforms such as single-cell RNA-seq, ATAC-seq, and proteomics for cell type identification and cell state inference, helping to reveal the molecular mechanisms of cell heterogeneity and cell fate determination. These application scenarios fully demonstrate the versatility and practical value of this technology in processing heterogeneous multimodal biomedical data. Next, taking cancer subtype discovery in multi-omics data integration as an example, we will provide a detailed explanation of the biomedical data inference method based on multimodal alignment.

[0044] First, initial measurement data for each data modality in a tumor cell cohort are determined, including gene expression profiles (transcriptome), DNA methylation profiles (epigmogenome), and copy number variations (genomics). Based on a pre-trained target biomedical data inference model, a corresponding target encoder is constructed for each modality. The initial measurement data for each modality are input into their respective encoders, and the initial embedding representation of each data modality in the latent space is output.

[0045] Next, the initial embedding channels in the target dual-channel data mechanism are invoked to perform constraint and cross-modal imputation operations on these initial embedding representations. Specifically, KL divergence regularization is used to make the latent distribution of each mode approximate the standard Gaussian distribution, while the missing or noisy parts are inferred and imputed with the help of the complete information of other modes, so as to obtain the imputed embedding representation that satisfies the standard Gaussian distribution.

[0046] Then, using the L2 normalized channel in the target dual-channel data mechanism, the L2 norm normalized is applied to the incomplete embedding representation of each modality. Then, the target embedding vector is mapped to the same unit hypersphere through cross-modal semantic alignment operation to generate the target embedding vector. Finally, the target embedding vectors of all modalities are concatenated by dimension and merged into a unified target embedding matrix.

[0047] Finally, unsupervised clustering of the target embedding matrix is ​​performed using K-means clustering to automatically divide the initial category clusters and generate initial cluster labels. Based on this, a linear classifier is trained using these initial labels as supervision signals to perform inference and prediction for each sample in the embedding matrix, outputting the category probability distribution and predicted category for each sample, thereby ultimately determining the molecular subtype to which each tumor cell belongs and the corresponding category accuracy score.

[0048] See Figure 2 As shown, this embodiment of the invention discloses a biomedical data inference device based on multimodal alignment, comprising: The model acquisition module 11 is used to acquire sample measurement data of each sample modality of the target biomedical sample, determine an initial data inference model for biomedical data inference, and train the initial data inference model using the sample measurement data of each sample modality through the KL divergence regularization mechanism to obtain the target data inference model. Embedded representation output module 12 is used to determine new target biomedical data, obtain initial measurement data of each data modality corresponding to the target biomedical data, construct a corresponding target encoder in the target biomedical data inference model based on each data modality, input the initial measurement data into the target encoder, and output the initial embedded representation of each data modality. Mechanism determination module 13 is used to determine the target dual-channel data mechanism in the target biomedical data inference model; the target dual-channel data mechanism includes an initial embedding channel and an L2 normalized channel; The embedding representation generation module 14 is used to perform cross-modal imputation operation on the initial embedding representation through the initial embedding channel, and to perform multimodal alignment on the initial embedding representation of each data modality using the L2 normalization channel to generate a target embedding vector. The category determination module 15 is used to merge the target embedding vectors of each data modality into a target embedding matrix, and use a linear classifier with a preset clustering algorithm to perform inference based on the target embedding matrix to determine the category of each data in the target biomedical data.

[0049] As described above, this application first acquires measurement data of each modality of the target biomedical sample, trains an initial data inference model using the KL divergence regularization mechanism to obtain the target data inference model, then acquires initial measurement data of each modality of the new target biomedical data, outputs the initial embedding representation of each modality through the corresponding target encoder, and then constrains and imputs the initial embedding representation across modalities using the initial embedding channel of the target dual-channel data mechanism. The imputed embedding representation is then aligned across modalities using the L2 normalization channel of this mechanism, generating target embedding vectors and merging them into a target embedding matrix. Finally, a pre-defined clustering algorithm and a linear classifier are used to infer the category of each sample based on the target embedding matrix and perform evaluation to obtain the clustering results and corresponding evaluation metrics. In this way, while maintaining the ability to compare and align multiple modalities, the introduction of the KL divergence regularization mechanism explicitly constrains the distribution of the embedding space to approximate the standard Gaussian distribution N(0,I), making the latent space have regular and smooth distribution characteristics, improving embedding quality and downstream task performance. Meanwhile, by leveraging a regularized embedding space distribution, the quality and stability of cross-modal imputation / generation are improved, making it possible to infer features from one modality to another. Furthermore, by constraining the embedding distribution to a standard Gaussian distribution, embeddings generated from different training runs and different datasets have comparable probabilistic semantics, improving the interpretability and cross-experimental transferability of the embedding representation.

[0050] In some specific implementations, the model acquisition module 11 may specifically include: An embedding representation generation unit is used to construct a corresponding initial encoder in the initial data inference model based on each of the sample modalities, and use the initial encoders corresponding to each of the sample modalities to map the corresponding sample measurement data to the latent space to generate sample embedding representations. The feature determination unit is used to determine the distribution characteristics of each sample mode based on the sample embedding representation using the initial embedding channel of the initial data inference model. The divergence determination unit is used to determine the target KL divergence between the distribution characteristics of each sample mode and the preset standard Gaussian distribution through the KL divergence regularization mechanism. The first loss acquisition unit is used to average the target KL divergence of each of the sample modes to obtain the target distribution regularization loss. The vector acquisition unit is used to perform L2 normalization on the sample embedding representation through the L2 normalization channel in the initial data inference model to obtain the normalized embedding vector of each sample mode. The vector concatenation unit is used to select any mode from each of the sample modes as the target master mode, and concatenate the normalized embedding vectors of the non-target master modes in the sample modes along the preset sample dimension into an auxiliary embedding matrix. A matrix determination unit is used to determine the target similarity matrix between the normalized embedding vector of the target main modality and the auxiliary embedding matrix; A matrix construction unit is used to construct a target soft label matrix based on a preset relationship between each of the sample modalities; The loss determination unit is used to determine the target cross-entropy loss corresponding to the target main modality based on the target similarity matrix and the target soft label matrix; The second loss acquisition unit is used to sum the target cross-entropy loss of each of the sample modes to obtain the target contrast loss; The third loss acquisition unit is used to perform a weighted summation of the target distribution regularization loss and the target comparison loss to obtain the total loss. The model acquisition unit is used to optimize and adjust the initial data inference model based on the total loss to obtain the target data inference model.

[0051] In some specific embodiments, the embedded representation generation module 14 may specifically include: The representation acquisition unit is used to perform cross-modal distribution alignment constraints on the initial embedding representations output by each target encoder, so as to map the initial embedding representations of each data mode to the target latent space and satisfy the preset distribution format, thereby obtaining the corresponding constrained embedding representations. A decoder determination unit is used to determine the target decoder corresponding to each of the target modes; The feature value generation unit is used to decode each of the constrained post-embedded representations using the decoder corresponding to the target mode, and generate the predicted feature value corresponding to the target mode; The data acquisition unit is used to fill in the initial measurement data in the target biomedical data that meets the preset data missing conditions based on the generated predicted feature values, so as to obtain the filled measurement data. An embedding representation generation unit is used to encode the filled measurement data using the target encoder to generate a filled target embedding representation.

[0052] In some specific embodiments, the embedded representation generation module 14 may specifically include: The data normalization unit is used to call the L2 normalization channel in the target dual-channel data mechanism to perform L2 norm normalization processing on the filled-in embedded representation of each data mode to generate the normalized embedded representation of each data mode. This represents a cross-modal alignment unit, used to map the normalized embedding representations of each of the data modalities to the target latent space, in order to perform cross-modal semantic alignment operations and generate semantic target embedding vectors; The vector generation unit is used to perform dimensional concatenation or weighted fusion on the semantic target embedding vectors of each of the data modalities to generate aligned target embedding vectors.

[0053] In some specific implementations, the category determination module 15 may specifically include: A hypersphere generation unit is used to project the target embedding vector of each of the data modes onto a unit hypersphere template to generate a target hypersphere. The vector transformation unit is used to transform and merge the embedding vectors in the target hyperplane sphere into a target embedding matrix.

[0054] In some specific implementations, the category determination module 15 may specifically include: The label generation unit is used to perform unsupervised clustering analysis on the target embedding matrix using a preset clustering algorithm, automatically divide the target biomedical samples into initial category clusters, and generate initial clustering labels corresponding to each sample in the target biomedical samples. The category probability output unit is used to perform inference and prediction on each sample in the target embedding matrix based on the initial clustering label through a linear classifier, and output the category probability distribution and predicted category corresponding to each sample; The category determination unit is used to determine the category of each data in the target biomedical data based on the category probability distribution and the predicted category.

[0055] In some specific embodiments, the biomedical data inference device based on multimodal alignment may further include: The first indicator determination module is used to determine the target F1 score of each data category in the target biomedical data, take the average of each target F1 score to generate the target score average, and determine the corresponding classification evaluation indicator based on the target score average. The second indicator determination module is used to obtain the target total variation distance of each sample in the target biomedical sample, take the average value of each target total variation distance to generate the target distance average value, and determine the corresponding cross-modal alignment evaluation index based on the target distance average value. The index generation module is used to generate the clustering results of each sample in the target biomedical sample and the target adjusted Rand index with preset labels, so as to use the target adjusted Rand index as a clustering evaluation index. The third indicator determination module is used to determine the corresponding result evaluation indicators of the clustering results based on the classification evaluation indicators, cross-modal alignment evaluation indicators, and clustering evaluation indicators.

[0056] Furthermore, embodiments of this application also disclose an electronic device, Figure 3This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the biomedical data inference method based on multimodal alignment disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0057] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0058] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon can include an operating system 221, computer programs 222, etc., and the storage method can be temporary storage or permanent storage.

[0059] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the multimodal alignment-based biomedical data inference method disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0060] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned biomedical data inference method based on multimodal alignment. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0061] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0062] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0063] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0064] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0065] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A biomedical data reasoning method based on multimodal alignment, characterized in that, include: Acquire sample measurement data for each sample modality of the target biomedical sample, determine an initial data inference model for biomedical data inference, and train the initial data inference model using the sample measurement data for each sample modality through the KL divergence regularization mechanism to obtain the target data inference model. New target biomedical data is identified, and initial measurement data of each data modality corresponding to the target biomedical data is obtained. Based on each data modality, a corresponding target encoder is constructed in the target biomedical data inference model. The initial measurement data is input into the target encoder to output the initial embedding representation of each data modality. Determine the target dual-channel data mechanism in the target biomedical data inference model; the target dual-channel data mechanism includes an initial embedding channel and an L2 normalized channel; The initial embedding representation is subjected to cross-modal imputation through the initial embedding channel, and the initial embedding representation of each data modality is multimodal aligned using the L2 normalization channel to generate the target embedding vector. The target embedding vectors of each data modality are merged into a target embedding matrix. A linear classifier is used with a preset clustering algorithm, and reasoning is performed based on the target embedding matrix to determine the category of each data in the target biomedical data.

2. The biomedical data inference method based on multimodal alignment according to claim 1, characterized in that, The step of training the initial data inference model using the sample measurement data of each of the sample modalities through the KL divergence regularization mechanism to obtain the target data inference model includes: Based on each of the sample modalities, a corresponding initial encoder is constructed in the initial data inference model. The corresponding sample measurement data is mapped to the latent space using the initial encoders corresponding to each of the sample modalities to generate sample embedding representations. Using the initial embedding channel of the initial data inference model, the distribution characteristics of each sample mode are determined based on the sample embedding representation; The target KL divergence between the distribution characteristics of each sample mode and the preset standard Gaussian distribution is determined by the KL divergence regularization mechanism. The target KL divergence of each of the sample modes is averaged to obtain the target distribution regularization loss; The L2 normalization channel in the initial data inference model is used to perform L2 normalization on the sample embedding representation to obtain the normalized embedding vector of each sample mode. Select any mode from each of the sample modes as the target master mode, and concatenate the normalized embedding vectors of the non-target master modes in the sample modes along the preset sample dimension to form an auxiliary embedding matrix; Determine the target similarity matrix between the normalized embedding vector of the target main modality and the auxiliary embedding matrix; Construct a target soft label matrix based on the preset relationships of each of the sample modalities; The target cross-entropy loss corresponding to the target dominant mode is determined based on the target similarity matrix and the target soft label matrix. The target contrast loss is obtained by summing the target cross-entropy loss for each of the sample modes; The total loss is obtained by weighted summing the target distribution regularization loss and the target contrast loss. The initial data inference model is optimized and adjusted based on the total loss to obtain the target data inference model.

3. The biomedical data inference method based on multimodal alignment according to claim 1, characterized in that, The initial embedding representation is subjected to cross-modal imputation via the initial embedding channel to generate a target embedding vector, including: Cross-modal distribution alignment constraints are applied to the initial embedding representations output by each target encoder to map the initial embedding representations of each data modality to the target latent space and satisfy a preset distribution format, thereby obtaining the corresponding constrained embedding representations. Determine the target decoder corresponding to each of the target modes; The decoder corresponding to the target mode is used to decode each of the constraint-embedded representations to generate the predicted feature value corresponding to the target mode; Based on the generated predicted feature values, the initial measurement data in the target biomedical data that meet the preset data missing conditions are filled in to obtain the filled measurement data. The infilled measurement data is encoded using the target encoder to generate an infilled target embedding representation.

4. The biomedical data inference method based on multimodal alignment according to claim 3, characterized in that, The step of using the L2 normalized channel to perform multimodal alignment on the initial embedding representation of each data modality to generate a target embedding vector includes: The L2 normalization channel in the target dual-channel data mechanism is invoked to perform L2 norm normalization processing on the initial embedding representation of each data mode, so as to generate the normalized embedding representation of each data mode. The normalized embedding representations of each of the data modalities are mapped to the target latent space to perform cross-modal semantic alignment operations and generate semantic target embedding vectors. The semantic target embedding vectors of each data modality are dimensionally concatenated or weighted and fused to generate aligned target embedding vectors.

5. The biomedical data inference method based on multimodal alignment according to claim 1, characterized in that, The step of merging the target embedding vectors of each data modality into a target embedding matrix includes: The target embedding vector of each of the data modes is projected onto the unit hypersphere template to generate the target hypersphere; The embedding vectors in the target hypersphere are transformed and merged into a target embedding matrix.

6. The biomedical data inference method based on multimodal alignment according to claim 1, characterized in that, The step of determining the category of each data point in the target biomedical data by using a linear classifier and reasoning based on the target embedding matrix through a preset clustering algorithm includes: An unsupervised clustering analysis is performed on the target embedding matrix using a preset clustering algorithm to automatically divide the target biomedical samples into initial category clusters, thereby generating initial clustering labels for each sample in the target biomedical samples. Based on the initial clustering labels, a linear classifier is used to infer and predict each sample in the target embedding matrix, and the category probability distribution and predicted category corresponding to each sample are output. The category of each data point in the target biomedical data is determined based on the category probability distribution and the predicted category.

7. The biomedical data inference method based on multimodal alignment according to any one of claims 1 to 6, characterized in that, After determining the category of each data point in the target biomedical data by using a linear classifier with a preset clustering algorithm and reasoning based on the target embedding matrix, the method further includes: Determine the target F1 score for each category of the target biomedical data, average the target F1 scores to generate the target score average, and determine the corresponding classification evaluation index based on the target score average. The target total variation distance of each sample in the target biomedical sample is obtained, and the average value of each target total variation distance is taken to generate the target distance average value. Based on the target distance average value, the corresponding cross-modal alignment evaluation index is determined. Generate clustering results and target adjusted Rand indexes for each sample in the target biomedical sample with preset labels, and use the target adjusted Rand indexes as clustering evaluation indicators. The corresponding result evaluation indicators for the clustering results are determined based on the classification evaluation indicators, cross-modal alignment evaluation indicators, and clustering evaluation indicators.

8. A biomedical data inference device based on multimodal alignment, characterized in that, include: The model acquisition module is used to acquire sample measurement data of each sample modality of the target biomedical sample, determine an initial data inference model for biomedical data inference, and train the initial data inference model using the sample measurement data of each sample modality through the KL divergence regularization mechanism to obtain the target data inference model. An embedding representation output module is used to determine new target biomedical data and obtain initial measurement data of each data modality corresponding to the target biomedical data. Based on each data modality, a corresponding target encoder is constructed in the target biomedical data inference model. The initial measurement data is input into the target encoder to output the initial embedding representation of each data modality. A mechanism determination module is used to determine the target dual-channel data mechanism in the target biomedical data inference model; the target dual-channel data mechanism includes an initial embedding channel and an L2 normalized channel. An embedding representation generation module is used to perform cross-modal imputation on the initial embedding representation through the initial embedding channel, and to perform multimodal alignment on the initial embedding representation of each data modality using the L2 normalization channel to generate a target embedding vector. The category determination module is used to merge the target embedding vectors of each data modality into a target embedding matrix, and use a linear classifier with a preset clustering algorithm to perform inference based on the target embedding matrix to determine the category of each data in the target biomedical data.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the biomedical data inference method based on multimodal alignment as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the biomedical data reasoning method based on multimodal alignment as described in any one of claims 1 to 7.

Images