Data processing method and device based on deep learning, equipment, medium and product

Abstract

The invention relates to a data processing method and device based on deep learning, equipment, a medium and a product. The method comprises the following steps: acquiring medical health data samples, and acquiring domain features which correspond to each medical health data sample and are used for reflecting semantic information; extracting sample features of the medical health data samples through a to-be-trained model; determining positive and negative pairs based on the domain features corresponding to the medical health data samples, and determining comparison loss according to the features of each sample in the positive and negative pairs; wherein the positive and negative pairs comprise at least one of a first positive and negative pair used for learning a local semantic relationship or a second positive and negative pair used for learning a global semantic relationship, and features of at least part of examples in the positive and negative pairs are obtained based on sample features; and training the to-be-trained model through the comparison loss, wherein the trained model is used for extracting the features of the medical health data. By adopting the method, the model training effect can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product based on deep learning. Background Art

[0002] With the development of artificial intelligence technology, deep learning algorithms have been increasingly widely used. For example, deep learning algorithms can automatically extract semantic information from raw data to achieve more accurate classification and prediction. At present, methods that use deep learning algorithms and are based on large-scale annotated data sets have achieved performance levels comparable to or even exceeding those of domain experts.

[0003] Generally, in the field of medical health, it is a major challenge to convert medical health data collected by detection equipment into actionable information suitable for back-end applications. In the field of medical health, manual annotation of medical health data collected by detection equipment (such as physiological signals collected by wearable devices, such as ECG (electrocardiogram) signals or EEG (brain wave) signals) consumes a lot of manpower and time costs, and usually requires a lot of clinical expertise. This has greatly hindered the expansion of the use of deep learning in healthcare applications. Therefore, traditional technologies often use SSCL (Self-Supervised Contrastive Learning) methods to avoid the inconvenience of manual annotation.

[0004] However, traditional SSCL methods often ignore key nuances in medical and health data and treat all samples equally, which may miss important relationships between data points that share similar medical concepts and lead to poor model training results. Summary of the invention

[0005] Based on this, it is necessary to provide a deep learning-based data processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the model training effect in response to the above technical problems.

[0006] In a first aspect, the present application provides a data processing method based on deep learning, comprising:

[0007] Acquire medical and health data samples, and obtain domain features corresponding to each medical and health data sample for reflecting semantic information;

[0008] Extracting sample features of each of the medical and health data samples through the model to be trained;

[0009] Determine positive and negative pairs based on the domain features corresponding to the medical and health data samples, and determine the contrast loss according to the features of each sample in the positive and negative pairs; wherein the positive and negative pairs include at least one of a first positive and negative pair for learning a local semantic relationship or a second positive and negative pair for learning a global semantic relationship, and the features of at least some samples in the positive and negative pairs are obtained based on the sample features;

[0010] The model to be trained is trained by using the contrast loss; the model obtained after the training is used to extract features of medical health data.

[0011] In a second aspect, the present application also provides a data processing device based on deep learning, comprising:

[0012] An acquisition module is used to acquire medical and health data samples and obtain domain features corresponding to each medical and health data sample for reflecting semantic information;

[0013] An extraction module, used to extract sample features of each of the medical and health data samples through a model to be trained;

[0014] A determination module, configured to determine positive and negative pairs based on domain features corresponding to the medical and health data samples, and determine contrast loss according to features of each sample in the positive and negative pairs; wherein the positive and negative pairs include at least one of a first positive and negative pair for learning a local semantic relationship or a second positive and negative pair for learning a global semantic relationship, and features of at least some samples in the positive and negative pairs are obtained based on the sample features;

[0015] A training model is used to train the model to be trained through the contrast loss; the model obtained after the training is used to extract the characteristics of medical health data.

[0016] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0017] Acquire medical and health data samples, and obtain domain features corresponding to each medical and health data sample for reflecting semantic information;

[0018] Extracting sample features of each of the medical and health data samples through the model to be trained;

[0019] Determine positive and negative pairs based on the domain features corresponding to the medical and health data samples, and determine the contrast loss according to the features of each sample in the positive and negative pairs; wherein the positive and negative pairs include at least one of a first positive and negative pair for learning a local semantic relationship or a second positive and negative pair for learning a global semantic relationship, and the features of at least some samples in the positive and negative pairs are obtained based on the sample features;

[0020] The model to be trained is trained by using the contrast loss; the model obtained after the training is used to extract features of medical health data.

[0021] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0022] Acquire medical and health data samples, and obtain domain features corresponding to each medical and health data sample for reflecting semantic information;

[0023] Extracting sample features of each of the medical and health data samples through the model to be trained;

[0024] Determine positive and negative pairs based on the domain features corresponding to the medical and health data samples, and determine the contrast loss according to the features of each sample in the positive and negative pairs; wherein the positive and negative pairs include at least one of a first positive and negative pair for learning a local semantic relationship or a second positive and negative pair for learning a global semantic relationship, and the features of at least some samples in the positive and negative pairs are obtained based on the sample features;

[0025] The model to be trained is trained by using the contrast loss; the model obtained after the training is used to extract features of medical health data.

[0026] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0027] Acquire medical and health data samples, and obtain domain features corresponding to each medical and health data sample for reflecting semantic information;

[0028] Extracting sample features of each of the medical and health data samples through the model to be trained;

[0029] Determine positive and negative pairs based on the domain features corresponding to the medical and health data samples, and determine the contrast loss according to the features of each sample in the positive and negative pairs; wherein the positive and negative pairs include at least one of a first positive and negative pair for learning a local semantic relationship or a second positive and negative pair for learning a global semantic relationship, and the features of at least some samples in the positive and negative pairs are obtained based on the sample features;

[0030] The model to be trained is trained by using the contrast loss; the model obtained after the training is used to extract features of medical health data.

[0031] The above-mentioned data processing method, device, computer equipment, computer-readable storage medium and computer program product based on deep learning obtain the domain features corresponding to each medical and health data sample for reflecting semantic information, wherein the domain features are based on the domain knowledge of clinical insights that have long guided data interpretation and feature engineering, and use the domain features to guide the selection of positive and negative pairs in the comparative learning process, so that the model can learn the local semantic relationship and / or global semantic relationship between samples. Among them, the local semantic relationship between samples can be understood as the semantic relationship between individuals (i.e., between samples), and the global semantic relationship between samples can be understood as the semantic relationship between individuals and clusters (i.e., between samples and sample clusters). In this way, the training effect of the model is improved, so that when the model extracts the features of medical and health data, it can well represent the semantic information therein in the feature space, thereby extracting high-quality features. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 An application environment diagram of a data processing method based on deep learning in one embodiment;

[0034] Figure 2 is a flowchart of a data processing method based on deep learning in one embodiment;

[0035] Figure 3 A flowchart of determining positive and negative pairs based on domain features corresponding to medical and health data samples and determining contrast loss according to features of each sample in the positive and negative pairs in one embodiment;

[0036] Figure 4 A flowchart of another embodiment of determining positive and negative pairs based on domain features corresponding to medical and health data samples, and determining contrast loss according to features of each sample in the positive and negative pairs;

[0037] Figure 5 A schematic diagram of a data processing method based on deep learning in one embodiment;

[0038] Figure 6 A schematic diagram of performance comparison of different methods in terms of class average F1, accuracy, AUC, recall rate and precision on different medical and health data in one embodiment;

[0039] Figure 7 is a schematic diagram of visualization results of waveform significance graphs of two representative samples in one embodiment;

[0040] Figure 8 t-SNE plot showing feature representations derived from a self-supervised contrastive learning backbone for ECG, IMU, and EEG data in one embodiment;

[0041] Fig. 9 is a schematic diagram of comparison results of semi-supervised learning based on ECG datasets in one embodiment;

[0042] Fig.10 A schematic diagram showing the importance of different domain features to direct logistic regression and the framework scheme of the present application in one embodiment;

[0043] Fig.11 is a structural block diagram of a data processing device based on deep learning in one embodiment;

[0044] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] In the field of artificial intelligence, a promising solution to the problem of scarce labeled data when it comes to model training is self-supervised learning (SSL), which reduces the reliance on annotations by learning useful representations from raw unlabeled data. In healthcare applications, SSL can simplify the analysis of medical health data collected by detection equipment, enabling faster and more accurate analysis / diagnosis without labor-intensive data labeling. Among traditional SSL techniques, contrastive learning has been particularly successful. Among SSL techniques, contrastive learning is popular for its ability to align similar pairs and alienate dissimilar pairs to learn representations.

[0047] Generally speaking, self-supervised contrastive learning requires identifying positive and negative pairs without direct access to downstream labels. Therefore, inappropriate or suboptimal pairing when applying self-supervised contrastive learning may lead to conflicts with the optimization objectives, which may limit the ability of the model to effectively generalize to real-world healthcare tasks. Various strategies have been proposed to alleviate this problem, including data augmentation techniques (such as adding noise or time flipping) and sampling strategies that group positive pairs based on shared medical concepts. Typical examples of the latter include treating samples from the same patient or from adjacent time frames as positive pairs. However, these methods often introduce inductive biases that may conflict with the underlying medical semantics of the data. For example, two electrocardiogram recordings of the same patient may differ greatly due to a sudden cardiac event, while recordings from different patients may have similar features if both patients are healthy. Therefore, traditional SSCL methods may not be consistent with the semantic knowledge required for practical downstream healthcare applications.

[0048] Unlike the success of deep learning, traditional supervised learning methods rely heavily on hand-crafted features based on specific domain knowledge. Data analysts (usually involving domain experts (e.g., clinicians)) design and extract these features by applying domain-specific expertise, who examine the data or use "textbook-level" empirical knowledge to interpret the signal. These domain features are carefully crafted by extensive reliance on domain expertise, encoding meaningful and clinically relevant concepts that are unique to the medical and health field. Although deep learning methods have far surpassed these traditional techniques in many aspects, domain features based on domain knowledge still have important semantic information and have great potential in guiding pair selection in self-supervised contrastive learning. We believe that while advanced self-supervised learning solutions, especially "new-school" contrastive methods for efficient learning of labels, have become increasingly popular, "old-school" domain knowledge-based feature engineering is still crucial for extracting valuable information from medical and health data.

[0049] Based on this, this application develops a new framework that combines and utilizes domain features that reflect semantic information to guide self-supervised contrastive learning. Specifically, when forming positive and negative pairs for self-supervised contrastive learning, domain knowledge of clinical insights that have long guided data interpretation and feature engineering is incorporated into self-supervised contrastive learning to help train the model. We tested our method in various medical and health testing devices (including cardiac, neural, and activity monitors) and found that it consistently outperformed traditional SSCL techniques (see below for detailed experimental details). These findings point to a novel and critical direction for developing efficient and scalable learning models in the field of digital health, highlighting the enduring value of domain-specific expertise in modern machine learning frameworks.

[0050] The data processing method based on deep learning provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the computer device 102 communicates with the electronic device 104 and / or the medical health detection device 106 through the network. The data storage system can store the data that the computer device 102 needs to process. The data storage system can be integrated on the computer device 102, or it can be placed on the cloud or other network servers. Exemplarily, a model to be trained can be pre-deployed in the computer device 102, and the computer device 102 can obtain medical and health data samples from at least one electronic device 104 and / or detection device 106 through network communication, and then extract the sample features of each medical and health data sample through the model to be trained; determine the positive and negative pairs based on the domain features corresponding to the medical and health data samples, and determine the contrast loss according to the features of each sample in the positive and negative pairs; train the model to be trained by contrast loss, and the model obtained after the training is completed is used to extract the features of medical and health data. Among them, the medical and health data can be collected by the detection device 106 or collected by the electronic device 104 and transmitted to the computer device 102.

[0051] Among them, the computer device 102 can be a terminal and / or a server. The terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The electronic device 104 can be a terminal and / or a server. The detection device 106 can be a medical imaging device or a signal measurement device, etc., which has the ability to detect biological medical health data. The medical imaging device can specifically be an X-ray device, an ultrasound imaging device, a magnetic resonance imaging device, etc.; the signal measurement device can specifically be a desktop measurement instrument or a wearable device, etc.

[0052] In an exemplary embodiment, Figure 2 As shown, a data processing method based on deep learning is provided, which is applied to Figure 1 The computer device 102 in the example is used as an example to illustrate, including the following steps 202 to 208. Among them:

[0053] Step 202: Acquire medical and health data samples, and obtain domain features corresponding to each medical and health data sample for reflecting semantic information.

[0054] Among them, the medical and health data samples are sample data used to train the model to be trained, and the sample data can specifically be medical and health data. Medical and health data is data that reflects the medical condition and / or health condition of the organism, and can specifically include at least one of the physiological signals, activity signals, medical images or medical texts of the organism. Among them, the physiological signal is a signal used to reflect the physiological state of the organism, such as electrocardiogram (such as electrocardiogram), electroencephalogram (such as electroencephalogram), electromyogram, pulse signal, etc. The activity signal is collected by sensors such as accelerometers and gyroscopes, and reflects the data of the external movement or behavior of the organism, such as the number of steps, walking, running, sitting, etc. Medical images are internal tissue images obtained in a non-invasive manner for a certain part of the organism, and can specifically include anatomical structure images (such as X-ray images, computed tomography (CT) images, magnetic resonance imaging (MRI) images, etc.), functional imaging data (such as positron emission tomography (PET) images, single photon emission computed tomography (SPECT) images, etc.), ultrasound imaging data, etc. Medical text is text data that describes biological and medical conditions, which may include medical records, test reports, etc., and its format can be structured or unstructured.

[0055] For example, if the medical health data sample Including physiological signal samples, the physiological signal samples of the present application can specifically be time series data collected from different medical care modes of signal measurement equipment. Consider an unlabeled data set, which is a collection of physiological signal samples, and express it as , from time series data Composition, of which Indicates the number of signal channels. represents the time step, Indicates the number of samples.

[0056] The model to be trained can be a machine learning model to be trained, such as a deep model to be trained. The model to be trained in the present application at least includes a neural network for feature extraction. The neural network for feature extraction can be a transformer-based neural network or a neural network based on other structures. The present application embodiment does not limit its structure. In some applications, the goal of the present application is to develop a feature encoder. (That is, the "model" trained in this article) (where, represents the real space dimension of feature z), which is capable of extracting semantically meaningful representations for downstream healthcare-related tasks after training.

[0057] In some embodiments, the medical health data sample includes at least an original sample. In some cases, the medical health data sample includes an original sample and an enhanced sample. Among them, the original sample is a medical health data sample directly collected by a detection device, and the enhanced sample is a sample obtained after the original sample is enhanced. The enhancement processing can specifically be cropping, rotation, masking, adding noise or time flipping, etc., which is not limited in the embodiments of the present application.

[0058] Domain features are features extracted based on specific domain knowledge. In this application, they can be features extracted based on medical domain knowledge. It should be noted that in order to use domain knowledge to improve the contrastive learning framework, for each medical health data sample , computer equipment can extract domain features based on domain knowledge These domain features are selected based on their relevance to the specific domain knowledge of the corresponding modality and are identified as important features based on specific domain expertise.

[0059] In some embodiments, these domain features (covering at least one of morphological, temporal, spectral, and signal quality dimensions, among others) have been developed and standardized and can be easily accessed through readily available tools and software packages.

[0060] It should be noted that the model training process usually involves multiple training rounds (epochs), and multiple iterations (iterations) are performed in each training round. In each iteration, a batch of samples are extracted based on the training data set for processing to update the parameters of the model (such as weights and / or biases). The entire training process is a continuous iterative cycle. It can be understood that the computer device can obtain the full amount of training data sets in advance and extract domain features for each medical and health data sample in the training data set. Then, in each iteration of each round of the model training process, the medical and health data samples and corresponding domain features required for the current iteration can be obtained from it, and the subsequent steps are performed to determine the contrast loss to train the model through the contrast loss. When the iteration is completed, the next iteration will be performed until the current round of training is completed. After the current round of training is completed, the next round will be entered, and this cycle will continue until the training end condition is reached and the trained model is obtained. The following will be described in detail for any iteration in any round of the training process.

[0061] Step 204: extracting sample features of each medical and health data sample through the model to be trained.

[0062] Specifically, each medical and health data sample (of the current batch) may be input into the model to be trained, and features of each medical and health data sample may be extracted by the model to be trained, where the extracted features may be referred to as sample features. In some embodiments, the computer device may further transform the sample features extracted by the model for subsequent loss determination. Exemplarily, the further transformation may specifically be projection and normalization processing.

[0063] Step 206, determine positive and negative pairs based on the domain features corresponding to the medical and health data samples, and determine the contrast loss according to the features of each sample in the positive and negative pairs; wherein the positive and negative pairs include at least one of a first positive and negative pair for learning local semantic relationships or a second positive and negative pair for learning global semantic relationships, and the features of at least some of the samples in the positive and negative pairs are obtained based on the sample features.

[0064] Among them, the positive and negative pairs include positive pairs and negative pairs. In the present application, positive pairs are samples that have semantic similarity to each other, and negative pairs are samples that do not have semantic similarity to each other. In some embodiments, the positive and negative pairs include at least one of a first positive and negative pair for learning local semantic relationships or a second positive and negative pair for learning global semantic relationships. Among them, the first positive and negative pairs include a first positive pair and a first negative pair. The first positive pair may include a medical and health data sample and a sample similar to it; the first negative pair may include a medical and health data sample and a sample different from it. The second positive and negative pairs include a second positive pair and a second negative pair. The second positive pair may include a medical and health data sample and a prototype of the cluster to which the sample belongs, and the second negative pair may include a medical and health data sample and a prototype of other clusters. Among them, the cluster can be obtained by clustering the full amount of medical and health data samples based on the domain features corresponding to each medical and health data sample in the full amount of medical and health data samples. The prototype of the cluster may refer to the centroid of the cluster or the centroid after iterative update.

[0065] Specifically, the computer device may determine positive and negative pairs based on the domain features corresponding to the medical and health data samples, thereby constructing a contrast loss according to the features of each sample in the positive and negative pairs. The contrast loss is used to minimize the distance between each sample in the positive pair and maximize the distance between each sample in the negative pair.

[0066] In some embodiments, if the positive-negative pair includes only the first positive-negative pair, or only the second positive-negative pair, the contrast loss can be constructed based on the features of each sample in the first positive-negative pair or the features of each sample in the second positive-negative pair. If the positive-negative pair includes the first positive-negative pair and the second positive-negative pair, the contrast loss can be obtained by combining the loss determined by the features of each sample in the first positive-negative pair and the loss determined by the features of each sample in the second positive-negative pair.

[0067] Step 208, training the model to be trained by contrast loss; the model obtained after training is used to extract features of medical health data.

[0068] Specifically, the computer device can iteratively train the model to be trained by contrast loss to continuously adjust the network parameters (such as weights and / or biases) of the model, stop training after the training end condition is met, and obtain a trained model, which can be put into use to extract the features of the medical and health data to be processed. Among them, the training end condition is the condition for ending the model training, which can specifically be reaching a preset training round or the model performance reaching a preset performance, etc.

[0069] As mentioned above, the model training process involves multiple training epochs, and multiple iterations are performed in each training epoch. The above steps are the steps required to be performed in one iteration. After one iteration is completed, the next iteration will be performed until the current epoch of training is completed. After the current epoch of training is completed, the next epoch will be entered. This cycle will continue until the training end condition is met, and a trained model will be obtained.

[0070] In some embodiments, the model trained by the present application can be applied to downstream tasks, for example, the model obtained by the present application can extract semantically meaningful features for downstream healthcare-related tasks. Exemplarily, these downstream tasks may include at least one of electrocardiogram-based cardiovascular disease classification, electroencephalogram-based sleep staging, or IMU (Inertial Measurement Unit)-based activity recognition.

[0071] The above-mentioned data processing method based on deep learning obtains the domain features corresponding to each medical and health data sample for reflecting semantic information, wherein the domain features are based on the domain knowledge of clinical insights that have long guided data interpretation and feature engineering, and use the domain features to guide the selection of positive and negative pairs in the comparative learning process, so that the model can learn the local semantic relationship and / or global semantic relationship between samples. Among them, the local semantic relationship between samples can be understood as the semantic relationship between individuals (i.e., between samples), and the global semantic relationship between samples can be understood as the semantic relationship between individuals and clusters (i.e., between samples and sample clusters). This improves the training effect of the model, so that when the model extracts the features of medical and health data, it can well represent the semantic information in the feature space, thereby extracting high-quality features.

[0072] It can be understood that in actual implementation, the domain features of each medical and health data sample in the training data set can be extracted in advance for subsequent model training. If the contrast loss also involves prototype loss, it is also possible to cluster the domain features of each medical and health data sample in advance to obtain multiple clusters for subsequent prototype-level contrast learning.

[0073] In some embodiments, domain features corresponding to each medical health data sample for reflecting semantic information are obtained, including: for any medical health data sample, extracting features of the medical health data sample or at least one of the cross-modal data samples associated with the medical health data sample on a preset dimension to obtain the domain features corresponding to the medical health data sample; wherein the preset dimension is determined based on domain knowledge.

[0074] Among them, the cross-modal data sample is a data sample whose modality is different from that of the medical health data sample. For example, when the medical health data sample is a medical imaging sample, its associated cross-modal data sample can specifically be at least one of a physiological signal sample, an activity signal sample or a medical text sample; when the medical health data sample is a physiological signal sample, its associated cross-modal data sample can specifically be at least one of an activity signal sample, a medical imaging sample or a medical text sample; when the medical health data sample is an activity signal sample, its associated cross-modal data sample can specifically be at least one of a physiological signal sample, a medical imaging sample or a medical text sample; when the medical health data sample is a medical text sample, its associated cross-modal data sample can specifically be at least one of a medical imaging sample, an activity signal sample or a physiological signal sample.

[0075] In some embodiments, for any biological object, multiple modal data from the biological object can be collected in advance to be used as training samples, such as collecting medical images, physiological signals, activity signals, and medical texts from the same biological object, and these multimodal data are stored in association. Exemplarily, for cardiovascular diseases, multiple modal data of the same biological object (such as patients with cardiovascular diseases) can be collected in advance, such as medical images (cardiac ultrasound), physiological signals (electrocardiogram), and medical texts (corresponding reports of cardiac ultrasound and electrocardiogram). Therefore, when necessary, cross-modal domain features can be applied and modeled based on the multimodal data corresponding to a large number of biological objects stored in association.

[0076] The following will provide a detailed description of the application and modeling of single mode and the application and modeling of multi-mode:

[0077] For single-modal applications and modeling: In some embodiments, for each medical and health data sample, the computer device can extract the features of the medical and health data sample on the preset dimensions of the corresponding modality and obtain the domain features of the medical and health data sample. When there are multiple preset dimensions, the features of multiple preset dimensions can be combined to obtain the domain features, such as splicing the features of each preset dimension in a preset order to obtain the domain features of the medical and health data sample. In this way, professional domain knowledge can be combined to extract domain features in the medical and health data sample that can reflect its intrinsic semantic information, which is convenient for guiding the selection of positive and negative pairs in the subsequent training process.

[0078] In other embodiments, for each medical and health data sample, the computer device may search for any cross-modal data sample associated with the medical and health data sample from a storage medium (such as a database or memory, etc.), and then extract the features of the cross-modal data sample on the preset dimension of the corresponding modality to obtain the domain features corresponding to the medical and health data sample. When there are multiple preset dimensions, the domain features can be combined with the features of multiple preset dimensions to obtain the domain features of the medical and health data sample, such as splicing the features of each preset dimension in a preset order to obtain the domain features of the medical and health data sample. In this way, it is possible to combine professional domain knowledge to extract the domain features that can reflect its inherent semantic information in any cross-modal data sample of the medical and health data sample, so as to facilitate the selection of positive and negative pairs in the subsequent training process. It can be understood that when the domain features cannot be extracted based on the medical and health data sample itself, or the domain features extracted based on the medical and health data sample itself are not accurate enough, by extracting the features of the cross-modal data sample associated with the medical and health data sample as the domain features, the defects caused by this situation can be well compensated, and the adaptability of the scheme is greatly improved.

[0079] For multimodal applications and modeling: In some embodiments, for any medical health data sample, the computer device can extract domain features on at least two modalities, and combine the domain features on at least two modalities to obtain the domain features corresponding to the medical health data sample. It can be understood that the at least two modalities may include the modality of the medical health data sample itself, or at least two modalities of the modality of the corresponding cross-modal data sample; the combination method can be splicing or fusion, etc., which is not limited to the embodiments of the present application. In this way, the domain features obtained by combining the professional domain knowledge of multiple modalities can more comprehensively reflect the intrinsic semantic information of the medical health data sample, which is convenient for guiding the selection of positive and negative pairs in the subsequent training process.

[0080] It should be noted that the preset dimensions of different modalities may be different, and the preset dimensions of the corresponding modality are determined based on the domain knowledge of the corresponding modality.

[0081] In some embodiments, the medical health data samples and / or cross-modal data samples may be any one or more of physiological signal samples, activity signal samples, medical image samples, and medical text samples. The following is an exemplary description of extracting features from physiological signal samples, activity signal samples, medical image samples, and medical text samples to obtain domain features. It can be understood that the domain features of the present application can be extracted based on at least one modality sample among physiological signal samples, activity signal samples, medical image samples, or medical text samples.

[0082] In some embodiments, the physiological signal samples include electrocardiogram samples, electroencephalogram samples, etc. The domain features extracted from the electrocardiogram samples may include at least one of AF (atrial fibrillation) related features, morphological features, RR interval related features, beat similarity related features, etc.; the domain features extracted from the electroencephalogram samples may include at least one of time domain statistical features, time domain feature entropy, frequency domain statistical features, frequency domain feature entropy, specific frequency band spectrum energy, etc.

[0083] In some embodiments, the activity signal samples include data collected by sensors such as accelerometers and gyroscopes, and the domain features extracted from the activity signal samples may include at least one of time statistical features, angle features, frequency domain features, peak features, etc.

[0084] In some embodiments, the medical image samples include ultrasound imaging data, magnetic resonance imaging images, etc. For example, when the medical image samples are cardiac ultrasound imaging data, the domain features extracted from the cardiac ultrasound imaging data may include the size of each atrium and ventricle, the maximum value of the ventricular septum thickness, the ascending aorta, the E peak, the maximum regurgitant pressure difference of the tricuspid valve, aortic valve degeneration, etc.; when the medical image samples are brain nuclear magnetic resonance images, the domain features extracted from the brain nuclear magnetic resonance images may include the size of gray and white matter of the brain, the thickness of the cerebral cortex, the distribution of cerebrospinal fluid, the perfusion flow rate, the functional connection, the intrinsic coherence volume fraction, the average diffusion rate, the isotropic volume fraction, etc.; when the medical image samples are lung CT, the domain features extracted from the lung CT may include lung density (HU (Hounsfield Unit) value distribution), lung volume, pulmonary artery diameter, lung nodule size, etc.

[0085] In some embodiments, the domain features extracted from the medical text samples may include diagnostic conclusions, local and / or global word / term frequencies, etc. These features may be obtained by processing an NLP (Natural Language Processing) model. Among them, the local word / term frequency may be extracted by the word2vec model (a related model for generating word vectors), and the global word / term frequency may be extracted by the Gram model (a statistical language model). In addition, in an embodiment of obtaining domain features based on medical text samples, we can use the similarity of medical text samples (such as test reports) to define the distance between patients, thereby determining positive and negative pairs, and performing clustering. For example, the similarity calculation method may include BLEU (Bilingual Evaluation Understudy) scores, cosine similarity, and Jaccard distance, etc.

[0086] In the above embodiment, combined with professional domain knowledge, the features of at least one modality sample can be extracted to obtain the domain features corresponding to the medical and health data samples, so that the domain features can accurately and comprehensively reflect their intrinsic semantic information, which is convenient for guiding the selection of positive and negative pairs in subsequent training processes.

[0087] It should be noted that traditional self-supervised contrastive learning methods often face challenges in healthcare applications due to sampling bias. Specifically, traditional instance-based contrastive learning usually assumes that all non-positive samples (i.e., negative samples) are dissimilar, which may lead to the incorrect separation of samples that actually have similar clinical or physiological characteristics. This problem is particularly serious in healthcare data. For example, different individuals with the same symptoms may have similar patterns in their healthcare data, such as patients with the same type of arrhythmia in ECG data. If based on traditional methods, misclassifying such semantically similar data as dissimilar may hinder the model's ability to learn clinically meaningful representations.

[0088] To overcome this limitation, the present application performs a nearest point search (NNS) in the domain feature space. By utilizing the domain knowledge encoded in the domain features, samples with higher semantic similarity are identified to be considered as additional positive pairs, and instance-level contrastive learning is guided so that the model can learn the local semantic relationship of samples. On the other hand, the present application can also perform unsupervised clustering of domain features to ensure that semantically similar samples remain clustered, and introduces domain knowledge-guided prototype contrastive learning so that the model can learn the global semantic relationship between samples and sample clusters.

[0089] It can be understood that in some embodiments, instance-level contrastive learning and / or prototype-level contrastive learning can be guided by domain knowledge.

[0090] refer to Figure 3 In some embodiments, when the positive and negative pairs include the first positive and negative pairs, the positive and negative pairs are determined based on the domain features corresponding to the medical and health data samples, and the contrast loss is determined according to the features of each sample in the positive and negative pairs, including steps 2062 and 2064, wherein:

[0091] Step 2062: Determine semantically similar samples of each medical and health data sample based on the domain features corresponding to the medical and health data sample.

[0092] It is understandable that during the model training process, it is usually trained batch by batch. For each anchor sample in the current batch (that is, the medical and health data samples currently being paid attention to), the computer device can identify the nearest neighbors in the batch as semantically similar samples of the anchor sample based on the distance in the domain feature space (such as Euclidean distance).

[0093] Exemplarily, the computer device may perform the nearest neighbor search by the following formula:

[0094] (Formula 1)

[0095] in, Indicates the number of anchor samples in the batch A collection of different medical and health data samples, also known as negative sample sets, Represents anchor samples By selecting the sample that is most similar to the anchor sample based on the domain features, it is ensured that semantically similar positive pairs are identified.

[0096] In some embodiments, it can be understood that the medical and health data sample and its enhanced sample can be considered to be semantically consistent, so the semantically similar samples that are semantically similar to the anchor sample can include A sample identified from a collection of different medical and health data samples, and an enhanced sample of the identified sample.

[0097] In some embodiments, after performing the nearest neighbor search, the computer device may search for the sample with the shortest distance from the anchor sample as the semantically similar sample, and may also select the first M (M is an integer greater than 1) samples with the shortest distance as the semantically similar samples.

[0098] Step 2064, determine the first positive and negative pairs based on the medical and health data samples and the corresponding semantically similar samples, and determine the instance-level contrast loss based on the sample features of each sample in the first positive and negative pairs; the instance-level contrast loss is used to determine the contrast loss.

[0099] Specifically, the computer device may use the medical health data sample and the corresponding semantically similar sample as the first positive pair, and use the medical health data sample and other medical health data samples as the first negative pair, so as to construct the first positive and negative pair. Then, the instance-level contrast loss is determined according to the sample features of each sample in the first positive and negative pair. It can be understood that the instance-level contrast loss is used to minimize the distance between the first positive pairs and maximize the distance between the first negative pairs.

[0100] It should be noted that the samples in the first positive and negative pairs are usually medical and health data samples. Therefore, when determining the instance-level contrast loss, it can be determined based on the sample features of each sample in the first positive and negative pairs. Among them, the sample features are features obtained by feature extraction of the model to be trained. In some embodiments, after the model to be trained extracts features from each medical and health data sample to obtain corresponding sample features, the computer device can perform nonlinear projection and normalization (such as l2 normalization) on the sample features for calculation of various contrast losses (including instance-level contrast loss and / or prototype-level contrast loss).

[0101] In some embodiments, a first positive and negative pair is determined based on medical and health data samples and corresponding semantically similar samples, including: determining a first positive pair based on an anchor sample and a semantically similar sample of at least the anchor sample, and determining a first negative pair based on the anchor sample and a medical and health data sample different from the anchor sample; wherein the anchor sample is any medical and health data sample in the training process, and the first positive pair and the first negative pair constitute the first positive and negative pair.

[0102] Exemplarily, for any anchor sample in the current training batch, the computer device may take the anchor sample and a semantically similar sample of the anchor sample as the first positive pair; or take the anchor sample and a semantically similar sample of the anchor sample as the first positive pair, and take the anchor sample and an enhanced sample of the anchor sample as the first positive pair; or take the anchor sample and a semantically similar sample of the anchor sample as the first positive pair, take the anchor sample and an enhanced sample of the anchor sample as the first positive pair, and take the anchor sample and an enhanced sample of the semantically similar sample as the first positive pair. If multiple first positive pairs are constructed, the model can learn the local semantic relationship between samples more fully and accurately during the training process, and the effect is better.

[0103] Furthermore, the computer device may construct at least one first negative pair with other medical and health data samples in the current training batch except those used to construct the first positive pair together with the anchor sample. It is understandable that the computer device may also construct multiple first negative pairs with other medical and health data samples in the current training batch except those used to construct the first positive pair and enhanced samples of other medical and health data samples together with the anchor sample.

[0104] In this way, at least one first positive pair and at least one first negative pair are constructed by using an anchor sample and at least a semantically similar sample to the anchor sample, so that a model trained based on the first positive pair and the first negative pair can learn local semantic relationships between different samples, such as local semantic information and local semantic structure.

[0105] Furthermore, after constructing the first positive and negative pairs, the computer device may determine the instance-level contrast loss based on the features of each sample in the first positive and negative pairs. In some embodiments, the computer device may use a variant of the InfoNCE (Information Noise Contrastive Estimation) loss to define the instance-level contrast loss in order to bring the features of the first positive pair closer while pulling the features of the first negative pair apart. Of course, the present application may also adopt other forms of contrast loss, such as triplet loss, etc., which are not limited in the present application.

[0106] For example, positive pairs can be generated based on the enhanced SSCL method, and considering that the medical and health data samples share the same domain features with their enhanced samples , so we can select the two most similar samples in the domain feature space to form (That is, a medical and health data sample whose domain characteristics are most similar to the domain characteristics of the anchor sample and the enhanced sample of this sample).

[0107] On this basis, the instance-level contrast loss can be determined by the following formula:

[0108] (Formula 2)

[0109] in, , Represent the indexes of anchor samples and positive samples respectively; The set of samples that form the first positive pair with the anchor sample may include the original positive pair and the sample that is most similar in terms of domain characteristics , that is, a set consisting of the enhanced samples of the anchor sample, the semantically similar samples of the anchor sample, and the enhanced samples of the semantically similar samples; and Represents the set of samples that form the first negative pair with the anchor sample, that is, the set of original negative samples excluding The resulting collection. Represents the features of the medical and health data samples obtained from the model to be trained. Specifically, it can be the sample features, or the sample features after nonlinear projection and normalization. By bringing the features of the positive sample and the anchor sample closer, it aims to guide the model to better align samples with meaningful semantic relationships.

[0110] It should be noted that the first positive pair constructed by the enhanced sample can be omitted in the instance-level contrast loss, or multiple semantically similar samples similar to the anchor sample can be selected to construct multiple first positive pairs, etc. The above examples are only used to illustrate the instance-level contrast loss of the present application and are not intended to limit the present application.

[0111] After constructing the instance-level contrast loss, the instance-level contrast loss can be directly used as the contrast loss of the model, that is, the contrast loss can be determined by the above formula 2. In other words, only domain features are used to guide the instance-level contrast loss.

[0112] In some embodiments, the computer device may further determine a prototype-level contrast loss, and obtain a comprehensive contrast loss by combining the instance-level contrast loss and the prototype-level contrast loss. The prototype-level contrast loss can be obtained by guiding the domain features in some cases (see the description below for details), and can be obtained by guiding the features learned by the model in other cases. The latter can be obtained by clustering the features of each medical and health data sample extracted using the model in the training process to obtain multiple clusters, so as to form positive and negative pairs with the medical and health data samples according to the prototypes of each cluster during the training process to determine the prototype-level contrast loss.

[0113] It can be understood that the above-mentioned various combinations are used to illustrate the technical concept of the present application, rather than to limit the present application.

[0114] In the above embodiment, when the positive and negative pairs include the first positive and negative pairs, enforcing the similarity between samples with similar domain features can ensure that the local semantic relationship is well represented in the learned feature space.

[0115] refer to Figure 4 In some embodiments, when the positive and negative pairs include a second positive and negative pair, the positive and negative pairs are determined based on the domain features corresponding to the medical and health data samples, and the contrast loss is determined according to the features of each sample in the positive and negative pairs, including steps 2066 and 2068, wherein:

[0116] Step 2066, determining the prototype of each cluster in the current iteration process; the cluster is obtained by clustering the domain features corresponding to each medical and health data sample in the full amount of medical and health data samples.

[0117] The cluster refers to a clustering cluster, or can also be called a sample cluster, which can be obtained by clustering the full amount of medical and health data samples based on the domain characteristics of the full amount of medical and health data samples. The prototype can be considered as the representative of the cluster, and in actual implementation, it can be the centroid of the cluster or the updated centroid.

[0118] In some embodiments, in the offline stage, the domain features of the entire amount of medical and health data samples can be clustered, and each medical and health data sample is assigned to the nearest centroid to obtain multiple clusters. The domain features corresponding to the medical and health data samples in the same cluster are relatively close, while the domain features corresponding to the medical and health data samples in different clusters are far away from each other.

[0119] For example, in this embodiment, an unsupervised k-means clustering method can be applied to obtain K (based on experience, K can be set to 128) clusters based on the similarity between medical and health data samples. Assigned to the nearest cluster centroid , which can reflect the proximity of similar domain features. This ensures that samples in the same cluster share domain-informed semantics and vice versa. Subsequently, for each cluster, an initial prototype is derived , for example, by using k-means clustering to obtain the characteristics of the initial prototype.

[0120] In some embodiments, the prototype of each cluster is updated during the training phase, that is, in the current iteration, the prototype of each cluster is updated based on the prototype obtained in the previous iteration, specifically, it can be updated based on the prototype obtained in the previous iteration based on the sample features generated during the learning process. It can be understood that in the first iteration of the first round, the initial prototype can be the centroid of the cluster.

[0121] In some embodiments, during the training process, the prototypes of each cluster can be updated online according to the sample clusters in each iteration process. Exemplarily, determining the prototype of each cluster in the current iteration process includes: obtaining the prototypes of each cluster in the previous iteration process; wherein the prototype of the cluster in the first iteration process is the centroid of the cluster; for the current iteration, according to the sample characteristics of the medical and health data samples belonging to each cluster in the current iteration, updating the prototypes of each cluster in the previous iteration process to obtain the prototypes of each cluster in the current iteration process.

[0122] It should be noted that the initial prototype of the cluster is the centroid of each cluster in the K clusters obtained after clustering, and the feature corresponding to the centroid of the cluster can be the average feature of the domain features of all medical and health data samples belonging to the cluster. It can be understood that in the offline stage, K clusters are obtained after clustering based on the full amount of medical and health data samples, and the prototype of each cluster is determined based on all medical and health data samples in the cluster. The training stage is based on batch iterative training. The sample size in each batch is less than the full sample size, so the clustering of the batch is substantially different from the clustering of the full sample. Therefore, when applied, the prototype can be updated according to the distribution of samples in the current batch. That is, for the current iteration, for any cluster, the sample features of the medical and health data samples belonging to the cluster in the medical and health data samples during the iteration can be used to update the prototype of the cluster to obtain an updated prototype, and the updated prototype will be used for the construction of the second positive and negative pairs during the iteration.

[0123] In this way, by updating the prototypes of each cluster according to the features learned by the model in the current iteration, the prototypes used in the training process can be closer to the samples in the current batch. This process ensures that the prototypes continue to evolve when generating new representations while maintaining consistency with past observations, allowing the model to learn the global semantic relationship between samples more flexibly.

[0124] In some embodiments, the prototype can be updated online through an exponential moving average (EMA) represented by the average of the clusters in each batch. For the current iteration, according to the sample features of the medical and health data samples belonging to each cluster in the current iteration, the prototypes of each cluster in the previous iteration are updated to obtain the prototypes of each cluster in the current iteration, including: for the current iteration, for any cluster, the sample features of the medical and health data samples belonging to the cluster in the current iteration are integrated to obtain the adjusted features, and the prototype of the cluster is updated based on the adjusted features.

[0125] Exemplarily, for any cluster, the average value of the features of the medical and health data samples belonging to the cluster in the current iteration can be calculated to obtain the adjusted features, and the prototype of the cluster can be updated based on the adjusted features. The features of the prototype of the updated cluster can be obtained by weighted summing the features of the prototype of the cluster generated in the previous iteration and the adjusted features.

[0126] For example, the prototype can be updated by the following formula:

[0127] (Formula 3)

[0128] in, represents the prototype of cluster j in the previous iteration, Represents the prototype of cluster j during the iteration; is the indicator function, Refers to the sample The cluster to which it belongs, when Returns 1 if yes, otherwise returns 0; Indicates the number of samples belonging to cluster j in each batch; is the momentum parameter (can be set to 0.5).

[0129] Step 2068, determine the second positive and negative pairs based on the medical and health data samples and the prototypes of each cluster, and determine the prototype-level contrast loss based on the characteristics of each sample in the second positive and negative pairs; wherein the characteristics of the prototype are determined based on the centroid of the corresponding cluster; the prototype-level contrast loss is used to determine the contrast loss.

[0130] In some embodiments, a second positive and negative pair is determined based on the medical health data sample and the prototypes of each cluster, including: determining a second positive pair based on the anchor sample and the prototype of the cluster to which the anchor sample belongs, and determining a second negative pair based on the anchor sample and the prototypes of other clusters; wherein the anchor sample is any medical health data sample in the training process, and the second positive pair and the second negative pair constitute a second positive and negative pair.

[0131] Furthermore, the computer device may determine the prototype-level contrast loss based on the features of each sample in the second positive and negative pairs. It can be understood that the prototype-level contrast loss is used to minimize the distance between the second positive pairs and maximize the distance between the second negative pairs. Among them, the features of the anchor sample in the second positive pair can be obtained by transforming the sample features extracted from the anchor sample by the model to be trained; (non-first iteration) the features of the prototype of the cluster to which the anchor sample belongs in the second positive pair are the features of the prototype updated in the iteration. (non-first iteration) The features of the prototypes of the other clusters in the second negative pair are also the features of the prototypes updated in the iteration.

[0132] In some embodiments, the features of the anchor samples may specifically be features obtained by extracting features from the anchor samples by the model to be trained, and then performing nonlinear projection and normalization (such as l2 normalization).

[0133] In this way, applying prototype-level contrastive learning to each sample can ensure that their features are closely consistent with the assigned prototypes while maintaining an appropriate distance from other clusters, which is conducive to the model learning the global semantic relationship between samples.

[0134] In some embodiments, the prototype-level contrast loss is determined based on the characteristics of each sample in the second positive-negative pair, including: obtaining the temperatures corresponding to each cluster; wherein the temperature corresponding to the cluster is determined by the number of samples in the cluster and the prototype obtained at the end of the previous round; the prototype-level contrast loss is determined based on the characteristics of each sample in the second positive-negative pair and the temperature corresponding to each cluster.

[0135] To optimize the learning process, the temperature can be dynamically adjusted according to the number of samples in each cluster, based on the observation that the lower the temperature, the higher the concentration of the corresponding prototype. This dynamic temperature strategy ensures that the latent feature distribution around each prototype has a similar concentration.

[0136] It should be noted that the temperature update can be performed once in an epoch, and the updated temperature of each cluster is applied to the next epoch.

[0137] Specifically, for any cluster, the temperature corresponding to the cluster can be determined according to the number of medical and health data samples in the cluster and the distance between each medical and health data sample and the prototype of the cluster. Exemplarily, the temperature is defined as follows:

[0138] (Formula 4)

[0139] in, is the number of samples belonging to cluster k in the total amount of medical and health data samples, and m is a smoothing parameter used to avoid Too large. That is, in any round, the characteristics of each medical and health data sample obtained after multiple iterations in the round have been completed. Features of the latest prototype in the current round.

[0140] It should be noted that at the beginning of the first epoch, since no updated prototypes were generated, all You can follow the constant To set, then in the second epoch and later, The value of each cluster can be calculated according to the above formula by continuously iterating and updating the prototype value. will be normalized so that all The average value is equal to .

[0141] It can be understood that the current round uses the Obtained through the previous round of updates; and the current round of updates Used for calculation of prototype contrast loss in the next round.

[0142] Exemplarily, the prototype level contrast loss may be determined by the following formula:

[0143] (Formula 5)

[0144] in, is the anchor sample obtained from the model to be trained Features, Anchor Sample The characteristics of the prototype of cluster i, is the characteristic of the prototype of cluster j, K is the total number of clusters, is the temperature corresponding to cluster i, is the temperature corresponding to cluster j.

[0145] In some embodiments, the prototype-level contrast loss may be directly used as the contrast loss, that is, the contrast loss may be determined by the above formula 5. That is, only the domain features are used to guide the prototype-level contrast loss.

[0146] In some embodiments, the computer device may further determine the instance-level contrast loss, and obtain a comprehensive contrast loss by combining the instance-level contrast loss and the prototype-level contrast loss. The instance-level contrast loss can be obtained by guiding the domain features in some cases (see the above description for details), and can be obtained by enhancing samples in other cases. The latter can be obtained by taking the enhanced view of the same instance as the positive pair. Exemplarily, the computer device can obtain the instance-level contrast loss in the following manner:

[0147] (Formula 6)

[0148] in, , Represent the indexes of anchor samples and positive samples respectively; Represents samples in a small batch (i.e., in a batch) The negative set of is the temperature that controls the sharpness of the similarity. For simplicity and without loss of generality, by default, another nonlinear projection is used to process the latent representation before similarity calculation. , and conduct Normalization. Usually, in the absence of actual ground truth, and is defined as augmented views of the same instance. Therefore, the loss function encourages the model to learn representations that make augmented views similar in the latent space while separating them from the representations of other samples in the mini-batch.

[0149] It can be understood that the above-mentioned various combinations are used to illustrate the technical concept of the present application rather than to limit the present application.

[0150] In the above embodiment, when the positive and negative pairs include the second positive and negative pairs, the global semantic relationship can be captured by guiding the determination of the prototype through the domain features. By applying unsupervised clustering to these domain features and then generating cluster-specific prototypes online, it can be ensured that the features are aligned with the global semantic clusters, so that the model can well learn the global semantic relationship between samples.

[0151] In some embodiments, the positive-negative pairs include a first positive-negative pair and a second positive-negative pair, and the contrast loss is determined according to the characteristics of each sample in the positive-negative pairs, including: determining an instance-level contrast loss according to the characteristics of each sample in the first positive-negative pair, and determining a prototype-level contrast loss according to the characteristics of each sample in the second positive-negative pair; combining the instance-level contrast loss and the prototype-level contrast loss to obtain the contrast loss.

[0152] In some embodiments, the computer device may determine a first positive-negative pair based on a medical and health data sample whose domain features meet similarity conditions, and determine an instance-level contrast loss based on the features of each sample in the first positive-negative pair, wherein the domain features meeting the similarity conditions may specifically be that the similarity between the domain features reaches a preset threshold or is ranked less than a preset ranking; determine a second positive-negative pair based on the medical and health data sample and the prototype of each cluster, and determine a prototype-level contrast loss based on the features of each sample in the second positive-negative pair, wherein the cluster is obtained based on the clustering of the domain features, and the prototype of the cluster is determined based on the centroid of the cluster; combine the instance-level contrast loss and the prototype-level contrast loss to obtain a comprehensive contrast loss. For example, the instance-level contrast loss and the prototype-level contrast loss may be weighted summed to obtain a comprehensive contrast loss. The coefficient of the weighted summation may be set as needed.

[0153] In some embodiments, determining a first positive and negative pair based on medical and health data samples whose domain features satisfy similarity conditions includes: determining a semantically similar sample for each medical and health data sample based on the domain features corresponding to the medical and health data samples; and determining a first positive and negative pair based on the medical and health data samples and the corresponding semantically similar samples.

[0154] The determination process of the instance-level contrast loss and the prototype-level contrast loss may refer to the above description. After determining the instance-level contrast loss and the prototype-level contrast loss, the computer device may combine the instance-level contrast loss and the prototype-level contrast loss to obtain a comprehensive contrast loss.

[0155] In this way, by jointly training the model with instance-level contrastive loss and prototype-level contrastive loss, the model can ensure that the features are not only locally aligned but also aligned with the global semantics of the cluster when encoding features.

[0156] In some embodiments, the instance-level contrast loss and the prototype-level contrast loss are combined to obtain the contrast loss, including: determining a dynamic weight based on the current round in the training process; the dynamic weight increases as the current round increases; applying the dynamic weight to the prototype-level contrast loss to perform weighted processing on the instance-level contrast loss and the prototype-level contrast loss to obtain the contrast loss.

[0157] Given that the representation of the prototype is usually of low quality in the initial training stage, prototype contrast learning can be gradually applied. The dynamic weight is greater than or equal to 0 and less than or equal to 1, and can be determined by the ratio of the difference between the current round and the starting round of applying the prototype-level contrast loss to the total training rounds. Exemplarily, the contrast loss function can be constructed by the following formula:

[0158] (Formula 7)

[0159] in, Refers to the current round (current epoch), Refers to the starting round (starting epoch) of applying the prototype-level contrastive loss, is the total number of training rounds (total number of epochs).

[0160] In the above embodiment, as the training rounds progress, the application of prototype-level contrast loss is gradually increased, and dynamic adjustment can be made based on the quality of each of the instance-level contrast loss and the prototype-level contrast loss, so that the comprehensive contrast loss is more accurate, thereby improving the training quality.

[0161] In one embodiment, the data processing method based on deep learning may include the following steps: obtaining a training data set, which includes multiple medical and health data samples. For any medical and health data sample, extract the features of the medical and health data sample or at least one of the cross-modal data samples associated with the medical and health data sample in a preset dimension to obtain the domain features corresponding to the medical and health data sample, and cluster all the medical and health data samples based on the domain features to obtain multiple clusters. The model to be trained is trained for multiple epochs, wherein multiple iterations are performed in each epoch. During each iteration, samples of the current batch are extracted from the training data set for processing, and sample features of each sample are extracted based on the model to be trained, which are recorded as ,based on Calculated Then, the first positive and negative pairs are determined according to the domain characteristics, and the instance-level contrast loss is determined according to the sample characteristics of each example in the first positive and negative pairs. The EMA updates the prototype of each cluster in each iteration, calculates the distance between the anchor sample and the prototype of each cluster, and calculates the temperature of each cluster to determine the prototype-level contrast loss. The comprehensive contrast loss is obtained based on the instance-level contrast loss and the prototype-level contrast loss, and the model is trained by the comprehensive contrast loss. After the last iteration in the epoch, the temperature of each cluster is updated, and an epoch is completed. Next, the next epoch is carried out, and the cycle continues until the last epoch is completed and the training is stopped to obtain the trained model.

[0162] In one embodiment, the data processing method based on deep learning includes: obtaining a training data set, which includes multiple medical and health data samples. For any medical and health data sample, extract the features of the medical and health data sample or at least one of the cross-modal data samples associated with the medical and health data sample in a preset dimension to obtain the domain features corresponding to the medical and health data sample, and cluster the full amount of medical and health data samples based on the domain features to obtain multiple clusters. For any iteration in any round, extract the sample features of the medical and health data samples used by the model to be trained, determine the semantically similar samples of each medical and health data sample based on the domain features of the medical and health data samples, determine the first positive and negative pairs according to the medical and health data samples and the corresponding semantically similar samples, and determine the instance-level contrast loss according to the sample features of each sample in the first positive and negative pairs. For any iteration in any round, determine the prototype of each cluster, cluster the full amount of medical and health data samples based on the domain features of the full amount of medical and health data samples, determine the second positive and negative pairs according to the prototypes of the medical and health data samples and each cluster, and determine the prototype-level contrast loss according to the features of each sample in the second positive and negative pairs. The instance-level contrast loss and the prototype-level contrast loss are combined to obtain a comprehensive contrast loss, and the model to be trained is trained by the comprehensive contrast loss. After all iterations in all rounds are completed, a model for extracting features of medical and health data is obtained.

[0163] In a specific embodiment, the processing method of the model includes the following steps:

[0164] Obtain a medical health data sample, and for any medical health data sample, extract features of the medical health data sample or at least one of the cross-modal data samples associated with the medical health data sample on a preset dimension to obtain a domain feature corresponding to the medical health data sample; wherein the preset dimension is determined based on domain knowledge.

[0165] The sample features of each medical and health data sample are extracted through the model to be trained.

[0166] Based on the domain features corresponding to the medical and health data samples, semantically similar samples of each medical and health data sample are determined.

[0167] The anchor sample and the semantically similar sample of the anchor sample are taken as the first positive pair, the anchor sample and the enhanced sample of the anchor sample are taken as the first positive pair, and the anchor sample and the enhanced sample of the semantically similar sample are taken as the first positive pair. The first negative pair is determined according to the anchor sample and the medical and health data sample different from the anchor sample; wherein the anchor sample is any medical and health data sample in the training process, and the first positive pair and the first negative pair constitute the first positive and negative pair. The instance-level contrast loss is determined according to the sample features of each sample in the first positive and negative pairs.

[0168] Obtain the prototype of each cluster in the previous iteration; wherein the prototype in the first iteration is the centroid of the cluster; for the current iteration, according to the sample characteristics of the medical and health data samples belonging to each cluster in the current iteration, update the prototype of each cluster in the previous iteration to obtain the prototype of each cluster in the current iteration. The cluster is obtained by clustering the full amount of medical and health data samples based on the domain characteristics corresponding to each medical and health data sample in the full amount of medical and health data samples. Determine the second positive pair according to the prototype of the anchor sample and the cluster to which the anchor sample belongs, and determine the second negative pair according to the prototype of the anchor sample and other clusters; wherein the anchor sample is any medical and health data sample in the training process, and the second positive pair and the second negative pair constitute the second positive and negative pair. Obtain the temperature corresponding to each cluster; wherein the temperature corresponding to the cluster is determined by the number of samples in the cluster and the prototype obtained at the end of the previous round; determine the prototype-level contrast loss according to the characteristics of each sample in the second positive and negative pair and the temperature corresponding to each cluster.

[0169] A dynamic weight is determined based on the current round in the training process; the dynamic weight increases as the current round increases; the dynamic weight is applied to the prototype-level contrast loss to weight the instance-level contrast loss and the prototype-level contrast loss to obtain the contrast loss. The model to be trained is trained by the contrast loss. Multiple iterations are performed in one round, and multiple rounds are performed during the entire training period, and a model for extracting features of medical health data is obtained after the training is completed.

[0170] This application uses the domain features of medical and health data samples to guide the nearest neighbor search when forming self-supervised comparison pairs. By enforcing the similarity between samples with similar domain features, it can ensure that local semantic relationships are well represented in the learned feature space. In addition, in order to capture global semantic relationships, unsupervised clustering is applied to these domain features in this application, and then cluster-specific prototypes are generated online. This prototype-based contrastive learning ensures that the representation is not only aligned locally, but also aligned with the global semantic clusters.

[0171] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0172] Next, combine Figure 5 One of the concepts of this application is described in detail: Figure 5 As shown in the figure, this application develops a domain knowledge-guided SSL framework that combines "old-school" domain feature engineering with "new-school" SSCL. We leverage domain features extracted through mature related techniques (such as biomedical signal processing methods) and integrate these domain features into the SSL process to guide model learning and improve downstream validation performance. Based on the enhanced SSCL method, we use the Euclidean distance between these domain features to guide the selection of positive pairs for instance-level contrastive learning. In addition, to further strengthen the learning of global semantic relationships, we introduce prototype-level contrastive learning. This will force each sample to be more closely aligned with its assigned prototype, which is generated by clustering based on domain features.

[0173] exist Figure 5 In the algorithm framework diagram, a. "Old-fashioned" domain feature engineering for interpreting medical and health data (such as physiological signals). Take electrocardiogram as an example, which is often used to detect cardiovascular abnormalities. A series of features have been developed to indicate abnormal presence / amplitude of heart rate variability (HRV) and P waves. b. Integrate domain features based on domain knowledge into the supervised contrastive learning paradigm. Compared with the traditional approach, under the guidance of domain features related to the expertise of cardiologists, we aim to maintain semantic relationships during contrastive learning. For example, by considering domain features indicating normal ECG records (such as heart rate variability and the presence of P waves), the model should position the anchor point (usually normal ECG) closer to another normal sample than to abnormal samples showing irregular RR intervals and missing P waves. c. Description of the algorithm framework guided by domain knowledge. In this application, conventional methods can be followed to generate two enhanced views of the same sample and derive their representations (i.e., sample features). At the same time, in accordance with standard medical and health data processing practices (such as biomedical signal processing practices), domain features reflecting underlying semantic information are obtained offline from all samples. In order to effectively utilize domain knowledge in self-supervised contrastive learning, the present application first identifies the nearest samples in the domain feature space to establish instance-level positive pairs, which together with the enhanced counterparts contribute to instance-level contrastive learning. On the other hand, in order to preserve the global and local semantic structure, the present application can also adopt prototype-level contrast. Specifically, it can be unsupervised clustering based on domain features to obtain multiple clusters and prototypes of each cluster, and then the features of the prototypes are derived online using the exponential moving average (EMA). The introduction of prototype-level contrast loss can encourage each sample to be closely aligned with its corresponding prototype while maintaining a distance from other samples. The importance of domain knowledge can be seen from this figure, not only in traditional machine learning methods, but also in advanced deep learning techniques used to interpret medical time series data.

[0174] In addition, the researchers also conducted experiments on one of the implementation schemes of this application, and the experimental results are as follows:

[0175] Experimental Settings:

[0176] It can be performed on any model of GPU (Graphics Processing Unit) (such as NVIDIAV100 GPU) through a deep learning framework (such as PyTorch). Exemplarily, for self-supervised training experiments, the model was trained for 1000 rounds with a batch size of 1024, using the Adam optimizer, an initial learning rate of 0.001, and a warm-up strategy for the first 5 rounds. Logistic regression was applied on the validation set every 10 rounds to monitor model selection. For transfer learning and semi-supervised learning, we reduced the learning rate to 0.0005 and limited the training to 40 rounds. Given the unbalanced nature of some datasets in downstream tasks, balanced Softmax (normalized exponential function) cross entropy can be used to handle class imbalance. The performance is evaluated using the class average F1 score. All data are from public resources, including MIMIC-IIIWDB, PhysioNet17, CPSC, SleepEDF, Capture24, etc.

[0177] Implementation results:

[0178] 1. Datasets: Please refer to Table 1. To evaluate the feasibility and versatility of the framework proposed in this application, experiments were conducted on three types of wearable sensors: electrocardiogram (ECG), electroencephalogram (EEG), and inertial measurement unit (IMU). The following is an explanation of the datasets in Table 1: The CinC17 dataset contains short ECG records collected from a portable chest patch, annotated into four categories, including normal sinus rhythm and atrial fibrillation. The CPSC dataset provides 12-lead ECG records annotated by cardiologists for nine different cardiac abnormalities, although we only use lead II to simulate wearable scenarios. The MIMIC-III-WDB dataset consists of ICU (Intensive Care Unit) ECG records from bedside monitors, which are mainly used for self-supervised training due to the lack of segment-level annotations. For EEG, we used the SleepEDF dataset (two EEG channels, namely Fpz-Cz and Pz-Oz), which includes long-term records segmented into 30-second windows and annotated with one of the five sleep stages. Finally, for IMU, we used the Capture24 dataset (xyz accelerometer), which contains wrist-worn accelerometer recordings segmented into 10-second windows for human activity recognition. In all datasets, standard preprocessing was applied, including resampling, bandpass filtering, segmentation, and z-normalization for consistency. All datasets can be randomly divided into training, validation, and test subsets in a 6:2:2 ratio, ensuring that there is no individual overlap. They are used to train the model, adjust hyperparameters, and evaluate performance, respectively.

[0179] Table 1 Datasets used in the experiment

[0180]

[0181] To benchmark the performance of our framework in various healthcare scenarios, we evaluated the discriminability of features derived from SSL using the class-averaged F1 score on the test set (see Table 2 for details). The discriminability of the learned features was directly evaluated using simple classifiers, including linear classifiers and KNN (K-Nearest Neighbors) classifiers (n=10), as these classifiers do not involve further complex feature abstraction. This allows us to evaluate the quality of the features themselves, rather than introducing additional layers of abstraction that may mask their inherent effectiveness. Our method was then compared to a range of SSCL methods, a domain feature-based model (Domain Feat.), and a fully supervised learning model with a randomly initialized backbone (FullySup.), all of which share the same architecture as our method. Our method shows superior performance compared to other SSL methods, even outperforming the fully supervised model in some cases, especially in IMU data, where it achieves a class-averaged F1 score of 0.526. This improvement may be attributed to the domain knowledge-guided approach in this application, which helps mitigate the impact of inter-subject variability, especially in physical activity data where individuals’ movement patterns can vary greatly. These results suggest that incorporating domain knowledge can help reduce bias and enhance the overall robustness of the model even in a self-supervised learning setting. In addition to the class-averaged F1, we also evaluated the F1 scores of the best performing models based on other metrics including precision, recall, AUC (Area Under Curve), and accuracy. Figure 6 ). Our method consistently outperforms other SSL methods on almost all of these metrics, further demonstrating its versatility and effectiveness in learning robust features from wearable device measurements. Compared to state-of-the-art SSCL methods, our method uses domain knowledge to drive features, which enhances the selection of contrast pairs and ensures that the model captures clinically meaningful patterns. This finding points to a promising direction for advancing SSL in time series data, especially in healthcare applications where domain knowledge plays a key role in improving model performance.

[0182] Table 2 Comparison results based on self-supervised learning

[0183]

[0184] To test the effectiveness of downstream knowledge learned during self-supervised learning, we froze the backbone into a feature extractor (the model mentioned above) and applied KNN or linear classifier for supervised training. Each method was run with five different random seeds to evaluate the performance. We compared the class average F1 scores of different models, and the results are shown in Table 2 above. Bold indicates the best result, and underlined indicates the second best result.

[0185] In addition, please refer to Figure 6 , Figure 6 The performance of different methods on different healthcare data (such as physiological signals and activity signals) in terms of class average F1, accuracy, AUC, recall, and precision is compared. We compare the performance of the model with the best F1 score on the validation subset on the test subset. These values ​​are obtained by normalizing the deviation of each corresponding metric from the best performance.

[0186] Please refer to the following Figure 7 , Figure 7 The visualization result of waveform significance map of two representative samples in one embodiment. The fully trained classifier attached to each self-supervised learning backbone (model) is used to classify the electrocardiogram, and the method of the present application is used to assign higher significance to clinically relevant areas, such as ST segment elevation, while avoiding noise and irrelevant waveform parts, thereby improving the interpretability and prediction accuracy of the model.

[0187] refer to Figure 8 , Figure 8 The figure shows a t-SNE (t-distributed stochastic neighbor embedding) plot of the feature representations derived from the self-supervised contrastive learning backbone for ECG, IMU, and EEG data in one embodiment. A. For the first three columns, the top row shows the results using SimCLR, while the bottom row shows the results using the method proposed in the present application. Each dot represents a sample, and different colors correspond to different categories. Compared with SimCLR, the clusters in the t-SNE plot of the present application's method are better separated and compact, indicating that the feature representations are more discriminative and coherent. This enhanced discriminability indicates that the model of the present application is better at capturing meaningful patterns from the data, which is critical for downstream classification tasks of medical wearable devices. B. The last column shows points colored according to the subject / individual. It can be observed that compared with the method proposed in the present application, the traditional SSCL method may tend to cluster samples from the same individual more prominently. This suggests that traditional methods may be more likely to learn individual-specific patterns, while our method can better generalize across individuals. For Figure 7In the atrial fibrillation case in , our method effectively captures clinically relevant morphological abnormalities, such as the absence of the P wave - a key indicator of atrial fibrillation - demonstrating its ability to focus on key features. Here, our method consistently assigns higher significance to waveform subsegments that directly reflect ST-segment elevation, while effectively ignoring irrelevant or noisy portions. This selective attention to clinically meaningful subsegments not only enhances the interpretability of the model’s decisions, but also improves its overall predictive accuracy. In addition, we visualize the t-SNE plots of the learned representations derived from the SSL framework on the test dataset, as shown in Figure 8 Even without access to the ground truth labels for training, the features learned by our method can show better separation and discrimination between different categories compared to the popular SimCLR method. This shows that the framework proposed in this application is effective in capturing meaningful features for downstream tasks.

[0188] refer to Fig. 9 , Fig. 9 Figure 2 shows the comparison results of semi-supervised learning based on ECG dataset in one embodiment. Fig. 9 It can be seen that the proposed method is always better than other methods, especially when labeled data is scarce. For example, when there is only 10% labeled data, the proposed method shows significant improvement over other methods. In addition, compared with other models, the proposed method is less sensitive to the proportion of labeled data and can maintain high performance even with a small amount of labeled data, reducing the dependence on large labeled datasets.

[0189] Furthermore, in order to evaluate the effectiveness of each module in the method proposed in this application, we conducted an ablation study, and the results are shown in Table 3. This study clearly demonstrates the effectiveness of each module and verifies their importance during training. The comparison based on NNS. (Nearest Neighbor Search) and Proto. (Prototype) helps to understand the semantic information at the local and global levels, respectively.

[0190] Table 3 Ablation study during model training

[0191]

[0192] Impact of Domain Knowledge Components on the Model:

[0193] We further compared the method of this application with direct regression as a pre-training proxy task, and the results are shown in Table 3. Direct regression performs poorly. Although domain features encode semantically meaningful information, they often lack sufficient separability. Relying solely on these features will degrade the overall performance, which further demonstrates the importance of integrating them into the SSL framework to enhance semantic understanding and discrimination capabilities. On the other hand, in order to understand the key role of domain features in the integrated framework of this application, we conducted a comprehensive ablation study on these features in the ECG domain. Specifically, we divided the features into different groups (AF-related features, morphology features, RR interval-related features, beat similarity-related features), and systematically ablated each group separately, as follows Fig.10 As shown, it can be seen that deleting a single feature group has a significant negative impact on direct logistic regression, which shows the sensitivity of traditional models to the loss of specific clinical features. In contrast, our SSCL framework shows resilience to such feature ablation, and can maintain stable performance even if certain clinically important features are deleted (that is, using the model trained by the training method of this application, feature extraction of input data with the above-mentioned feature missing can also extract features with better performance). From a clinical perspective, this robustness is particularly important. This shows that our framework is more capable of handling realistic changes in clinical data, such as due to noise, artifacts, or incomplete data collection.

[0194] refer to Fig.10 , which shows the importance of different domain features for direct logistic regression and the framework scheme of this application. The heat map on the left illustrates which feature combinations (AF features, morphology, RR interval, and similarity) are used, where the purple cubes indicate that the features are used. The bar chart on the right shows the class average F1 score for each feature combination, where direct logistic regression is represented by blue (hatched) and the method of this application is represented by green. Higher values ​​indicate better performance, which highlights the advantage of integrating domain features into the self-supervised learning framework for downstream classification tasks.

[0195] Based on the same inventive concept, the embodiment of the present application also provides a deep learning-based data processing device for implementing the deep learning-based data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the deep learning-based data processing device provided below can refer to the limitations on deep learning-based data processing above, and will not be repeated here.

[0196] In an exemplary embodiment, Fig.11As shown, a data processing device based on deep learning is provided, including: an acquisition module 1101, an extraction module 1102, a determination module 1103 and a training module 1104, wherein:

[0197] The acquisition module 1101 is used to acquire medical and health data samples, and obtain the domain features corresponding to each medical and health data sample for reflecting semantic information.

[0198] The extraction module 1102 is used to extract sample features of each medical and health data sample through the model to be trained.

[0199] Determination module 1103 is used to determine positive and negative pairs based on domain features corresponding to medical and health data samples, and determine contrast loss according to the features of each sample in the positive and negative pairs; wherein the positive and negative pairs include at least one of a first positive and negative pair for learning local semantic relationships or a second positive and negative pair for learning global semantic relationships, and the features of at least some of the samples in the positive and negative pairs are obtained based on the sample features.

[0200] The training module 1104 is used to train the model to be trained by contrast loss; the model obtained after the training is completed is used to extract the features of medical health data.

[0201] In some embodiments, the determination module is also used to determine the semantically similar samples of each medical and health data sample based on the domain characteristics corresponding to the medical and health data samples; determine the first positive and negative pairs according to the medical and health data samples and the corresponding semantically similar samples, and determine the instance-level contrast loss according to the sample characteristics of each sample in the first positive and negative pairs; the instance-level contrast loss is used to determine the contrast loss.

[0202] In some embodiments, the determination module is also used to determine a first positive pair based on an anchor sample and a semantically similar sample of at least the anchor sample, and to determine a first negative pair based on the anchor sample and a medical health data sample different from the anchor sample; wherein the anchor sample is any medical health data sample in the training process, and the first positive pair and the first negative pair constitute a first positive-negative pair.

[0203] In some embodiments, the determination module is further used to take the anchor sample and the semantically similar sample of the anchor sample as the first positive pair; take the anchor sample and the enhanced sample of the anchor sample as the first positive pair; take the anchor sample and the enhanced sample of the semantically similar sample as the first positive pair.

[0204] In some embodiments, the determination module is also used to determine the prototype of each cluster in the current iteration process; the clusters are clustered based on the domain features corresponding to each medical and health data sample in the full amount of medical and health data samples; the second positive and negative pairs are determined according to the medical and health data samples and the prototypes of each cluster, and the prototype-level contrast loss is determined according to the features of each sample in the second positive and negative pairs; wherein the features of the prototype are determined based on the centroid of the corresponding cluster; the prototype-level contrast loss is used to determine the contrast loss.

[0205] In some embodiments, the determination module is also used to obtain the prototype of each cluster in the previous iteration process; wherein, the prototype of the cluster in the first iteration process is the centroid of the cluster; for the current iteration, according to the sample characteristics of the medical and health data samples belonging to each cluster in the current iteration, the prototype of each cluster in the previous iteration process is updated to obtain the prototype of each cluster in the current iteration process.

[0206] In some embodiments, the determination module is also used to determine a second positive pair based on the anchor sample and the prototype of the cluster to which the anchor sample belongs, and to determine a second negative pair based on the prototype of the anchor sample and other clusters; wherein the anchor sample is any medical health data sample in the training process, and the second positive pair and the second negative pair constitute a second positive and negative pair.

[0207] In some embodiments, the determination module is also used to obtain the temperatures corresponding to each cluster; wherein the temperature corresponding to the cluster is determined by the number of samples in the cluster and the prototype obtained at the end of the previous round; and the prototype-level contrast loss is determined based on the characteristics of each sample in the second positive and negative pairs and the temperature corresponding to each cluster.

[0208] In some embodiments, the determination module is also used to determine the instance-level contrast loss according to the characteristics of each sample in the first positive and negative pairs, and determine the prototype-level contrast loss according to the characteristics of each sample in the second positive and negative pairs; and combine the instance-level contrast loss and the prototype-level contrast loss to obtain the contrast loss.

[0209] In some embodiments, the determination module is also used to determine the dynamic weight based on the current round in the training process; the dynamic weight increases as the current round increases; the dynamic weight is applied to the prototype-level contrast loss to perform weighted processing on the instance-level contrast loss and the prototype-level contrast loss to obtain the contrast loss.

[0210] In some embodiments, the acquisition module is also used to extract, for any medical health data sample, features of the medical health data sample or at least one of the cross-modal data samples associated with the medical health data sample in a preset dimension to obtain domain features corresponding to the medical health data sample; wherein the preset dimension is determined based on domain knowledge.

[0211] Each module in the above-mentioned data processing device based on deep learning can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0212] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.12 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store training data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data processing method based on deep learning is implemented.

[0213] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0214] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0215] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0216] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0217] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0218] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0219] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0220] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data processing method based on deep learning, characterized in that: The method comprises: Acquire medical and health data samples, and obtain domain features corresponding to each medical and health data sample for reflecting semantic information; Extracting sample features of each of the medical and health data samples through the model to be trained; Determine positive and negative pairs based on the domain features corresponding to the medical and health data samples, and determine the contrast loss according to the features of each sample in the positive and negative pairs; wherein the positive and negative pairs include at least one of a first positive and negative pair for learning a local semantic relationship or a second positive and negative pair for learning a global semantic relationship, and the features of at least some samples in the positive and negative pairs are obtained based on the sample features; The model to be trained is trained by using the contrast loss; the model obtained after the training is used to extract features of medical health data.

2. The method according to claim 1, characterized in that In the case where the positive and negative pairs include the first positive and negative pairs, determining the positive and negative pairs based on the domain features corresponding to the medical and health data samples, and determining the contrast loss according to the features of each sample in the positive and negative pairs include: Determining semantically similar samples of each medical and health data sample based on the domain features corresponding to the medical and health data sample; A first positive-negative pair is determined according to the medical health data sample and the corresponding semantically similar sample, and an instance-level contrast loss is determined according to the sample features of each sample in the first positive-negative pair; the instance-level contrast loss is used to determine the contrast loss.

3. The method according to claim 2, characterized in that The determining of the first positive and negative pairs according to the medical health data sample and the corresponding semantically similar samples includes: A first positive pair is determined based on an anchor sample and a semantically similar sample of at least the anchor sample, and a first negative pair is determined based on the anchor sample and a medical health data sample different from the anchor sample; wherein the anchor sample is any medical health data sample in the training process, and the first positive pair and the first negative pair constitute the first positive and negative pair.

4. The method according to claim 3, characterized in that The determining a first positive pair according to the anchor sample and at least a semantically similar sample of the anchor sample comprises: Taking the anchor sample and a semantically similar sample of the anchor sample as the first positive pair; Taking the anchor sample and the enhanced sample of the anchor sample as the first positive pair; The anchor sample and an enhanced sample of the semantically similar sample are taken as the first positive pair.

5. The method according to claim 1, characterized in that In the case where the positive and negative pairs include a second positive and negative pair, determining the positive and negative pairs based on the domain features corresponding to the medical and health data samples, and determining the contrast loss according to the features of each sample in the positive and negative pairs include: Determine the prototype of each cluster in the current iteration process; the cluster is obtained by clustering the domain features corresponding to each medical and health data sample in the full amount of medical and health data samples; A second positive-negative pair is determined according to the medical health data sample and the prototype of each cluster, and a prototype-level contrast loss is determined according to the characteristics of each sample in the second positive-negative pair; wherein the characteristics of the prototype are determined based on the centroid of the corresponding cluster; and the prototype-level contrast loss is used to determine the contrast loss.

6. The method according to claim 5, characterized in that Determining the prototype of each cluster in the current iteration process includes: Obtain the prototype of each cluster in the previous iteration process; wherein, the prototype of the cluster in the first iteration process is the centroid of the cluster; For the current iteration, according to the sample characteristics of the medical and health data samples belonging to each cluster in the current iteration, the prototypes of each cluster in the previous iteration are updated to obtain the prototypes of each cluster in the current iteration.

7. The method according to claim 5, characterized in that The determining of the second positive and negative pairs according to the medical health data sample and the prototypes of each cluster includes: A second positive pair is determined based on the anchor sample and the prototype of the cluster to which the anchor sample belongs, and a second negative pair is determined based on the anchor sample and the prototypes of other clusters; wherein the anchor sample is any medical health data sample in the training process, and the second positive pair and the second negative pair constitute the second positive and negative pair.

8. The method according to claim 5, characterized in that The step of determining the prototype level contrast loss according to the features of each sample in the second positive and negative pairs includes: Obtain the temperature corresponding to each cluster; the temperature corresponding to the cluster is determined by the number of samples in the cluster and the prototype obtained at the end of the previous round; The prototype-level contrast loss is determined according to the characteristics of each sample in the second positive-negative pair and the temperature corresponding to each cluster.

9. The method according to claim 1, characterized in that: The positive and negative pairs include a first positive and negative pair and a second positive and negative pair, and determining the contrast loss according to the features of each sample in the positive and negative pairs includes: Determine an instance-level contrast loss according to the features of each example in the first positive-negative pair, and determine a prototype-level contrast loss according to the features of each example in the second positive-negative pair; The instance-level contrast loss and the prototype-level contrast loss are combined to obtain a contrast loss.

10. The method according to claim 9, characterized in that The combining the instance-level contrast loss and the prototype-level contrast loss to obtain the contrast loss includes: Determining a dynamic weight based on a current round in a training process; the dynamic weight increases as the current round increases; The dynamic weight is applied to the prototype-level contrast loss to perform weighted processing on the instance-level contrast loss and the prototype-level contrast loss to obtain a contrast loss.

11. The method according to any one of claims 1 to 10, characterized in that The obtaining of the domain features corresponding to each medical and health data sample for reflecting semantic information includes: For any medical health data sample, extract the features of the medical health data sample or at least one of the cross-modal data samples associated with the medical health data sample on a preset dimension to obtain the domain features corresponding to the medical health data sample; wherein the preset dimension is determined based on domain knowledge.

12. A data processing device based on deep learning, characterized in that: The device comprises: An acquisition module, used to acquire medical and health data samples, and obtain domain features corresponding to each medical and health data sample for reflecting semantic information; An extraction module, used to extract sample features of each of the medical and health data samples through a model to be trained; A determination module, configured to determine positive and negative pairs based on domain features corresponding to the medical and health data samples, and determine contrast loss according to features of each sample in the positive and negative pairs; wherein the positive and negative pairs include at least one of a first positive and negative pair for learning a local semantic relationship or a second positive and negative pair for learning a global semantic relationship, and features of at least some samples in the positive and negative pairs are obtained based on the sample features; A training model is used to train the model to be trained through the contrast loss; the model obtained after the training is used to extract the characteristics of medical health data.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images