A wearable automated diagnostic system and method for mental disorders based on contrastive learning

Abstract

This invention discloses a wearable automatic diagnostic system and method for mental disorders based on contrastive learning. The system is implemented using a wearable device and includes: a data acquisition unit for acquiring the user's multimodal physiological data; a user registration unit, used to, when the user is determined to be a new user, perform the following steps: fine-tuning a pre-trained offline feature encoder using the multimodal physiological data through self-supervised contrastive learning to obtain a second feature encoder; extracting data features from the labeled multimodal physiological data via the second feature encoder; using these data features as input to train a personalized classifier to obtain a mental disorder identification model; and an identification unit, used to, when the user is determined to be a non-new user, using the mental disorder identification model to obtain an identification result. This invention can detect the user's mental state in real time, thereby intervening early to alleviate mental disorder symptoms.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and more specifically, to a wearable automatic diagnostic system and method for mental disorders based on contrastive learning. Background Technology

[0002] For disorders such as depression, anxiety, and mood disorders, current researchers use different modalities of data for identification, including textual information in the form of questionnaires, behavioral information captured using mobile sensing technology and real-time ecological assessments, and deep physiological information obtained using specialized medical devices. However, these identification methods suffer from problems such as privacy leaks, high costs of specialized equipment, difficulty in obtaining data labels, and low real-time performance. In recent years, wearable devices have developed rapidly, enabling the acquisition of a large amount of superficial, unlabeled physiological information about users anytime, anywhere, such as heart rate variability and blood oxygen saturation.

[0003] In existing technologies, solutions for identifying different mental disorders can be mainly divided into three categories. The first category uses traditional identification methods such as questionnaires or specialized medical equipment. These methods face problems such as privacy leaks, high costs, and lack of universality. The second category uses universally available devices to collect user behavioral information for mental disorder identification and assessment. This method typically obtains user activity and sleep data, but suffers from low real-time performance. The third category uses self-supervised learning for mental disorder identification. This method usually obtains the user's single-modal signal for pre-training, but suffers from problems such as limited modality, difficulty in obtaining data labels, and lack of joint identification of multiple mental disorders.

[0004] Studies have shown that mental disorders often trigger physiological responses, leading to changes in superficial physiological information. This superficial physiological information directly reflects the body's immediate physiological state, thus allowing for timely monitoring of the user's mental state. Currently, research on technologies using contrastive learning for mental disorder identification is developing rapidly. However, this approach does not utilize multimodal fusion in constructing positive and negative samples, and these studies typically focus on only one condition. Because different mental disorders may share common symptoms, it is difficult to identify which specific disease an individual suffers from.

[0005] Analysis reveals that existing solutions for identifying mental disorders have the following main shortcomings:

[0006] 1) Most studies on mental disorder identification focus on only one mental disorder, rarely identifying multiple mental disorders simultaneously. Therefore, users can only determine if they have the specific mental disorder being studied, but cannot determine if they have other mental illnesses, thus hindering early detection and potentially worsening their condition.

[0007] 2) Using specialized equipment to collect users' deep physiological information is not only costly, complex to operate, and not universally applicable, but also cannot meet the needs of real-time monitoring in daily life.

[0008] 3) Using mobile sensing technology to collect user behavior information usually requires collecting user behavior information over a long period of time, and then analyzing this information to obtain results. It has low real-time performance and it is difficult to capture the user's instantaneous psychological state.

[0009] 4) Currently, in schemes using self-supervised contrastive learning, the contrastive learning strategies used are usually only for one modality of data. This single modality of physiological information is limited and cannot extract deeper information, resulting in poor recognition performance.

[0010] In summary, there is currently limited research, both domestically and internationally, on the identification of various mental disorders using physiological data collected by ubiquitous wearable devices through multimodal feature fusion. Furthermore, existing research requires large amounts of labeled data and does not support user personalization. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of the prior art and provide a wearable automatic diagnostic method and system for mental disorders based on contrastive learning.

[0012] According to a first aspect of the present invention, a wearable automated diagnostic system for mental disorders based on contrastive learning is provided. The system is implemented using a wearable device and includes:

[0013] Data acquisition unit: used to acquire the user's multimodal physiological data;

[0014] User registration unit: When determining that the user is a new user, it performs the following actions: fine-tuning the offline pre-trained first feature encoder using the multimodal physiological data in a self-supervised contrastive learning manner to obtain a second feature encoder; extracting data features from the stored labeled multimodal physiological data via the second feature encoder; and training a personalized classifier using the data features as input to obtain a mental disorder recognition model, which is constructed based on the second feature encoder and the trained personalized classifier.

[0015] Identification unit: used to obtain identification results by utilizing the stored mental disorder identification model when it is determined that the user is not a new user.

[0016] According to a second aspect of the present invention, a wearable automated diagnostic method for mental disorders based on contrastive learning is provided. The method includes the following steps:

[0017] Acquire the user's multimodal physiological data and determine whether the user is a new user;

[0018] If the user is determined to be a new user, the following steps are performed: fine-tuning the offline pre-trained first feature encoder using the multimodal physiological data in a self-supervised contrastive learning manner to obtain a second feature encoder; extracting data features from the stored labeled multimodal physiological data via the second feature encoder; and training a personalized classifier using the data features as input to obtain a mental disorder recognition model, which is constructed based on the second feature encoder and the trained personalized classifier.

[0019] If the user is determined to be a non-new user, the stored mental disorder recognition model is used to obtain the recognition result.

[0020] Compared with existing technologies, the advantages of this invention are that the wearable automatic diagnosis scheme for mental disorders based on contrastive learning and multimodal physiological information fusion is generally divided into two stages. In the first stage, a feature encoder is pre-trained using a large amount of unlabeled superficial physiological information from patients and healthy individuals through self-supervised contrastive learning. In the second stage, the feature encoder from the first stage is fine-tuned using a very small amount of labeled superficial physiological information to obtain a personalized model specific to each user. This invention utilizes a universal wearable device to collect users' superficial unlabeled physiological information and integrates it through contrastive learning to identify various mental disorders, achieving the goals of real-time monitoring, early detection, and intervention.

[0021] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0023] Figure 1 This is an overall framework diagram of a wearable automatic diagnosis scheme for mental disorders based on contrastive learning according to an embodiment of the present invention;

[0024] Figure 2 This is a frame diagram of a wristband-type smart wearable device according to an embodiment of the present invention;

[0025] Figure 3 This is an external view of a wristband-type smart wearable device according to an embodiment of the present invention;

[0026] Figure 4 This is an internal structural diagram of a wristband-type smart wearable device according to an embodiment of the present invention;

[0027] Figure 5 This is a schematic diagram of the sampling rules used according to an embodiment of the present invention;

[0028] Figure 6 This is a schematic diagram of a backbone network and multimodal fusion strategy according to an embodiment of the present invention;

[0029] Figure 7 This is a flowchart of a new user registration process according to an embodiment of the present invention;

[0030] Figure 8 This is a schematic diagram of the process of identifying mental disorders according to an embodiment of the present invention;

[0031] Figure 9 This is a schematic diagram of the mental disorder identification results according to an embodiment of the present invention. Detailed Implementation

[0032] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0033] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0034] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0035] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0036] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0037] In general, the provided wearable automatic diagnostic system for mental disorders based on contrastive learning is implemented using a wearable device, including: a data acquisition unit for acquiring the user's multimodal physiological data; a user registration unit for, when determining that the user is a new user, performing the following: fine-tuning a pre-trained offline feature encoder using the multimodal physiological data in a self-supervised contrastive learning manner to obtain a second feature encoder; extracting data features from stored labeled multimodal physiological data via the second feature encoder; training a personalized classifier using the data features as input to obtain a mental disorder recognition model, which includes the second feature encoder and the trained personalized classifier; and a recognition unit for, when determining that the user is not a new user, obtaining a recognition result using the stored mental disorder recognition model. Each unit in the system can be implemented using a general-purpose processor, a dedicated processor, or a combination of FPGA, general-purpose sensors, etc., and software. Furthermore, various methods can be used to divide the units, as long as the functionality of this invention can be achieved.

[0038] This invention proposes a multimodal shallow physiological information fusion mental disorder identification system based on contrastive learning, utilizing ubiquitous wearable devices. The system consists of a two-stage training process. In the first stage, pre-training, a feature encoder is pre-trained in the cloud using contrastive learning with a large amount of unlabeled shallow physiological information from different users. In the second stage, users collect unlabeled shallow physiological information anytime, anywhere via ubiquitous wearable devices. This information is then used to personalize a user model using the feature encoder pre-trained in the first stage to identify whether a user suffers from a mental disorder. Contrastive learning learns high-quality, discriminative feature representations by maximizing the similarity of positive samples and minimizing the similarity of negative samples, enabling the model to more clearly distinguish subtle differences between different mental disorders.

[0039] Combination Figure 1As shown, the proposed wearable automatic diagnosis scheme for mental disorders based on contrastive learning consists of two stages. In the first stage, a general feature encoder is pre-trained in the cloud using multimodal fusion contrastive learning, based on a large amount of unlabeled superficial physiological information collected by the smart wearable device. This general feature encoder is then deployed to the smart wearable device. In the second stage, when a new user uses the smart wearable device, a small amount of unlabeled superficial physiological information is collected from the new user. This information is combined with previously imported model parameters and a small amount of data to quickly fine-tune a model suitable for the new user, thus completing the customization of a personalized model for that user. Next, the smart wearable device collects the user's superficial physiological information anytime, anywhere. After data preprocessing, this information is fed into the personalized model. The user's physiological indicators and the result of identifying whether they have a mental disorder can be displayed on the smart wearable device. This data and results can also be transmitted to other devices for display via wireless transmission protocols (such as Bluetooth). If the user has a mental disorder, the smart wearable device's voice module can also issue an alarm.

[0040] In one embodiment, a wristband-style smart wearable device is employed, using an STM32 microcontroller as the main control board to connect to various physiological sensors and run system algorithms. The smart wearable device is equipped with a display screen to show the recognition results and physiological indicators. This information can also be transmitted via Bluetooth to a terminal device for intuitive observation by the user, such as a mobile phone or tablet.

[0041] Figure 2 This is a schematic diagram of the overall framework of a wristband-style smart wearable device, which mainly consists of a data acquisition board and a main control board. The data acquisition board houses a bioelectrical impedance sensor, a blood oxygen sensor, a photoplethysmography (PPG) sensor, a skin conductance sensor, a skin temperature sensor, and a display module. The main control board contains a microcontroller, a voice module, a Bluetooth module, and a power module. The main control board communicates with the modules on the data acquisition board.

[0042] For example, the bioelectrical impedance sensor is connected to the main control board via an analog input pin, and the main control board's analog-to-digital converter (ADC) reads the respiratory information values. The main control board communicates asynchronously with the blood oxygen sensor via its serial port to read blood oxygen saturation data. The main control board communicates with the photoplethysmography (PPG) sensor using its ADC to read heart rate variability values. The main control board communicates with the electrodermal conductivity (EDA) sensor using its ADC to read changes in skin conductivity, thus monitoring EDA. The main control board communicates with the skin temperature sensor using its ADC to read skin temperature values. The main control board communicates with the Bluetooth module using the UART serial protocol, transmitting data from each sensor to the Bluetooth BLE module every 50ms. The Bluetooth BLE module then transmits the data to other terminals, for example, via Bluetooth 5.3. The main control board and the display module communicate via a low-speed serial connection using the main control board's I2C (or IIC) bus, displaying the values ​​from each sensor and the processing results on the screen. When the main control board receives data from each sensor and the output results from the model, it sends commands and data to the voice broadcast module via the main control board's UART serial port to broadcast the overall data situation and the final recognition results for the monitoring period.

[0043] Figure 3 This is an overall view of the wristband-style wearable device. Figure 4 This is a diagram of its internal structure. The top of the device has a display screen that shows the values ​​from the various sensors processed by the main control board, model recognition results, etc. Since these sensors need to be in close contact with the skin, they can be embedded in the bottom of the device, fitting snugly against the wrist. The Bluetooth module and voice broadcast module are embedded in the main control board's circuitry.

[0044] The following sections will focus on describing the processes of pre-training, customizing individual models, and applying the models.

[0045] I. Pre-trained Feature Encoder Stage

[0046] In daily life, unlabeled physiological data of individuals is readily available. However, labeled multimodal physiological information of individuals with mental disorders is much harder to obtain, and different mental disorders often share common symptoms, making it difficult to identify which disease an individual actually suffers from. Contrastive learning learns high-quality, discriminative feature representations by maximizing the similarity of positive samples and minimizing the similarity of negative samples, enabling the model to more clearly distinguish subtle differences between different mental disorders. Contrastive learning models are pre-trained in a self-supervised manner using unlabeled data from the source domain dataset, and can be rapidly generalized to different individuals through supervised fine-tuning with a small amount of labeled data.

[0047] 1. Sampling of positive and negative sample pairs

[0048] A key aspect of contrastive learning is the sampling of positive and negative sample pairs. Standard contrastive learning methods only coarsely divide data into positive and negative pairs along the time dimension, without finely dividing the data into positive and negative pairs across different modalities of physiological data. In one embodiment of this invention, coarse-grained and fine-grained positive and negative sample pairs are combined to segment multimodal physiological data, thereby enhancing representational power and achieving more comprehensive feature capture.

[0049] Figure 5 This illustrates the sampling rules for positive and negative samples in contrastive learning. With temporal alignment, a modality is selected as the anchor modality at time T1. Physiological data from other modalities at the same time are considered positive samples. Samples from the anchor modality at different time points are considered strong negative samples, and samples from other modalities at different time points are considered weak negative samples. The ultimate goal of contrastive learning is to maximize the similarity between positive samples and minimize the similarity between negative samples.

[0050] For example, coarse-grained sampling can be viewed as being implemented along the time dimension of a 1-second sample window. Positive samples are different modalities within the same 1-second window at the same time, while negative samples are 1-second windows at different times. Fine-grained sampling is implemented along the modalities within the 1-second window. Positive samples are mutual sampling of different modalities at the same time, and negative samples are divided into two types: strong negative samples are samples of the same modality at different times, and weak negative samples are samples of different modalities at different times. Combining this sampling method with residual variable convolution fusion achieves coarse-grained sample representation along the time dimension and fine-grained representation across different physiological modalities. This sampling method significantly improves the accuracy of the recognition results.

[0051] 2. Backbone Network and Multimodal Fusion Strategy

[0052] Variable convolution learns offsets, enabling the convolution kernel to adaptively adjust its sampling position and capture local details. When processing time-series data, variable convolution can flexibly adapt to different rhythms and fluctuations in time-series physiological data, capturing more accurate temporal features. In fusing multimodal physiological data, variable convolution can capture dynamic changes between different modalities or between time points within the same modality, thereby enhancing feature extraction and more accurately fusing data from different modalities.

[0053] In one embodiment, a residual variable convolutional network fusion method is proposed, combining the above sampling methods, to obtain local fusion features of physiological data from different modalities, as well as global unique features of each modality. Figure 6This diagram illustrates the backbone network and multimodal fusion strategy, where SpO2 represents blood oxygen saturation, BIA represents respiratory data, EDA represents skin conductance data, SKT represents skin temperature data, and PPG represents heart rate variability data. For example, using a 1-second window as the sample size for different modalities of physiological information, each modality first passes through different Bidirectional Long Short-Term Memory (BiLSTM) layers to extract its specific temporal features. After passing through a linear layer, these specific features are concatenated along the channel dimension to obtain a vector of (batch size, 5, 100). Next, the concatenated vector is fed into a Residual Variable Convolutional Network (DCN) to extract local fusion features and global features specific to each modality. Different modalities of physiological data are learned together using both coarse and fine granular approaches, and fused using the Residual Variable Convolutional Network, which helps to better represent the characteristics of patients and control groups for each disease.

[0054] A feature encoder f is pre-trained in the first stage using a large amount of unlabeled shallow physiological information processed by a filter bank. This stage can be completed in the cloud or on a server, thus saving computing resources for smart wearable devices.

[0055] In summary, this invention utilizes residual variable convolution for multimodal fusion, thereby extracting local fusion features and global features specific to each modality. By fusing physiological information from different channels through a residual variable convolutional network, both global information of each channel and local information of the fused modality are preserved. The fusion of physiological data from different modalities using coarse-grained and fine-grained learning combined with the residual variable convolutional network allows the system to better represent the characteristics of patients and control groups for each disease.

[0056] II. New User Personalization Model Phase

[0057] After pre-training the feature encoder, the trained feature encoder and a small amount of labeled multimodal physiological data are deployed into the wearable device. In everyday life scenarios, new users need to register when using this system to obtain a personalized model tailored to their needs. The registration process is as follows: Figure 7As shown, in this process, the wearable device worn by the user dynamically collects multimodal physiological data over a period of time (e.g., a few minutes). The local software then preprocesses this data, and the processed multimodal physiological data is fine-tuned using self-supervised contrastive learning to refine the feature encoder f from the pre-training stage. After fine-tuning, a new feature encoder g is obtained. Next, the local software randomly selects a small amount of labeled multimodal physiological data from within the system, processes the labeled data through encoder g to obtain features, and then inputs these features into a classifier composed of a lightweight transformer, a multilayer perceptron (MLP), a dropout layer, and a softmax layer. Finally, a personalized feature encoder g1 and classifier c1 are created for the new user. This completes the new user's registration, enabling real-time monitoring of the user's physiological indicators and mental state, and determining whether the user suffers from a mental disorder.

[0058] If a new user wears the device throughout the personalization process, they can obtain a diagnosis of their illness within minutes of the process ending. If the user does not wear the device, they will need to wear it to collect, for example, ten minutes of multimodal physiological data before obtaining a diagnosis.

[0059] In summary, this invention uses a filter bank combining Butterworth bandpass filtering and bandstop filtering to preprocess multimodal physiological information, and then uses a deep learning model to identify different categories of mental disorders. This not only improves the accuracy of identification, but also saves resources for wearable devices.

[0060] III. The Process of Systematic Identification of Mental Disorders

[0061] The process of systematically identifying mental disorders, such as Figure 8 As shown, firstly, in the cloud, a feature encoder is pre-trained using a large amount of unlabeled multimodal shallow physiological information processed by a filter bank through self-supervised contrastive learning. This feature encoder, along with a small amount of labeled multimodal shallow physiological information processed by the filter bank, is then embedded into a smart wearable device. Next, when a user uses the system, it first determines whether they are a new user. If so, the user is registered, and the unlabeled shallow physiological information of the new user is collected through the smart wearable device to fine-tune the feature encoder within the device. Then, a model specific to the new user is trained using the small amount of labeled shallow physiological information already present in the device. This allows for the monitoring of the new user's physiological information and the identification of whether they have a mental disorder. If the user is not a new user, the system directly monitors their physiological information and identifies whether they have a mental disorder.

[0062] This system allows users to monitor their mental and physical state anytime, anywhere. Smart wearable devices can also transmit the recognition results and physiological indicators to a mobile phone via Bluetooth for better analysis of changes in these states. If the system identifies a mental disorder, the result can be read aloud via voice to promptly alert the user. This enables early detection and intervention for mental disorders, thereby alleviating psychological and physical symptoms.

[0063] Accordingly, the present invention also provides a wearable automatic diagnostic method for mental disorders based on contrastive learning. The method includes the following steps: acquiring a user's multimodal physiological data and determining whether the user is a new user; if the user is a new user, performing the following: fine-tuning a pre-trained offline feature encoder using the multimodal physiological data in a self-supervised contrastive learning manner to obtain a second feature encoder; extracting data features from stored labeled multimodal physiological data via the second feature encoder; training a personalized classifier using the data features as input to obtain a mental disorder recognition model, which is constructed based on the second feature encoder and the trained personalized classifier; if the user is not a new user, obtaining a recognition result using the stored mental disorder recognition model.

[0064] Furthermore, to verify the feasibility of the present invention, preliminary experiments were conducted. In the experiments, 18 patients with depressive disorders, 20 patients with anxiety disorders, 16 patients with mood and affective disorders, and 18 normal individuals were used. The test accuracy for each individual in each category was obtained using the leave-one-out method. The results are as follows: Figure 9 As shown, where Figure 9 (a) is the accuracy rate of identifying depressive disorders. Figure 9 (b) is the accuracy rate of anxiety disorder identification. Figure 9 (c) is the accuracy rate of identifying mood and affective disorders. Figure 9 (d) is the recognition accuracy rate of normal people. As can be seen from Figure (9), the average recognition accuracy rate of depressive disorders is 82.51%, the average recognition accuracy rate of anxiety disorders is 81.97%, the average recognition accuracy rate of mood and affect disorders is 82.04%, and the average accuracy rate of normal people is 80.97%.

[0065] It should be noted that, without departing from the spirit and scope of this invention, those skilled in the art can make appropriate changes or modifications to the above embodiments. For example, for the preprocessing of the dataset, various statistical features of physiological signals, such as mean, variance, standard deviation, median, mode, maximum, and minimum, can be used, and these statistical features can then be used as substitutes for mental disorder identification. Alternatively, the physiological signals can be converted into time-frequency graphs for processing, and the processed features can be transmitted to the system framework for mental disorder identification. Furthermore, for the sampling rules of positive and negative sample pairs, strong and weak negative sample pairs can be combined without weighting coefficients to construct negative samples. In addition, a bidirectional GRU can be used instead of BiLSTM. With improvements, this invention is also applicable to the identification of other types of diseases, such as cardiovascular diseases, as long as multimodal physiological data of other types of diseases can be collected.

[0066] In summary, compared with the prior art, the present invention has the following advantages:

[0067] 1) This invention utilizes universal and convenient smart wearable devices to collect users' multimodal superficial physiological information. Since mental disorders often cause physiological reactions that lead to changes in superficial physiological information, which directly reflects the body's immediate physiological state, universal wearable devices, compared to dedicated devices or non-physiological data collection devices, are better able to meet the real-time requirements of individuals in daily life, enabling early detection and intervention of mental disorders and timely relief of psychological and physical symptoms.

[0068] 2) This invention employs a novel contrastive learning approach combined with residual variable convolution for multimodal fusion. By leveraging the characteristics of contrastive learning, it maximizes the similarity between positive samples and minimizes the similarity between negative samples. The residual variable convolution then extracts local fusion features as well as globally unique features for each modality. Physiological data from different modalities are fused using a combination of coarse-grained and fine-grained learning with a residual variable convolution network, enabling the system to better represent the characteristics of patients and control groups for each disease.

[0069] 3) Existing self-supervised fine-tuning strategies directly train classifiers for downstream tasks using a small amount of labeled data. This invention, however, addresses the different data for each user by first fine-tuning the feature encoder using unlabeled data through contrastive learning, and then training classifiers specific to each user using a small amount of labeled data, thereby obtaining a personalized model for each user.

[0070] 4) Most existing work focuses on identifying only one type of mental disorder, while a few work identify multiple types of mental disorders. However, in terms of the amount of data labels required and system performance, these works are inferior to contrastive learning. The novel contrastive learning method adopted in this invention enables the identification of multiple types of mental disorders and solves the problem of insufficient labels. The model trained using this method performs better than models trained with a large amount of labeled data.

[0071] 5) Different mental disorders often share common symptoms, making it difficult to identify which disease an individual has. This invention uses a novel contrastive learning strategy to construct positive and negative sample pairs, enabling in-depth analysis of multimodal physiological data at both coarse-grained and fine-grained levels across the entire temporal dimension. This allows for the joint identification of multiple categories of mental disorders (such as depressive disorders, anxiety disorders, and mood disorders), significantly improving the system's generalization ability compared to existing research and providing a more comprehensive identification of users' psychological states.

[0072] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0073] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0074] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0075] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0076] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0077] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0078] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0079] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0080] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A wearable automated diagnostic system for mental disorders based on contrastive learning, the system being implemented using wearable devices, comprising: Data acquisition unit: used to acquire the user's multimodal physiological data; User registration unit: When it is determined that the user is a new user, it performs the following: Fine-tuning the offline pre-trained first feature encoder using the multimodal physiological data in a self-supervised contrastive learning manner to obtain the second feature encoder; The stored labeled multimodal physiological data is used to extract data features via a second feature encoder; the data features are used as input to train a personalized classifier to obtain a mental disorder identification model, which is constructed based on the second feature encoder and the trained personalized classifier. Identification unit: used to obtain identification results using the mental disorder identification model when it is determined that the user is not a new user; In the comparative learning, the sampling rules for positive and negative samples are set as follows: for the multimodal physiological data, under the condition of temporal alignment, a modality is selected as the anchor modality at a set time, the physiological data of other modalities at the same time are regarded as positive samples, the sampling of the anchor modality at different time points is regarded as strong negative samples, and the sampling of other modalities at different time points is regarded as weak negative samples. The first feature encoder includes multiple bidirectional long short-term memory networks, linear layers, and residual variable convolutional networks. Each of the bidirectional long short-term memory networks is used to extract specific time-series features corresponding to a certain modality of physiological data. After passing through the linear layers, the specific time-series features of each modality of physiological data are concatenated in the channel dimension to obtain a concatenation vector. This concatenation vector is passed to the residual variable convolutional network to extract local fusion features and global features of each modality of physiological data.

2. The system according to claim 1, characterized in that, The personalized classifier includes a transformer network, a multilayer perceptron, a dropout layer, and a softmax layer.

3. The system according to claim 1, characterized in that, The multimodal physiological data include blood oxygen saturation, respiratory information, skin conductance, skin temperature, and heart rate variability.

4. The system according to claim 1, characterized in that, The offline pre-training process of the first feature encoder is carried out in the cloud or on a server, and offline pre-training is performed using unlabeled multimodal physiological data from different users in a comparative learning manner.

5. The system according to claim 1, characterized in that, The various modal physiological data monitored by the system and the recognition results of the mental disorder recognition model are transmitted to the terminal device via Bluetooth for display.

6. The system according to claim 3, characterized in that, The system is a wristband-type wearable device, including a main control board, a bioelectrical impedance sensor, a blood oxygen sensor, a photoplethysmography (PPG) sensor, a skin conductance sensor, a skin temperature sensor, a display module, a voice broadcast module, and a Bluetooth module. The bioelectrical impedance sensor is connected to the main control board via an analog input pin, and the main control board's analog-to-digital converter is used to read respiratory information values. The blood oxygen sensor communicates asynchronously via the main control board's serial port. The main control board communicates with the Bluetooth module using the UART serial protocol. The main control board communicates serially with the display module using an I2C bus. Commands and data are sent to the voice broadcast module via the main control board's UART serial port to broadcast physiological data information for the monitoring period and the recognition results of the mental disorder recognition model.

7. A wearable automated diagnostic method for mental disorders based on contrastive learning, comprising the following steps: Acquire the user's multimodal physiological data and determine whether the user is a new user; If the user is determined to be a new user, the following steps are performed: fine-tuning the offline pre-trained first feature encoder using the multimodal physiological data in a self-supervised contrastive learning manner to obtain the second feature encoder; The stored labeled multimodal physiological data is used to extract data features via a second feature encoder; the data features are used as input to train a personalized classifier to obtain a mental disorder identification model, which is constructed based on the second feature encoder and the trained personalized classifier. If the user is determined to be a non-new user, the mental disorder identification model is used to obtain the identification result; In the comparative learning, the sampling rules for positive and negative samples are set as follows: for the multimodal physiological data, under the condition of temporal alignment, a modality is selected as the anchor modality at a set time, the physiological data of other modalities at the same time are regarded as positive samples, the sampling of the anchor modality at different time points is regarded as strong negative samples, and the sampling of other modalities at different time points is regarded as weak negative samples. The first feature encoder includes multiple bidirectional long short-term memory networks, linear layers, and residual variable convolutional networks. Each of the bidirectional long short-term memory networks is used to extract specific time-series features corresponding to a certain modality of physiological data. After passing through the linear layers, the specific time-series features of each modality of physiological data are concatenated in the channel dimension to obtain a concatenation vector. This concatenation vector is passed to the residual variable convolutional network to extract local fusion features and global features of each modality of physiological data.

8. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the steps of the method according to claim 7.

Images