Mental health cloud service platform system and method based on multi-modal data analysis

Abstract

The invention discloses a psychological health cloud service platform system and method based on multi-modal data analysis. The system comprises a multi-modal data acquisition module; a heterogeneous data fusion analysis module; a dynamic intervention decision module; a block chain evidence tracking module; an edge computing terminal; the output end of the multi-modal data acquisition module is electrically connected with the input end of an edge computing terminal, the output end of the edge computing terminal is electrically connected with the input end of a heterogeneous data fusion analysis module, and the output end of the heterogeneous data fusion analysis module is electrically connected with the input end of a dynamic intervention decision module. According to the method, multi-modal data are integrated to construct a dynamic psychological portrait, a double-flow network is adopted to model spatio-temporal characteristics, an intervention scheme is matched based on risk levels, block chain evidence storage closed-loop management is performed, and an edge computing terminal performs cooperative computing through 5G encryption, so that real-time security is guaranteed, and the method is suitable for basic education scenes.

Description

Technical Field

[0001] The present invention relates to the field of mental health monitoring, and in particular to a mental health cloud service platform system and method based on multimodal data analysis. Background Art

[0002] In the context of basic education, there are three major technical bottlenecks in mental health monitoring and intervention: First, traditional screening tools are mainly based on static scales, which make it difficult to capture dynamic characteristics such as adolescents' emotional fluctuations and abnormal behaviors, resulting in high rates of missed diagnosis and misdiagnosis; second, physiological signals (such as heart rate variability), behavioral data (such as social media interaction patterns) and psychological assessment results have been in a state of separation for a long time, and there is a lack of multimodal data fusion and analysis mechanisms; third, existing intervention methods only stay at the risk warning level, and fail to form a full-process closed loop of "screening-intervention-tracking", resulting in a low conversion rate of high-risk cases.

[0003] In response to the above problems, the existing technology has the following defects: 1. Single-modal data collection cannot construct a complete psychological portrait; 2. The static screening model cannot adapt to the stage characteristics of adolescent psychological development; 3. Intervention measures are disconnected from individual characteristics and lack dynamic adaptation capabilities. Therefore, a mental health cloud service platform system and method based on multimodal data analysis is proposed. Summary of the Invention

[0004] In response to the deficiencies of the existing technology, the present invention provides a mental health cloud service platform system and method based on multimodal data analysis to solve the problems raised in the above background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solutions: a mental health cloud service platform system based on multimodal data analysis, comprising: A multimodal data acquisition module, which integrates a physiological sensor array, a behavior log acquisition unit, and a psychological scale engine. The output of the physiological sensor array is electrically connected to the input of the data preprocessing unit. The behavior log acquisition unit communicates with the school management system through an API interface. Heterogeneous data fusion analysis module, including a feature extraction submodule, a spatiotemporal correlation modeling submodule and a three-level warning submodule. The feature extraction submodule uses a two-stream convolutional network to process physiological signals and behavioral sequence data; A dynamic intervention decision module, the input of which is connected to the output of the three-level warning submodule. The dynamic intervention decision module has a built-in intelligent grading engine and an intervention plan library. The intelligent grading engine calls the corresponding intervention strategy template according to the risk level; A blockchain evidence tracking module, which records the entire process operation log through smart contracts and forms a two-way data exchange with the dynamic intervention decision module; Edge computing terminals are deployed in school computer rooms. They establish encrypted transmission channels with cloud platforms via 5G networks to perform collaborative computing for local preprocessing and cloud-based in-depth analysis. The output end of the multimodal data acquisition module is electrically connected to the input end of the edge computing terminal, the output end of the edge computing terminal is electrically connected to the input end of the heterogeneous data fusion analysis module, the output end of the heterogeneous data fusion analysis module is electrically connected to the input end of the dynamic intervention decision module, and the branches of the multimodal data acquisition module, edge computing terminal, heterogeneous data fusion analysis module, and dynamic intervention decision module are all electrically connected to the input end of the blockchain evidence tracking module; The behavior log collection unit uses a standardized API interface to periodically synchronize with the school management system's student attendance, course schedule, and grade record databases to ensure the real-time and accuracy of behavior log data, providing comprehensive behavioral context support for subsequent psychological analysis. The heterogeneous data fusion analysis module adopts advanced dual-stream convolutional network processing technology to effectively extract key features from physiological signals and behavioral sequences. Combined with spatiotemporal correlation modeling, it realizes dynamic tracking and early warning of students' mental states. The dynamic intervention decision-making module intelligently calls intervention strategies according to risk levels to ensure the targetedness and effectiveness of intervention measures. The introduction of the blockchain evidence tracking module not only enhances the security and traceability of data, but also realizes transparent management of the entire process operation through smart contracts. The deployment of edge computing terminals realizes the coordination of local preprocessing and cloud-based deep analysis, improves data processing efficiency, reduces latency, and provides strong technical support for students' mental health services.

[0006] Preferably, the multimodal data acquisition module further includes an environment sensing unit, which collects environmental parameters such as classroom light intensity and noise decibels through an Internet of Things device, and the output end of the environment sensing unit forms a data fusion link with the physiological sensor array; After preliminary processing, the collected data is transmitted to the data fusion interface of the physiological sensor array via wired or wireless means to achieve synchronous fusion of environmental parameters and physiological data, providing more comprehensive data support for subsequent analysis; Light intensity and noise decibels are important environmental parameters that can directly or indirectly affect students' emotions, attention, and learning efficiency. Combining these data with physiological data collected by physiological sensor arrays can more comprehensively assess students' mental state and improve the accuracy of early warning and intervention. In addition, this multimodal data fusion method helps to discover the potential correlation between environmental factors and psychological state, providing a scientific basis for subsequent environmental optimization and psychological intervention, thereby more effectively promoting students' mental health.

[0007] Preferably, the heterogeneous data fusion analysis module further includes an attention mechanism weighting unit, which assigns dynamic weights to physiological indicators, behavioral characteristics, and scale results, and the weight values ​​of the dynamic weights are adaptively adjusted according to individual historical data; This unit builds a dynamic weight model by analyzing the fluctuations of physiological indicators, changes in behavioral characteristics, and trends in scale results in individual historical data. During each data fusion, the unit automatically adjusts the weight of each data source based on the matching degree between the current data characteristics and the historical pattern, ensuring that the fusion result is more in line with the individual's actual situation. This unit can dynamically assign weights to physiological indicators, behavioral characteristics and scale results, and these weights can be adaptively adjusted based on individual historical data. This feature enables the platform to more accurately capture changes in individual psychological states when processing complex multimodal data, thereby improving the accuracy of risk assessment. At the same time, the dynamic weight adjustment mechanism enhances the flexibility and adaptability of the platform, enabling it to better serve user groups with different backgrounds and different needs.

[0008] Preferably, the dynamic intervention decision module includes an intervention effect evaluation submodule, which parses the intervention record text using NLP technology and generates a quantitative evaluation report based on the physiological signal change curve; In the dynamic intervention decision-making module, the intervention effect evaluation submodule first uses NLP technology to perform semantic analysis on the intervention record text, extracting key intervention measures and effect descriptions. At the same time, this submodule receives real-time physiological signal data from the physiological sensor array and generates physiological signal change curves. By comparing the physiological signal differences before and after the intervention and combining the text analysis results, it automatically generates an evaluation report containing quantitative indicators such as the effect of the intervention measures and the degree of improvement in physiological responses. This sub-module can not only parse the intervention record text and extract key information, but also combine the physiological signal change curve to quantitatively evaluate the actual effect of the intervention measures. This comprehensive evaluation method helps educators and parents understand the changes in students' mental state more accurately, adjust intervention strategies in a timely manner, and improve the pertinence and effectiveness of mental health services. At the same time, the quantitative evaluation report provides valuable data support for subsequent mental health research, which helps promote the continuous optimization and innovation of mental health services.

[0009] Preferably, the blockchain evidence tracking module further includes a privacy protection submodule, which uses homomorphic encryption technology to desensitize sensitive psychological data, and the privacy protection submodule is deployed on an edge computing terminal; When sensitive psychological data is transmitted to the edge computing terminal, the privacy protection submodule immediately activates the homomorphic encryption algorithm to encrypt the data and generate ciphertext data. When the ciphertext data is transmitted to the blockchain network for evidence storage, it remains encrypted to ensure the security of the data during transmission and storage. When the data needs to be queried or verified, it is decrypted using the corresponding decryption key to restore the original data for authorized users to use. It ensures the encryption status of psychological data throughout the entire process of collection, transmission, storage and query, effectively preventing data leakage and abuse. At the same time, the privacy protection sub-module is deployed on the edge computing terminal to realize localized pre-processing of data, reduce the risk of data transmission, and improve the response speed of the system. In addition, the application of homomorphic encryption technology enables data to be calculated and verified in an encrypted state, which not only ensures the availability of data, but also maintains the privacy of users, providing a solid guarantee for the data security of the mental health cloud service platform.

[0010] Preferably, it also includes a home-school collaboration terminal, which supports multi-terminal access of the parent-side APP and the teacher-side management platform, and the home-school collaboration terminal communicates with the dynamic intervention decision module in real time through the WebSocket protocol; The home-school collaboration terminal has a built-in WebSocket client, and the dynamic intervention decision module deploys a WebSocket server. The terminal initiates a connection request, and the server establishes a persistent connection after identity verification. Both parties exchange data through JSON format messages. For example, when the terminal sends a query command, the server returns the student's psychological intervention progress, achieving real-time communication. It breaks down information barriers, allowing parents and teachers to obtain the latest progress in student mental health interventions at any time, enhancing communication and collaboration between home and school. The real-time nature of the WebSocket protocol ensures the instant transmission of information, allowing both parties to quickly respond to changes in students' mental states. In addition, the multi-terminal access feature meets the needs of different user groups and improves the convenience and flexibility of use.

[0011] Preferably, the home-school collaboration terminal includes an emotion visualization submodule, which uses a three-dimensional emotion coordinate system to display the trajectory of changes in an individual's psychological state, and the coordinate system dimensions include activation, valence, and control; The emotion visualization submodule in the home-school collaboration terminal integrates a three-dimensional emotion computing model to map the collected student psychological data into three dimensions: activation, valence, and control. The model calculates the value of each dimension in real time based on the psychological characteristic map and dynamically updates the position in the three-dimensional coordinate system to intuitively display the changing trajectory of the student's psychological state. The introduction of the emotion visualization sub-module in the home-school collaboration terminal provides parents and teachers with an intuitive and scientific psychological state monitoring tool. Through the three-dimensional emotional coordinate system, it can clearly display the psychological changes of students in the three key dimensions of activation, valence and control, helping parents and teachers to grasp the students' psychological state more accurately. This visualization method not only enhances the readability of information, but also promotes effective communication between home and school, enabling both parties to formulate more personalized and targeted intervention measures based on a common understanding of the psychological state, thereby more effectively supporting students' mental health development.

[0012] The method of the mental health cloud service platform based on multimodal data analysis is based on the above-mentioned mental health cloud service platform system based on multimodal data analysis, and includes the following steps: Step 1: Data collection: Through the multimodal data collection module, students' physiological data, behavioral log data and psychological scale data are collected simultaneously; Step 2: Data preprocessing and fusion: The collected data is transmitted to the edge computing terminal for pre-processing, and then encrypted and transmitted to the cloud platform for deep fusion analysis to generate a comprehensive psychological feature map; Step 3: Early warning and intervention decision-making: Conduct risk assessment based on comprehensive psychological profiles, generate early warning signals, and invoke corresponding intervention strategy templates; Step 4: Evidence Storage and Tracking: Record the entire process operation log through the blockchain evidence tracking module; Step 5: Home-school collaboration: Real-time communication between parents and teachers is achieved through the home-school collaboration terminal, showing the trajectory of changes in individual psychological states; The edge computing terminal first cleans the received raw data to remove noise and invalid data. It then normalizes the physiological data and performs structured conversion on the behavioral log data. Finally, it uses a lightweight machine learning model to perform preliminary feature extraction on the pre-processed data, preparing for subsequent deep fusion analysis on the cloud. Through the multimodal data acquisition module, the mental health cloud service platform can synchronously and comprehensively collect students' physiological, behavioral and psychological scale data, providing a rich and multi-dimensional information basis for subsequent analysis. The data preprocessing and fusion steps are achieved through the collaborative work of edge computing terminals and cloud platforms, realizing the organic combination of localized data preprocessing and cloud-based in-depth analysis, which not only ensures the real-time data processing, but also improves the accuracy of analysis. The early warning and intervention decision-making mechanism can generate early warning signals in a timely manner based on the comprehensive psychological characteristic map and call the corresponding intervention strategy template, providing strong support for the early detection and intervention of mental health problems. The introduction of the blockchain evidence tracking module ensures the traceability and non-tamperability of the entire process operation log, enhancing the security and credibility of the system.

[0013] Preferably, the multimodal data acquisition module supports access to multiple data sources, the edge computing terminal has localized computing capabilities to process and analyze data in real time, the cloud platform uses a data fusion algorithm to generate a comprehensive psychological feature map, and the dynamic intervention decision module has a built-in intervention strategy template; The multimodal data acquisition module supports access to multiple data sources. By defining a standard data interface protocol, it is compatible with access to data sources such as physiological sensors and behavioral log collection devices of different brands and types, thus achieving unified data collection. The edge computing terminal uses a high-performance processor and a dedicated algorithm chip to optimize the allocation of local computing resources and achieve real-time processing and analysis. The cloud platform uses the multimodal fusion algorithm in deep learning to extract, align, and fuse features of different types of data to generate a comprehensive psychological feature map. The dynamic intervention decision module has built-in intervention strategy templates. Based on different risk levels and types of psychological problems, a team of experts formulates a variety of standardized intervention strategies. The multimodal data acquisition module supports access to multiple data sources and can comprehensively collect students' multi-dimensional data such as physiology, behavior, and psychology, providing rich materials for accurate assessment. The edge computing terminal has localized computing capabilities, which can process and analyze data in real time, reduce data transmission delays, and quickly respond to emergencies. The cloud platform uses data fusion algorithms to generate comprehensive psychological feature maps, deeply mine data associations, and accurately present students' psychological states. The dynamic intervention decision-making module has built-in intervention strategy templates, which can quickly call appropriate strategies according to risk levels to achieve timely and effective intervention, and jointly provide comprehensive, efficient, and accurate protection for students' mental health.

[0014] Preferably, an encrypted transmission protocol is used between the edge computing terminal and the cloud platform, the heterogeneous data fusion analysis module supports online learning and adaptive adjustment, and the dynamic intervention decision module supports user-defined intervention strategies; Using the SSL / TLS encryption protocol, data is encrypted before transmission to generate encrypted data packets with digital signatures to verify data integrity. During transmission, the data packets are transmitted to the cloud platform through a secure 5G network channel. The cloud platform decrypts and verifies the data after receiving it to ensure the confidentiality, integrity, and authenticity of the data transmission. The heterogeneous data fusion analysis module supports online learning and adaptive adjustment, enabling the system to continuously optimize the analysis model, improve the accuracy of the psychological characteristic map, and better adapt to the psychological changes of different students. The dynamic intervention decision-making module supports user-defined intervention strategies, providing flexibility for schools and parents. They can adjust the intervention plan according to actual conditions to meet personalized needs and improve the pertinence and effectiveness of mental health services. These features jointly promote the intelligent, secure and personalized development of the mental health cloud service platform system, providing more comprehensive and reliable protection for students' mental health.

[0015] In summary, compared with the prior art, the present invention provides a mental health cloud service platform system and method based on multimodal data analysis, which has the following beneficial effects: This invention achieves dynamic modeling of psychological portraits through a multimodal data fusion analysis mechanism, breaking through the limitations of traditional single-modal screening. The system integrates physiological sensor arrays, behavioral log collection units, and psychological scale engines to build a three-dimensional correlation analysis system of physiological signals-behavioral patterns-psychological assessments. It uses a two-stream convolutional network to extract spatiotemporal features and perform correlation modeling on multi-source heterogeneous data, synchronously capturing the emotional fluctuation characteristics and social behavior pattern changes of adolescents, forming the ability to dynamically track individual psychological states, and solving the problem of high missed diagnosis rate of traditional static scales. Based on the risk levels output by the three-level early warning submodule, the intelligent grading engine automatically matches the corresponding strategy templates in the intervention plan library. Combined with the full-process operation logs recorded by the blockchain evidence tracking module, this forms a traceable intervention effect evaluation chain. This mechanism ensures that intervention measures are dynamically matched to individual characteristics, significantly improving the intervention conversion rate for high-risk cases. The collaborative architecture of edge computing and blockchain is adopted to ensure the real-time performance and data security of the system. The edge computing terminals deployed in the school computer room realize the collaborative computing of local pre-processing and cloud analysis through 5G encrypted channels. While reducing data transmission delay, the operation logs are ensured to be tamper-proof through blockchain smart contracts. This hybrid architecture design not only meets the needs of basic education scenarios for real-time intervention, but also protects the security of students' privacy data through a distributed evidence storage mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of the mental health cloud service platform system of the invention based on multimodal data analysis.

[0017] Figure 2This is a step diagram of the method of the mental health cloud service platform based on multimodal data analysis of the invention. DETAILED DESCRIPTION

[0018] The present invention provides a technical solution, a mental health cloud service platform system based on multimodal data analysis, please refer to Figure 1 ,include: Multimodal data acquisition module, which integrates a physiological sensor array, a behavior log collection unit, and a psychological scale engine. The output of the physiological sensor array is electrically connected to the input of the data preprocessing unit, and the behavior log collection unit communicates with the school management system through an API interface. Heterogeneous data fusion analysis module, including feature extraction submodule, spatiotemporal correlation modeling submodule and three-level warning submodule. The feature extraction submodule uses a two-stream convolutional network to process physiological signals and behavioral sequence data; Dynamic intervention decision module: The input of the dynamic intervention decision module is connected to the output of the three-level warning submodule. The dynamic intervention decision module has a built-in intelligent grading engine and intervention plan library. The intelligent grading engine calls the corresponding intervention strategy template according to the risk level; The blockchain evidence tracking module records the entire process operation log through smart contracts, and forms a two-way data interaction with the dynamic intervention decision module; Edge computing terminals are deployed in school computer rooms. They establish encrypted transmission channels with cloud platforms via 5G networks to perform collaborative computing for local pre-processing and cloud-based in-depth analysis. The output of the multimodal data acquisition module is electrically connected to the input of the edge computing terminal, the output of the edge computing terminal is electrically connected to the input of the heterogeneous data fusion and analysis module, the output of the heterogeneous data fusion and analysis module is electrically connected to the input of the dynamic intervention decision module, and the branches of the multimodal data acquisition module, the edge computing terminal, the heterogeneous data fusion and analysis module, and the dynamic intervention decision module are all electrically connected to the input of the blockchain evidence tracking module; The behavior log collection unit uses a standardized API interface to periodically synchronize with the school management system's student attendance, course schedule, and grade record databases to ensure the real-time and accuracy of behavior log data, providing comprehensive behavioral context support for subsequent psychological analysis. The heterogeneous data fusion analysis module adopts advanced dual-stream convolutional network processing technology to effectively extract key features from physiological signals and behavioral sequences. Combined with spatiotemporal correlation modeling, it realizes dynamic tracking and early warning of students' mental states. The dynamic intervention decision-making module intelligently calls intervention strategies according to risk levels to ensure the targetedness and effectiveness of intervention measures. The introduction of the blockchain evidence tracking module not only enhances the security and traceability of data, but also realizes transparent management of the entire process operation through smart contracts. The deployment of edge computing terminals realizes the coordination of local preprocessing and cloud-based deep analysis, improves data processing efficiency, reduces latency, and provides strong technical support for students' mental health services.

[0019] See also Figure 1 ,The multimodal data acquisition module also includes an environment perception unit, which ,collects environmental parameters such as classroom light intensity and noise decibels through IoT devices. The output ,end of the environment perception unit forms a data fusion link with the physiological ,sensor array; After preliminary processing, the collected data is transmitted to the data fusion interface of the physiological sensor array via wired or wireless means to achieve synchronous fusion of environmental parameters and physiological data, providing more comprehensive data support for subsequent analysis; Light intensity and noise decibels are important environmental parameters that can directly or indirectly affect students' emotions, attention, and learning efficiency. Combining these data with physiological data collected by physiological sensor arrays can more comprehensively assess students' mental state and improve the accuracy of early warning and intervention. In addition, this multimodal data fusion method helps to discover the potential correlation between environmental factors and psychological state, providing a scientific basis for subsequent environmental optimization and psychological intervention, thereby more effectively promoting students' mental health.

[0020] See also Figure 1 ,The heterogeneous data fusion and analysis module also includes an ,attention mechanism weighting unit, which assigns dynamic weights to physiological ,indicators, behavioral characteristics and scale results, and the weight values ​​of ,dynamic weights are adaptively adjusted based on individual historical ,data; This unit builds a dynamic weight model by analyzing the fluctuations of physiological indicators, changes in behavioral characteristics, and trends in scale results in individual historical data. During each data fusion, the unit automatically adjusts the weight of each data source based on the matching degree between the current data characteristics and the historical pattern, ensuring that the fusion result is more in line with the individual's actual situation. This unit can dynamically assign weights to physiological indicators, behavioral characteristics and scale results, and these weights can be adaptively adjusted based on individual historical data. This feature enables the platform to more accurately capture changes in individual psychological states when processing complex multimodal data, thereby improving the accuracy of risk assessment. At the same time, the dynamic weight adjustment mechanism enhances the flexibility and adaptability of the platform, enabling it to better serve user groups with different backgrounds and different needs.

[0021] See also Figure 1 ,The dynamic intervention decision module includes an intervention effect evaluation submodule, which ,parses the intervention record text using NLP technology and generates a ,quantitative evaluation report in combination with the physiological signal ,change curve; In the dynamic intervention decision-making module, the intervention effect evaluation submodule first uses NLP technology to perform semantic analysis on the intervention record text, extracting key intervention measures and effect descriptions. At the same time, this submodule receives real-time physiological signal data from the physiological sensor array and generates physiological signal change curves. By comparing the physiological signal differences before and after the intervention and combining the text analysis results, it automatically generates an evaluation report containing quantitative indicators such as the effect of the intervention measures and the degree of improvement in physiological responses. This sub-module can not only parse the intervention record text and extract key information, but also combine the physiological signal change curve to quantitatively evaluate the actual effect of the intervention measures. This comprehensive evaluation method helps educators and parents understand the changes in students' mental state more accurately, adjust intervention strategies in a timely manner, and improve the pertinence and effectiveness of mental health services. At the same time, the quantitative evaluation report provides valuable data support for subsequent mental health research, which helps promote the continuous optimization and innovation of mental health services.

[0022] See also Figure 1 ,The blockchain evidence tracking module also includes a privacy protection submodule, which uses homomorphic encryption technology to desensitize sensitive psychological data. The privacy protection submodule is deployed on the edge computing terminal; When sensitive psychological data is transmitted to the edge computing terminal, the privacy protection submodule immediately activates the homomorphic encryption algorithm to encrypt the data and generate ciphertext data. When the ciphertext data is transmitted to the blockchain network for evidence storage, it remains encrypted to ensure the security of the data during transmission and storage. When the data needs to be queried or verified, it is decrypted using the corresponding decryption key to restore the original data for authorized users to use. It ensures the encryption status of psychological data throughout the entire process of collection, transmission, storage and query, effectively preventing data leakage and abuse. At the same time, the privacy protection sub-module is deployed on the edge computing terminal to realize localized pre-processing of data, reduce the risk of data transmission, and improve the response speed of the system. In addition, the application of homomorphic encryption technology enables data to be calculated and verified in an encrypted state, which not only ensures the availability of data, but also maintains the privacy of users, providing a solid guarantee for the data security of the mental health cloud service platform.

[0023] See also Figure 1 , also includes a home-school collaboration terminal, which supports multi-terminal access of the parent-side APP and the teacher-side management platform. The home-school collaboration terminal communicates with the dynamic intervention decision module in real time through the WebSocket protocol; The home-school collaboration terminal has a built-in WebSocket client, and the dynamic intervention decision module deploys a WebSocket server. The terminal initiates a connection request, and the server establishes a persistent connection after identity verification. Both parties exchange data through JSON format messages. For example, when the terminal sends a query command, the server returns the student's psychological intervention progress, achieving real-time communication. It breaks down information barriers, allowing parents and teachers to obtain the latest progress in student mental health interventions at any time, enhancing communication and collaboration between home and school. The real-time nature of the WebSocket protocol ensures the instant transmission of information, allowing both parties to quickly respond to changes in students' mental states. In addition, the multi-terminal access feature meets the needs of different user groups and improves the convenience and flexibility of use.

[0024] See also Figure 1 ,The home-school collaborative terminal includes an emotion visualization sub-module, which uses a three-dimensional emotion coordinate system to ,display the trajectory of individual psychological state changes, and the coordinate ,system dimensions include activation, valence, and control; The emotion visualization submodule in the home-school collaboration terminal integrates a three-dimensional emotion computing model to map the collected student psychological data into three dimensions: activation, valence, and control. The model calculates the value of each dimension in real time based on the psychological characteristic map and dynamically updates the position in the three-dimensional coordinate system to intuitively display the changing trajectory of the student's psychological state. The introduction of the emotion visualization sub-module in the home-school collaboration terminal provides parents and teachers with an intuitive and scientific psychological state monitoring tool. Through the three-dimensional emotional coordinate system, it can clearly display the psychological changes of students in the three key dimensions of activation, valence and control, helping parents and teachers to grasp the students' psychological state more accurately. This visualization method not only enhances the readability of information, but also promotes effective communication between home and school, enabling both parties to formulate more personalized and targeted intervention measures based on a common understanding of the psychological state, thereby more effectively supporting students' mental health development.

[0025] Method of mental health cloud service platform based on multimodal data analysis, based on the above mental health cloud service platform system based on multimodal data analysis, please refer to Figure 1 and Figure 2 , including the following steps: Step 1: Data collection: Through the multimodal data collection module, students' physiological data, behavioral log data and psychological scale data are collected simultaneously; Step 2: Data preprocessing and fusion: The collected data is transmitted to the edge computing terminal for pre-processing, and then encrypted and transmitted to the cloud platform for deep fusion analysis to generate a comprehensive psychological feature map; Step 3: Early warning and intervention decision-making: Conduct risk assessment based on comprehensive psychological profiles, generate early warning signals, and invoke corresponding intervention strategy templates; Step 4: Evidence Storage and Tracking: Record the entire process operation log through the blockchain evidence tracking module; Step 5: Home-school collaboration: Real-time communication between parents and teachers is achieved through the home-school collaboration terminal, showing the trajectory of changes in individual psychological states; The edge computing terminal first cleans the received raw data to remove noise and invalid data. It then normalizes the physiological data and performs structured conversion on the behavioral log data. Finally, it uses a lightweight machine learning model to perform preliminary feature extraction on the pre-processed data, preparing for subsequent deep fusion analysis on the cloud. Through the multimodal data acquisition module, the mental health cloud service platform can synchronously and comprehensively collect students' physiological, behavioral and psychological scale data, providing a rich and multi-dimensional information basis for subsequent analysis. The data preprocessing and fusion steps are achieved through the collaborative work of edge computing terminals and cloud platforms, realizing the organic combination of localized data preprocessing and cloud-based in-depth analysis, which not only ensures the real-time data processing, but also improves the accuracy of analysis. The early warning and intervention decision-making mechanism can generate early warning signals in a timely manner based on the comprehensive psychological characteristic map and call the corresponding intervention strategy template, providing strong support for the early detection and intervention of mental health problems. The introduction of the blockchain evidence tracking module ensures the traceability and non-tamperability of the entire process operation log, enhancing the security and credibility of the system.

[0026] See also Figure 1 and Figure 2,The multimodal data acquisition module supports access to multiple data sources, the edge computing terminal has local computing capabilities for real-time data processing and analysis, the cloud platform uses data fusion algorithms to generate comprehensive psychological feature maps, and the dynamic intervention decision module has built-in intervention strategy templates; The multimodal data acquisition module supports access to multiple data sources. By defining a standard data interface protocol, it is compatible with access to data sources such as physiological sensors and behavioral log collection devices of different brands and types, thus achieving unified data collection. The edge computing terminal uses a high-performance processor and a dedicated algorithm chip to optimize the allocation of local computing resources and achieve real-time processing and analysis. The cloud platform uses the multimodal fusion algorithm in deep learning to extract, align, and fuse features of different types of data to generate a comprehensive psychological feature map. The dynamic intervention decision module has built-in intervention strategy templates. Based on different risk levels and types of psychological problems, a team of experts formulates a variety of standardized intervention strategies. The multimodal data acquisition module supports access to multiple data sources and can comprehensively collect students' multi-dimensional data such as physiology, behavior, and psychology, providing rich materials for accurate assessment. The edge computing terminal has localized computing capabilities, which can process and analyze data in real time, reduce data transmission delays, and quickly respond to emergencies. The cloud platform uses data fusion algorithms to generate comprehensive psychological feature maps, deeply mine data associations, and accurately present students' psychological states. The dynamic intervention decision-making module has built-in intervention strategy templates, which can quickly call appropriate strategies according to risk levels to achieve timely and effective intervention, and jointly provide comprehensive, efficient, and accurate protection for students' mental health.

[0027] See also Figure 1 and Figure 2 ,An encrypted transmission protocol is used between the edge computing terminal and the cloud platform, the heterogeneous data fusion and analysis module supports online learning and adaptive adjustment, and the dynamic intervention decision module supports user-defined intervention strategies; Using the SSL / TLS encryption protocol, data is encrypted before transmission to generate encrypted data packets with digital signatures to verify data integrity. During transmission, the data packets are transmitted to the cloud platform through a secure 5G network channel. The cloud platform decrypts and verifies the data after receiving it to ensure the confidentiality, integrity, and authenticity of the data transmission. The heterogeneous data fusion analysis module supports online learning and adaptive adjustment, enabling the system to continuously optimize the analysis model, improve the accuracy of the psychological characteristic map, and better adapt to the psychological changes of different students. The dynamic intervention decision-making module supports user-defined intervention strategies, providing flexibility for schools and parents. They can adjust the intervention plan according to actual conditions to meet personalized needs and improve the pertinence and effectiveness of mental health services. These features jointly promote the intelligent, secure and personalized development of the mental health cloud service platform system, providing more comprehensive and reliable protection for students' mental health.

[0028] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0029] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A mental health cloud service platform system based on multimodal data analysis, characterized by: include: A multimodal data acquisition module, which integrates a physiological sensor array, a behavior log acquisition unit, and a psychological scale engine. The output of the physiological sensor array is electrically connected to the input of the data preprocessing unit. The behavior log acquisition unit communicates with the school management system through an API interface. Heterogeneous data fusion analysis module, including a feature extraction submodule, a spatiotemporal correlation modeling submodule and a three-level warning submodule. The feature extraction submodule uses a two-stream convolutional network to process physiological signals and behavioral sequence data; A dynamic intervention decision module, the input of which is connected to the output of the three-level warning submodule. The dynamic intervention decision module has a built-in intelligent grading engine and an intervention plan library. The intelligent grading engine calls the corresponding intervention strategy template according to the risk level; A blockchain evidence tracking module, which records the entire process operation log through smart contracts and forms a two-way data exchange with the dynamic intervention decision module; Edge computing terminals are deployed in school computer rooms. They establish encrypted transmission channels with cloud platforms via 5G networks to perform collaborative computing for local preprocessing and cloud-based in-depth analysis. The output end of the multimodal data acquisition module is electrically connected to the input end of the edge computing terminal, the output end of the edge computing terminal is electrically connected to the input end of the heterogeneous data fusion analysis module, the output end of the heterogeneous data fusion analysis module is electrically connected to the input end of the dynamic intervention decision module, and the branches of the multimodal data acquisition module, edge computing terminal, heterogeneous data fusion analysis module and dynamic intervention decision module are all electrically connected to the input end of the blockchain evidence tracking module.

2. The mental health cloud service platform system based on multimodal data analysis according to claim 1 is characterized by: The multimodal data acquisition module also includes an environment perception unit, which collects environmental parameters such as classroom light intensity and noise decibels through an Internet of Things device. The output end of the environment perception unit forms a data fusion link with the physiological sensor array.

3. The mental health cloud service platform system based on multimodal data analysis according to claim 1 is characterized by: The heterogeneous data fusion analysis module also includes an attention mechanism weighting unit, which assigns dynamic weights to physiological indicators, behavioral characteristics and scale results, and the weight values ​​of the dynamic weights are adaptively adjusted according to individual historical data.

4. The mental health cloud service platform system based on multimodal data analysis according to claim 1 is characterized by: The dynamic intervention decision module includes an intervention effect evaluation submodule, which parses the intervention record text through NLP technology and generates a quantitative evaluation report based on the physiological signal change curve.

5. The mental health cloud service platform system based on multimodal data analysis according to claim 1 is characterized by: The blockchain evidence tracking module also includes a privacy protection submodule, which uses homomorphic encryption technology to desensitize sensitive psychological data. The privacy protection submodule is deployed on an edge computing terminal.

6. The mental health cloud service platform system based on multimodal data analysis according to claim 1 is characterized by: It also includes a home-school collaboration terminal, which supports multi-terminal access of the parent-side APP and the teacher-side management platform. The home-school collaboration terminal communicates with the dynamic intervention decision module in real time through the WebSocket protocol.

7. The mental health cloud service platform system based on multimodal data analysis according to claim 6 is characterized by: The home-school collaboration terminal includes an emotion visualization submodule, which uses a three-dimensional emotion coordinate system to display the trajectory of changes in an individual's psychological state. The coordinate system dimensions include activation, valence, and control.

8. A method for a mental health cloud service platform based on multimodal data analysis, based on the mental health cloud service platform system based on multimodal data analysis according to any one of claims 1 to 7, characterized in that: The steps include: Step 1: Data collection: Through the multimodal data collection module, students' physiological data, behavioral log data and psychological scale data are collected simultaneously; Step 2: Data preprocessing and fusion: The collected data is transmitted to the edge computing terminal for pre-processing, and then encrypted and transmitted to the cloud platform for deep fusion analysis to generate a comprehensive psychological feature map; Step 3: Early warning and intervention decision-making: Conduct risk assessment based on comprehensive psychological profiles, generate early warning signals, and invoke corresponding intervention strategy templates; Step 4: Evidence Storage and Tracking: Record the entire process operation log through the blockchain evidence tracking module; Step 5: Home-school collaboration: Real-time communication between parents and teachers is achieved through the home-school collaboration terminal, showing the changing trajectory of individual psychological states.

9. The method of the mental health cloud service platform based on multimodal data analysis according to claim 8, characterized in that: The multimodal data acquisition module supports access to multiple data sources, the edge computing terminal has localized computing capabilities to process and analyze data in real time, the cloud platform uses a data fusion algorithm to generate a comprehensive psychological characteristic map, and the dynamic intervention decision module has a built-in intervention strategy template.

10. The method of the mental health cloud service platform based on multimodal data analysis according to claim 8, characterized in that: An encrypted transmission protocol is used between the edge computing terminal and the cloud platform. The heterogeneous data fusion analysis module supports online learning and adaptive adjustment. The dynamic intervention decision module supports user-defined intervention strategies.

Images