An implementation method of multi-modal four-end collaborative mental health and behavior safety management based on an AI large model

Abstract

The application provides an implementation method of multi-modal four-end collaborative mental health and behavior safety management based on an AI large model, and relates to the technical field of mental health services. The method collects five types of heterogeneous data, including standardized scales, behavior labels, physiological indicators, voice texts and time sequence history, through a service object end, an executor end, an intelligent sensing end and a family member end. After data cleaning, feature extraction and time alignment, a cross-modal attention fusion network is used to calculate a comprehensive psychological risk index and sub-risk probabilities such as depression, anxiety, violent tendency and fall risk. A large language model fine-tuned by domain knowledge is called to automatically generate three versions of differentiated and personalized intervention schemes, including a service object version, an executor version and a family member version. The method adopts a modular general architecture and can seamlessly adapt to two core scenes, namely, adolescent campus moral education and nursing homes, effectively solving the defects of single data dimension, low evaluation accuracy, homogeneous schemes and information silos in the prior art.

Description

Technical Field

[0001] This invention relates to the field of mental health service technology, specifically to a method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI big data model. Background Technology

[0002] As society continues to pay increasing attention to mental health, psychological crises among teenagers on campuses and mental health and behavioral safety issues among the elderly are becoming increasingly prominent. Currently, various psychological assessment and monitoring systems have emerged in the industry, but existing technologies generally suffer from the following core shortcomings, making it difficult to meet the needs of refined, closed-loop, and large-scale management: First, the data collection dimensions are limited, resulting in insufficient assessment accuracy. Existing systems mostly rely on standardized single-item assessments or monitoring by a single physiological sensor such as heart rate and body movement, lacking the integration of multi-dimensional data such as emotion, behavior, social interaction, voice, and historical trends. Single data sources are easily affected by subjective responses and environmental interference, leading to high rates of underreporting and false positives in psychological risks. The early identification rate of depression risk is generally less than 30%, making precise early intervention impossible.

[0003] Second, the multi-source heterogeneous data is fragmented and lacks fusion capabilities. Structured scale data and behavioral label data are processed independently from unstructured speech and text data, without deep cross-modal fusion; the data from different systems are not interconnected, forming information silos, making it impossible to comprehensively assess individual psychological states, and the assessment results are highly one-sided.

[0004] Third, intervention programs suffer from severe homogenization and poor implementation. Most systems output template-based and generic suggestions, such as "spend more time with your family" and "communicate more," without generating differentiated programs based on the age, personality, cognitive level, and historical circumstances of the service recipients. Furthermore, they fail to differentiate between the three target audiences: service recipients, frontline implementers, and family members, resulting in poor content targeting and an actual adoption rate of less than 40%, significantly diminishing the intervention's effectiveness.

[0005] Fourth, the lack of a closed-loop feedback mechanism hinders continuous optimization. Existing systems mostly follow a one-way process of "assessment - report generation," with no follow-up on the effects after intervention, and the solutions cannot be automatically iterated based on feedback; the systems lack self-learning capabilities, their effectiveness diminishes after long-term use, and they cannot adapt to the dynamically changing psychological states of individuals.

[0006] Fifth, multi-party collaboration is inefficient, and early warning response is delayed. Data is not shared among service recipients, teachers / caregivers, administrators, and family members, making it impossible to transmit abnormal events in real time. Early warnings are only pushed to a single position, lacking a timeout escalation mechanism, which can easily lead to missing the best intervention opportunity and even causing safety accidents and disputes.

[0007] Sixth, poor scenario adaptability and high cost of redundant construction. The campus psychological system and the elderly psychological system are developed independently, and the underlying architecture is not universal; the same technical framework cannot be reused across scenarios, resulting in high R&D and deployment costs and making it difficult to achieve economies of scale.

[0008] In summary, the industry urgently needs an intelligent management method for mental health and behavioral safety that can achieve deep integration of multimodal data, personalized intervention for each individual, closed-loop management across four terminals, and cross-scenario applicability, in order to address many pain points of existing technologies. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a method for implementing multimodal, four-terminal collaborative mental health and behavioral safety management based on an AI large-scale model. It improves risk identification accuracy through multimodal attention fusion, generates three personalized solutions through domain-specific fine-tuning of the large model, breaks down information silos through a four-terminal collaborative architecture, constructs a continuous optimization loop through reinforcement learning, and adopts a modular architecture to achieve applicability in both campus and elderly care scenarios. Ultimately, it achieves full-process, intelligent, and closed-loop management of mental health and behavioral safety. This solves the problems of existing technologies, such as single data dimensions, low assessment accuracy, homogenized solutions, lack of closed-loop optimization, information silos, and inapplicability across scenarios.

[0010] To achieve the above objectives, the present invention provides the following technical solution: A method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model includes the following steps: Step S1: Multi-source data collection. Through four data entry points—the service recipient end, the executor end, the intelligent sensing end, and the family member end—scale assessment data, behavioral observation tags, physiological monitoring indicators, voice and text data, and historical risk time series data are collected simultaneously to form a multi-source heterogeneous raw dataset. Step S2: Data Preprocessing and Feature Extraction The raw data is cleaned, desensitized, and time-aligned. Structured scale features, behavioral level features, physiological quantitative features, voice emotion vector features, and historical trend features are extracted and mapped to a unified dimension of input feature vector. Step S3: Multimodal fusion risk prediction Input the input feature vector into the multimodal fusion risk prediction model based on cross-modal attention mechanism, and output a comprehensive psychological risk index in the range of 0 to 1, as well as the probability of four sub-risks: depression, anxiety, violent tendencies, and fall risk. Step S4: Generate Personalization Scheme for Large Model By invoking a large language model finely tuned with knowledge from the psychology / care field, and combining individual profiles, risk indices, and trends, three versions of differentiated intervention guidance plans are automatically generated for service recipients, implementers, and family members. Step S5: Four-terminal collaborative early warning and task scheduling Based on the comprehensive risk index, a three-tiered warning system of blue, yellow, and red is triggered, automatically generating corresponding intervention tasks and pushing them to the corresponding terminals; through the linkage of four terminals—the service recipient terminal, the executor terminal, the manager terminal, and the family terminal—task execution check-in, status tracking, and automatic escalation upon timeout are realized; Step S6: Closed-loop feedback and model optimization Data on changes in emotion and behavior after intervention are collected, intervention reward values ​​are calculated, and the policy network is updated through deep reinforcement learning algorithms to continuously optimize the accuracy of intervention program recommendations, forming a complete closed loop of "monitoring-early warning-intervention-feedback-optimization".

[0011] Further, in step S1, the scale assessment data includes the Adolescent Learning Stress Scale, the Self-Rating Depression Scale, the Geriatric Depression Scale GDS-5, and the Psychological Resilience Scale; the physiological monitoring indicators include heart rate, heart rate variability (HRV), blood oxygen, sleep efficiency, frequency of getting out of bed, and respiratory rate; the voice-text data is transcribed into text by automatic speech recognition (ASR) and the emotion probability distribution vector is extracted.

[0012] Furthermore, in step S3, the processing flow of the multimodal fusion risk prediction model is as follows: 1) Independent encoding of each modality: The features of the five modalities are mapped to a unified 256-dimensional feature space through a multilayer perceptron (MLP); 2) Cross-modal attention fusion: Using speech modal features as the query vector Query and other modal features as key vectors Key and value vectors Value, a weighted fusion feature is calculated through a multi-head attention mechanism, and residual connections and layer normalization are added; 3) Multi-task output: The comprehensive psychological risk index, the probability of four types of subdivided risks, and the risk contribution attribution results based on the SHAP algorithm are output through three fully connected branches.

[0013] Furthermore, in step S4, the large language model uses a domestically developed basic model combined with LoRA low-rank adaptation technology for domain fine-tuning. The fine-tuning training set includes mental health dialogue data, elderly care standards, school moral education manuals, and historical effective intervention cases. The generated three versions of the scheme are as follows: Targeted version: Includes daily tasks, emotion regulation methods, and positive guidance content; Implementer's version: Includes a care / counseling checklist, key points to focus on, and step-by-step implementation instructions; Family version: Includes a guide to companionship, communication techniques, and suggestions for family cooperation; After the plan is generated, it undergoes three post-processing steps: security verification, age and cognitive consistency verification, and feasibility verification.

[0014] Furthermore, in step S5, the functions of each terminal in the four-terminal collaborative architecture are as follows: On the user side: It provides functions such as self-assessment scale, voice diary, mood check-in, and personalized task reception; On the executor side: It provides functions such as behavior observation and recording, rapid voice reporting, intervention task reception and check-in, and one-click abnormality reporting; Administrator side: Provides functions such as psychological weather map data dashboard, early warning center, personnel management, report export, and task assignment; Family member side: Provides life briefings, tiered early warning notifications, companionship guidance, and satisfaction feedback functions.

[0015] Furthermore, in step S5, the three-level graded early warning rule is as follows: Blue Level 1 Warning: The overall risk index is ≥0.5 and <0.7, or the probability of a single sub-category of risk is >0.6. The task is pushed to the executor, requiring the handling to be completed within 24 hours. Yellow Level 2 Alert: If the comprehensive risk index is ≥0.5 twice consecutively, or the comprehensive risk index is ≥0.7, the task will be pushed to both the executor and the manager, requiring completion within 12 hours; Red Level 3 Alert: Comprehensive risk index ≥ 0.85, or violent tendency / probability of fall > 0.8. The task is simultaneously pushed to the executor, manager, and family members, requiring a response and handling within 2 hours. All tasks that are not checked in and confirmed within the time limit will be automatically upgraded to the next higher warning level.

[0016] Furthermore, in step S6, the deep reinforcement learning algorithm employs a DQN deep Q-network or a PPO proximal policy optimization algorithm, with the specific rules being: State st: The multimodal feature vector at time t and the ID of the preceding intervention scheme; Action at: The ID of the intervention plan type recommended by the system, with a total of M discrete plans; The reward rt = w1・Δemotional score + w2・Δbehavioral level - w3・execution cost, where Δemotional score is the change in self-reported emotion before and after intervention, Δbehavioral level is the improvement in observed behavior, and execution cost is the time spent by caregivers / teachers and the difficulty of cooperation with family members; Set up an experience replay pool with a capacity of 10000 (s_t, a_t, r_t, s_{t+1}). Update the policy network once every 100 new experience samples. The target network adopts a soft update method with an update coefficient τ=0.01.

[0017] Furthermore, the method adopts a modular and universal architecture, sets up a scenario adaptation layer, and can switch between the youth campus scenario and the elderly care institution scenario with one click by loading different scenario configuration files. When switching scenarios, the core AI engine, data flow logic, and four-terminal architecture remain unchanged, and only the scale system, behavior label system, solution script template and risk threshold parameters are replaced.

[0018] Furthermore, the risk contribution attribution results output in step S3 clearly indicate the contribution ratio of each dimension—scale, behavior, physiology, voice, and history—to the risk results, generating an interpretable risk report for review by professionals.

[0019] Furthermore, all raw data is stored using AES-256 encryption, and the original content of voice and text data is deleted immediately after feature extraction; federated learning mode is supported, allowing multiple institutions to train locally, only uploading encrypted model gradients, and not sharing original user data, thus meeting data privacy compliance requirements.

[0020] This invention provides a method for implementing multimodal, four-terminal collaborative mental health and behavioral safety management based on a large AI model. It has the following beneficial effects: 1. This invention employs multimodal attention fusion to achieve complementary data across all dimensions, including structured, unstructured, physiological, phonological, and historical data, thereby increasing the early identification rate of depression risk from 28% to 84% and significantly reducing the false positive and false negative rates.

[0021] 2. The AI ​​risk prediction model used in this invention can identify risks such as violent conflict, falls, and worsening depression in advance, and push forward prevention plans, which can reduce violent conflict by an average of 75% per month and prevent falls by 83%.

[0022] 3. This invention uses a domain-fine-tuned large model to generate three differentiated solutions, allowing for personalized solutions for each user. The solution adoption rate reaches 92%, which is much higher than the 40% of traditional template-based solutions, resulting in a significant improvement in effectiveness.

[0023] 4. This invention uses voice input for rapid recording and AI for automatic feature extraction, which reduces the time spent on data recording by frontline staff from 35 minutes per day to 9 minutes, improving efficiency by 74%.

[0024] 5. This invention uses reinforcement learning and experience replay mechanisms to achieve autonomous iteration of the solution. The effect of secondary intervention is 30% higher than that of the initial solution, making the system more and more accurate with use. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the implementation method of multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model according to the present invention. Figure 2 This is a diagram of the overall four-terminal collaborative architecture of the system of the present invention; Figure 3 This is a system data flow diagram of the present invention; Figure 4 This is a flowchart of the multimodal data fusion and AI large model analysis process of the present invention; Figure 5 This is a flowchart of the closed-loop feedback and reinforcement learning optimization process of the present invention; Figure 6 This is a flowchart illustrating the early warning classification and automatic allocation process of the present invention; Figure 7 This is a schematic diagram illustrating the deployment of the system of the present invention in a school setting; Figure 8 This is a schematic diagram illustrating the deployment of the system of the present invention in a nursing home setting. Detailed Implementation

[0026] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0027] In the description of this invention, unless otherwise explicitly specified and limited, the terms "connected," "linked," and "fixed" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0028] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0029] In the description of this embodiment, the terms "upper," "lower," "right," etc., refer to the orientation or positional relationship shown in the accompanying drawings. They are used only for ease of description and simplification of operation, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. In addition, the terms "first" and "second" are used only for distinction in description and have no special meaning.

[0030] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0031] Example 1: like Figures 1-6 As shown, this embodiment of the invention provides a method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model. The core technical architecture is divided into five layers: data acquisition layer, preprocessing layer, AI analysis engine layer, four-terminal application layer, and closed-loop optimization layer. At the same time, a cross-scenario adaptation layer is set up to achieve multi-scenario reuse. Multi-source data acquisition layer: Achieves full-dimensional data acquisition through four types of terminals, covering four dimensions: subjective self-assessment, objective observation, physiological perception, and family feedback. On the service recipient side: data such as standardized questionnaire responses, daily voice diaries, and mood check-in are collected through tablets, mobile phones, or dedicated voice terminals; for adolescent scenarios, learning stress scales and depression self-rating scales are adapted, and for elderly scenarios, the GDS-5 geriatric depression scale is adapted; the collection frequency is daily mood check-in and weekly scale assessment.

[0032] On the implementer side: Through a mobile APP, the head teacher / caregiver can enter behavioral observation tags, including irritability level, appetite, sleep, social status, safety risks, etc. It supports voice input for quick entry and automatic transcription of tags; the collection frequency is once per shift, and abnormal situations are reported in real time.

[0033] Intelligent sensing terminal: Through devices such as smart bracelets, smart mattresses, and millimeter-wave radar, physiological indicators such as heart rate, HRV heart rate variability, blood oxygen, sleep efficiency, frequency of getting out of bed, and respiratory rate are collected in real time; the collection frequency is on the minute level.

[0034] For family members: Collect visit records, companionship feedback, and satisfaction evaluation data through WeChat mini-program; submit as needed, and provide feedback at least once a week.

[0035] Data preprocessing and feature extraction layer: Standardizes multi-source heterogeneous data to form a unified feature vector that can be input into the model. Data cleaning: Invalid scale samples with a response time of <2 seconds or >60 seconds were removed, and outliers with abnormal physiological signals were removed using the 3σ principle; Speech-to-text processing: The voice diary is transcribed into text using ASR automatic speech recognition, and then a 128-dimensional emotion vector is extracted by the fine-tuned BERT emotion classification model, corresponding to the probability distribution of five emotions: happiness, sadness, anxiety, anger, and calmness. Physiological indicator calculation: Calculate quantitative indicators such as HRV stress index LF / HF, sleep efficiency, and frequency of getting out of bed at night; Time alignment and anonymization: All data is standardized to three time granularities: 15 minutes, 1 hour, and daily, and stored in a time-series database; all personal information is anonymized using AES-256 encryption.

[0036] AI analysis engine layer: The AI ​​analysis engine consists of three parts: a multimodal fusion risk prediction model, a large language model scheme generator, and a reinforcement learning policy network. 1) Multimodal fusion risk prediction model The input feature vector X = [x_scale, x_behavior, x_physio, x_voice, x_history] corresponds to the scale standardized score, behavioral observation level, physiological quantitative index, voice emotion vector, and historical risk trend, respectively.

[0037] The model employs a multimodal fusion network based on an attention mechanism: ① Independent encoding: The five MLP sub-networks respectively map the features of the five modalities to a unified 256-dimensional dimension; ② Cross-modal attention fusion: Using speech modal features as queries and other modal features as keys / values, weighted fusion features are calculated through multi-head attention, and residual connections and LayerNorm layers are added for normalization; ③ Multi-task output layer: The three fully connected branches output the comprehensive psychological risk index R (0~1 continuous value), the probability of four subdivided risks (depression, anxiety, violent tendencies, falls), and the SHAP interpretability attribution results.

[0038] The model is trained in a supervised manner using historical data with expert annotations, and the loss function is weighted cross-entropy to balance positive and negative samples.

[0039] 2) Domain-wide large language model solution generator The base model uses domestic large-scale models (such as DeepSeek and Tongyi Qianwen), and LoRA low-rank adaptation technology is used for efficient domain fine-tuning. The fine-tuning datasets include: public mental health dialogue datasets, the Ministry of Civil Affairs' elderly care standards, the Ministry of Education's moral education manual, and effective intervention cases accumulated by pilot institutions.

[0040] When generating the model, input an individual profile JSON (scenario, age, cognitive level, anomaly dimension, risk index, risk type, trend). The model is then guided by preset prompt templates to output three versions of a structured JSON format. Target audience version: Designed for students / seniors, it includes daily tasks, positive guidance, and methods for regulating emotions, using simple and gentle language; Implementer version: Designed for teachers / caregivers, it includes a care / coaching checklist, key concerns, and step-by-step implementation instructions, making it professional and practical. Family version: Designed for parents / family members, it includes a guide to accompanying their children, communication techniques, and suggestions for family cooperation, balancing professionalism with ease of understanding.

[0041] After generation, it undergoes three verifications: security verification (filtering negative guiding language), consistency verification (matching age and cognitive level), and feasibility verification (adapting to the physical / ability conditions of the service recipient).

[0042] 3) Reinforcement learning strategy network Construct a closed loop of "monitoring-intervention-feedback-optimization" using DQN or PPO reinforcement learning algorithms: State s_t: Current multimodal feature vector and ID of the previous intervention scheme; Action a_t: The type of intervention plan recommended by the system, with a total of M discrete plans; The reward r_t = w1・Δemotional score + w2・Δbehavioral level - w3・execution cost, which comprehensively measures the intervention effect and the execution cost; Set up an experience replay pool with a capacity of 10,000 to store quadruples (s_t, a_t, r_t, s_{t+1}); every 100 new experiences are accumulated, randomly sample batch=64 to update the policy network, and the target network uses soft update (τ=0.01).

[0043] After 30 intervention iterations, the accuracy of the individualized model in recommending treatment plans can be improved from 75% to 89%.

[0044] Four-terminal collaborative application layer: Construct a four-terminal interconnected architecture encompassing service recipients, implementers, managers, and family members to achieve data interoperability, task collaboration, and synchronized early warnings. Service recipients: Receive personalized daily tasks and emotional regulation guidance, and complete self-assessment and check-in; On the executor's side: Receive early warning tasks and care lists, enter behavioral observations, check in execution progress, and report anomalies with one click; Managerial side: Use the psychological weather map cockpit to grasp the overall situation across the entire domain, manage personnel and tasks, and export statistical reports; Family members: Receive daily life briefings, tiered early warning notifications, and companionship guidance; submit feedback and satisfaction ratings.

[0045] A three-tiered early warning and task scheduling mechanism is in place: Blue Alert (Mild Attention): Only pushed to the executor, with a 24-hour response time limit; Yellow alert (moderate emergency): Notify executors and managers, with a 12-hour response time limit; Red Alert (High Urgency): Notify executors, managers, and their families; 2-hour response time limit. If no action is taken within the specified time, the warning level will be automatically upgraded to form a closed-loop tracking system.

[0046] Cross-scenario general adaptation layer It adopts a modular architecture of "core engine and scenario configuration": the core AI algorithm, data flow logic, and four-terminal architecture remain universal; by loading different scenario configuration files, replacing scale system, behavior tags, solution script templates, and risk threshold parameters, you can switch between youth campus scenarios and elderly care institution scenarios with one click without redevelopment, reducing deployment costs by 70%.

[0047] Example 2: Scenario of Adolescent Campus Mental Health and Behavioral Safety Management like Figure 7 As shown, this embodiment is applied to a school in Beijing, covering 1246 students, 32 classes, and 32 homeroom teachers, and is connected to four terminals: student-side mini-program, homeroom teacher-side APP, principal-side web backend, and parent-side WeChat mini-program.

[0048] ①System Deployment and Data Acquisition Hardware and terminal deployment: Students use their own mobile phones / tablets, homeroom teachers use their work phones to install a dedicated APP, the principal deploys a web management backend, and parents access a WeChat mini program; no additional wearable devices are required, reducing deployment costs.

[0049] Data collection rules: Student side: Complete a scale assessment of academic stress, depression, anxiety, etc. once a week, and submit a voice diary and mood check-in daily; Class teacher's side: Enter student behavior observation tags daily (class participation, irritability level, social status, academic fluctuations), and support voice input for quick recording and automatic tag conversion; Parents can receive weekly student progress reports and submit feedback on their time spent with their children and their satisfaction ratings. All data is automatically synchronized to the cloud every day at midnight and stored using AES-256 encryption.

[0050] ② Model Operation and Risk Assessment Every day at midnight, the system automatically preprocesses all data: standardizes scale data, quantifies behavioral labels, extracts 128-dimensional emotion vectors from voice data, calculates historical risk trends over the past 7 days, and generates an input vector composed of 5 types of features.

[0051] Input the multimodal fusion risk prediction model to calculate the comprehensive psychological risk index for each student, as well as the probability of four sub-risks: depression, anxiety, interpersonal conflict, and academic pressure. Simultaneously output the SHAP attribution results, marking the main sources of risk (e.g., "academic pressure contributes 62%, interpersonal sensitivity contributes 28%").

[0052] Risk classification: Comprehensive risk 0~0.3 is normal, 0.3~0.5 is of concern, 0.5~0.7 is blue warning, 0.7~0.85 is yellow warning, and ≥0.85 is red warning.

[0053] ③ Large-scale model scheme generation and four-terminal push For students with blue or higher warning levels, a large language model, fine-tuned by educational psychology, is used. Student profiles (age, grade, personality tags, risk type, historical intervention records) are input to generate three versions of the treatment plan: Student version: Generates a "three-week growth plan", which includes daily 3-minute emotion regulation tasks, cognitive reshaping exercises, and positive psychological suggestion techniques; Class Teacher Version: Generates a tutoring checklist, including key points to focus on, communication techniques, classroom guidance methods, and suggestions for home-school cooperation; Parent version: Generates a guide to spending time with your child, including taboos in family communication, suggestions for parent-child interaction, and key points for observing emotions.

[0054] After the plan has been verified for safety and feasibility, it will be pushed out according to the warning level: blue alerts will be pushed to the homeroom teacher, yellow alerts will be pushed to the homeroom teacher and the moral education director, and red alerts will be pushed to the homeroom teacher, the moral education director, and the parents.

[0055] ④ Closed-loop execution and optimization After receiving the task, the homeroom teacher will carry out intervention according to the checklist and check in the APP to record the execution status; the system will automatically record the processing time, and if it is not processed within the time limit, it will automatically escalate the warning and push it to the superior manager.

[0056] On the 3rd and 7th days after the intervention, the system automatically triggers the remeasurement scale and emotion check-in, calculates the changes in emotion and behavior, and generates reward values ​​to be stored in the experience replay pool.

[0057] For every 100 accumulated intervention experiences, the system automatically updates the reinforcement learning strategy network to optimize the accuracy of subsequent solution recommendations.

[0058] ⑤ Implementation Results After running continuously for one semester, the core metrics of this system are as follows: The early identification rate of depression risk has increased from the traditional 28% to 84%, enabling early intervention in multiple high-risk psychological crisis events; The time spent on recording psychological work by homeroom teachers has been reduced from an average of 32 minutes per day to 8 minutes, significantly reducing their workload. Parent satisfaction increased by 24%, and the number of parent-school disputes decreased by 60%. Students' overall emotional stability improved by 55%, and interpersonal conflicts in the class decreased by 70%.

[0059] Typical case: Li, a junior high school boy, had an initial anxiety index of 0.67. After the system pushed a personalized intervention plan of "five-minute start method", and the homeroom teacher and parents implemented it together for 4 weeks, his anxiety index dropped to 0.42 and his emotional state improved significantly.

[0060] Example 3: Psychological Health and Behavioral Safety Management Scenario in Elderly Care Institutions like Figure 8 As shown, this embodiment is applied to a nursing home in Tianjin, covering 50 elderly residents, 15 caregivers, and 1 director, and is equipped with 10 tablet terminals, 30 smart bracelets, and 20 smart mattresses.

[0061] ①System Deployment and Data Acquisition Hardware and terminal deployment: Tablet terminals are set up in the public activity area for the elderly to self-assess and record voice diaries, caregivers are equipped with work mobile phone APP, the director deploys a web management backend, and family members access WeChat mini program; smart bracelets collect heart rate and HRV, and smart mattresses collect sleep, getting out of bed and breathing data.

[0062] Data collection rules: For seniors: The GDS-5 geriatric depression scale is used once a week, and a daily voice diary (such as "How am I feeling today") is kept, with the assistance of a caregiver. Caregiver's app: Each shift records behavioral tags such as the elderly person's irritability level, appetite, sleep, social interaction, and fall risk, and abnormal situations can be reported with one click; Smart device terminal: Collects physiological data in minutes and monitors bed exit and respiratory abnormalities in real time; Family members: Receive daily life briefings and alerts for any abnormalities from the elderly, and submit feedback on visits.

[0063] ② Model Operation and Risk Assessment The system synchronizes physiological data every hour, summarizes scale, behavioral and voice data daily, and extracts feature vectors after unifying the time granularity. The multimodal fusion model outputs a comprehensive psychological risk index, as well as the probabilities of three sub-categories of risks: depression, irritability and violent tendencies, and fall risk. To address the risk of falls, additional data on nighttime bed-leaning frequency and gait behavior characteristics are incorporated to improve the accuracy of early warnings.

[0064] ③ Large-scale model scheme generation and four-terminal push For elderly individuals with early warning signs, a large language model, fine-tuned for elderly care, is used. Inputting the elderly person's age, cognitive level (e.g., mild cognitive impairment), risk type, and physical condition, three versions of the plan are generated: Version for seniors: Daily tasks, such as "smile in the mirror after listening to 15 minutes of opera," with simple, easy-to-understand, positive, and gentle language; Caregiver version: Care checklist, such as "actively check in every 2 hours and arrange 30 minutes of music relaxation activities", including operation steps and precautions; Family version: Companionship guidelines, such as "reduce lecturing visits, bring your favorite pastries and spend 15 minutes listening to a play together", and provide specific communication suggestions.

[0065] Tiered alert system: Blue alert alerts are sent to caregivers; yellow alert alerts are sent to caregivers and hospital directors; red alert alerts are sent to caregivers, hospital directors, and family members.

[0066] ④ Closed-loop execution and optimization After receiving a task, the caregiver executes it and clocks in. If the task is not completed within the time limit, it will be automatically escalated to the hospital director. Daily reassessment of behavior and emotional state after intervention, calculation of reward values, and updating of reinforcement learning model; For elderly people with a high tendency towards violence, the system automatically generates management suggestions such as relocating accommodations and diverting activities to resolve conflicts in advance.

[0067] ⑤ Implementation Results After the system has been running for 3 months: Two potential violent conflicts involving elderly people were successfully warned and resolved, and dormitory changes were made in advance to prevent the conflicts from escalating. The number of violent conflicts within the hospital decreased from an average of 2.4 per month to 0.6, a reduction of 75%. It can prevent falls by 83% and reduce the incidence of depression among the elderly by 40%; Improved efficiency of caregivers, significantly enhanced family members' right to know and satisfaction, and a 65% reduction in elderly care disputes.

[0068] The technical effectiveness and cross-scenario versatility of the method described in Examples 2 and 3 above are fully verified. This invention, through multimodal data fusion, large-scale model personalization, four-terminal collaboration, and reinforcement learning closed loop, can simultaneously address the pain points of mental health and behavioral safety management in two scenarios: youth campuses and elderly care institutions. It possesses extremely high promotional value and socio-economic benefits.

[0069] The implementation method of multimodal four-terminal collaborative mental health and behavioral safety management based on AI big model in this application is based on the core of "assessment-diagnosis-intervention or development". This closed-loop logic has strong universality and can be extended to more industries and scenarios.

[0070] Here are some potential areas for expansion: ① Education: From "Cultivating Grades" to "Cultivating People" Education is one of the most widely used fields for scale assessment, which can extend from traditional academic assessment to deeper student development support.

[0071] Learning Ability and Cognitive Diagnosis: Develop assessments for cognitive abilities such as attention, memory, and logical thinking to provide precise intervention plans for students with learning difficulties. This aligns with the trend of shifting from an 'academic achievement' to a 'competency' and 'well-being' orientation.

[0072] Career planning education: In primary, secondary and university stages, tools such as the Holland Occupational Themes Test and MBTI are used to help students understand their own interests and strengths, and to assist in choosing subjects, majors and career planning.

[0073] • Student comprehensive quality evaluation: Construct an evaluation system covering multiple dimensions such as academics, psychology, social skills, and morality, and generate personalized growth reports and development suggestions.

[0074] ② Organization and Human Resource Management: From "Selecting the Right People" to "Utilizing the Right People" In addition to enhancing talent capabilities as mentioned above, scale assessments can play a greater role in the entire human resources chain.

[0075] • High-potential talent identification and development: Through customized scales, high-potential employees are accurately identified from dimensions such as professional qualities, emotional qualities, and mental abilities, and personalized development paths are developed for them.

[0076] • Organizational health diagnosis: Regularly assess organizational atmosphere and team dynamics using scales such as employee satisfaction and engagement to provide data support for management decisions.

[0077] • Job fit and team configuration: Build competency models for specific positions (such as blue-collar workers and IT business analysts), conduct person-job matching assessments, and optimize team composition.

[0078] ③ Health and Medical Field: From "Treating Disease" to "Preventing Disease" Scale assessments are the cornerstone of mental health services and can extend from screening to health management.

[0079] Mental health screening and early warning: In schools, enterprises, communities and other settings, use AI psychological assessment systems to conduct large-scale and routine mental health screenings to achieve early warning.

[0080] Integrated management of physical and mental health: Integrating psychological assessments (such as stress and mood) into routine physical examinations as an important component of health management. Using clinical scales such as the PHQ-9 to assist in the assessment.

[0081] Health promotion for specific populations: Developing exclusive scales for specific populations such as the elderly and pregnant women to provide targeted physical and mental health guidance.

[0082] ④ Family and Life Services: From "Individual" to "Relationship" Assessment can expand from focusing on individuals to focusing on the systems of their lives.

[0083] Marriage and family relationship counseling: We conduct personality and marriage and love concept assessments for newlyweds, evaluate their marital compatibility, and provide personalized communication and relationship advice.

[0084] Parent-child relationships and family education: Assessments are conducted on both parents and children to diagnose parent-child interaction patterns and provide scientific guidance for family education.

[0085] Elderly care and the silver economy: Assessing the cognitive function, emotional state, and life satisfaction of older adults to provide personalized solutions for elderly care services.

[0086] ⑤ Public Safety and Special Industries: From "Management" to "Prevention" In some specialized industries, scale assessments are a crucial step in ensuring safety and preventing risks.

[0087] Judicial and community corrections: Conduct psychological assessments of correctional subjects, identify risk factors (such as aggression and addiction), and develop personalized correction and support plans.

[0088] High-risk occupation screening: In industries such as military, police, and fire protection, psychological assessments are used to select personnel with strong psychological qualities to achieve the best match between people and positions.

[0089] Driving safety assessment: Conducting psychological and fatigue assessments of professional drivers (such as passenger and freight drivers) to prevent safety accidents.

[0090] ⑥ Emerging and cutting-edge fields By combining new technologies, scale-based assessments can create entirely new service models.

[0091] AI-driven dynamic assessment: Utilizing AI to analyze multimodal data (such as facial expressions and voice) to achieve more dynamic and natural psychological and behavioral assessments.

[0092] Gamified assessment: By embedding assessment tasks into game scenarios, users' abilities and traits can be evaluated in a relaxed atmosphere, improving the experience and accuracy.

[0093] User Experience and Product Evaluation: Evaluate products or services from the perspectives of "user experience" and "emotional value" to provide optimization suggestions for brands.

[0094] Intelligent question answering and interactive assessment: By interacting with a large language model, assessments are completed in dialogue and instant feedback is provided.

[0095] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

Claims

1. A method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model, characterized in that, Includes the following steps: Step S1: Multi-source data acquisition Through four data entry points—the service recipient end, the implementer end, the intelligent sensing end, and the family end—scale assessment data, behavioral observation tags, physiological monitoring indicators, voice and text data, and historical risk time series data are collected simultaneously to form a multi-source heterogeneous raw dataset. Step S2: Data Preprocessing and Feature Extraction The raw data is cleaned, desensitized, and time-aligned. Structured scale features, behavioral level features, physiological quantitative features, voice emotion vector features, and historical trend features are extracted and mapped to a unified dimension of input feature vector. Step S3: Multimodal fusion risk prediction Input the input feature vector into the multimodal fusion risk prediction model based on cross-modal attention mechanism, and output a comprehensive psychological risk index in the range of 0 to 1, as well as the probability of four sub-risks: depression, anxiety, violent tendencies, and fall risk. Step S4: Generate Personalization Scheme for Large Model By invoking a large language model finely tuned with knowledge from the fields of psychology or care, and combining individual profiles, risk indices and trends, three versions of differentiated intervention guidance plans are automatically generated for service recipients, implementers and their families. Step S5: Four-terminal collaborative early warning and task scheduling Based on the comprehensive risk index, a three-tiered warning system of blue, yellow, and red is triggered, automatically generating corresponding intervention tasks and pushing them to the corresponding terminals; through the linkage of four terminals—the service recipient terminal, the executor terminal, the manager terminal, and the family terminal—task execution check-in, status tracking, and automatic escalation upon timeout are realized; Step S6: Closed-loop feedback and model optimization Data on changes in emotion and behavior after intervention are collected, intervention reward values ​​are calculated, and the policy network is updated through deep reinforcement learning algorithms to continuously optimize the accuracy of intervention program recommendations, forming a complete closed loop of "monitoring-early warning-intervention-feedback-optimization".

2. The method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model according to claim 1, characterized in that, In step S1, the scale assessment data includes the Adolescent Learning Stress Scale, the Self-Rating Depression Scale, the Geriatric Depression Scale GDS-5, and the Psychological Resilience Scale; the physiological monitoring indicators include heart rate, heart rate variability (HRV), blood oxygen, sleep efficiency, frequency of getting out of bed, and respiratory rate; the voice-text data is transcribed into text by automatic speech recognition (ASR) and the emotion probability distribution vector is extracted.

3. The method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model according to claim 1, characterized in that, In step S3, the processing flow of the multimodal fusion risk prediction model is as follows: 1) Independent encoding of each modality: The features of the five modalities are mapped to a unified 256-dimensional feature space through a multilayer perceptron (MLP); 2) Cross-modal attention fusion: Using speech modal features as the query vector Query and other modal features as key vectors Key and value vectors Value, a weighted fusion feature is calculated through a multi-head attention mechanism, and residual connections and layer normalization are added; 3) Multi-task output: The comprehensive psychological risk index, the probability of four types of subdivided risks, and the risk contribution attribution results based on the SHAP algorithm are output through three fully connected branches.

4. The method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model according to claim 1, characterized in that, In step S4, the large language model uses a domestically developed basic model combined with LoRA low-rank adaptation technology for domain fine-tuning. The fine-tuning training set includes mental health dialogue data, elderly care standards, school moral education manuals, and historical effective intervention cases. The generated three versions of the scheme are as follows: Targeted version: Includes daily tasks, emotion regulation methods, and positive guidance content; Implementer's version: Includes a care / counseling checklist, key points to focus on, and step-by-step implementation instructions; Family version: Includes a guide to companionship, communication techniques, and suggestions for family cooperation; After the plan is generated, it undergoes three post-processing steps: security verification, age and cognitive consistency verification, and feasibility verification.

5. The method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model according to claim 1, characterized in that, In step S5, the functions of each terminal in the four-terminal collaborative architecture are as follows: On the user side: It provides functions such as self-assessment scale, voice diary, mood check-in, and personalized task reception; On the executor side: It provides functions such as behavior observation and recording, rapid voice reporting, intervention task reception and check-in, and one-click abnormality reporting; Administrator side: Provides functions such as psychological weather map data dashboard, early warning center, personnel management, report export, and task assignment; Family member side: Provides life briefings, tiered early warning notifications, companionship guidance, and satisfaction feedback functions.

6. The method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model according to claim 1, characterized in that, In step S5, the three-level hierarchical early warning rule is as follows: Blue Level 1 Warning: The overall risk index is ≥0.5 and <0.7, or the probability of a single sub-category of risk is >0.

6. The task is pushed to the executor, requiring the handling to be completed within 24 hours. Yellow Level 2 Alert: If the comprehensive risk index is ≥0.5 twice consecutively, or the comprehensive risk index is ≥0.7, the task will be pushed to both the executor and the manager, requiring completion within 12 hours; Red Level 3 Alert: Comprehensive risk index ≥ 0.85, or violent tendency / probability of fall > 0.

8. The task is simultaneously pushed to the executor, manager, and family members, requiring a response and handling within 2 hours. All tasks that are not checked in and confirmed within the time limit will be automatically upgraded to the next higher warning level.

7. The method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model according to claim 1, characterized in that, In step S6, the deep reinforcement learning algorithm employs either a DQN deep Q-network or a PPO near-end policy optimization algorithm, with the specific rules as follows: State st: the multimodal feature vector at time t and the ID of the preceding intervention scheme; Action at: The ID of the intervention plan type recommended by the system, with a total of M discrete plans; The reward rt = w1・Δemotional score + w2・Δbehavioral level - w3・execution cost, where Δemotional score is the change in self-reported emotion before and after intervention, Δbehavioral level is the improvement in observed behavior, and execution cost is the time spent by caregivers / teachers and the difficulty of cooperation with family members; Set up an experience replay pool with a capacity of 10000 (s_t, a_t, r_t, s_{t+1}). Update the policy network once every 100 new experience samples. The target network adopts a soft update method with an update coefficient τ=0.

01.

8. The method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model according to claim 1, characterized in that, The method adopts a modular and universal architecture, sets up a scenario adaptation layer, and can switch between the youth campus scenario and the elderly care institution scenario with one click by loading different scenario configuration files. When switching scenarios, the core AI engine, data flow logic, and four-terminal architecture remain unchanged, and only the scale system, behavior label system, solution script template and risk threshold parameters are replaced.

9. The method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model according to claim 3, characterized in that, The risk contribution attribution results output in step S3 clearly indicate the contribution ratio of each dimension (scale, behavior, physiology, voice, and history) to the risk results, and generate an interpretable risk report for review by professionals.

10. The method for implementing multimodal four-terminal collaborative mental health and behavioral safety management based on an AI large model according to claim 1, characterized in that, All raw data is stored using AES-256 encryption. The original content of voice and text data is deleted immediately after feature extraction. Federated learning mode is supported, allowing multiple institutions to train locally and only upload encrypted model gradients without sharing original user data, thus meeting data privacy compliance requirements.

Images