Intelligent system and method for realizing multi-mode psychological counseling and seamless connection based on education robot

Abstract

The invention discloses an intelligent system and method for realizing multi-mode psychological counseling and seamless connection based on an education robot. A multi-mode psychological counseling system is constructed aiming at the problems of insufficient privacy protection of a traditional campus psychological counseling room, shortage of psychological counseling resources of mountainous schools and the like. The system provides psychological counseling modes such as an education robot, real person remote and man-machine collaboration, can be combined with counseling environments such as a simple psychological counseling room and a VR counseling room to become multi-modal psychological counseling, supports seamless switching of each modal and spatial form, and achieves the intelligent scheduling technology through a multi-modal data real-time synchronization engine and reinforcement learning driving. And data encryption transmission and millisecond-level service form switching are realized. A dynamic auxiliary screen system is innovatively integrated, and psychological archives are dynamically associated with a knowledge graph through a domain-customized named entity recognition model (NER, the accuracy rate is greater than or equal to 95%), so that the remote real expert key information retrieval time is less than or equal to 2 seconds, and the decision-making efficiency is improved by 78%. The system meets the requirements of various laws and regulations such as the personal information protection law.

Description

Technical Field

[0001] This invention pertains to the application of artificial intelligence in the field of psychological education, and also relates to the fields of mental health analysis and data encryption technology. It complies with the requirements of the Personal Information Protection Law and the Educational Data Security Management Standards. Background Technology

[0002] 1. The low utilization rate of school counseling rooms and the prominent mental health problems of students. A report by China Youth Daily in June 2024 pointed out that although the demand for mental health care among teenagers has increased significantly, the actual utilization rate of school counseling rooms is still low.

[0003] A 2023 survey by Red Star News found that over 50% of students believed that school counselors could not completely maintain confidentiality. Some students chose to conceal the truth or forgo counseling for fear that their homeroom teacher or parents would find out about their consultations.

[0004] 2. According to the "2024 Report on the Mental Health of Rural Students in Underdeveloped Areas of my country" by the China Development Research Foundation, mental health education in mountainous schools currently faces severe challenges: 72% of township primary schools lack dedicated mental health teachers. Currently, most domestic mental health experts are located in cities, making remote mental health counseling a valuable solution for schools in mountainous areas.

[0005] 3. Legal Compliance Statement: This system complies with the Mental Health Law and relevant regulations of the Ministry of Education, and only provides mental health support services, without involving the diagnosis of mental illnesses. Summary of the Invention

[0006] This invention aims to address the shortcomings of traditional school counseling rooms, such as insufficient privacy protection and a lack of diverse counseling methods. It also seeks to provide more comprehensive counseling services for schools in mountainous areas. As an improvement on existing technologies, this invention aims to solve at least one of the aforementioned technical problems by constructing a multimodal counseling system to provide diversified counseling service pathways for students.

[0007] Software Educational Robot: The software educational robot described in this invention refers to an intelligent program system that runs on operating systems of devices such as computers, mobile terminals, and smart wearables. This system integrates core technologies such as natural language processing, machine learning, and robotic process automation. Its hardware carriers cover mobile terminal devices such as computers and mobile phones, as well as smart wearable devices, supporting installation and deployment on various types of mobile terminals used by teachers, students, parents, and other members of the public.

[0008] Physical Educational Robots: Physical educational robots are intelligent terminal devices deployed in key areas of a school campus (including but not limited to school gates, dormitory entrances, teaching building entrances, and risk monitoring points). These devices integrate sensor technology and artificial intelligence algorithms, and incorporate an educational resource database, possessing core functions such as teaching assistance and support. They feature hot-swappable interfaces, allowing integration with foldable, simple psychological counseling room equipment, VR glasses, smart bracelets, and brain-computer interface helmets. These devices connect to the physical educational robot via hot-swappable interfaces.

[0009] Home-use physical educational robots: These are physical educational robots that can be placed in the home or carried by students. They are smaller versions of physical educational robots.

[0010] 1. This system establishes a privacy and information security protection framework. 1.1. Addressing privacy and information security issues at their source 1.1.1. Anonymity Mechanism Regardless of the form of psychological counseling used, this system implements an anonymity mechanism. Users only need to define a unique alias and set a complex password. The alias and password must consist of a mixture of uppercase and lowercase letters, numbers, and symbols. The system has preset naming rules and provides real-time prompts. All alias IDs must pass the system's uniqueness verification and must not contain any personalized information. If they do not meet the requirements, users must re-enter and carefully remember their unique alias ID and password. All information flow within the system is based on the alias ID for circulation and storage, cutting off the path of personal privacy leakage at the data source.

[0011] 1.1.2. Encryption Mechanism When a user accesses the system for the first time, an anonymous electronic psychological profile will be generated, linked only to a unique ID. The profile undergoes anonymization, removing all sensitive personal information and retaining only basic information such as the specific time and duration of psychological counseling, which cannot be traced back to personal privacy. The electronic psychological profile employs a double encryption method combining an algorithm certified by the State Cryptography Administration with quantum key distribution. Furthermore, federated learning and blockchain technologies are incorporated to construct a security protection matrix, ensuring that the information is neither leaked nor cracked. The system implements a strict access control mechanism; without the user's authorization, no third party, including schools, can access the profile information. Schools can only request access to statistical reports containing common characteristics, and the data in these reports has been technically processed to completely eliminate the possibility of personal information targeting.

[0012] 1.2. Technical Solution 1.2.1. Multi-factor hybrid anonymous identifier generation and circulation module To address the issue of easily leaked user identities, a full-process mechanism of "generating, verifying, and transferring anonymous identifiers without personalization" is designed: (1) Anonymous identifier generation rules: Users need to customize "code ID" and set "complex password", where: Code ID: length ≥ 10 characters, composed of uppercase and lowercase letters, numbers, and symbols (such as !@#), the system has a preset naming framework (such as "letter + number + symbol" structured prompt), and strictly requires that it does not contain any personalized information (such as name pinyin, school abbreviation, birthday, etc.); Complex password: consistent with the code ID rules, adopts the same multi-element mixing strategy, ensures that the password entropy value (Entropy) ≥ 128 bits, and resists brute-force attacks.

[0013] (2) Uniqueness verification and dynamic correction: The system uses an artificial intelligence identity verification model (based on hash algorithm and comparison with the database) to verify the global uniqueness of the code ID in real time; if the verification fails (such as duplicate or containing personalized information), the system will automatically prompt the user to fill in the information again and record the correction process to the blockchain for evidence storage.

[0014] (3) Anonymized information flow: All data in the system (including consultation records, chat content, and file transfers) are uniquely associated with a code ID, completely severing the mapping relationship with real identity (such as name, ID number, and contact information), thus blocking the privacy leakage path from the source of data.

[0015] 1.2.2. Secure storage module for electronic psychological profiles based on multi-layered encryption and federated learning To address the issues of insufficient strength in traditional encryption technologies and the potential for privacy leaks during statistical analysis, a three-tiered protection system of "de-identification - encryption - privacy computation" is designed: (1) Desensitization: When a user visits the system for the first time, an electronic psychological profile is automatically generated that is only associated with the ID code. The system uses a natural language processing (NLP) model to identify and remove sensitive information (such as home address, medical history, social relationships, etc.), and only retains metadata that cannot be traced back to the individual, such as the time and duration of psychological counseling and keywords in the dialogue text (non-sensitive content).

[0016] (2) Double encryption storage: The desensitized files are encrypted at the first level using algorithms certified by the State Cryptography Administration (such as SM4 symmetric encryption algorithm and SM2 asymmetric encryption algorithm), and the key is stored by the system security chip; a quantum key is superimposed for secondary encryption, and an absolutely secure one-time encryption key is generated by using the no-cloning theorem of quantum mechanics to ensure that the ciphertext cannot be cracked by brute force or quantum computing attack.

[0017] (3) Privacy-preserving statistical analysis driven by federated learning: When schools or other third parties apply to access statistical reports, the system constructs a privacy-preserving statistical model based on federated learning technology: Each user's electronic psychological profile runs the statistical model only on the local device (such as the user terminal), and only uploads the encrypted intermediate results (such as feature vectors and statistics) to the server; the server integrates all intermediate results through the Secure Aggregation algorithm to generate a statistical report containing common features (such as consultation frequency at different times and common psychological problem types); this process does not require the original data to flow out, completely avoiding the risk of inferring personal information from statistical data.

[0018] 1.2.3. Blockchain-Driven Access Control and Evidence Storage Module To address the issue of unauthorized access by third parties, a blockchain-based permission tracing and control mechanism is designed: (1) Hierarchical access control: The system permissions are divided into “user level”, “school level” and “administrator level”. Among them, user level: only has the right to read, delete and authorize the electronic psychological profile of the user; school level: can only apply for the statistical report generated by federated learning, and must be authorized by the user terminal (such as SMS verification code confirmation); administrator level: can only maintain the operation of the system, and cannot directly access user profiles or statistical raw data.

[0019] (2) Blockchain Evidence Storage: All permission operations (such as authorization, access, and modification) are recorded in the form of transactions on the Consortium Blockchain, including the operation time, operator ID, operation type (such as "apply for statistical report" or "modify password"), etc. The immutability of the blockchain ensures that permission operations are traceable and auditable, preventing unauthorized parties from maliciously obtaining information.

[0020] 1.2.4 Data management follows the principle of minimization: Apart from network security devices such as firewalls and intrusion detection systems, all data to be stored is encrypted using algorithms from the National Cryptography Administration combined with quantum key distribution and stored on a local campus server. Physical isolation is implemented daily through a network gateway, with network access only during data usage. Blockchain technology is used throughout the entire process to prevent data tampering and monitor intrusions in real time. Upon detecting any anomalies, the network gateway is immediately notified to physically disconnect information transmission.

[0021] 1.3. Numerous innovations have been made in the field of psychological education and counseling, mainly reflected in the following aspects: 1.3.1. Multi-element hybridization and AI verification for anonymous identifier generation: Breaking through the traditional real-name identification mode of "mobile phone number + verification code", this method uses a non-personalized code ID with more than ten digits of uppercase and lowercase letters, numbers and symbols, combined with an artificial intelligence hash verification model, to achieve "de-real informationization" and global uniqueness of identity identification, thus blocking privacy leaks from the source of psychological counseling.

[0022] 1.3.2. Dual encryption system of national cryptographic algorithm and quantum key: Unlike single encryption technology, it adopts superimposed encryption of algorithm certified by the State Cryptography Administration and quantum key. By utilizing the absolute security of quantum key and the compliance of national cryptographic algorithm, a "crackable" psychological counseling data storage protection layer is constructed.

[0023] 1.3.3. Privacy-preserving statistical analysis driven by federated learning: Through a federated learning model that combines local computation and secure aggregation, common statistical reports are generated while preventing the outflow of raw data, thus solving the technical problem that traditional statistical methods are prone to causing the leakage of personal information.

[0024] 1.3.4. Blockchain Access Control and Evidence Storage: Utilizing the immutability of blockchain, the entire process of access control operations can be traced, ensuring that third parties (such as schools) can only obtain statistical reports authorized by users, thus eliminating the risk of unauthorized access.

[0025] 1.4. Technical Effects The present invention, through the above-mentioned security technical solution, can achieve the following technical effects: 1.4.1. User identity and real information are completely decoupled. Verified by pilot schools and tested with 200 users, the time for remote expert key information retrieval is reduced from 15 seconds to 2 seconds, improving decision-making efficiency by 78%; through simulated attack testing, the risk of privacy leakage is reduced by 92% compared to traditional solutions. 1.4.2. The encryption strength of electronic psychological profiles reaches the security standard of the quantum computing era (resistant to quantum attacks of 1024 bits or more); 1.4.3. The statistical report generation process requires no original data transmission, and the success rate of personal information traceability is 0%; 1.4.4. Authorization operations can be traced back to specific user IDs, and the unauthorized access interception rate is 100%.

[0026] 1.5. Compliance Summary: The system comprehensively meets the following legal requirements: 1.5.1. Identity De-identification: Complies with Article 73 of the Personal Information Protection Law and Article 3.10 of GB / T 35273-2020; 1.5.1. Data Encryption Strength: Complies with Article 27 of the Cryptography Law and Article 5.3 of GB / T 39725-2023; 1.5.1. Privacy Computing Compliance: Complies with Article 27 of the Personal Information Protection Law and Article 3.10 of GB / T 35273-2020; 1.5.1. Access Traceability: Complies with Article 21 of the Cybersecurity Law and blockchain evidence audit requirements.

[0027] 2. Diversified psychological counseling service model This invention constructs a psychological counseling system that, while retaining the traditional offline psychological counseling room, innovatively provides multi-dimensional optional psychological counseling models. All services adhere to the principle of voluntariness, with parents or students with full civil capacity signing an authorization agreement in advance. Students only need to log in to the system using a code ID, without exposing personalized information throughout the process, thus eliminating privacy concerns from a mechanism perspective.

[0028] There are generally three consultation models: educational robots, remote human consultation, and human-machine collaboration. These can be combined with simple psychological counseling rooms, VR counseling rooms, and other consultation environments to become multimodal psychological counseling, as detailed below: 2.1 Independent Psychological Counseling Model Using Software-Based Educational Robots Students can launch the educational robot via mobile applications (App), WeChat mini-programs, Alipay mini-programs, and other mobile clients. Login requires only an ID and password. The system has been professionally certified by psychological counselors, generating a personalized intelligent psychological counseling service unit for each user. Given the uniqueness of individual psychological characteristics, the service unit provides customized and unique counseling services based on the user's personalized psychological profile data, integrating multi-dimensional parameters such as personality traits, interests, and biological characteristics. During the counseling process, the system senses the user's emotional state in real time and dynamically adjusts the counseling strategy to achieve precise psychological intervention.

[0029] 2.2 Remote Psychological Counseling Model with Real Experts The mobile app features a dedicated portal for live psychological counseling experts. All participating experts have undergone rigorous qualification certification to ensure their services meet professional standards. Students can use this portal to conduct real-time remote psychological counseling with live experts, overcoming geographical limitations to enjoy a face-to-face communication experience.

[0030] 2.3 Human-Machine Collaborative Hybrid Psychological Counseling Model The software-based educational robot collects and analyzes users' psychological profile data in real time, proactively recommending suitable psychological test scales or intelligently embedding test questions within the dialogue, which users can participate in voluntarily. Initial consultation content, after being anonymized, is transmitted to a real expert's client via a secure data channel. The real expert accesses the platform via computer or mobile terminal and conducts in-depth consultations based on the robot's pre-processed information. This model is the primary solution, leveraging both the scalable service capabilities of the software robot and the professional insights of real experts to form a highly efficient and collaborative service system.

[0031] 2.4 Seamless Integration Mode of Software and Physical Educational Robots When consultations with software robots or human experts are interrupted due to time and space constraints, the dialogue content, emotional parameters, and other process data are synchronized in real time to the anonymized psychological profile system. With data security guaranteed, the physical educational robot can access encrypted and authorized profile information, allowing users to continue consultations on physical terminals. This mechanism supports seamless bidirectional switching—whether smoothly transitioning from a physical robot to a software robot, or vice versa, the system provides end-to-end data synchronization support to ensure service continuity.

[0032] 2.5 Local Psychological Counseling Model Using Physical Education Robots Deployed on campus, these physical educational robots are equipped with hot-swappable interfaces, allowing them to connect to devices such as smart bracelets and brain-computer interface helmets. They can engage in interactive psychological counseling with users or act as assistants to remote human experts to better serve the counseling process. With user authorization and voluntary use, the devices integrate physiological monitoring modules such as heart rate sensors, blood oxygen gauging devices, and blood pressure monitors. These modules can collect biological data in real time, combining it with historical psychological profiles and real-time dialogue to construct a multi-dimensional psychological assessment model, providing data support for precise counseling.

[0033] 2.6 Multi-terminal collaborative in-depth expert consultation model After completing basic data collection and preliminary analysis, the physical educational robot transmits encrypted psychological profiles and biological detection data (including physiological indicators such as eye movement tracking, heart rate, pulse, and blood oxygen saturation) to a live expert client. The expert's interface uses split-screen technology: the main screen displays real-time video interaction, while the secondary screen dynamically presents the psychological profile and physiological state analysis results. The system continuously performs semantic parsing of the dialogue content, automatically triggering the highlighted and magnified display of corresponding profile fields (such as family background information); these are presented visually on the secondary screen, providing experts with precise auxiliary information.

[0034] All of the above consultation models can be used independently. Tests in pilot schools have shown that the human-machine collaborative model of "robot preprocessing + remote in-depth consultation with real experts" can improve efficiency by 50%, breaking the limitations of traditional single services. This model is the main psychological counseling model, and it is particularly effective for mountain schools with scarce psychological counseling resources.

[0035] 3. Diverse Psychological Counseling Room Scenarios 3.1. Foldable, simple psychological counseling room This invention relates to a foldable, simple psychological counseling room module, comprising a foldable main frame, a sound insulation layer disposed within the frame, and psychological counseling equipment. The main frame is unfolded / folded via a linkage mechanism driven by a servo motor, which solves the problems of time-consuming construction and low space utilization in traditional psychological counseling rooms.

[0036] Furthermore, the unfolding process of the foldable, simplified psychological counseling room module employs an intelligent scheduling algorithm, specifically including the following steps and logic: 3.1.1. Environmental Data Acquisition Module: Pressure sensors are installed at the folding joints of the module to collect stress values ​​of the folding structure in real time. Fi ( i =1,2,..., n , n (Total number of folding joints); A laser rangefinder is installed around the module to measure the distance between the module and surrounding obstacles. Dj ( j =1,2,..., m , m (Total number of rangefinders) 3.1.2. Status Assessment Logic: The Central Controller, based on the collected data... Fi and Dj The risk coefficient of the calculation module is expanded. R The formula is: R = α ⋅max( Fi )+ β ⋅min( Dj 1) in, α Stress weighting coefficient (0 < α <1), β Obstacle distance weighting coefficient ( α + β =1); 3.1.3. Priority scheduling strategy: If R≤ R Threshold ( R (The threshold is a preset safety threshold), then the priority of using the psychological counseling room will be considered. Pk ( k =1,2,..., p , p Generate an expansion order for the total number of modules; the priority Pk Based on the urgency of the user's appointment time Tk (Values ​​range from 0 to 1, with higher values ​​indicating greater urgency) and space requirement levels. Sk (Values ​​range from 0 to 1; larger values ​​require more space for expansion) Weighted calculation, the formula is: Pk = γ ⋅ Tk +(1− γ )⋅ Sk, in γ The time urgency weighting coefficient (0 < γ <1); 3.1.4. Motion Execution Control: The central controller sends commands to the servo motors of the folding structure, unfolding the modules sequentially in descending priority; unfolding speed... v With risk coefficient R Inversely proportional, the calculation formula is: v =1+ Rv 0 ( v 0 is the base speed.

[0037] 3.2. Physical robot integrated with a simplified psychological counseling room model The physical educational robot or its charging cabinet has a folding mechanism and is equipped with an unfoldable virtual counseling room module. When unfolded, it forms a simple counseling scene including a folding chair, a privacy curtain, and an interactive screen, along with auxiliary equipment such as tissues and a punching bag. Users can choose educational robots with different counseling styles as their interaction partners in this space to obtain an immersive counseling experience.

[0038] 3.3 Scenario Model of a Simple Psychological Counseling Room with Real Experts In a simplified psychological counseling room environment, students establish a communication connection with remote, live experts via a display screen. The system synchronizes biometric data and dialogue content to the expert's end, supporting real-time consultations across distances and creating a closed and confidential professional service space.

[0039] 3.4 VR Immersive Virtual Psychological Counseling Room Mode The physical educational robot has a hot-swappable interface, which can be used to add VR devices.

[0040] Users can construct two virtual consultation scenarios using VR glasses: one is to accurately replicate the physical environment of a real-world school counseling room; the other is to generate personalized virtual spaces (such as Arctic ice fields, grassland ranches, Minecraft game scenes, Mars bases, nebulae, etc.) based on voice commands or touch input. Customization of the consultation expert's image is also supported; users can choose real-life figures, cartoon characters, or science fiction characters as their interaction objects, and customize their faces and voices to create a highly personalized consultation environment.

[0041] 4. Innovative technology for seamless switching between all forms of psychological counseling The system supports seamless switching between various consultation providers, including software robots, physical robots, and human experts, as well as spatial formats such as simple consultation rooms and VR virtual consultation rooms, achieving seamless integration of all service formats and closed-loop data management. The core technical solutions are as follows: 4.1 Real-time synchronization and intelligent analysis module for multimodal process data By analyzing the semantic and emotional features of dialogue text, speech tone, and micro-expressions using Natural Language Processing (NLP) and Bidirectional Long Short-Term Memory (Bi-LSTM) networks, and combining this with physiological data such as heart rate and blood oxygen collected from smart bracelets / EEG helmets, a "multi-dimensional process data package" containing semantic features, physiological indicators, and environmental parameters is generated and anonymized with a code ID. Employing a timestamp-based event-driven synchronization mechanism and distributed edge computing, the data is encrypted, packaged, and compressed (72% compression rate) within 0.3 seconds. It is then transmitted to a blockchain distributed ledger for storage via Secure Hypertext Transfer Protocol (HTTPS) and a quantum key distribution channel, ensuring data immutability.

[0042] 4.2 Intelligent Scheduling Engine for Full-Mode Switching The system monitors terminal status, network environment, and user operations in real time. By integrating a rule engine with reinforcement learning algorithms, it determines the switching intention (active / passive) within 500 milliseconds and selects the optimal path based on a switching cost model built using a deep neural network (DNN). When switching between terminals, it automatically extracts historical data to generate a "consultation scenario summary" (including key dialogue nodes and emotion fluctuation curves) and pushes it to the expert interface within 3 seconds. When switching between spatial forms, it reconstructs an equivalent virtual environment (such as an Arctic ice field scene) within 1 second and supports reverse smooth switching triggered by body language.

[0043] 4.3 Three-Dimensional Adaptive Security Authentication System A role-based access control (RBAC) model implements terminal permission tiers, requiring secondary authentication for both human experts and VR devices. A dynamic desensitization engine uses a generative adversarial network (GAN) to transform sensitive data in real time, preserving data utility (regional feature retention rate >95%). All switching operations generate blockchain transaction records, and multi-factor biometric authentication (anonymous voiceprint / fingerprint code) is initiated during cross-terminal switching, reducing the false acceptance rate (FAR) to 10⁻. 6 At a scale that allows data to be "usable but not visible".

[0044] 4.4 Two-way smooth switching execution process Forward switching (physical → virtual): When interrupted, edge nodes capture data, generate an emotion feature matrix and store it on the blockchain. After biometric authentication is passed, the VR scene loads historical context. The total time is <8 seconds.

[0045] Reverse switching (virtual → physical): Parses VR scene parameters and transmits them in encrypted form. The physical device adaptively adjusts its hardware configuration (lighting / sound effects synchronization), with a coordinate matching error of less than 2 cm, ensuring seamless service integration.

[0046] 4.5 Technological Innovations in the Field of Psychological Counseling 4.5.1 Multimodal Seamless Switching and Intelligent Context Reconstruction Technology Technological breakthrough: For the first time, dynamic scheduling of all forms has been achieved, solving the problem of fragmented forms in traditional systems.

[0047] Cross-modal context-aware engine: generates a composite matrix of 128-dimensional semantic features + 64-dimensional emotion vector, introduces an attention mechanism to dynamically focus on key nodes, and has an end-to-end switching latency of ≤1 second (traditionally 5.2 minutes).

[0048] Reinforcement learning-driven switching decision: Define a state space of 12 environmental parameters, and use deep reinforcement learning (DRL) to achieve millisecond-level optimal path selection (e.g., prioritizing terminals with bandwidth > 100Mbps).

[0049] VR reverse switching spatiotemporal anchor point technology: Preloading is triggered 0.1 seconds after the action of removing glasses is detected, and the transition error of virtual entity scene is <5%.

[0050] 4.5.2 AI-driven multimodal data synchronization and compression technology Technological breakthrough: Building a full-link intelligent data pipeline to solve data loss and bandwidth bottlenecks.

[0051] Emotion perception fusion algorithm: Multimodal data generates dynamic emotion maps, with an accuracy improvement of 14.3% compared to traditional single-modal data; Generative Adversarial Network (GAN) completes the semantics of interrupted dialogues, ensuring logical coherence.

[0052] Edge computing optimization: Variational autoencoder (VAE) compresses data (compression ratio 1:3.6), reducing transmission bandwidth to 2.24Mbps (traditional 8Mbps), and data loss rate <0.03% (traditional 2.3%).

[0053] 4.5.3 Three-Dimensional Adaptive Security Authentication and Privacy Protection System Technological breakthrough: Building a comprehensive protection network to address the risk of privacy leaks.

[0054] Dynamic desensitization and access control: Generative adversarial networks (GANs) transform sensitive information in real time (regional feature retention rate >95%), and access verification latency <200ms (60% efficiency improvement over traditional ACL models).

[0055] Multi-factor biometric authentication: dual verification using "password anonymous voiceprint code / fingerprint code", blockchain evidence storage combined with zero-knowledge proofs (ZK-SNARKs) to achieve second-level traceability of operations.

[0056] 4.5.4 Knowledge Graph-Based Consultation Logic Inheritance Technology Technological Breakthrough: For the first time, knowledge graphs are applied to psychological counseling to solve the problem of semantic gaps.

[0057] Knowledge graph construction: Map dialogue, test, and physiological data as graph nodes (average 2000+ entities / users). Graph neural network (GNN) evaluates context similarity, and automatically completes semantics when similarity is <0.9.

[0058] Intelligent context summarization: When switching to a real expert, a visual summary of key nodes is generated in 3 seconds (traditionally it takes 5 minutes); VR scenes are matched with historical preferences according to the "scene emotion" rule, with an accuracy rate of 91%.

[0059] This module utilizes four core technologies—multimodal scheduling, AI data processing, three-dimensional security, and knowledge graphs—to form the first seamless switching solution across all modes in the field of psychological counseling. It achieves orders-of-magnitude breakthroughs in indicators such as switching latency, data integrity, and security level, combining technological integration, problem-specificity, and performance breakthroughs.

[0060] 5. Innovative dynamic secondary screen technology can be used in remote, live expert psychological counseling. The system integrates an intelligent split-screen system into the remote operation interface of a live expert: the main split screen is used for real-time video interaction, while the secondary split screen dynamically and intelligently presents psychological profile information. It features an intelligent highlighting and retrieval function, using natural language processing technology to perform real-time semantic analysis of the dialogue content. When the conversation involves personality analysis, the system integrates multi-source data from the psychological profile's personality assessment, historical test results, and the current dialogue text, highlighting comprehensive personality traits in a visual chart format. If the user voluntarily wears a physiological monitoring device, the system analyzes biological data such as blood pressure, heart rate, and blood oxygen in real time. In addition to displaying the raw readings, it automatically generates quantitative analysis conclusions such as "Stress Index Level 3" and "Stress Value Exceeds Standard by 20%", which are simultaneously displayed on the secondary screen to assist expert decision-making. This mechanism, through multimodal data fusion and intelligent visualization technology, significantly improves the accuracy and professionalism of live experts providing remote psychological counseling.

[0061] The core technology solution consists of four innovative modules, forming a complete technology chain of "data acquisition, semantic analysis, fusion, visualization, and decision support": 5.1. Intelligent Split-Screen Dynamic Interaction System Architecture 5.1.1. Dual-screen collaborative interactive interface, supporting split-screen display on the client of a remote real expert.

[0062] Main split screen (enhanced video interaction area): It adopts WebRTC technology to achieve high-definition video streaming, integrates an audio-visual synchronization optimization algorithm (AV Synchronization Optimization Algorithm) to ensure lip-sync error <50ms, and supports real-time interactive functions such as electronic whiteboard annotation and screen sharing; end-to-end encryption adopts the national cryptographic SM4 algorithm, and the video stream latency is controlled within 200ms.

[0063] Sub-screen (multimodal data visualization area): Deploys a dynamic presentation engine for psychological profiles, using HTML5 Canvas technology to achieve interactive chart rendering. Through Semantic-Driven Display Technology, it intelligently matches the profile information display logic according to the dialogue process; supports mouse hover to view data source annotations (such as "Family Background March 2024 Assessment Results"), and provides a timeline-related display function to automatically annotate the profile data corresponding to key events.

[0064] 5.1.2. Multimodal data input interface By associating data through code IDs, the system accesses electronic psychological profiles (including historical assessment data such as the MMPI scale and projective tests), real-time dialogue text streams (speech-to-text API with a recognition accuracy of ≥95%), and Bluetooth data interfaces of physiological testing devices (smart bracelets, EEG devices), forming a multi-source dataset containing text, numerical values, and time-series signals.

[0065] 5.2. NLP-based Deep Semantic Analysis Module 5.2.1. Domain-Specific Semantic Parsing System A specialized thesaurus for the psychological counseling field containing over 5000 core words was constructed. Named Entity Recognition (NER) was performed using a Bi-LSTM network combined with a Conditional Random Field (CRF) model, achieving an entity recognition accuracy of 95%, which is 18% higher than that of general models. A mapping relationship between "keyword profile fields" (e.g., "Parents" → "Family Communication Patterns" chapter) was established through a knowledge graph. When a keyword is detected, the corresponding field in the sub-screen is automatically highlighted with adaptive color temperature within 0.3 seconds (the font size is increased by 40%, and the background color is dynamically adjusted to yellow / orange according to the sensitivity level).

[0066] 5.2.2. Multi-source personality feature fusion algorithm It integrates psychological assessment data (such as MMPI scale scores), dialogue text features (12-dimensional language style features such as lexical complexity and the proportion of emotional words), and facial expression recognition results (micro-expression frequencies extracted through computer vision) to generate a dynamic personality radar chart through a feature fusion network. It displays the comprehensive scores of five personality dimensions such as "extroversion" and "neuroticism" in real time, and uses principal component analysis (PCA) for dimensionality reduction, with an error rate controlled within 8%. It also supports historical data comparison display.

[0067] 5.3. Intelligent Biosignal Analysis and Decision Support Module 5.3.1. Multi-source physiological signal processing flow On a voluntary basis, data such as heart rate (HR), blood oxygen saturation (SpO2), and skin conductance response (GSR) are collected in real time. The fluctuation range of the indicators is calculated using a sliding window algorithm (window size 30 seconds). Combined with the individual physiological baseline model established during the first test (including parameters such as resting heart rate and mean blood oxygen), an abnormal value alarm is generated. A "physiological and psychological" correlation model is constructed using a recurrent neural network (RNN). Based on features such as heart rate variability (HRV) and GSR change rate, a "stress index" grading model is trained to output a quantitative conclusion of 1-5 levels (classification accuracy ≥92%). When systolic blood pressure is consistently >140 mmHg and the HRV low frequency / high frequency ratio (LF / HF) is >3, a three-level warning system (red / yellow / blue warning boxes) is triggered. After desensitization, this information can be viewed by remote live experts through a secondary screen during consultations.

[0068] 5.3.2. Visual Decision Support Interface Physiological Data Dashboard: Real-time Waveform Graph: Dynamically displays 30-second heart rate and blood oxygen trend curves, with abnormal fluctuation areas automatically highlighted in red; Quantitative Analysis Panel: Displays conclusions such as "Stress Index Level 3 (25% higher than baseline)" and "Stress Value Exceeds Standard by 20%", and indicates the data source (e.g., "Current GSR is 30% higher than the user's average level"); Intervention Suggestion Pop-up Window: When the stress index is ≥ Level 3, it automatically pushes standardized intervention scripts (e.g., "Please follow my rhythm and take deep breaths"), which can be customized and edited by real experts.

[0069] 5.4. System Integration and Adaptive Optimization Mechanism 5.4.1. Multimodal data synchronization technology A message queue (RabbitMQ) is used to achieve asynchronous decoupling of video stream, text stream and physiological data stream. A timestamp alignment algorithm (error <100ms) is used to ensure synchronous display of data. The transmission of psychological records adopts dual encryption of HTTPS and national cryptographic SM4. Sensitive information in the sub-screen is dynamically de-identified (e.g., home address is only displayed to the district and county level, and key medical history information is blurred).

[0070] 5.5. Pioneering Points in the Field of Psychological Counseling 5.5.1. Semantic-driven intelligent highlighting retrieval and dynamic association technology Technical principle: A natural language processing (NLP) engine specifically designed for the field of psychological counseling is constructed. Through multi-level semantic parsing, real-time linkage between dialogue content and psychological profiles is achieved, solving the inefficiency problem of traditional systems relying on manual retrieval.

[0071] (1) Domain-specific Named Entity Recognition (NER) model: Employing a hybrid architecture combining Bi-LSTM and Conditional Random Field (CRF), this study labeled over 50,000 dialogue texts for psychological counseling scenarios, training a dedicated model encompassing 12 entity categories, including "academic pressure" and "emotional symptoms," achieving an entity recognition accuracy of 95.2% (compared to only 77.3% for general NER models). It innovatively introduces a "context-sensitive weighting mechanism," dynamically assigning 1.8 times the weight to crisis keywords such as "suicide" and "harm," and combining this with an attention mechanism to focus on emotionally charged words in the dialogue (such as "despair" and "collapse"), ensuring keyword detection and risk level classification are completed within 0.3 seconds.

[0072] (2) Knowledge graph-driven intelligent positioning system: We constructed a knowledge graph for the psychological counseling field containing 2000+ entity nodes and 5000+ relationships. We established a three-level mapping relationship of "keywords → profile fields → historical data" through graph neural networks (GNN) (e.g., "declining grades" → "academic performance → exam scores → the curve of the last three math scores"). We developed a "semantic distance calculation algorithm" to calculate the semantic similarity between dialogue keywords and profile fields based on Word2Vec word vectors, and dynamically adjusted the display priority of related data (e.g., when "parents" and "arguments" are mentioned at the same time, "family communication pattern assessment results" are displayed first instead of general family background information).

[0073] (3) Multi-dimensional highlighting visualization technology: The system employs a four-color highlighting hierarchy (blue - general association, yellow - key clues, orange - sensitive information, red - crisis warning), utilizing HTML5 Canvas hardware acceleration rendering to achieve millisecond-level redrawing of highlighted areas. It also supports displaying data source paths on mouse hover (e.g., "family background → parent-child conflict record in March 2024 → mentioned at the 12th minute of the current conversation"). For complex concepts such as "personality traits" and "sources of stress," a progressive highlighting strategy is adopted: basic fields are highlighted in yellow when keywords are mentioned for the first time, and the association analysis results (e.g., the "neuroticism" dimension score in personality assessments) are overlaid when keywords are mentioned repeatedly or discussed in depth.

[0074] Innovation Summary: For the first time, a domain-customized NLP model was combined with a knowledge graph to build a complete technical chain of "dialogue semantic parsing → intelligent file location → multi-dimensional highlighting on the secondary screen". This improved the efficiency of key information retrieval by 89% and reduced the rate of missed detection of sensitive information from 35% to 4.7%, providing real-time and accurate information navigation for remote human experts.

[0075] 5.5.2. Dynamic Visualization Decision Support Technology Based on Multimodal Data Fusion Technical principle: Integrating three types of heterogeneous data—psychological assessment data, real-time dialogue text, and physiological signals—and constructing a three-dimensional decision support system through feature fusion algorithms and interactive visualization technology.

[0076] (1) Cross-modal feature fusion analysis framework: Data layer fusion: The MMPI scale score (numerical) is converted into a standardized Z-score, the dialogue text is converted into a 300-dimensional word vector through word embedding, and the micro-expression frequency (such as the number of frowns per minute) is extracted from the video stream using computer vision (CV), and then uniformly converted into a 256-dimensional feature vector. Model layer fusion: A feature fusion network is constructed using a multilayer perceptron (MLP). A comprehensive evaluation vector is generated through dynamic weight allocation (e.g., emotional dialogue text accounts for 60% of the weight in anxiety assessment, and physiological signals account for 30%). After dimensionality reduction by principal component analysis (PCA), the real-time scores of the five personality dimensions are output (error rate ≤ 8%). Results layer fusion: Generates an interactive dynamic radar chart on the sub-screen, supporting the overlay and comparison of historical data (the last 3 consultations) and current data. When the score of a certain dimension fluctuates by more than 15%, a flashing warning is triggered (such as when the "extroversion" dimension drops sharply, the edge of the radar chart flashes red).

[0077] (2) Timeline-related display and semantic origination: Time phrases (such as "three months ago" and "last Tuesday") in the dialogue are extracted using regular expression matching and date recognition models. A ternary association of "event time point → psychological profile fragment → physiological indicator snapshot" is established by combining the archive timestamp. A scalable timeline is generated at the bottom of the sub-screen, and each time point displays the data type in the form of an icon (document icon - archive record, heart rate icon - physiological data). Clicking on it will pop up a floating window to display the specific content (such as "2024.04.15: Heart rate 120 beats / min after math exam, consultation record marked 'exam anxiety'").

[0078] (3) Development of interactive visualization components: A custom chart library based on D3.js is developed, including: Keyword Cloud: Real-time generation of dynamic cloud maps of high-frequency words in conversations, with font size positively correlated with frequency of occurrence (e.g., words such as "exam" and "fear" are significantly enlarged); Physiological waveform graph: The smooth curve algorithm (cubic spline interpolation) is used to display the trend of heart rate and blood oxygen changes over 30 seconds, and abnormal areas are automatically filled with red semi-transparent masks; Intervention suggestion carousel card: Based on the current analysis results, the three most relevant standardized statements are pushed, which can be switched by experts by swiping left and right and referenced with one click.

[0079] Innovation Summary: For the first time, we have achieved deep integration and analysis of psychological assessments, dialogue texts, and physiological signals. Through a three-layer integration framework, we have improved the accuracy of multi-source data association to 92%. Visual decision support has shortened the time for expert problem localization by 78%. We have also constructed the first three-dimensional dynamic analysis model in the field of psychological counseling.

[0080] 5.5.3. Physiological-Psychological Cross-Modal Analysis and Intelligent Early Warning Technology Technical principle: Based on biosignal processing and machine learning algorithms, a quantitative correlation model of "physiological response-psychological state" is established to realize real-time assessment of stress level and crisis intervention support.

[0081] (1) Dynamic modeling of individual physiological baseline: During the initial test, 5 minutes of resting state data were collected to calculate the baseline values ​​(μ) and standard deviation (σ) of heart rate (HR), blood oxygen saturation (SpO2), and skin conductance (GSR) to establish an individual physiological model (normal fluctuation range is μ±2σ). The baseline model is updated in real time using a sliding window technique (window size 30 seconds, step size 10 seconds). Noise is filtered out by a Kalman filter algorithm to adapt to changes in individual stress thresholds during long-term counseling (e.g., after 3 counseling sessions, the resting heart rate baseline decreased from 85 beats / min to 78 beats / min).

[0082] (2) Recurrent Neural Network (RNN) stress grading model: Input layer: Extracts 6-dimensional physiological features such as heart rate variability (HRV), GSR change rate, and blood oxygen fluctuation amplitude, and combines them with the frequency of stress keywords in the dialogue text (statistically counted in real time through an NLP model). Hidden layer: Two layers of long short-term memory network (LSTM) (128 neurons each) capture the temporal dependence of physiological signals (e.g., a sustained increase in the low-frequency component of HRV predicts the accumulation of chronic stress). Output layer: The Softmax function outputs a stress index of 1-5 levels, with a classification accuracy of 92.3% (the test set contains 2000 samples, and the Kappa consistency coefficient is 0.89). Crisis warning mechanism: When HR > 100 beats / min and GSR > baseline 1.5σ are detected for 3 consecutive windows, the three-level warning system is triggered (blue - attention, yellow - warning, red - emergency), the corresponding color warning light is displayed in the upper right corner of the sub-screen, and the voice prompt "Current physiological indicators show a state of high stress" is given.

[0083] (3) Standardized intervention recommendation generation system: Establish an intervention library containing 200+ professional scripts, categorized and stored according to problem type (academic / family / emotional) and stress level, and achieve intelligent matching through a combination of rule-based expert system and reinforcement learning model; It supports experts manually marking the effectiveness of suggestions (effective / ineffective), and optimizes the matching strategy through backpropagation algorithm, which improves the accuracy of suggestions from the initial 75% to 88%, and automatically generates personalized intervention plans (such as pushing "progressive muscle relaxation guidance + sharing of historical successful coping cases" for users with "exam anxiety + high heart rate").

[0084] Innovation Summary: Breaking through the limitations of traditional physiological data that only displays raw readings, this study achieves precise quantification of stress state through individual baseline modeling and RNN hierarchical modeling, improving the accuracy of acute psychological crisis identification by 21.3%, and constructing a closed-loop technical system of "physiological signal acquisition - cross-modal analysis - intelligent early warning - intervention suggestions".

[0085] 5.5.4. Reinforcement Learning-Driven System Adaptive Interaction Optimization Techniques Technical principle: Based on expert interaction behavior data, the layout of interface elements, information recommendation strategies and operation processes are dynamically optimized through reinforcement learning to achieve intelligent interaction that "adapts to the operating habits of experts".

[0086] (1) Interactive behavior log analysis system: Record three types of core interaction data: Information retrieval behaviors: number of clicks on highlighted fields, frequency of switching between profile modules, and duration of viewing physiological dashboards; Decision support behaviors: adoption rate of intervention suggestions, data annotation operations (such as manually marking "important" fields), and number of times charts are zoomed / dragged; Efficiency indicators: time to locate key information in a single consultation, total duration of interface operations, and number of unnecessary window switching.

[0087] Hidden Markov Model (HMM) was used to analyze the operation sequence and identify the high-frequency operation patterns of experts (such as an expert's habit of checking physiological data before locating the file).

[0088] (2) Reinforcement learning optimization engine: State space: Defines 20+ optimizable parameters, including interface layout (e.g., the ratio of left and right partitions in the sub-screen), highlighting strategy (e.g., keyword weight coefficient), and data push frequency (e.g., updating analysis results every 5 minutes / 10 minutes). Action space: 50+ operation actions such as adjusting field highlight colors, reorganizing data module layout, and dynamically increasing or decreasing the density of recommendation information; Reward function: A multi-objective reward mechanism is constructed based on information retrieval efficiency (-20%→+20%), operation smoothness (interface switching delay <100ms, reward +10 points), and expert satisfaction (manual scoring interface). The optimal interaction strategy is trained through a deep Q network (DQN), and the optimization scheme is automatically updated once every 100 consultations.

[0089] Innovation Summary: This project is the first to apply reinforcement learning technology to the optimization of the psychological counseling interface. Through behavioral log analysis and personalized strategy generation, the efficiency of expert operation gradually increases by 20%-30% over time, and the subjective score of interface interaction fluency increases by 40%. A closed-loop optimization system of "user behavior data → intelligent algorithm optimization → interactive experience upgrade" has been constructed.

[0090] 5.6. Technological Effects in Pilot Schools |Technical Specifications|Invention|Traditional System|Improvement Amount| Key information retrieval | Time ≤ 2 seconds | 15-20 seconds | Efficiency improvement of 89% | |Multimodal data|Association accuracy 92% | 65% | Improvement 41.5% | |Tension Index|Classification Accuracy ≥92% |75% |Improvement 22.7% | |Expert preparation time per consultation| 3 seconds| 5 minutes| 99% efficiency improvement| |Acute psychological crisis|Identification accuracy rate 91% | 75% | Improvement 21.3% | |Concordance in adolescent stress diagnosis| 0.89 | 0.61 | Improvement of 45.9% | |Sensitive data leakage risk| Leakage rate 0% |Manual de-identification (missed de-identification rate 5%) |Risk cleared to zero| 6. Intelligent psychological counseling service selection mechanism Students can choose their own or have the system intelligently recommend a psychological counseling mode. The core innovations are as follows: 6.1 Two-way guided autonomous selection mode Visualized Decision Support: A visualized service format matrix (horizontal axis represents the consultation subject, including software robots, physical robots, and human experts; vertical axis represents the scene type, including VR virtual rooms, simple consultation rooms, and offline clinics) supports gesture or voice switching, displays the characteristics of each mode in real time, and dynamically highlights recommendations based on dialogue keywords (e.g., when "fear of communicating with people" is mentioned, the VR virtual room icon flashes). When a user clicks on a mode, the system retrieves historical consultation effect data for that mode (e.g., anxiety index decreased by 20% after the last VR consultation) and provides a "one-click comparison" function, visualizing the ratings of different modes in dimensions such as "privacy protection" and "emotional support" through a radar chart. This reduces user selection time from an average of 2 minutes to 30 seconds, decreasing the churn rate caused by selection difficulties by 40%.

[0091] 6.2 Intelligent Recommendation Mode Based on Multimodal Fusion For students with unclear consultation needs, the system uses Natural Language Processing (NLP) to parse dialogue text, Computer Vision (CV) to recognize facial micro-expressions and voice tone, and integrates multi-source data such as psychological tests and historical records to construct a 128-dimensional input vector containing dialogue semantics, micro-expressions, physiological indicators (such as heart rate variability and skin conductance), and historical features. After dimensionality reduction using Principal Component Analysis (PCA), a deep neural network (DNN) combined with an attention mechanism is used for hierarchical matching: the first-level coarse matching uses the K-Nearest Neighbors (K-NN) algorithm to filter candidate patterns, the second-level fine matching outputs matching scores, and a reinforcement learning model is introduced to update matching weights in real time based on user feedback. This algorithm is the first to integrate four types of data, improving the accuracy of traditional single-label recommendations from 65% to 92%, and can dynamically adjust the data modality weights based on emotional intensity (e.g., micro-expression features account for 50% of the weight in a high-anxiety state), solving the recommendation problem of "contradiction between user language and true emotion".

[0092] 6.3 Reinforcement Learning-Driven Adaptive Mechanism Real-time status monitoring and mode switching: The system continuously monitors the user's status during the consultation process. When a change in needs is detected (such as a clear signal of needing help and an increase in emotional intensity), a mode switching suggestion is automatically triggered. The switching process synchronizes the conversation history and emotional parameters to ensure service continuity. If a crisis signal is detected (such as the keyword "I don't want to live anymore" + a sudden increase in heart rate), the "live expert + offline emergency consultation" mode is prioritized within 0.5 seconds, improving crisis intervention response efficiency by 60%.

[0093] Closed-loop optimization system: The matching model is continuously trained by user feedback data on the recommendation mode. The recommendation accuracy automatically increases by 5%-8% after every 100 consultations, achieving intelligent evolution of "the more you use it, the more accurate it becomes". Attached Figure Description

[0094] Figure 1 : This is a flowchart illustrating the principle of this application. Core process: Users select various consultation modes (educational robot, remote human, human-computer collaboration, etc.), and can choose environments such as ordinary environment, simple psychological counseling room, VR counseling room, etc. → Anonymous collection of multimodal data (text / voice / video, etc.) → Data is transmitted encrypted through a multimodal data real-time synchronization engine → Intelligent scheduling technology driven by reinforcement learning enables seamless mode switching → Dynamic secondary screen technology intelligently links psychological profiles to assist decision-making → Data is stored securely and complies with regulatory requirements.

[0095] Figure 2: Privacy Protection System Architecture Flowchart The core process includes anonymous identifier generation and dual encryption protection. Innovation: It integrates QuantumKey, Federated Learning, and Blockchain technologies to achieve "data usable but not visible," reducing the risk of privacy breaches by more than 90% and meeting the requirements of laws and regulations such as the Personal Information Protection Law.

[0096] Figure 3: Flowchart of the technical architecture for seamless switching between all forms This diagram illustrates the technical process for seamless switching between consultation modes, involving data acquisition and encoding, intelligent scheduling and compression, etc. Innovation: For the first time, it achieves full-form dynamic scheduling of "software robot, physical robot, real-person expert, and virtual scene," with a switching latency of less than 1 second and a data loss rate of less than 0.03%, solving the service interruption problem of traditional systems.

[0097] Figure 4: Data Flow Diagram of Remote Live Expert Intelligent Split-Screen Interface This diagram illustrates the multimodal data fusion mechanism of the remote consultation interface, including a split-screen interactive architecture and semantic-driven highlighting. Innovation: It constructs a three-dimensional support system of "dialogue semantic analysis, file intelligent positioning, and physiological indicator quantification," and for the first time applies knowledge graphs and dynamic visualization technology to remote psychological consultation, improving key information retrieval efficiency by 89%. Detailed Implementation

[0098] 1. Implementation details of the privacy protection module 1.1. Implementation of Anonymity Mechanism Upon initial registration, users are prompted to enter a unique ID (≥10 characters long) conforming to the structure "letters + numbers + symbols" and a complex password, such as "Abc123!@#Xyz". The system verifies in real-time whether it contains personalized information such as the user's name in pinyin or school abbreviation. A unique identifier is generated using the SHA-256 hash algorithm and linked to the blockchain distributed ledger. All consultation records and chat content are stored using this unique ID. Real identity information is completely removed through anonymization, retaining only metadata such as time and duration. The ID is bound to a biometric hash value and stored in a secure chip. If the AI ​​model detects residual personalized information, it triggers a blockchain correction process. In addition to usernames and passwords, the system also supports two-factor authentication using anonymous voiceprint / fingerprint codes.

[0099] 1.2. Double Encryption and Federated Learning The electronic psychological profile first identifies and removes sensitive information such as home address and medical history using a Natural Language Processing (NLP) model. The remaining data is then encrypted using the SM4 algorithm and further encrypted using a quantum key. The quantum key is generated in real time via quantum communication equipment and destroyed after each encryption. When a school requests a statistical report, the system triggers a federated learning process: each user profile runs a statistical model on its local terminal, uploading only the encrypted feature vectors (such as consultation frequency and question type) to the server. A report containing common features is generated through a secure aggregation algorithm. The original data is retained on the user's terminal throughout the process, with a statistical error of <0.1%. All data usage traces are stored on the blockchain.

[0100] 1.3. Blockchain Permission Management Permissions are divided into user level, school level, and administrator level. Schools need to be authenticated by the user to request access to statistical reports. All operations are stored in the form of blockchain transactions, including operation time, code ID and type (such as "application report"). The immutability of blockchain ensures traceability and auditability.

[0101] 2. Implementation details of the multimodal consulting module 2.1. Application of Physical Educational Robots The physical robots deployed at campus gates, dormitory entrances, and other areas are equipped with Type-C hot-swappable interfaces, allowing them to connect to VR glasses, smart bracelets, and brain-computer interface helmets. For example, when a user wears a smart bracelet, the robot collects heart rate and blood oxygen data in real time, and combines this data with the conversation to generate a multi-dimensional psychological assessment model. If it detects a heart rate greater than 100 beats per minute and mentions "stress" in the conversation, it automatically recommends VR relaxation scenes.

[0102] The foldable, simple psychological counseling room module is normally stored in the robot charging cabinet. When triggered by the user, it unfolds to form a 1.5m x 1.5m private space, equipped with a curtain, folding chair, and touch screen. It supports the selection of software robots, physical robots, or remote human experts for consultation. When connected externally, it can unfold to a 2m³ private space. The device driver uses national cryptographic algorithm signature to ensure firmware security.

[0103] 2.2. VR Immersive Consultation Scenarios Users enter a virtual consultation room through VR glasses and can choose between a "replica of reality" mode (restoring a school counseling room) or a "custom" mode. Scenes such as "grassland ranch" or "Mars base" can be generated through voice commands. The expert's avatar can be customized as a cartoon character or a science fiction element, and the voice is synthesized using text-to-speech (TTS) technology to reduce users' social anxiety.

[0104] 3. Implementation details of the full-mode switching module 3.1 Multimodal Data Synchronization Process When a user switches from a software chatbot to a physical chatbot, edge computing nodes capture conversation snippets and emotional parameters (such as an anxiety index of 0.8), generate a 128-dimensional feature matrix using a Bidirectional Long Short-Term Memory (Bi-LSTM) network model, and transmit it to the blockchain via HTTPS + quantum key distribution, storing it in an encrypted block named with a unique ID. Upon receiving an authorization request, the physical chatbot retrieves the data using anonymous voiceprint (or fingerprint) authentication, reconstructs the consultation context within 3 seconds, and achieves a conversation backtracking error of <5%.

[0105] When the VR scene switches to the physical robot, the system analyzes the VR spatiotemporal anchor points (such as virtual ambient lighting and sound effect parameters), controls the physical robot hardware to adjust the light color and play ambient sound effects, and realizes the offline restoration of scenes such as the "Arctic ice field", with a coordinate matching error of less than 2 cm.

[0106] When the action of the user removing the VR glasses is detected (through image recognition via convolutional neural network (CNN)), the edge node completes the compression and encryption of the dialogue segment within 0.3 seconds, synchronously parses the spatiotemporal anchor point environmental parameters, and the physical robot synchronously completes the configuration of lighting / sound effects, with an end-to-end switching latency of <8 seconds.

[0107] 3.2. Intelligent Scheduling Engine Switching intent recognition is achieved through a fusion of a rule engine and reinforcement learning. For example, if the system detects a user removing their VR glasses, a preloading process is triggered within 0.1 seconds, prioritizing currently online physical robot terminals. The switching cost model is based on 12 parameters, including network latency and device load, and dynamically selects the optimal path using a deep neural network (DNN), such as prioritizing the use of 5G networks with bandwidth >100Mbps for VR transmission.

[0108] 4. Real-person expert intelligent assistance module 4.1. Operation of the intelligent split-screen system The main split screen uses WebRTC technology to achieve 1080P video transmission with a latency of <200ms, supports electronic whiteboard annotation, and the video stream is encrypted using the SM4 algorithm; the secondary split screen is based on HTML5 Canvas to render psychological profiles in real time, including: real-time personality radar chart (five-dimensional scoring); physiological waveform chart (30-second trend analysis) and heart rate variability (HRV) low-frequency / high-frequency ratio (LF / HF) monitoring. When LF / HF>3, a red warning box pops up on the secondary screen, indicating "stress value exceeds the standard by 20%", and pushes standardized intervention scripts; crisis warning lights (red / yellow / blue three colors).

[0109] When keywords such as "parent-child conflict" and "exam stress" are mentioned in the conversation, the Natural Language Processing (NLP) model triggers a knowledge graph retrieval, automatically highlighting relevant chapters in the file (such as "family communication patterns" and "academic performance") and historical assessment data, enlarging the font by 40% and displaying it in yellow, while simultaneously pushing relaxation techniques or intervention suggestions.

[0110] 5. Implementation details of the intelligent recommendation module 5.1. Multi-dimensional matching algorithm The input data includes: dialogue text word vectors (64-dimensional), micro-expression features (32-dimensional, extracted through a residual network (ResNet) model), physiological indicators (HRV, skin conductance response (GSR, etc., 16-dimensional), and historical profile tags (16-dimensional). After being reduced to 32 dimensions by principal component analysis (PCA), the data is weighted by an attention mechanism (e.g., micro-expression weight is 50% when emotions are intense) and then input into a deep neural network (DNN) model to calculate the matching scores for each pattern.

[0111] For example, when a user repeatedly mentioned "exams" and "nervousness" in their consultation, and their baseline heart rate increased by 15%, the system strongly recommended the "VR virtual room + software robot" mode (matching score of 92 points), because this combination had previously reduced the anxiety index of similar users by 28%.

[0112] 6. Examples of Implementation Methods Case 1: Exam Stress and Family Communication Problems When a user states in a video consultation, "My heart races whenever I have an exam lately, and my parents always say I'm not working hard enough": 6.1. Semantic Analysis Module: The Bi-LSTM+Conditional Random Field (CRF) model is used to identify the keywords "exam" and "parents", triggering the color temperature adaptive highlighting (yellow background + enlarged font) of the "Academic Performance" and "Family Communication Mode" fields in the sub-screen. 6.1.1 Multi-dimensional Semantic-Sentence Joint Analysis Engine A three-tiered, progressive semantic processing architecture is constructed to overcome the limitations of traditional keyword matching: The basic entity recognition layer employs a Bi-LSTM + Conditional Random Field (CRF) joint model (200 word embedding dimensions, 300 hidden layer nodes), combined with a custom dictionary for the psychological counseling domain (containing 2000+ professional terms such as "exam anxiety" and "parent-child communication"), to achieve accurate entity boundary localization (F1 score of 93.7%). When core entities such as "exam" and "parents" are recognized, a dynamic context window expansion mechanism is triggered, automatically capturing the context of the preceding and following 5 words (e.g., "heart racing" corresponds to the physiological reaction dimension, and "not trying hard enough" corresponds to the emotional evaluation dimension).

[0113] The deep semantic encoding layer introduces a pre-trained bidirectional Transformer model (BERT - a fine-tuned version for the psychological counseling domain) to perform deep semantic encoding of the context surrounding entities (768-dimensional character vectors). Through a dynamic emotional attention mechanism, the emotional contribution of each word to the core entity is calculated. An anxiety weight of 0.8 was assigned to "heart racing" (based on an emotion intensity dictionary trained on 50,000 historical dialogues); a resentment weight of 0.7 was assigned to "not trying hard enough" (combined with a negation word detection algorithm to identify the emotion reversal effect of "not"). The final result is a composite representation containing 128 semantic features and a 32-dimensional emotion vector, achieving an 18% improvement in emotion classification accuracy compared to traditional models (from 75% to 93%).

[0114] A cross-modal association reasoning layer constructs a semantic-archival association knowledge graph. When the keywords "exam" and "parents" are detected, a graph neural network (GNN) is used to query the co-occurrence patterns of similar keywords in historical archives: automatically associating abnormal physiological indicators that are often accompanied by "exam stress" (heart rate increase >20% probability 89%); and identifying the assessment tags corresponding to "family communication" problems ("high expectations and low support" type matching degree 91%).

[0115] 6.1.2 Intelligent Visual Interactive Decision-Making System (1) Based on semantic-emotion composite representation, a dynamic visual feedback algorithm is designed to achieve three-level intelligent highlighting: basic information positioning uses a rule engine and machine learning fusion model to map the keywords "exam" and "parents" to the preset fields of the sub-screen ("academic performance" and "family communication mode"), with a mapping accuracy of 98% (based on 100,000 labeled data training).

[0116] The emotional intensity visualization system utilizes a dual-mapping algorithm for color temperature and font: background color is dynamically calculated using the HSV color space (an anxiety value of 0.82 corresponds to a yellow chroma value of 60°, with saturation increasing linearly to 70% with emotional intensity); font size is proportionally scaled up to the magnitude of the emotional vector (formula: font size = 14px + 6px × emotional intensity, currently calculated at 17px). This algorithm has been validated through eye-tracking experiments with 200 users, reducing the time for acquiring key information by 40% compared to a fixed highlight mode (from 2.5 seconds to 1.5 seconds).

[0117] (2) Dynamic association prompts introduce the knowledge graph heat radiation mechanism: A miniature correlation map is displayed next to the highlighted field, annotating the historical co-occurrence probability of "exam anxiety" and "family communication patterns" (83%, data from statistics of 100,000 consultation records); through an attention-guided algorithm, when the user's gaze lingers on a field for more than 2 seconds, it automatically expands to display 3 typical cases of that field (such as "85% of similar cases have improved through family system intervention").

[0118] 6.1.3 Incremental Learning Optimization Mechanism Build an online semantic adaptation system to enable continuous model evolution: When consultants manually adjust the highlighted fields (e.g., adding the label "heart rate increase" to "physiological response"), the interaction data is added to the training set through the active learning algorithm, triggering a micro-update of the model parameters (retraining after every 100 manual annotations, taking <30 seconds).

[0119] Domain knowledge is dynamically injected by regularly capturing the latest psychology literature (such as the new term "parental expectations" in "Family Factors Influencing Test Anxiety"), and the domain dictionary is automatically expanded through remote supervised entity extraction technology to ensure the model's ability to identify emerging concepts (an average of 50+ new professional terms are added per month).

[0120] 6.1.4. Summary of Technological Innovations Dynamic Emotional Attention Mechanism: Breaking through the semantic isolation problem of traditional keyword matching, it enables quantitative calculation of contextual emotional intensity, improving the accuracy of core question localization to 93%; Knowledge graph-based associative reasoning: This is the first time that graph neural networks have been applied to semantic-archive association in a consultation scenario, which increases the speed of making implicit question associations explicit by 5 times; Multimodal visual feedback algorithm: By jointly encoding color temperature, font, and heat map, a visual decision support system that conforms to cognitive psychology is constructed, improving the efficiency of key information acquisition by 40%.

[0121] 6.2. Data Fusion Module: Synchronously retrieves final exam results (showing a math score of 72, ranking in the bottom 30% of the class) and family assessment results (showing a communication type of "high expectations and low support"), displays them in a card-like overlay, and generates a keyword cloud containing "academic pressure" and "parent-child conflict"; 6.3. Physiological Analysis Module: If the user voluntarily wears the smart bracelet, the current heart rate is calculated in real time as 112 beats / min (25% higher than the baseline) and the GSR fluctuation exceeds the threshold. The conclusion of "Stress Index Level 2" is generated, the abnormal area of ​​the heart rate curve is highlighted in red on the physiological dashboard, and the intervention suggestion of "guiding the user to describe the specific feelings before and after the exam" is pushed. 6.4. Expert Decision Support: Based on visualized data, the core problem is quickly identified as "academic pressure compounded by poor family communication." Cognitive and behavioral interventions are then implemented using system-recommended scripts, reducing the time for single problem identification from the traditional 5 minutes to 30 seconds.

[0122] 7. Case Study 2: Anxious Users with Unclear Needs When a student with unclear needs initiates counseling: 7.1. Data Collection: The software education robot detected an increase in the frequency of the user's frowning (6 times / minute) through the camera, and the voice tone analysis showed that the speech rate increased by 20%, with keywords such as "what to do" and "I can't do it" appearing multiple times in the dialogue; 7.2. Intelligent Recommendation: The multi-dimensional matching algorithm integrates the above real-time data and historical records (anxiety index decreased by 30% during consultation with a real expert), and calculates a matching score of 92 points for "Remote Mode with Real Expert" and 85 points for "VR Virtual Room Mode". 7.3. Recommended Presentation: The interface prioritizes displaying the live expert mode, with the recommendation reason "combining current anxiety with historical intervention effects," and provides a VR mode as an alternative, stating that it is "suitable for relaxation needs requiring immersive scenarios." 7.4. Dynamic Adjustment: If a user selects VR mode but actively switches back after 10 minutes, the system records this behavior and reduces the recommendation weight of VR mode in similar scenarios. In the next similar situation, the real-person expert mode will be recommended first.

[0123] 8. Summary of overall system innovations and technical effects Breakthrough in privacy protection: Through a four-dimensional system of "multi-element anonymity + double encryption + federated learning + blockchain", the risk of privacy leakage is eliminated, which complies with the requirements of laws and regulations such as the Personal Information Protection Law and the Cryptography Law.

[0124] Service model innovation: We have built eight types of consultation models (including VR, human-machine collaboration, etc.) to cover different scenario needs, improving service continuity by 95% and reducing user churn rate due to privacy concerns by 80%.

[0125] Improved technical performance: Full-mode switching latency <1 second, data loss rate <0.03%, key information retrieval efficiency improved by 89%, significantly better than traditional systems.

[0126] Advantages of cross-modal analysis: Through AI technologies such as NLP semantic analysis, multimodal data fusion, and physiological signal analysis, cross-modal correlation analysis of psychological and physiological indicators can be achieved, improving the accuracy and professionalism of remote consultation and meeting the requirements of medical-grade psychological services.

[0127] Intelligent Recommendation Mechanism: Constructing the first intelligent service selection mechanism in the field of psychological counseling.

[0128] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multimodal campus psychological counseling system, characterized in that, It includes a privacy protection module, a multimodal consultation module, a full-form switching module and an intelligent interaction module; the privacy protection module achieves complete decoupling of user identity and real information through the following sub-modules: (1) Anonymous identifier generation module (101): It uses multi-element hybrid anonymity technology to generate a code ID without personalized information, and combines hash algorithm and blockchain evidence storage to ensure uniqueness; (2) Federated learning encryption module (102): It uses the national cryptographic SM4 / SM2 algorithm and quantum key dual encryption, superimposed federated learning local computing mode to achieve zero outflow of original data; (3) Blockchain permission management system: It implements three-level permission control based on consortium blockchain and zero-knowledge proof (ZK-SNARKs), with a 100% unauthorized access interception rate.

2. The system according to claim 1, characterized in that, The anonymous identifier generation module (101) adopts a multi-element hybrid anonymity technology: the user ID is composed of ≥10 uppercase and lowercase letters, numbers and symbols, without personalized information, and global uniqueness is verified by combining the hashing algorithm with blockchain notarization; the information flow only uses the ID as the associated identifier, and sensitive information is dynamically transformed through generative adversarial network (GAN).

3. The system according to claim 1, characterized in that, The federated learning encryption module (102) constructs a three-tier protection system: it uses natural language processing (NLP) to strip sensitive information and generate desensitized electronic psychological profiles; it employs dual encryption using the national cryptographic SM4 / SM2 algorithm and quantum key (the quantum key is generated using the quantum no-cloning theorem); and it combines federated learning local computation with a secure aggregation mode to generate statistical reports. The original data does not need to be leaked, and the success rate of personal information traceability is 0%. The privacy protection module also includes a blockchain-driven access control system: it adopts a three-tier access control system (user-level / school-level / administrator-level), and operation records are stored as transaction evidence on a consortium blockchain. It combines zero-knowledge proofs (ZK-SNARKs) and biometric (anonymous voiceprint / fingerprint recognition) dual-factor authentication, achieving a 100% unauthorized access interception rate.

4. The system according to claim 1, characterized in that, The multimodal consultation module includes: a software education robot standalone mode: providing customized consultation based on the user's multidimensional psychological profile (including personality traits and biological data); a physical education robot local mode: integrating VR glasses, brain-computer interface helmets, and smart bracelets through hot-swappable interfaces to collect physiological data such as heart rate and blood oxygen in real time, and supporting the rapid deployment of an integrated foldable simple psychological counseling room module (including privacy curtains, interactive screens, and punching venting kits); and a VR immersive mode: allowing customization of virtual scenes (such as the Arctic ice field and nebula universe) and expert images (real people / cartoons / science fiction characters).

5. The foldable, simplified psychological counseling room module according to claim 4, characterized in that: The unfolding process adopts an intelligent scheduling algorithm, and the steps are as follows: (1) Data acquisition: The stress value F_i of the folding structure is collected by the pressure sensor (Pressure Sensor) (i=1~n), and the distance D_j of the obstacle is collected by the laser rangefinder (Laser Rangefinder) (j=1~m); (2) Risk assessment: The risk coefficient R = α・max (F_i) + β・min (1 / D_j) is calculated, where α+β=1 (0<α,β<1); (3) Priority calculation: If R≤R threshold, the unfolding order is determined by P_k = γ・T_k + (1-γ)・S_k (k=1~p), where T_k is the time urgency (0-1), S_k is the space requirement (0-1), 0<γ<1; (4) Motion control: The central controller unfolds the modules in descending order of priority through the servo motor (Servo Motor), with a speed v = v_0 / (1+R).

6. The system according to claim 4, characterized in that, The hot-swappable interface supports real-time desensitized acquisition of biosensor data, secure transmission of brain-computer interface signals, and human-machine collaborative services between physical robots and remote human experts. Biometric data and dialogue content are synchronized to the expert's end to assist in decision-making.

7. The system according to claim 1, characterized in that, The full-form switching module achieves seamless connection through a multi-terminal switching engine: based on a timestamp event-driven mechanism, it completes the encrypted packaging and blockchain storage of dialogue content and emotion parameters within 0.3 seconds; it uses a deep reinforcement learning (DRL) algorithm to evaluate 12 environmental parameters (such as network bandwidth and device load), dynamically selects the optimal switching path within 500 milliseconds, and combines spatio-temporal anchor technology to achieve VR scene reconstruction error <5% and end-to-end switching latency ≤1 second.

8. The system according to claim 7, characterized in that, The multimodal data fusion technology includes: parsing dialogue text through natural language processing (NLP), analyzing speech tone and micro-expressions by combining bidirectional long short-term memory network (Bi-LSTM), and generating a composite feature matrix containing semantic features and physiological indicators through edge computing optimization (compression rate 72%, bandwidth requirement ≤2.24Mbps), supporting cross-modal context coherence inheritance.

9. The system according to claim 1, characterized in that, The intelligent interaction module includes an integrated intelligent split-screen system: the main split-screen uses WebRTC technology to achieve 1080P high-definition video stream (latency ≤200ms, lip-sync error <50ms), and the secondary split-screen uses a domain-customized named entity recognition model (NER, accuracy ≥95%) and a knowledge graph to dynamically highlight psychological profile fields, combined with a multimodal data fusion radar chart (five personality dimensions analysis), with key information retrieval time ≤2 seconds.

10. The system according to claim 1, characterized in that, The system also includes an intelligent recommendation module: supporting visualized guidance of service mode matrices, historical effect comparison radar charts, and real-time mode switching suggestions for crisis signals. A multimodal matching model with 128-dimensional feature vectors is constructed, and matching results are generated through a hierarchical recommendation algorithm using K-Nearest Neighbors (K-NN) and Deep Neural Networks (DNN). Reinforcement learning is used to dynamically optimize the matching strategy, achieving intelligent service mode recommendation.

Images