A method and system for automatic supervision of psychological counseling

Abstract

The application discloses a kind of psychological counseling automatic supervision method and system, belong to artificial intelligence and mental health technical field.The method is by real-time acquisition counseling dialogue audio and video and text data, utilize trained multi-modal emotion recognition model output emotion vector and main emotion value, according to this detect emotion mutation event;When negative emotion mutation occurs, corresponding audio and video slice is carried out hard cut and text alignment, sensitive entity is identified using named entity recognition and regular expression dual-engine, and it is mapped into 8-bit reversible token to generate desensitization evidence package using hash;Based on desensitization evidence package, counselor response text is extracted in negative emotion period, input three-dimensional supervision scoring model carries out three-dimensional scoring of emotion reflection, cognitive reconstruction and safety assessment, and when not up to standard, demonstration correction sentence is pushed in real time.The present application realizes second-level emotion early warning, privacy evidence and closed-loop supervision, effectively improves the real-time, safety and supervision refinement level of psychological counseling.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and mental health technology, in particular to a psychological counseling automatic supervision method and system. BACKGROUND

[0002] At present, multi-modal emotion recognition and dialogue quality evaluation, as an important application of artificial intelligence in the field of mental health, has made significant progress in emotion detection, risk warning, supervision training, etc., and has become a key means to improve counseling efficiency and visitor safety, and has achieved good results in online education and remote psychological intervention.

[0003] However, how to capture negative emotions in real time in the counseling dialogue flow, and conduct privacy forensics and analysis supervision on the evidence, is a technical problem that needs to be broken through at present. The traditional method mainly relies on manual playback of audio and video or offline text analysis, which has the problems of high delay, high risk of privacy leakage, and coarse supervision granularity, and it is difficult to realize second-level closed loop.

[0004] "Early warning-recording-postmortem" is a necessity for counseling quality management, risk prevention and control, and skill growth. Current research includes speech emotion recognition, video micro-expression analysis, and automatic supervision based on LLM, but the organic integration of emotion recognition, privacy desensitization, and analysis supervision is still a blank, and there are defects such as sensitive information residue and single evaluation dimension.

[0005] Therefore, in the psychological counseling scene, it is a prominent technical difficulty to realize "second-level emotion early warning-privacy forensics-analysis supervision" full-link automation.

[0006] The prior art has the following technical limitations:

[0007] (1) Lack of second-level negative emotion capture and visual feedback. Existing emotion recognition is mostly based on offline analysis of the entire section, which sends the entire recording or video into the model at once, and cannot locate 20s-level mutation frames, nor can it give immediate visual feedback on intervention effectiveness. This "post-batch" mode is easy to miss the crisis window and is difficult to use for real-time warning.

[0008] (2) Lack of privacy protection. Current supervisors analyze the original verbatim transcript, which can easily lead to naked exposure of original sensitive information, such as name, mobile phone, and address leakage risk. If there is no record at all, it is impossible to support accurate forensics for judicial or institutional quality inspection.

[0009] (3) Supervision evaluation dimension is limited, and skill improvement is insufficient. The existing method only gives a general grade in the counselor response score, which is easy to have a "good / bad" binary label problem; meanwhile, it lacks a three-dimensional fine-grained mechanism of "emotion reflection-cognitive reconstruction-safety evaluation", which cannot locate the violation points sentence by sentence and give a landable demonstration sentence, affecting the immediate correction and subsequent growth of the counselor. SUMMARY

[0010] To solve the technical problems in the background art, the present application provides a psychological counseling automatic supervision method and system.

[0011] The psychological counseling automatic supervision method provided by the present application comprises the following steps:

[0012] S1, obtaining real-time audio and video streams and automatic speech recognition text data of a counseling dialogue;

[0013] S2, inputting the audio and video streams and the text data into a trained multi-modal emotion recognition model to output a seven-dimensional emotion vector and a main emotion value in a preset time sliding window unit;

[0014] S3, detecting a negative emotion mutation event and a positive emotion mutation event according to the change of the main emotion value;

[0015] S4, when the negative emotion mutation event occurs, audio and video slices corresponding to the time period are intercepted from the audio and video streams and the text data are aligned, a named entity recognition model and a regular expression engine are used to identify sensitive entities in the text data, the identified sensitive entities are hashed and mapped into 8-bit reversible tokens, and the original text is replaced to generate a desensitization evidence package;

[0016] S5, based on the desensitization evidence package, in the negative emotion time period, the counselor response text is extracted and input into a trained three-dimensional supervision scoring model together with the dialogue context, and emotion reflection dimension score, cognitive reconstruction dimension score and safety evaluation dimension score are output, and when any dimension score does not reach a safety threshold, a demonstration correction sentence is generated and pushed immediately.

[0017] Preferably, the use of the named entity recognition model and the regular expression engine to identify the sensitive entities in the text data specifically comprises:

[0018] The text data is input into the named entity recognition model and the regular expression engine in parallel, the named entity recognition model is based on a large language model fine-tuned on a psychological counseling corpus, and identifies and labels text segments belonging to four types of entities, i.e., names, addresses, institutions and mobile phone numbers; the regular expression engine synchronously runs pre-configured regular rules for identifying and capturing formatted sensitive information strings including ID numbers, bank card numbers, QQ numbers and WeChat numbers;

[0019] Merge the recognition results of the named entity recognition model and the regular expression engine to form a complete list of sensitive entities, and record the location information of each entity in the text data.

[0020] Preferably, the identified sensitive entities are hashed into 8-bit reversible tokens and the original text is replaced to generate a desensitized evidence package, specifically including:

[0021] Each identified sensitive entity is calculated by the HMAC-SHA256 keyed hash message authentication code algorithm to generate a 32-byte hash value, the 32-byte hash value is converted into a hexadecimal string, and the first 8 characters are taken as the core summary of the entity. The core summary and the corresponding entity type identifier are spliced to generate a standard format reversible token;

[0022] In the same session, the same reversible token is forced to be reused for the same entity to maintain context consistency, all generated reversible tokens are used to replace the corresponding sensitive entity in the original text, a completely desensitized text copy is generated, the desensitized text copy, the corresponding audio and video clip, and the emotion data are packaged into a desensitized evidence package, and the original text and the token mapping table are separated and encrypted.

[0023] Preferably, it also includes:

[0024] S6, after the end of the consultation session, generate a double-track time axis visualization chart according to the emotion value sequence and the counselor response score sequence, generate an overall evaluation and improvement suggestion based on the counselor response score of the entire session using the trained overall supervision score model, and integrate the double-track time axis visualization chart, the improvement suggestion and the overall evaluation to generate a structured supervision report.

[0025] Preferably, the seven-dimensional emotion vector includes seven dimensions of anger, happiness, sadness, fear, surprise, disgust and neutrality, wherein the main emotion value is determined by selecting the dimension with the largest value in the seven-dimensional emotion vector; the training process of the multi-modal emotion recognition model specifically includes:

[0026] Obtain a training data set, which contains a large number of labeled multi-modal sample data, each sample includes a synchronous audio and video clip and its corresponding text transcription, and is labeled with a seven-dimensional emotion vector true value;

[0027] Construct a multi-modal emotion recognition model, which includes an audio feature extraction network, a video feature extraction network, a text feature extraction network, and a multi-modal feature fusion module;

[0028] Supervised training of the multi-modal emotion recognition model using the training data set, optimizing model parameters by minimizing the cross-entropy loss function between the seven-dimensional emotion vector output by the model and the true value;

[0029] During the training process, the training data set is divided into a training set, a validation set and a test set according to a preset ratio, and when the performance of the model on the validation set reaches a preset standard, the training is stopped, and the trained multi-modal emotion recognition model is obtained.

[0030] Preferably, the change in the main emotion value is used to detect negative emotion mutation events and positive emotion mutation events, specifically including:

[0031] The negative emotion mutation event refers to the change of the main emotion value from a positive value to a negative value, wherein the positive value corresponds to happy emotion and neutral emotion, and the negative value corresponds to surprise emotion, disgust emotion, sadness emotion, fear emotion and anger emotion; the positive emotion mutation event refers to the change of the main emotion value from a negative value to a positive value, wherein the positive value corresponds to happy emotion and neutral emotion, and the negative value corresponds to surprise emotion, disgust emotion, sadness emotion, fear emotion and anger emotion.

[0032] Preferably, the training process of the three-dimensional supervision score model specifically includes:

[0033] A training data set is constructed, which contains a large number of psychological counseling dialogue samples, each sample is composed of a client statement, a counselor response statement and a dialogue context, and is independently labeled by multiple experienced supervisors according to three dimensions of emotion reflection, cognitive reconstruction and safety evaluation, and the labeling result of each dimension is a binary label;

[0034] A three-dimensional supervision score model based on a large language model is constructed, and a prompt word engineering is used to solidify the psychological counseling supervision rules into a model input template;

[0035] Supervised training of the three-dimensional supervision score model using the training data set, optimizing model parameters by minimizing the cross-entropy loss between the prediction results of the model in three dimensions and the supervisor labeling results;

[0036] During the training process, the training data set is divided into a training set, a validation set and a test set according to a 7:2:1 ratio;

[0037] When the accuracy of the model on the validation set and the consistency with the supervisor labeling reach a preset threshold, the training is stopped, and the trained three-dimensional supervision score model is obtained.

[0038] Preferably, the training process of the overall supervision score model specifically includes:

[0039] construct a total training data set, which contains a large number of data samples of complete psychological counseling sessions, each sample including an emotion value sequence of the whole process of the session, a counselor response score sequence and a corresponding double-track time axis visualization chart, and is annotated by a plurality of senior supervisors based on the sequence and the chart to comprehensively evaluate the overall performance of the counselor;

[0040] construct a total supervision scoring model based on a large language model, and use a structured prompt template to solidify psychological counseling supervision rules into a model input format;

[0041] use the total training data set to supervise the training of the total supervision scoring model, and optimize the model parameters by minimizing the difference between the comprehensive evaluation results output by the model and the supervision teacher annotation results;

[0042] In the training process, the total training data set is divided into a training subset, a validation subset and a test subset according to a predetermined proportion, the training is stopped when the consistency of the comprehensive evaluation results of the model on the validation subset and the supervision teacher annotation reaches a predetermined standard, and the trained total supervision scoring model is obtained.

[0043] The present application provides a kind of psychological counseling automatic supervision system, comprising:

[0044] Multi-modal seven-dimensional emotion recognition module, for obtaining real-time audio and video stream and automatic speech recognition text data of counseling dialogue, inputting the audio and video stream and text data into the trained multi-modal emotion recognition model, outputting seven-dimensional emotion vector and main emotion value in a predetermined time sliding window unit;

[0045] Negative emotion desensitization recording module, for detecting negative emotion mutation event and positive emotion mutation event according to the change of the main emotion value, when a negative emotion mutation event occurs, audio and video slices of the corresponding time period are intercepted from the audio and video stream and aligned with the text data, sensitive entities in the text data are identified using a named entity recognition model and a regular expression engine, the identified sensitive entities are hashed into 8-bit reversible tokens and the original text is replaced to generate a desensitization evidence package;

[0046] Three-dimensional supervision intelligent agent module, for extracting counselor response text based on the desensitization evidence package and inputting the counselor response text and the dialogue context into the trained three-dimensional supervision scoring model to output emotion reflection dimension score, cognitive restructuring dimension score and safety assessment dimension score, and generating and pushing a demonstration correction sentence in real time when any dimension score does not reach a safety threshold.

[0047] Preferably, the multi-modal seven-dimensional emotion recognition module further comprises a color mapping unit configured to map the main emotion value to a predetermined RGB color value and display a negative warning identifier and an intervention success identifier in the emotion time sequence chart.

[0048] In the present application, the proposed psychological counseling automatic supervision method and system realizes three breakthroughs in the field of psychological counseling quality supervision through the organic integration of multi-modal emotion recognition, privacy desensitization recording and three-dimensional supervisor agent: in terms of real-time, the system realizes the immediate capture of negative emotion mutation and intuitive feedback of intervention effectiveness through sliding window emotion analysis and visual warning signs, significantly shortening the response delay caused by traditional offline analysis, ensuring that the counselor can timely grasp the key intervention opportunity; in terms of security, the dual-engine desensitization and reversible token technology is adopted to complete the automatic replacement and encryption isolation of sensitive information in the streaming process, which not only ensures the integrity and availability of the evidence chain, but also realizes zero exposure of the original privacy data, and builds a privacy protection system that meets the judicial requirements; in terms of supervision refinement, through the three-dimensional scoring mechanism and instant demonstration sentence pushing, the traditional binary evaluation of coarse granularity is upgraded to a closed-loop supervision mode of traceable sentence by sentence and multi-dimensional improvable, and with the help of double-track time axis and structured report generation, a systematic and visual growth path is provided for the counselor skill improvement, effectively solving the industry problems of single evaluation dimension and lack of improvement suggestions in the supervision process. BRIEF DESCRIPTION OF DRAWINGS

[0049] Fig. 1 A workflow schematic diagram of a psychological counseling automatic supervision method proposed by the present application;

[0050] Fig. 2 An emotion timing diagram of an embodiment of a psychological counseling automatic supervision method proposed by the present application;

[0051] Fig. 3 A system architecture schematic diagram of a psychological counseling automatic supervision method and system proposed by the present application. DETAILED DESCRIPTION

[0052] REFERENCE Figs. 1-3 The psychological counseling automatic supervision method proposed by the present application comprises the following steps:

[0053] S1, obtaining real-time audio and video streams and automatic speech recognition text data of a counseling dialogue.

[0054] S2, inputting the audio and video streams and the text data into a trained multi-modal emotion recognition model to output a seven-dimensional emotion vector and a main emotion value in a preset time sliding window unit.

[0055] In the present embodiment, the seven-dimensional emotion vector includes seven dimensions of anger, happiness, sadness, fear, surprise, disgust and neutrality, and the main emotion value is determined by selecting the dimension with the largest value in the seven-dimensional emotion vector; the training process of the multi-modal emotion recognition model specifically comprises:

[0056] obtain a training data set, the training data set containing a large number of labeled multi-modal sample data, each sample including a synchronous audio-video segment and its corresponding text transcription, and being labeled with a seven-dimensional emotion vector true value;

[0057] construct a multi-modal emotion recognition model, the model including an audio feature extraction network, a video feature extraction network, a text feature extraction network, and a multi-modal feature fusion module;

[0058] use the training data set to supervise the training of the multi-modal emotion recognition model, and optimize the model parameters by minimizing the cross-entropy loss function between the seven-dimensional emotion vector output by the model and the true value;

[0059] In the training process, the training data set is divided into a training set, a validation set and a test set according to a preset proportion, and when the performance of the model on the validation set reaches a preset standard, the training is stopped, and a trained multi-modal emotion recognition model is obtained.

[0060] Specifically, the preset time is 20s, in order to obtain the current main emotion value, the latest 20s video stream is first sliced with a fixed window, and the audio-video-text three-modal frame-level features are sent into the multi-modal large model, so that the model outputs the main emotion value, forming a seven-dimensional emotion vector and a main emotion value, to ensure that the subsequent decision is based on interpretable reasoning. For the tthsecond window, the seven-dimensional emotion vector is represented as:

[0061] wherein, i=angry (angry), happy (happy), sad (sad), fear (fear), surp (surprise), disgust (disgust), neut (neutral);

[0062] In this embodiment, the main emotion value is converted into an RGB lighthouse value according to a fixed color mapping table. The one-to-one correspondence between the RGB lighthouse value and the emotion in the fixed color mapping table is as follows: angry--deep red, happy--emerald green, sad--lake blue, fear--purple, surprise--orange, disgust--brown, and neutral--gray. The binary tuple (t, ) writes the corresponding emotion into the cache in real time, and a smooth curve is fitted to form an emotion timing diagram containing an accurate emotion straight line diagram and a fitted emotion curve diagram.

[0063] It should be noted that when the emotion is changed from neutral / happiness to negative emotion (anger, sadness, fear, surprise, disgust), the negative early warning sign is started to prevent the counselor from missing; when the emotion is changed from negative emotion to neutral / happiness, the intervention success sign is started for subsequent supervision highlighting. At the same time, it is updated in real time according to the intervention sequence, which not only intuitively prompts the current emotion, but also immediately feeds back the intervention effect. Thus, the "output time-main emotion-negative emotion sign-intervention success sign" four continuous flows provide visual, interpretable, and closed-loop real-time basis for high-risk slice triggering and intervention quality assessment.

[0064] Specifically, the audio-visual text of the visitor is captured in a 20-second sliding window, a multimodal large model outputs a seven-dimensional emotion vector, and is mapped into a "lighthouse" color to draw a retroactive time curve in real time. When the curve is changed from green / grey to red, blue, purple and other negative colors, the "negative early warning" lighthouse is immediately lit; if the color is changed to green / grey again, the "intervention success" white star sign is popped up. The counselor can immediately determine the emotion main signal and intervention effect with one eye, which provides closed-loop visual basis for subsequent high-risk slice interception and supervision evaluation. The emotion positive and negative mutation points are captured, the corresponding audio and video are hard cut, and the emotion-text block is synchronized and aligned, which is saved without frame loss. The slice starts the "NER+regular" dual-engine desensitization, which hashes sensitive entities such as names and mobile phones into 8-bit reversible codes, which are consistent in context and available for quality inspection. The original data and mapping table are isolated by AES-256-GCM encryption, and only the desensitized copy is circulated. When authorized by the court or the client, two-factor decryption can reconstruct the plaintext to realize fast evidence collection and full-link privacy protection in high-sensitivity counseling scenarios.

[0065] In the negative emotion interval, the three-dimensional scoring prompt word Prompt-Scorer is immediately called for each response of the counselor, which is scored 0 / 1 according to "emotion reflection-cognitive reconstruction-safety evaluation" three-dimensional, and the demonstration correction sentence is pushed immediately if the safety threshold is not reached. The emotion and score double-track time axis are recorded synchronously throughout the process, and the visual fold line is automatically aligned and generated when the conversation ends, and then the three-dimensional overall evaluation Prompt-Total-Scorer is output, and the PDF supervision report is obtained, that is, the fold line graph, the improvement list, and the overall evaluation. The "early warning-recording-review" full-link closed loop is realized, which not only guarantees the safety of the visitor, but also quantitatively drives the continuous improvement of the counselor's skills.

[0066] S3, detecting a negative emotion mutation event and a positive emotion mutation event according to a change in the main emotion value.

[0067] In this embodiment, step S3 specifically includes:

[0068] The negative emotion mutation event refers to a change of the main emotion value from a positive value to a negative value, wherein the positive value corresponds to happy emotion and neutral emotion, and the negative value corresponds to surprise emotion, disgust emotion, sadness emotion, fear emotion and anger emotion; the positive emotion mutation event refers to a change of the main emotion value from a negative value to a positive value, wherein the positive value corresponds to happy emotion and neutral emotion, and the negative value corresponds to surprise emotion, disgust emotion, sadness emotion, fear emotion and anger emotion.

[0069] It should be noted that the negative mutation criterion is defined as emotion value from a positive value (1, 0) to a negative value (-1, -2, -3, -4, -5); the positive mutation criterion is defined as emotion value from a negative value (-1, -2, -3, -4, -5) to (1, 0). Once triggered, record the negative mutation frame (UTC millisecond timestamp), record the positive mutation frame (UTC millisecond timestamp), immediately hard-cut the original video stream V_raw and the synchronous audio A_raw, intercept the clip, generate VidClip (video clip) and AudClip (audio clip). At the same time, extract the emotion label sequence E_clip and the ASR text sequence T_clip in the same segment, time-align them in 100ms granularity, splice them into "emotion-text" blocks , and write them into a temporary zero-loss frame storage for real-time consumption by the downstream evidence chain module. The entire slicing process is completed within 300ms, ensuring zero-loss frames for subsequent forensics, quality inspection and quality inspection review.

[0070] S4, when a negative emotion mutation event occurs, audio and video clips corresponding to the time period are intercepted from the audio and video stream, and text data are aligned, sensitive entities in the text data are identified using a named entity recognition model and a regular expression engine, the identified sensitive entities are hashed into 8-bit reversible tokens and the original text is replaced to generate a desensitized evidence package.

[0071] In the embodiment, the named entity recognition model and the regular expression engine are used to identify sensitive entities in the text data, which specifically includes:

[0072] The text data are input in parallel to the named entity recognition model and the regular expression engine. The named entity recognition model is based on a large language model fine-tuned on a psychological counseling corpus, and identifies and labels text segments belonging to four types of entities, namely, name, address, institution and mobile phone number. The regular expression engine synchronously runs pre-configured regular rules to identify and capture formatted sensitive information strings including ID number, bank card number, QQ number and WeChat number.

[0073] The recognition results of the named entity recognition model and the regular expression engine are combined to form a complete sensitive entity list, and the position information of each entity in the text data is recorded.

[0074] In this embodiment, the identified sensitive entities are hashed into 8-bit reversible tokens and replaced in the original text to generate a desensitized evidence package, specifically including:

[0075] The original text of each identified sensitive entity is calculated by the HMAC-SHA256 keyed hash message authentication code algorithm to generate a 32-byte hash value, the 32-byte hash value is converted into a hexadecimal string, and the first 8 characters are taken as the core summary of the entity. The core summary and the corresponding entity type identifier are spliced to generate a standard format reversible token;

[0076] In the same session, the same entity that appears repeatedly is forced to reuse the same reversible token to maintain context consistency, the generated all reversible tokens are used to replace the corresponding sensitive entity original text in the original text, and a completely desensitized text copy is generated. The desensitized text copy, the corresponding audio and video slices, and the emotion data are packaged into a desensitized evidence package, and the original text and the token mapping table are separately encrypted and stored.

[0077] Specifically, in the psychological counseling scene, the original audio and video and text contain a large amount of sensitive information (such as customer name, mobile phone number, address, bank card number, etc.), and the sensitive entities are desensitized in real time to ensure that the "data is available but not visible". While meeting the business needs, the privacy risk is reduced to a safe level of "auditable, recoverable, and irreversible", ensuring that the quality inspector can see the desensitized evidence package in time, and the customer's privacy is completely exposed.

[0078] It should be noted that the present application adopts "NER+regular" dual-engine desensitization:

[0079] (1) Entity recognition:

[0080] NER model: based on a large model, fine-tuned on call corpus, supporting four types of entities: "name, mobile phone number, address, and institution"; Regular supplement: 46 high-precision regular expressions for ID cards, bank cards, QQ, WeChat, etc.

[0081] (2) Hash mapping:

[0082] The original text of the identified sensitive entity is converted into a fixed 32-byte "fingerprint" through the HMAC-SHA256 algorithm, and only the first 8 characters are left when the result is converted into hexadecimal, generating an 8-bit hexadecimal summary, and then splicing the reversible code template, such as @Name_8a3f@, @Phone_3e7b@. The same entity in the same call context is forced to reuse the same code to ensure context consistency and facilitate subsequent manual tracing by quality inspectors.

[0083] (3) Copy separation:

[0084] After desensitization, the text T anon is packaged as Evidence Package (desensitization evidence package) together with VidClip (video clip) and E clip (emotion clip); the original text T raw and the entity mapping table are encrypted by AES-256-GCM, and the key is uniformly managed by the server and stored in the desensitization evidence copy storage; the evidence chain circulation only uses the desensitized copy, and any third-party quality inspection, modeling and auditing links cannot access the plaintext privacy. Among them, the entity mapping table includes 50 entities and their corresponding mapped codes, and the 50 entities include 4 entity classes identified by large models and 46 entity classes identified by high-precision regular identification, and the 4 entity classes identified by large models include name, mobile phone, address and institution.

[0085] (4) Reversible recovery:

[0086] When and only when the court investigates or the client authorizes, the system administrator decrypts the entity mapping table through double-factor + sharding key, and the original text can be reconstructed within 30s, meeting the requirements of compliance audit and judicial evidence.

[0087] S5, based on the desensitization evidence package, in the negative emotion time period, the counselor response text is extracted and input into the trained three-dimensional supervision scoring model together with the dialogue context, and the emotion reflection dimension score, the cognitive reconstruction dimension score and the safety evaluation dimension score are output, and when any dimension score does not reach the safety threshold, the demonstration correction sentence is generated and pushed.

[0088] In this embodiment, the training process of the three-dimensional supervision scoring model specifically includes:

[0089] A training data set is constructed, which contains a large number of psychological counseling dialogue samples, each sample is composed of a visitor statement, a counselor response statement and a dialogue context, and is independently labeled by multiple experienced supervisors according to three dimensions of emotion reflection, cognitive reconstruction and safety evaluation, and the labeling result of each dimension is a binary label;

[0090] A three-dimensional supervision scoring model based on a large language model is constructed, and a prompt word engineering is used to solidify the psychological counseling supervision rules into a model input template;

[0091] The three-dimensional supervision scoring model is supervised and trained using the training data set, and the model parameters are optimized by minimizing the cross-entropy loss between the prediction results of the model in the three dimensions and the supervisor's labeling results;

[0092] In the training process, the training data set is divided into training set, validation set and test set according to the ratio of 7:2:1;

[0093] When the accuracy of the model on the validation set reaches the preset threshold consistent with the annotations of the supervisor, the training is stopped, and a trained three-dimensional supervision scoring model is obtained.

[0094] In this embodiment, it also includes:

[0095] S6, after the end of the consultation session, a double-track timeline visualization chart is generated according to the emotion value sequence and the counselor response score sequence, an overall evaluation and improvement suggestion are generated based on the counselor response score of the whole session using the trained overall supervision scoring model, and a structured supervision report is generated by integrating the double-track timeline visualization chart, the improvement suggestion and the overall evaluation.

[0096] In this embodiment, the training process of the overall supervision scoring model specifically includes:

[0097] An overall training data set is constructed, which contains a large number of complete psychological counseling session data samples, each sample including an emotion value sequence, a counselor response score sequence and a corresponding double-track timeline visualization chart for the whole session, and the overall performance of the counselor is evaluated and labeled by multiple experienced supervisors based on the sequence and chart;

[0098] An overall supervision scoring model based on a large language model is constructed, and psychological counseling supervision rules are solidified into a model input format using a structured prompt template;

[0099] The overall supervision scoring model is supervised trained using the overall training data set, and the model parameters are optimized by minimizing the difference between the comprehensive evaluation results output by the model and the annotations of the supervisors;

[0100] In the training process, the overall training data set is divided into a training subset, a validation subset and a test subset according to a preset proportion, and when the consistency of the comprehensive evaluation results of the model on the validation subset with the annotations of the supervisors reaches a predetermined standard, the training is stopped, and a trained overall supervision scoring model is obtained.

[0101] Embodiment 1:

[0102] Taking a negative emotion mutation segment in a real psychological counseling call as an example, a complete demonstration of how a multi-modal seven-dimensional emotion recognition, negative emotion desensitization record and three-dimensional supervision intelligent agent complete second-level recognition, privacy evidence collection and instant supervision is performed.

[0103] In this embodiment, the latest 20s video stream is received at 5:35, window number W_755. The audio-visual-text three-modal frame-level features are sent to the multi-modal large model, and the output main emotion value m_W755=“sad”, corresponding to the lighthouse color “lake blue”. The color binary tuple (5:35, lake blue) is written into the cache and a smooth curve is drawn, and the curve color is changed from “emerald green” of the last window to “lake blue”, triggering the “negative warning” mark.

[0104] At this time, the negative warning signal as an input event, the emotion value is detected to change from 0→-1 (neutral→sad) at frame t_neg=5:35:17.324UTC. Immediately start recording the original video V_raw, audio A_raw, generate VidClip755 and AudClip755; At the same time, extract the segment ASR text T_clip=“I… really can't hold on anymore” and the corresponding region emotion m_clip=[-3,-3], zero loss frame saving.

[0105] NER+regular dual engine identifies entity “xxxxxxxxxxx”→@Phone_9e4b@ in T_clip; HMAC--SHA256 generates an 8-bit code, the same entity reuses the same code to ensure context consistency; After desensitization, the text T_anon and E_clip are packaged into Evidence_Package755; The original text and entity mapping table are encrypted by AES-256-GCM, and the key fragments are hosted to the location of the desensitized evidence copy storage. Only the desensitized copy is seen downstream of the evidence chain, with zero privacy exposure.

[0106] Evidence_Package755 is pushed to the supervisor intelligent agent Coach-Agent. The three-dimensional scoring prompt Prompt-Scorer scores the following text as the dialogue content:

[0107] Visitor: “I… really can't hold on anymore.”;

[0108] Consultant: “Don't be sad, think positively.”

[0109] Output: Emotional reflection | 0 | unnamed sad emotion | “I feel your deep despair at this moment, can you say a little more?”; Cognitive restructuring | 0 | no challenge to catastrophic belief | “What do you think is the most worried behind 'can't hold on'? We can develop a safety plan together.”; Safety assessment | 0 | no self-harm risk screening | “Have you thought about hurting yourself? We can develop a safety plan together.”; Three-dimensional average 0, immediate score S=0, no safety threshold reached, background storage of related evidence chain and sentence-by-sentence coaching suggestions.

[0110] The 50min conversation ends, and the system splices the dual-track timeline:

[0111] Track--A: emotion sequence {t, m_t};

[0112] Track--B: counselor reply score sequence {t, S_t};

[0113] Generate a visual line chart aligned by absolute timestamps, output overall evaluation:

[0114] Emotion reflection | partially in place | still 3 times of un-named sadness | "use 'it sounds like you are very sad' to empathize first";

[0115] Cognitive restructuring | weak | not challenging core catastrophic beliefs | "ask 'is the worst outcome really 100% happening?'";

[0116] Safety assessment | qualified | complete risk check throughout the process | "continue to maintain assessment + safety plan";

[0117] One-key export PDF supervision report, including line chart, improvement strategy list, overall evaluation, complete "early warning - record - review" full-link closed loop.

[0118] As Fig. 2 shown, the emotion changes from "happy - sad - angry - sad - neutral", and the counselor's average score in the negative emotion interval is concentrated at 0, so that a red line is formed in [5min, 37min], and some scores are scattered in (-1 / 3, -2 / 3, -1) values, indicating that most of the counselor's replies meet the rules, and a small part of the replies have errors that violate the principles, which need to be further optimized according to the suggestions of the supervision report.

[0119] Referring to Figs. 1-3 , the present application provides a kind of psychological counseling automatic supervision system, comprising:

[0120] Multi-modal seven-dimensional emotion recognition module, for obtaining the real-time audio and video stream and automatic speech recognition text data of counseling dialogue, input audio and video stream and text data into trained multi-modal emotion recognition model, output seven-dimensional emotion vector and main emotion value in preset time sliding window unit;

[0121] Negative emotion desensitization recording module, for detecting negative emotion mutation event and positive emotion mutation event according to the change of main emotion value, when negative emotion mutation event occurs, audio and video slice corresponding to the time period is intercepted from audio and video stream and aligned text data, sensitive entity in text data is identified using named entity recognition model and regular expression engine, the identified sensitive entity is hashed mapped to 8-bit reversible token and replaces original text to generate desensitization evidence package;

[0122] The three-dimensional supervisor intelligent agent module is configured to extract the counselor response text and input the counselor response text and the dialogue context into the trained three-dimensional supervisor scoring model based on the desensitization evidence package during the negative emotion time period, and output emotion reflection dimension scores, cognitive reconstruction dimension scores and safety evaluation dimension scores, and generate and push a demonstration correction sentence in real time when any dimension score fails to reach a safety threshold.

[0123] In the embodiment, the multi-modal seven-dimensional emotion recognition module further includes a color mapping unit configured to map the main emotion value to a preset RGB color value, and display a negative warning mark and an intervention success mark in the emotion timing diagram.

[0124] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can make equivalent replacements or changes within the technical range disclosed by the present application according to the technical solution and the inventive concept of the present application, which should be covered within the protection scope of the present application.

Claims

1. A method of automatically supervising psychological counseling, characterized by, The method comprises the following steps: S1, obtaining real-time audio and video stream and automatic speech recognition text data of a counseling session; S2, inputting the audio and video stream and text data into a trained multi-modal emotion recognition model to output a seven-dimensional emotion vector and a main emotion value in a preset time sliding window unit; S3, detecting negative and positive emotion mutation events according to the change of the main emotion value; S4, when a negative emotion mutation event occurs, audio and video slices corresponding to the time period are intercepted from the audio and video stream, and the text data is aligned, a named entity recognition model and a regular expression engine are used to identify sensitive entities in the text data, the identified sensitive entities are hashed into 8-bit reversible tokens, and the original text is replaced to generate a desensitization evidence package; S5, based on the desensitization evidence package, in the negative emotion time period, the counselor's response text is extracted and input into a trained three-dimensional supervision scoring model together with the context of the session, and the emotion reflection dimension score, the cognitive restructuring dimension score and the safety assessment dimension score are output, and when any dimension score does not reach the safety threshold, a demonstration correction sentence is generated and pushed in real time; The identified sensitive entities are hashed into 8-bit reversible tokens and the original text is replaced to generate a desensitization evidence package, specifically comprising: The original text of each identified sensitive entity is calculated by a keyed HMAC-SHA256 hash message authentication code algorithm to generate a 32-byte hash value, the 32-byte hash value is converted into a hexadecimal string, and the first 8 characters are taken as the core summary of the entity. The core summary and the corresponding entity type identifier are spliced to generate a standard format reversible token; In the same session, the same reversible token is forced to be reused for the same entity to maintain context consistency, all generated reversible tokens are used to replace the corresponding sensitive entity original text in the original text to generate a completely desensitized text copy, and the desensitized text copy, corresponding audio and video slices and emotion data are packaged into a desensitization evidence package. The original text and the token mapping table are stored separately and encrypted.

2. The method of claim 1, wherein, The use of a named entity recognition model and a regular expression engine to identify sensitive entities in the text data specifically includes: The text data is input into the named entity recognition model and the regular expression engine in parallel, the named entity recognition model is based on a large language model fine-tuned on a psychological counseling corpus, and identifies and labels text segments belonging to four types of entities, namely names, addresses, institutions and mobile phone numbers. The regular expression engine runs a pre-configured regular rule in synchronization to identify and capture formatted sensitive information strings including ID numbers, bank card numbers, QQ numbers and WeChat numbers; The identification results of the named entity recognition model and the regular expression engine are merged to form a complete sensitive entity list, and the position information of each entity in the text data is recorded.

3. The method of claim 1, wherein the method further comprises: Further comprising: S6. After the consultation session ends, a dual-track timeline visualization chart is generated based on the emotion value sequence and the counselor response score sequence. Based on the counselor response score of the entire session, a trained overall supervision scoring model is used to generate an overall evaluation and improvement suggestions. The dual-track timeline visualization chart, improvement suggestions and overall evaluation are integrated to generate a structured supervision report.

4. The method of claim 1, wherein the method further comprises: The seven-dimensional emotion vector includes seven dimensions: anger, happiness, sadness, fear, surprise, disgust, and neutrality. The dominant emotion value is determined by selecting the dimension with the largest value among the seven-dimensional emotion vectors. The training process of the multimodal emotion recognition model specifically includes: Obtain a training dataset containing a large number of labeled multimodal sample data. Each sample includes synchronized audio and video segments and their corresponding text transcriptions, and is labeled with the ground truth value of a seven-dimensional sentiment vector. A multimodal emotion recognition model is constructed, which includes an audio feature extraction network, a video feature extraction network, a text feature extraction network, and a multimodal feature fusion module; The multimodal emotion recognition model is trained under supervision using the training dataset, and the model parameters are optimized by minimizing the cross-entropy loss function between the seven-dimensional emotion vector output by the model and the ground truth. During the training process, the training dataset is divided into a training set, a validation set, and a test set according to a preset ratio. When the performance of the model on the validation set reaches a preset standard, training is stopped, and the trained multimodal emotion recognition model is obtained.

5. The method of claim 1, wherein the method further comprises: Based on the changes in the primary emotion value, negative and positive emotion abrupt events are detected, specifically including: The negative emotion mutation event refers to the change of the main emotion value from positive to negative, where positive values ​​correspond to happy and neutral emotions, and negative values ​​correspond to surprise, disgust, sadness, fear, and anger; the positive emotion mutation event refers to the change of the main emotion value from negative to positive, where positive values ​​correspond to happy and neutral emotions, and negative values ​​correspond to surprise, disgust, sadness, fear, and anger.

6. The method of claim 1, wherein, The training process of the three-dimensional supervision and scoring model specifically includes: A training dataset was constructed, which contains a large number of psychological counseling dialogue samples. Each sample consists of the client's statements, the counselor's response statements, and the dialogue context. Multiple senior supervisors independently labeled the samples according to three dimensions: emotional response, cognitive reconstruction, and safety assessment. The labeling results for each dimension are binary labels. A three-dimensional supervision scoring model based on a large language model was constructed, and the rules of psychological counseling supervision were solidified into model input templates using prompt word engineering; The three-dimensional supervisory scoring model is trained under supervision using the training dataset, and the model parameters are optimized by minimizing the cross-entropy loss between the model's prediction results and the supervisor's annotation results in the three dimensions. During the training process, the training dataset is divided into a training set, a validation set, and a test set in a ratio of 7:2:

1. Training stops when the model's accuracy on the validation set reaches a preset threshold consistent with the supervisor's annotations, thus obtaining the trained 3D supervisory scoring model.

7. The method of claim 3, wherein the method further comprises: The training process of the overall supervision and scoring model specifically includes: constructing a total training dataset containing a large number of complete psychological counseling session data samples, each sample including an emotion value sequence of the whole session, a counselor response score sequence, and a corresponding double-track time axis visualization chart, and being annotated with a comprehensive evaluation of the counselor's overall performance based on the sequences and charts by multiple experienced supervisors; constructing a total supervision scoring model based on a large language model, and using a structured prompt template to solidify psychological counseling supervision rules into a model input format; supervised training of the total supervision scoring model using the total training dataset, and optimizing model parameters by minimizing the difference between the comprehensive evaluation results output by the model and the supervisor's annotations; during training, dividing the total training dataset into a training subset, a validation subset, and a test subset according to a predetermined ratio, stopping training when the consistency of the model's comprehensive evaluation results on the validation subset with the supervisor's annotations reaches a predetermined standard, and obtaining the trained total supervision scoring model.

8. A system for automatic supervision of psychological counseling, characterized in that, comprise: a multi-modal seven-dimensional emotion recognition module for obtaining real-time audio and video streams and automatic speech recognition text data of a counseling dialogue, inputting the audio and video streams and text data into a trained multi-modal emotion recognition model, and outputting a seven-dimensional emotion vector and a primary emotion value in a predetermined time sliding window unit; a negative emotion desensitization recording module for detecting negative emotion mutation events and positive emotion mutation events according to changes in the primary emotion value, and when a negative emotion mutation event occurs, extracting audio and video slices of the corresponding time period from the audio and video streams and aligning the text data, identifying sensitive entities in the text data using a named entity recognition model and a regular expression engine, hashing the identified sensitive entities into 8-bit reversible tokens, and replacing the original text to generate a desensitization evidence package; a three-dimensional supervision intelligent agent module for extracting counselor response text based on the desensitization evidence package and inputting the text and the dialogue context into a trained three-dimensional supervision scoring model to output emotion reflection dimension scores, cognitive restructuring dimension scores, and safety assessment dimension scores, and generating and pushing demonstration correction sentences in real time when any dimension score does not reach a safety threshold; the hashing of the identified sensitive entities into 8-bit reversible tokens and the replacement of the original text to generate a desensitization evidence package specifically comprises: calculating a 32-byte hash value for each identified sensitive entity original text using a keyed HMAC--SHA256 hash message authentication code algorithm, converting the 32-byte hash value into a hexadecimal string, and taking the first 8 characters as the core summary of the entity, and concatenating the core summary with the corresponding entity type identifier to generate a standard format reversible token; in the same session, the same reversible token is forced to be reused for the same entity to maintain context consistency, all generated reversible tokens are used to replace the corresponding sensitive entity original text in the original text to generate a completely desensitized text copy, the desensitized text copy, the corresponding audio and video slices, and the emotion data are packaged into a desensitization evidence package, and the original text and the token mapping table are stored separately and encrypted.

9. The automated supervision system for psychological counseling according to claim 8, wherein The multi-modal seven-dimensional emotion recognition module further comprises a color mapping unit configured to map the primary emotion value into a preset RGB color value, and display a negative early warning mark and an intervention success mark in the emotion time sequence diagram.

Images