AI-assisted psychological assessment method based on conversation and electronic equipment

Abstract

The invention discloses a conversation-based AI auxiliary psychological assessment method and an electronic device, and aims at feature extraction analysis and assessment of a user in combination with cooperation of a device end and a cloud end, and the method specifically comprises the following steps: firstly, the device end extracts lightweight features in a conversation process in real time; the lightweight features comprise sound features extracted from the conversation sound signals and content features extracted from the conversation content signals; the cloud end receives the lightweight features transmitted by the equipment end, cross-round dynamic tracking and multi-modal fusion analysis are performed on the lightweight features through a deep analysis model, a quantitative psychological assessment result is output, and the multi-modal fusion analysis of the deep analysis model is realized based on a preset psychological dimension-feature mapping matrix; the cloud side generates feedback information according to the psychological assessment result, the feedback information comprises conventional guide information or risk intervention information, the feedback information is transmitted to the equipment side, and the feedback information is output by the equipment side. According to the invention, information can be effectively obtained through a conversation mechanism for psychological assessment and analysis.

Description

Technical Field

[0001] This invention belongs to the field of information and data processing technology, specifically relating to an AI-assisted psychological assessment method and electronic device based on conversation. Background Technology

[0002] The fast pace of modern life, coupled with heavy workloads, family responsibilities, and social pressures, leads to widespread hidden psychological stress among individuals. Some also experience anxiety, sleep-related disturbances, and other potential mental health issues. These problems often lurk beneath the surface of daily life, without obvious external symptoms. While individuals have a need to understand their own mental health, there is a significant barrier to seeking professional psychological screening or counseling. On one hand, visiting a professional mental health institution requires additional time investment, and the appointment and communication processes are relatively cumbersome. On the other hand, some people are influenced by the societal perception that mental health issues equate to abnormality, fearing negative stigma associated with professional psychological assessments. This leads to resistance to proactively seeking professional help, ultimately discouraging them from understanding their own mental state. Consequently, hidden psychological problems remain undetected and may gradually develop into more serious psychological distress.

[0003] To lower the barrier to entry for psychological assessments, lightweight psychological assessment solutions relying on apps or portable electronic devices have emerged in the current technology landscape. For example, some apps infer psychological states by collecting user sleep data, step counts, or simple text diaries, while some devices attempt to use voice emotion recognition to assist in judging emotional tendencies. However, these solutions have significant limitations: First, their ability to integrate multi-source data is weak, mostly relying on a single type of data (such as judging emotions solely by sleep duration or identifying negative emotions solely by text keywords). They fail to achieve collaborative analysis of voice features and content features in conversational scenarios, and cannot capture key correlation signals such as sudden changes in speech rate and negative vocabulary, prolonged pauses and self-deprecating expressions, resulting in a single analytical dimension. Second, the user guidance mechanism is rigid, with guidance content mostly consisting of pre-set fixed questions, such as "Are you happy today?" or "Have you been stressed lately?" It lacks the ability to dynamically adapt to real-time dialogue content, cannot simulate empathetic guidance in natural interpersonal interactions, and is difficult to stimulate users' willingness to express themselves. This leads to insufficient quality of collected psychological data, and the final analysis results are superficial, lack specificity, and cannot accurately reflect the individual's true psychological state. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides an AI-assisted psychological assessment method and electronic device based on conversation, aiming to form a system and method for analyzing psychological conditions based on conversation-based information acquisition through multimodal fusion with weak perception.

[0005] The technical solution adopted in this invention is as follows: Firstly, this invention provides an AI-assisted psychological assessment method based on conversation, which combines on-device and cloud-based approaches for feature extraction, analysis, and assessment of users, as detailed below: S100: First, the device extracts lightweight features during the conversation in real time. Lightweight features include sound features extracted from the conversation sound signal and content features extracted from the conversation content signal. S200 then receives lightweight features transmitted from the device via the cloud, and performs cross-round dynamic tracking and multimodal fusion analysis on the lightweight features through a deep analysis model, outputting quantitative psychological assessment results. The multimodal fusion analysis of the deep analysis model is based on a preset psychological dimension-feature mapping matrix. The S300 and cloud-based system generate feedback information based on the psychological assessment results. The feedback information includes routine guidance information or risk intervention information, and is transmitted to the device for output.

[0006] In conjunction with the first aspect, the present invention provides a first implementation of the first aspect, wherein in step S100, the content features include text data after ASR transcription processing, surface sentiment tags corresponding to the text data, and demand keywords in the text data; The preprocessing of the content features includes: The ASR-transcribed text data is denoised to remove meaningless interjections and repeated characters; the denoised text data is matched based on a preset emotion lexicon to determine the surface emotion tags corresponding to the text data; and the required keywords are extracted from the denoised text data based on a preset demand lexicon.

[0007] The voice features include speech rate, fundamental frequency of pitch, and duration of continuous silence; the preprocessing of the voice features includes: The conversational audio signal is segmented into preset time frames, and the number of effective syllables in each time frame within a single turn of dialogue is calculated. The ratio of the number of effective syllables in a single turn of dialogue to the dialogue duration is used as the speech rate. The fundamental frequency data of each time frame within a single turn of dialogue is extracted, and the standard deviation of the fundamental frequency data is calculated as the pitch fundamental frequency. The time period in a single turn of dialogue where there is no effective audio signal is detected, and the time period with a duration greater than 2 seconds is recorded as the continuous silence duration.

[0008] In conjunction with the first aspect, the present invention provides a second embodiment of the first aspect, wherein in step S200, the deep analysis model includes a dynamic tracking module and a multimodal fusion module; The dynamic tracking module uses a long short-term memory network. The dynamic tracking module receives cross-round lightweight features transmitted from the device and tracks the changing trends of the cross-round lightweight features. The changing trends include the amplitude of speech rate fluctuations, the frequency of surface emotion label conversions, and the frequency of occurrence of demand keywords. Among them, the speech rate fluctuation range is the percentage of the difference between the speech rate of the current round and the speech rate of the previous round relative to the speech rate of the previous round; the surface emotion label switching frequency is the number of times the surface emotion label changes per unit time. The multimodal fusion module receives the cross-cycle lightweight feature change trend output by the dynamic tracking module, and performs fusion calculation on the voice features and content features based on the psychological dimension-feature mapping matrix, outputting psychological assessment results including at least crisis risk value, mental health value, behavioral tendency value, and personality trait value.

[0009] In conjunction with the second embodiment of the first aspect, the present invention provides a third embodiment of the first aspect, wherein the process of obtaining the psychological dimension-feature mapping matrix includes: First, an initial mapping matrix is ​​constructed. In the initial mapping matrix, each psychological dimension corresponds to at least one voice feature and at least one content feature, and an initial weight is assigned to each corresponding feature. The psychological dimensions include crisis risk, mental health, behavioral tendencies, and personality traits. Obtain a labeled dataset, which includes multiple sets of conversation samples and corresponding psychological dimension labeling values ​​for each set of conversation samples. Each set of conversation samples contains corresponding voice feature data and content feature data. The sound feature data and content feature data in the labeled dataset are input into the initial mapping matrix. The feature weights in the initial mapping matrix are iteratively trained using the backpropagation algorithm to obtain the trained mapping matrix. The output of the training mapping matrix is ​​verified by psychology professionals based on the evaluation criteria. If the deviation between the output and the evaluation criteria is not less than the preset deviation threshold, the feature weights are adjusted and the third step is repeated until the deviation between the output and the evaluation criteria is less than the preset deviation threshold, thus obtaining the final psychological dimension-feature mapping matrix.

[0010] In conjunction with the second embodiment of the first aspect, the present invention provides a fourth embodiment of the first aspect, wherein when the multimodal fusion module performs fusion calculation, it further includes a feature association judgment step, the feature association judgment step including: A time window threshold T is set, and sound features and content features extracted at different times during the same conversation are obtained. The time difference Δt between the sound features and content features is calculated. If Δt is not greater than T, the sound features and content features are determined to be directly related, and the multimodal fusion module directly performs fusion calculation on the sound features and content features based on the feature weights in the psychological dimension-feature mapping matrix. If Δt is greater than T, the sound features and content features are determined to be indirectly related. The multimodal fusion module first uses the topic matching unit to determine whether the conversation content corresponding to the sound features and the conversation content corresponding to the content features belong to the same topic. If they belong to the same topic, the weights of the sound features are adjusted based on a preset time decay formula, and then the fusion calculation is performed in combination with the content feature weights in the psychological dimension-feature mapping matrix.

[0011] In conjunction with the fourth implementation of the first aspect, the present invention provides a fifth implementation of the first aspect, wherein the topic matching unit has a built-in topic tag library, the topic tag library contains at least four core topics and a set of keywords corresponding to each core topic; the topic matching unit matches the conversation text data corresponding to the sound features and the text data corresponding to the content features with the set of keywords in the topic tag library respectively; if the number of keywords matching the same core topic is greater than a preset threshold, then the two are determined to belong to the same topic. The preset time decay formula is: in For the adjusted sound feature weights, The initial weights for this sound feature in the psychological dimension-feature mapping matrix. Δt The time difference between sound features and content features. τ For the preset attenuation coefficient, τ The value range is 30-60 seconds.

[0012] In conjunction with the first aspect, the present invention provides a sixth embodiment of the first aspect. In step S100, the device further includes a guidance generation module. The guidance generation module determines whether the amount of data currently provided by the user has reached a preset basic data amount based on the extracted lightweight features. If the preset basic data amount has not been reached, the guidance generation module generates guidance text containing self-narrative scene content based on the keywords in the currently extracted content features, and outputs the guidance text through the device. The self-narrative scene content is a lifelike scene description content corresponding to the keywords. The preset basic data amount is at least three rounds of effective dialogue, and each round of effective dialogue must satisfy that the number of effective words in the text data is greater than five and there is a clear surface emotion label.

[0013] In conjunction with the sixth embodiment of the first aspect, the present invention provides a seventh embodiment of the first aspect, wherein the process of the guide generation module generating guide text includes: Extract core keywords from the current content features. The core keywords are words that appear more than twice in the text data and belong to a preset core vocabulary. Based on the core keywords, call a preset scene template library, which contains scene description frameworks corresponding to different core keywords. Fill the scene description framework with content to generate a self-narrative scene content with a first-person perspective, forming guiding text. The generation delay of the guiding text does not exceed 0.8 seconds, and the language style of the guiding text conforms to the norms of colloquial expression, with the sentence length controlled within the range of 8-15 Chinese characters.

[0014] Secondly, the present invention provides an electronic device for implementing the AI-assisted psychological assessment method based on conversation as described in any of the above claims. The electronic device includes a device terminal and a cloud terminal. The device terminal establishes a data interaction connection with the cloud terminal. The device terminal is configured with a functional module for collecting sound signals and content signals during conversation and outputting feedback information. The cloud terminal is configured with a functional module for receiving data transmitted by the device terminal and performing in-depth analysis to generate quantitative psychological assessment results.

[0015] The beneficial effects of this invention are as follows: (1) This invention breaks through the limitations of specialized psychological assessment scenarios. Psychological state assessment can be completed through daily conversation, effectively reducing the threshold for individual participation. Users do not need to actively go to professional institutions or cooperate with specific screening processes. During natural conversation, the device collects sound and content characteristics in real time, which avoids the deliberate feeling of psychological assessment and eliminates users' resistance to negative labels, making the early detection of hidden psychological problems more feasible. (2) This invention realizes the deep fusion analysis of multi-source features in conversation scenarios, solving the problems of single analysis dimension and one-sided results in existing technologies. By clarifying the specific types and preprocessing rules of voice features and content features, and combining the cloud-based deep analysis model and psychological dimension-feature mapping matrix, the two types of features are calculated according to logical association. Key association signals such as tone change, self-harm words, speech rate fluctuation, and negative word density can be captured. The output quantitative psychological assessment results are more in line with the individual's true psychological state, and the accuracy of analysis is significantly improved. (3) This invention solves the problems of rigid guidance and low user willingness to express themselves in the existing technology by using dynamic guidance and virtual object interaction mechanism. The device-side guidance generation module dynamically generates empathic self-narration guidance text based on real-time extracted high-frequency core words and emotion tags. The virtual object generation module combines user information to build personalized interactive objects, simulates timely interruption and topic deepening in a friend-like manner, effectively stimulates users' willingness to share, ensures the integrity and quality of psychological data, and provides data support for accurate analysis. (4) This invention relies on the end-to-cloud collaborative architecture, which ensures the depth of analysis while taking into account real-time performance and privacy security. The device only handles lightweight feature extraction and guidance text generation, while the core in-depth analysis is completed by the cloud. This avoids the delay problem caused by insufficient computing power on the device and reduces the risk of privacy leakage because only lightweight features are transmitted. At the same time, through the dual-path feedback of risk intervention information in the cloud, it ensures that high-risk psychological states can be triggered in a timely manner, thereby improving the practical application value of psychological assessment. Detailed Implementation

[0016] The present invention will be further explained below with reference to specific embodiments.

[0017] Example 1: This embodiment discloses an AI-assisted psychological assessment method based on conversation. The method relies on an assessment system, the hardware of which mainly includes the device and the cloud. It can use existing common mobile terminal device systems, in the form of a mini program or a standalone APP, to communicate and connect with the backend server as the cloud network, which can implement the method in a low-cost manner.

[0018] Specifically, the device uses a smartphone and is equipped with a sound acquisition module, a content processing module, a lightweight feature extraction module, and a feedback output module. Its core function is to acquire sound and content signals during conversations in real time, extract lightweight features, and output information from the cloud.

[0019] The cloud-based system features a deep analysis model and a pre-defined psychological dimension-feature mapping matrix. Its core function is to receive lightweight features transmitted from the device, perform cross-round dynamic tracking and multimodal fusion analysis, and generate quantitative psychological assessment results and corresponding feedback information.

[0020] The evaluation method for the above system in this embodiment is as follows: Step 1: Real-time extraction of lightweight features during the conversation on the device. Users can activate the AI-powered psychological healing function on their smartphones and engage in natural conversations with the system.

[0021] Then, the device's sound acquisition module captures the user's conversational audio signal in real time, and the lightweight feature extraction module preprocesses the audio signal. The signal is segmented at 20ms / frame, and the ratio of the number of effective syllables in a single turn of dialogue to the dialogue duration is calculated to obtain the speech rate.

[0022] Extract the fundamental frequency data of each frame and calculate the standard deviation to obtain the pitch fundamental frequency; detect the time period of continuous silence and mark the duration of continuous silence. The above data constitute the sound features.

[0023] The device-side content processing module uses an ASR transcription unit to transcribe the user's speech into text data in real time. After the text data is denoised, the lightweight feature extraction module matches the surface emotion tag of depression based on a preset emotion lexicon and extracts demand keywords such as overtime and lack of energy based on a preset demand lexicon. The above data constitutes the content features. The device integrates the extracted sound features and content features into lightweight feature data and transmits it to the cloud at a frequency of 500ms / time.

[0024] Step 2: Receive lightweight features in the cloud and perform deep analysis to output quantitative psychological assessment results. The cloud receives lightweight feature data across rounds transmitted from the device. The dynamic tracking module (using an LSTM network in this embodiment) processes the data: calculates the fluctuation range of the user's speech rate in each round of dialogue, the frequency of surface emotion label conversion, and the frequency of occurrence of demand keywords, thus completing the tracking of feature change trends across rounds.

[0025] The multimodal fusion module calls a preset psychological dimension-feature mapping matrix and matches the dynamically tracked feature change trends with the weight rules in the matrix. Taking the mental health dimension as an example, it combines the fundamental frequency of 12Hz and the density of negative words of 0.3 for fusion calculation, while also referring to the calculation results of speech rate fluctuation amplitude of 40.6% and missing future tense sentences under the behavioral tendency dimension, and finally outputs a quantitative psychological assessment result.

[0026] For example, this user's mental health score is 0.62, behavioral tendency score is 0.58, and crisis risk score is 0.35, where the crisis risk score is in the range of 0-1.

[0027] Step S3: Generate feedback information in the cloud and output it through the device. Based on the quantitative psychological assessment results, the cloud platform determined that the crisis risk value was 0.35 < 0.85, indicating no high risk. It then generated standard guiding messages, such as, "From the chat, I can sense you've been a bit tired due to working overtime lately. Besides work, are there any small things that help you relax a little?"

[0028] The cloud transmits the above-mentioned conventional guiding information to the device side. The device side displays text information through the display screen and plays voice information through the speaker to complete the feedback output. Subsequently, steps 1 to 3 can be repeated based on the user's response to continuously perform psychological evaluation and dynamic guidance.

[0029] Based on the foregoing embodiments of the user's conversation evaluation with the system through a smart phone, in order to further improve the accuracy and reliability of lightweight feature extraction, the following preferred limitations are made on the preprocessing process of content features and voice features to ensure that the extracted features are more in line with the clinical needs of psychological evaluation and provide higher-quality basic data for in-depth analysis in the cloud.

[0030] Further preferred implementation of content feature preprocessing On the basis of the original embodiments of ASR transcription, denoising, emotion label matching, and requirement keyword extraction, by refining the processing rules, optimizing the lexicon design and matching logic, the feature error is reduced, as follows: Denoising processing of text data after ASR transcription For the common colloquial redundant expressions in the user's conversation, the denoising processing is further divided into two steps: redundant character filtering and context semantic verification. Among them, meaningless modal particles specifically include those, bar, ah, oh, ne when not used at the end of a sentence and when there is no actual semantic meaning, and repeated characters are defined as words that appear continuously twice or more and have no semantic progression.

[0031] Taking the user's conversation content as an example, the user mentions that I have been working overtime recently bar and it's really really tiring. After redundant character filtering, I have been working overtime recently and it's really tiring is initially obtained.

[0032] Subsequently, the lightweight semantic analysis unit on the device side is called. Based on the pruned BERT model, it is judged whether there is semantic breakage in the filtered text.

[0033] If the original sentence is I have been working overtime recently um and it's really tiring, after filtering um and it's really tiring, I have been working overtime recently is obtained and the semantics are complete, so it is retained; if the original sentence is I that it's really tiring, after filtering, the semantics are broken, so that is retained as a temporary connecting word and marked as to be verified, and will be further corrected in combination with the conversation context later to avoid losing key semantics due to excessive denoising.

[0034] After the above preferred processing, the final denoising result of the user's text data is I have been working overtime recently and it's really tiring, without redundancy and with complete semantics.

[0035] Determination of surface emotion labels The pre-defined emotion lexicon is further subdivided into a core emotion lexicon and a related emotion lexicon. A weighted voting mechanism is used to determine emotion tags, rather than single-keyword matching. The core emotion lexicon includes five basic emotions: low mood, anxiety, irritability, calmness, and positivity. Each emotion category corresponds to ten to fifteen core keywords. For example, low mood corresponds to tiredness, fatigue, lack of energy, and reluctance to move; anxiety corresponds to tension, worry, restlessness, and palpitations. The related emotion lexicon consists of extended scenario-based words for the core emotions. For instance, working late, poor sleep, and failing to complete tasks are related words for low mood, while approaching deadlines and exams are related words for anxiety.

[0036] During matching, the matching weight of keywords in the core emotion lexicon is set to 0.8, and the matching weight of keywords in the associated emotion lexicon is set to 0.5. The total weight of all matching keywords in the text is calculated, and the emotion with the highest total weight is the surface emotion label.

[0037] Taking the user's denoised text "I've been working late lately and I'm so tired" as an example, "tired" is the core keyword of "low mood" with a matching weight of 0.8, and "working late late" is a related keyword of "low mood" with a matching weight of 0.5. The total weight is 1.3, which is higher than the matching weight of other emotions. Finally, the surface emotion label is determined to be "low mood".

[0038] Extraction of demand keywords The pre-defined demand vocabulary is further divided into a basic demand vocabulary and a deep demand vocabulary. The basic demand vocabulary includes words related to direct needs such as rest, talking, help, and answering questions, while the deep demand vocabulary includes words related to indirect needs such as relieving stress, improving sleep, regulating emotions, and solving work problems.

[0039] During extraction, the denoised text is first matched with two types of required thesaurus to filter out preliminary matching words; then, the frequency of occurrence and semantic importance of the words in the text are combined, and words that appear more than once and support the core semantics of the text are identified as required keywords.

[0040] Taking the user's text as an example, "overtime" corresponds to work-related scenarios, and "tired" corresponds to a state of physical fatigue. After matching, "overtime" is associated with solving work-related problems in the deep needs vocabulary, and "tired" is associated with rest in the basic needs vocabulary. Both words are supporting words for the core semantics of the text. Finally, the key needs are determined to be "overtime" and "rest".

[0041] Further preferred implementation of sound feature preprocessing Based on the original implementation's time frame segmentation, speech rate calculation, pitch fundamental frequency extraction, and continuous silence duration detection, the accuracy of voice feature quantification is improved by clearly defining parameters and optimizing calculation logic, as follows: Calculation of speech rate First, it is clear that the preset time frame is 20 milliseconds per frame. The conversation voice signal is segmented into several frame data according to this time frame. Subsequently, calculate the number of effective syllables in each time frame within a single-round conversation. An effective syllable is defined as a syllable that is clearly recognizable and not corresponding to a filler word. Syllables corresponding to filler words such as "um", "ah", "oh", etc. are not included in the effective syllables. Count the total number of effective syllables in all time frames within a single-round conversation, and then divide it by the total duration of the single-round conversation to obtain the speech rate.

[0042] This embodiment is preferably adapted to the Chinese scenario. An effective syllable is defined as a syllable that conforms to the "Scheme for the Chinese Phonetic Alphabet", excluding syllables corresponding to meaningless filler words such as "um", "ah", "oh", "ne", etc. If adapting to other languages, it is necessary to adjust in advance based on the syllable division standard of the target language. A single-round conversation is defined as a continuous speech of the user and the silent duration after the end is not less than 2 seconds. The system response period is not included in the user's single-round conversation duration.

[0043] Taking the user's single-round conversation as an example, the total duration of the user's conversation in this round is 15 seconds. After segmentation at 20 milliseconds per frame, 750 frame data are obtained. The total number of effective syllables is counted as 48, and the speech rate is 48 divided by 15, which is 3.2 words per second.

[0044] Extraction of the fundamental frequency of intonation The autocorrelation method is used to extract the fundamental frequency data of each time frame within a single-round conversation. The calculation window of the autocorrelation method is set to 10 milliseconds to ensure the extraction accuracy of the fundamental frequency data. After extracting the fundamental frequency data of all time frames, calculate the standard deviation of this set of data, and the standard deviation result is the fundamental frequency of intonation.

[0045] Taking the user's single-round conversation as an example, the fundamental frequency data of each extracted time frame is distributed between 180 Hz and 220 Hz. The standard deviation of this set of data is calculated as 12 Hz, and finally, the fundamental frequency of intonation of this round of conversation is determined to be 12 Hz.

[0046] Detection of continuous silent duration First, set the threshold of the voice signal intensity. This threshold is 10% of the average voice signal intensity within a single-round conversation. The voice signal below this threshold is regarded as an ineffective voice signal. Real-time detect the time period of continuously being in the state of ineffective voice signal within a single-round conversation, and time this time period. If the timing result is greater than 2 seconds, then record this time period as the continuous silent duration; if the timing result is less than or equal to 2 seconds, it is not recorded as the continuous silent duration.

[0047] Taking the user's conversation process as an example, during a certain conversation of the user, there is a time period of ineffective voice signal once, and the timing result is 1.8 seconds, which is less than 2 seconds, so it is not recorded as the continuous silent duration; there is no time period of ineffective voice signal exceeding 2 seconds in the subsequent conversation. Therefore, there is no effective record of the continuous silent duration of this round of conversation.

[0048] To further clarify the working logic of the deep analysis model and the construction and application process of the psychological dimension-feature mapping matrix, and based on actual user conversation data, the following supplementary explanations are provided regarding relevant technical details: The process of obtaining the psychological dimension-feature mapping matrix Construction of the initial mapping matrix First, four core psychological dimensions are identified: crisis risk, mental health, behavioral tendencies, and personality traits. For each psychological dimension, corresponding voice and content features are matched, and initial weights are assigned.

[0049] Crisis risk dimension: The corresponding voice feature is the fundamental frequency of tone, with an initial weight of 0.7; the corresponding content feature is self-harm keywords, with an initial weight of 0.9. Psychological health dimension: The corresponding voice feature is speech rate, with an initial weight of 0.6; the corresponding content feature is negative word density, with an initial weight of 0.8. Behavioral tendency dimension: The corresponding voice feature is the duration of continuous silence, with an initial weight of 0.5; the corresponding content feature is the proportion of future tense sentences, with an initial weight of 0.6. Personality trait dimension: The corresponding voice feature is the range of intonation fluctuation, with an initial weight of 0.4; the corresponding content feature is the frequency of first-person use, with an initial weight of 0.7.

[0050] The constructed initial mapping matrix is ​​stored in a cloud database in tabular form, with each dimension's features and weights clearly labeled, providing a basic framework for subsequent training.

[0051] Acquisition of labeled datasets Multiple conversation samples from different groups were collected. Each sample included complete vocal feature data, content feature data, and corresponding psychological dimension annotation values. The samples covered people of different ages, occupations, and psychological states. For example, a conversation sample of a working professional, Mr. Li, was included. The vocal feature data in this sample were: speech rate 3.5 words / second, fundamental frequency of tone 15Hz, duration of continuous silence 1.2 seconds, and tone fluctuation range 45Hz. The content feature data were: surface emotion label anxiety, demand keyword deadline, no self-harm keywords, negative vocabulary density 0.4, future tense sentence ratio 0.2, and first person pronoun usage frequency 0.6. Three psychology professionals jointly evaluated the sample and determined the corresponding psychological dimension annotation values ​​as follows: crisis risk value 0.4, mental health value 0.55, behavioral tendency value 0.5, and personality trait value 0.65.

[0052] Through the above methods, 1000 valid conversation samples were finally obtained, forming a labeled dataset. The dataset was divided into a training set and a validation set in a 7:3 ratio.

[0053] Iterative training of the initial mapping matrix The sound feature data and content feature data in the training set are input into the initial mapping matrix, and the initial weights of each feature are iteratively trained using the backpropagation algorithm.

[0054] The training batch is set to 50 rounds. After each round of training, the mean square error between the matrix output and the sample label value is calculated. If the mean square error is greater than 0.05, the weight of the corresponding feature is adjusted.

[0055] For example, in one round of training, when inputting sample data of a working professional named Li, the matrix output a mental health value of 0.48, which deviated significantly from the labeled value of 0.55. Analysis revealed that the initial weight of speech rate at 0.6 was too low and could not fully reflect the impact of speech rate on mental health. Therefore, the speech rate weight was adjusted to 0.65, and training was repeated. After 50 iterations, the mean squared error between the matrix output and the labeled values ​​in the training set decreased to 0.03, yielding the post-trained mapping matrix.

[0056] Professional validation of the mapping matrix after training A validation team of five psychology professionals validated the output of the training mapping matrix based on clinical psychological assessment standards (such as the core assessment indicators of the SCL-90 scale).

[0057] The conversation samples from the validation set are input into the training mapping matrix to obtain the psychological dimension values ​​output by the matrix. These values ​​are then compared with the results of professional assessments based on clinical standards to calculate the deviation value.

[0058] The preset deviation threshold is 0.08. If the deviation value of a sample is not less than 0.08, the weights of the corresponding features are adjusted and retraining is performed. For example, if the validation set includes a conversation sample of student Wang, the crisis risk value output by the matrix is ​​0.3, while the professional assessment result is 0.4, resulting in a deviation value of 0.1, which is greater than 0.08. Analysis revealed that the weight of self-harm keywords (0.9) was too high, excessively suppressing the crisis risk value for samples without self-harm keywords. Therefore, the weight of self-harm keywords was adjusted to 0.85, and after three iterations in the training set, the deviation value of this sample was again verified to have decreased to 0.06, which is less than the preset deviation threshold. After multiple rounds of verification and adjustment, the psychological dimension-feature mapping matrix was finally obtained. The deviation of the output results of this matrix from the clinical assessment standard is less than 0.08, meeting the accuracy requirements of psychological assessment.

[0059] Implementation process of deep analysis model The dynamic tracking module tracks cross-round features. The user's conversation is divided into 3 rounds. The device transmits the extracted lightweight features to the cloud dynamic tracking module according to the round. This module uses a long short-term memory network to track the changing trend of lightweight features across rounds.

[0060] The speech rate fluctuation was calculated as follows: in the first round of dialogue, the user's speech rate was 3.2 words / second, in the second round it was 3.8 words / second, and in the third round it was 3.5 words / second. The speech rate fluctuation of the second round relative to the first round was calculated as (3.8-3.2) / 3.2×100%=18.75%; the speech rate fluctuation of the third round relative to the second round was calculated as (3.5-3.8) / 3.8×100%≈-7.89%. The dynamic tracking module recorded the fluctuation amplitude and the corresponding round for each round. A negative result indicates that the speech rate in the current round has decreased compared to the previous round; a positive result indicates that the speech rate has increased.

[0061] The surface emotion label switching frequency was calculated with a unit time of 5 minutes. The total duration of the user's three rounds of dialogue was 8 minutes. The surface emotion label in the first and second rounds was "low" and the label in the third round was also "low". No label switching occurred. Therefore, the surface emotion label switching frequency was 0 times / 5 minutes.

[0062] The frequency of demand keywords is calculated. The demand keywords are "overtime" and "rest". In the first round of dialogue, "overtime" appeared once and "rest" did not appear. In the second round, "overtime" appeared twice and "rest" appeared once. In the third round, "overtime" appeared once and "rest" appeared once. The dynamic tracking module records the frequency of the two keywords by round, forming a frequency change curve across rounds.

[0063] Feature fusion calculation of multimodal fusion module The multimodal fusion module receives the cross-round lightweight feature change trend output by the dynamic tracking module, calls the final psychological dimension-feature mapping matrix, and fuses the voice features and content features according to the weights in the matrix to output the psychological assessment result: In calculating the crisis risk value, the average fundamental frequency of the user's voice across rounds was 12Hz, with no significant abrupt changes. No self-harm keywords were detected in the content features; the value corresponding to self-harm keywords was 0, while a value of 1 was obtained if detected. Based on the matrix weights, the crisis risk value = min[0.7 × (fundamental frequency of tone / 200) + 0.9 × value of self-harm keywords, 1.0], where the fundamental frequency of tone / 200 is a normalization process, adapting to the common range of 80-200Hz for Chinese users' conversational fundamental frequency, ensuring that this sub-item's result does not exceed 1. The min function is used to limit the final result to an upper limit of 1.0 and a lower limit of 0. Substituting the data, we get: 0.7 × (12 / 200) + 0.9 × 0 = 0.042.

[0064] The mental health score was calculated based on an average speaking speed of 3.5 words per second across rounds. The negative word density in the content features was 0.3, meaning there were two negative words in the text: "tired" and "too tired," for a total of six effective words. The density was approximately 2 / 6 ≈ 0.3. According to the matrix weights, the mental health score = 0.4 × (1 - speaking speed / 10) + 0.9 × (1 - negative word density). Substituting the data, we get 0.4 × (1 - 0.35) + 0.9 × (1 - 0.3) = 0.4 × 0.65 + 0.9 × 0.7 = 0.26 + 0.63 = 0.89.

[0065] For behavioral tendency calculation, the duration of continuous silence across rounds is less than 2 seconds, corresponding to a value of 0. In terms of content features, the proportion of future tense statements is 0.1, meaning there are no future tense statements in the text. With a total of 10 statements, the proportion is 1 / 10 = 0.1. Based on the matrix weights, the behavioral tendency value = 0.5 × (1 - duration of continuous silence / 10) + 0.6 × proportion of future tense statements. Substituting the data, we get 0.5 × (1 - 0 / 10) + 0.6 × 0.1 = 0.5 + 0.06 = 0.56.

[0066] To calculate the personality trait value, assuming the user's tone fluctuation range is 40Hz and the first-person pronoun usage frequency is 0.7, the formula is: Personality trait value = 0.4 × (tone fluctuation range / 100) + 0.7 × first-person pronoun usage frequency. Substituting these values, we get 0.4 × 0.4 + 0.7 × 0.7 = 0.16 + 0.49 = 0.65, ensuring that the assessment results cover all initial psychological dimensions.

[0067] The final multimodal fusion module outputs the user's psychological assessment results as follows: crisis risk value 0.084, mental health value 0.89, behavioral tendency value 0.56, and personality trait value 0.65, providing data support for the generation of subsequent feedback information.

[0068] The topic matching unit has a built-in topic tag library, which contains at least four core topics and a set of keywords for each core topic. The topic matching unit matches the conversation text data corresponding to the sound features and the text data corresponding to the content features with the set of keywords in the topic tag library. If the number of keywords that match the same core topic is greater than the preset threshold, then the two are determined to belong to the same topic. The preset time decay formula is: in For the adjusted sound feature weights, The initial weights for this sound feature in the psychological dimension-feature mapping matrix. Δt The time difference between sound features and content features. τ For the preset attenuation coefficient, τ The value range is 30-60 seconds.

[0069] The value of τ was determined based on psychological experimental data. The experiment showed that the short-term memory duration of emotion-language feature association in Chinese users' conversations is about 45 seconds. At this time, the evaluation bias after cross-time feature fusion is the smallest, so τ=45 seconds is preferred. After testing with 1000 samples, the evaluation results with τ=45 seconds have a deviation from the clinical standard deviation of ≤0.03, which is better than 30 seconds and 60 seconds.

[0070] Furthermore, the topic matching unit's built-in topic tag library contains four core topics: work, family, health, and social, with each core topic corresponding to a specific set of keywords.

[0071] The keyword set for work-related topics includes overtime, projects, boss, colleagues, tasks, deadlines, workplace, meetings, performance, and salary; The keyword set for family-themed topics includes parents, children, spouse, housework, family gatherings, arguments, caregiving, hometown, relatives, and family responsibilities. The set of keywords related to health includes fatigue, insomnia, pain, physical examination, medication, exercise, diet, illness, energy, and rest; The keyword set for social topics includes friends, parties, strangers, communication, conflict, loneliness, dinner parties, social circles, human relationships, and interactions.

[0072] The preset threshold is 2, meaning that if the number of keywords matching the same core theme is greater than 2, they are considered to be on the same theme.

[0073] Taking a user's conversation as an example, the user mentioned overtime and project progress in the first round of conversation, uncooperative colleagues and heavy workload in the second round, and criticism from the boss and being too tired in the third round. The topic matching unit matched the text data of each round with the topic tag library. The number of keywords matching for the work topic was 6, that is, overtime, project, colleagues, task, and boss are all work topic keywords, which is much greater than the preset threshold of 2. Therefore, it was determined that all of the user's conversation text belonged to the work topic.

[0074] The application process of the preset time decay formula is as follows: the preset decay coefficient τ is set to 45 seconds, meaning that for every 45 seconds increase in the feature time difference, the weight decays to approximately 37% of the initial value. Taking a user's conversation as an example, when the user mentions the pressure of the project at the 2-minute mark, there is a sudden change in the fundamental frequency of their tone. The initial weight ω of this feature in the psychological dimension-feature mapping matrix is ​​0.7. When the user mentions criticism from their boss at the 3-minute and 20-second mark, negative words are used.

[0075] The adjusted sound feature weights are calculated using the time decay formula: ω'=0.7×e^(-80 / 45)≈0.7×e^(-1.78)≈0.7×0.17≈0.12. The adjusted weight of 0.12 is then combined with the weight of negative words in the content features (0.8) to ensure that the correlation analysis of cross-time features considers both the influence of historical features and avoids excessive interference from outdated features with the current evaluation results.

[0076] Furthermore, the device is also equipped with a boot generation module, which performs the following operations: First, preset a basic data volume threshold, which includes: The text data must contain at least 30 valid words, and the cumulative sampling time for sound features must be at least 60 seconds, covering at least 3 surface emotion tags. Then, the difference between the currently acquired data volume and the aforementioned basic data volume threshold is calculated in real time to obtain the data gap value. If the data gap value is greater than 0, the dynamic guided generation process is initiated.

[0077] Extract high-frequency core words from the current content features. High-frequency core words are words with a frequency of no less than 3 times and a TF-IDF value of no less than 0.6. Call the lightweight generative model deployed on the device, input the high-frequency core words and the current surface sentiment label, and generate 3-5 candidate guiding texts.

[0078] Candidate guidance texts are filtered based on preset evaluation indicators, and one optimal guidance text is selected. The control device outputs the optimal guidance text during dialogue pauses, with an output duration not exceeding 5 seconds.

[0079] The training process for lightweight generative models includes: A training dataset was constructed, containing 100,000 psychological counseling dialogue samples. Each sample was labeled with: trigger keywords, emotion tags, high-quality guiding text, and the corresponding BLEU value.

[0080] We employ knowledge distillation technology, using the T5-small model as the base model and a miniaturized version of cloud-based GPT-3.5 as the teacher model, to distill and train the base model.

[0081] The distilled model is compressed using INT8 quantization, resulting in a compressed file size of no more than 50MB, while ensuring the perplexity of the generated text is no greater than 30. Real-time inference optimization is performed on the device side. For mid-to-high-end devices, optimization using the ARM Neon instruction set ensures that the generation time for a single text line is no more than 0.3 seconds and memory usage is no more than 80MB. For mid-to-low-end devices, the generation time is no more than 0.8 seconds and memory usage is no more than 100MB. For low-end devices, the perplexity of the generated text is reduced to no more than 40, memory usage is no more than 80MB, and the generation time is no more than 1.0 second.

[0082] Meanwhile, the device is also equipped with a virtual object generation module, which includes an attribute generation unit, a dynamic update unit, and an interaction control unit.

[0083] The attribute generation unit executes during the initial phase of the dialogue (within the first 30 seconds): Receive initial information (name, gender) from the user and generate a virtual object name based on preset naming rules.

[0084] Randomly generate basic attributes for virtual objects, such as age (user's age ± 5 years) and gender (same as or complementary to the user, each with a 50% probability), and store them as the initial attribute vector.

[0085] The dynamically updated unit performs an attribute iteration every 3 minutes during the dialogue: User profile feature vectors are retrieved from the cloud, including occupational tendencies, interest tags, and emotional fluctuation patterns. The cosine similarity between the current attribute vector of the virtual object and the user profile feature vector is calculated. If the similarity is <0.6, the virtual object's occupation and experience tags are updated using an attribute adjustment algorithm to increase the similarity by at least 0.2. The interaction control unit monitors the dialogue status in real time and triggers simulated interaction when any of the following conditions are detected: (1) If the user's voice pauses for no less than 1.5 seconds and the current topic has not ended, the topic coherence score is no less than 0.7. (2) A high emotional intensity word was detected, with an emotional intensity value of not less than 0.8 and no user voice input within the next 5 seconds.

[0086] Once triggered, it generates a 15-20 word interruptive question, with the interruption time controlled within 0.3-0.8 seconds after the user's voice pauses.

[0087] Furthermore, the generation of experience tags for virtual objects follows these rules: Build an event experience library containing 100 categories of life scenario event templates. Each category contains 5-8 variable parameters, and each event template is labeled with: scenario keywords, emotional relevance, and suitable age range.

[0088] Extract experience-related entities from user dialogue text, and use the BERT entity recognition model to achieve an accuracy of no less than 92%, including time, location, and event type; calculate the matching degree between the extracted entities and templates in the experience event library, and select the 3 templates with the highest matching degree.

[0089] Based on the user's age and emotional characteristics, the variable parameters of the selected template are instantiated to generate an experience description that conforms to the attributes of the virtual object; the language style of the experience description is controlled by the perplexity index (the perplexity difference with the user's language style is no greater than 5), and the proportion of first-person pronouns is no less than 80%.

[0090] The bootloader generation module also includes a feedback adjustment mechanism: After each output of the introductory text, user response characteristics are collected in real time, including: Response time Effective vocabulary increment in response text Number of changes in emotion labels The guidance effect score is calculated based on the response features, where a valid response scores 3 points, a vocabulary increment of at least 5 words scores 3 points, and a new sentiment tag of 1 word scores 4 points. If the guidance effect score is no greater than 5 points for 3 consecutive times, the model parameters are adjusted: the weight of the current high-frequency core words is increased (by 0.2), and the probability of repetition of similar guidance texts is reduced; when the data gap value is no greater than 0, the guidance generation module switches to maintenance mode, the frequency of generated text is reduced to 50% of the initial frequency, and the text length is shortened to 60%-80% of the initial length.

[0091] This invention is not limited to the optional embodiments described above, and anyone can derive other various forms of products based on the inspiration of this invention. The specific embodiments described above should not be construed as limiting the scope of protection of this invention; the scope of protection of this invention should be determined by the claims, and the specification can be used to interpret the claims.

Claims

1. A conversation-based AI-assisted psychological assessment method, combining device-side and cloud-side collaboration for user feature extraction, analysis, and assessment, characterized by: The specific steps are as follows: S100: First, the device extracts lightweight features during the conversation in real time. Lightweight features include sound features extracted from the conversation sound signal and content features extracted from the conversation content signal. S200 then receives lightweight features transmitted from the device via the cloud, and performs cross-round dynamic tracking and multimodal fusion analysis on the lightweight features through a deep analysis model, outputting quantitative psychological assessment results. The multimodal fusion analysis of the deep analysis model is based on a preset psychological dimension-feature mapping matrix. The S300 and cloud-based system generate feedback information based on the psychological assessment results. The feedback information includes routine guidance information or risk intervention information, and is transmitted to the device for output.

2. The AI-assisted psychological assessment method based on conversation according to claim 1, characterized in that: In step S100, the content features include text data after ASR transcription, surface sentiment tags corresponding to the text data, and demand keywords in the text data. The preprocessing of the content features includes: Denoising is performed on the text data after ASR transcription to remove meaningless interjections and repeated characters; The denoised text data is matched based on a pre-set emotion lexicon to determine the surface emotion tags corresponding to the text data; keywords are extracted from the denoised text data based on a pre-set demand lexicon to obtain the demand keywords.

3. The voice features include speech rate, fundamental frequency of pitch, and duration of continuous silence; the preprocessing of the voice features includes: The conversational audio signal is segmented into preset time frames, and the number of effective syllables in each time frame within a single turn of dialogue is calculated. The ratio of the number of effective syllables in a single turn of dialogue to the dialogue duration is used as the speech rate. The fundamental frequency data of each time frame within a single turn of dialogue is extracted, and the standard deviation of the fundamental frequency data is calculated as the pitch fundamental frequency. The time period in a single turn of dialogue where there is no effective audio signal is detected, and the time period with a duration greater than 2 seconds is recorded as the continuous silence duration.

4. The AI-assisted psychological assessment method based on conversation according to claim 1, characterized in that: In step S200, the deep analysis model includes a dynamic tracking module and a multimodal fusion module; The dynamic tracking module uses a long short-term memory network. The dynamic tracking module receives cross-round lightweight features transmitted from the device and tracks the changing trends of the cross-round lightweight features. The changing trends include the amplitude of speech rate fluctuations, the frequency of surface emotion label conversions, and the frequency of occurrence of demand keywords. Among them, the speech rate fluctuation range is the percentage of the difference between the speech rate of the current round and the speech rate of the previous round relative to the speech rate of the previous round; the surface emotion label switching frequency is the number of times the surface emotion label changes per unit time. The multimodal fusion module receives the cross-cycle lightweight feature change trend output by the dynamic tracking module, and performs fusion calculation on the voice features and content features based on the psychological dimension-feature mapping matrix, outputting psychological assessment results including at least crisis risk value, mental health value, behavioral tendency value, and personality trait value.

5. The AI-assisted psychological assessment method based on conversation according to claim 3, characterized in that: The process of obtaining the psychological dimension-feature mapping matrix includes: First, an initial mapping matrix is ​​constructed. In the initial mapping matrix, each psychological dimension corresponds to at least one voice feature and at least one content feature, and an initial weight is assigned to each corresponding feature. The psychological dimensions include crisis risk, mental health, behavioral tendencies, and personality traits. Obtain a labeled dataset, which includes multiple sets of conversation samples and corresponding psychological dimension labeling values ​​for each set of conversation samples. Each set of conversation samples contains corresponding voice feature data and content feature data. The sound feature data and content feature data in the labeled dataset are input into the initial mapping matrix. The feature weights in the initial mapping matrix are iteratively trained using the backpropagation algorithm to obtain the trained mapping matrix. The output of the training mapping matrix is ​​verified by psychology professionals based on the evaluation criteria. If the deviation between the output and the evaluation criteria is not less than the preset deviation threshold, the feature weights are adjusted and the third step is repeated until the deviation between the output and the evaluation criteria is less than the preset deviation threshold, thus obtaining the final psychological dimension-feature mapping matrix.

6. The AI-assisted psychological assessment method based on conversation according to claim 3, characterized in that: When the multimodal fusion module performs fusion calculations, it also includes a feature association judgment step, which includes: A time window threshold T is set, and sound features and content features extracted at different times during the same conversation are obtained. The time difference Δt between the sound features and content features is calculated. If Δt is not greater than T, the sound features and content features are determined to be directly related, and the multimodal fusion module directly performs fusion calculation on the sound features and content features based on the feature weights in the psychological dimension-feature mapping matrix. If Δt is greater than T, the sound features and content features are determined to be indirectly related. The multimodal fusion module first uses the topic matching unit to determine whether the conversation content corresponding to the sound features and the conversation content corresponding to the content features belong to the same topic. If they belong to the same topic, the weights of the sound features are adjusted based on a preset time decay formula, and then the fusion calculation is performed in combination with the content feature weights in the psychological dimension-feature mapping matrix.

7. The AI-assisted psychological assessment method based on conversation according to claim 5, characterized in that: The topic matching unit has a built-in topic tag library, which contains at least four core topics and a set of keywords corresponding to each core topic. The topic matching unit matches the conversation text data corresponding to the sound features and the text data corresponding to the content features with the set of keywords in the topic tag library. If the number of keywords matching the same core topic is greater than a preset threshold, then the two are determined to belong to the same topic. The preset time decay formula is: in For the adjusted sound feature weights, The initial weights for this sound feature in the psychological dimension-feature mapping matrix. Δt The time difference between sound features and content features. τ For the preset attenuation coefficient, τ The value range is 30-60 seconds.

8. The AI-assisted psychological assessment method based on conversation according to claim 1, characterized in that: In step S100, the device also includes a guidance generation module. The guidance generation module determines whether the amount of data currently provided by the user has reached the preset basic data amount based on the extracted lightweight features. If the preset basic data amount has not been reached, the guidance generation module generates guidance text containing self-narrative scene content based on the keywords in the currently extracted content features, and outputs the guidance text through the device. The self-narrative scene content is a lifelike scene description content corresponding to the keywords. The preset basic data volume is at least three rounds of effective dialogue. Each round of effective dialogue must meet the requirement that the number of effective words in the text data is greater than five and there are clear surface emotion labels.

9. The AI-assisted psychological assessment method based on conversation according to claim 7, characterized in that: The process by which the guide generation module generates guide text includes: Extract core keywords from the current content features. The core keywords are words that appear more than twice in the text data and belong to a preset core vocabulary. Based on the core keywords, call a preset scene template library, which contains scene description frameworks corresponding to different core keywords. Fill the scene description framework with content to generate a self-narrative scene content with a first-person perspective, forming guiding text. The generation delay of the guiding text does not exceed 0.8 seconds, and the language style of the guiding text conforms to the norms of colloquial expression, with the sentence length controlled within the range of 8-15 Chinese characters.

10. An electronic device, characterized in that, For implementing the conversation-based AI-assisted psychological assessment method according to any one of claims 1-8, the electronic device includes a device terminal and a cloud terminal; the device terminal establishes a data interaction connection with the cloud terminal, the device terminal is configured with a functional module for collecting sound signals and content signals during the conversation and outputting feedback information, and the cloud terminal is configured with a functional module for receiving data transmitted by the device terminal and performing in-depth analysis to generate quantitative psychological assessment results.

Images