Sleep collaborative psychological health assessment and training method and device based on man-machine interaction

Through the sleep collaborative mental health assessment and training method based on human-computer interaction, and using multimedia interaction system and biometric system, the problem of low sleep health assessment and training efficiency in the existing technology is solved, achieving more accurate assessment and efficient training The effect meets the vast demand.

CN119092052APending Publication Date: 2024-12-06ZHUHAI KAIYUE SLEEP HOME TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310653236.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-05
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing sleep health psychological counseling methods are susceptible to personal factors, resulting in inaccurate evaluation results and poor training results. The limitations of manpower, time and space lead to low efficiency and inability to meet the general needs.

Method used

Adopt sleep collaborative mental health assessment and training method based on human-computer interaction, and through the built-in multimedia interaction system and biometric system of terminal devices, multi-modal, multi-dimensional, and multi-scene data are collected, and thematic training program library is constructed, and the sleep collaborative psychological theme is recommended. Training and human-computer interaction through various information transmission methods to reduce the impact of personal factors of professionals and improve evaluation and training efficiency.

Benefits of technology

Through the combination of multimedia interaction system and information processing system, the impact of personal factors of professionals on evaluation and training is reduced, the efficiency and effectiveness of evaluation and training is improved, and the accuracy and training effect of sleep quality assessment is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119092052A_ABST
    Figure CN119092052A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of man-machine interaction, in particular to a sleep collaboration mental health assessment and training method and device based on man-machine interaction, and the method comprises the following steps: S1, presetting a user information storage module in equipment for obtaining and storing account registration information of a user; s2, a multimedia interaction system and a biological recognition system are arranged in the user information storage module, and the user information storage module is used for collecting multi-modal, multi-dimensional, multi-scene and continuous data of a user in a use rest state; s3, processing and analyzing the registration information and data; s4, constructing a theme training scheme library; and S5, recommending sleep collaborative psychological theme training to the user based on a theme recommendation algorithm. According to the method, multimedia data such as voice, videos and games are played through the multimedia playing system, interaction with the user is completed, the influence of personal factors of professionals in the sleep and psychological fields on the evaluation and training effect is reduced, and meanwhile the evaluation and training efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction, and in particular to a sleep-coordinated mental health assessment and training method and a device thereof based on human-computer interaction. Background Art

[0002] Sleep quality includes two aspects: sleep depth and sleep time. High-quality sleep will make you feel less tired and more energetic after waking up. You can engage in various work, study and life. Therefore, deep sleep and sufficient sleep are called good sleep. The evaluation of sleep quality includes both subjective and objective evaluation.

[0003] 1. Subjective evaluation, including: 1. A scale of sleep quality index, which mainly assesses the sleep quality in the last month. It consists of 19 self-assessment items and 5 other-assessment items. The total score ranges from 0 to 21 points. The higher the score, the worse the sleep quality.

[0004] 2. In daily life, sleep quality can be measured by some simple standards, such as falling asleep within 30 minutes, deep sleep, deep breathing, no snoring, no waking up at night, few night sleeps, few nightmares, and feeling energetic after getting up in the morning, clear mind during the day, high work efficiency, etc. 2. Objective evaluation is mainly through polysomnography, which includes electroencephalogram, electromyogram and oculogram. Through the detection of these relevant objective data, we can make a detailed evaluation of the patient's sleep quality and the relationship between each indicator is of great significance for the identification of sleep disorders.

[0005] In the existing related technologies, when it comes to psychological counseling on sleep health, professionals in the fields of sleep and psychology need to communicate face-to-face with visitors. Professionals in the fields of sleep and psychology determine how to communicate with visitors based on the state of the visitors, so as to achieve the purpose of improving the mental health of the visitors. However, this method is easily affected by various factors, resulting in poor evaluation and training effects. For example, when a psychological counselor is tired, it will affect the counselor's ability to obtain and process information, resulting in ignoring some key information that is very useful for guidance expressed by the visitor through body language or language, resulting in inaccurate evaluation results and poor training effects. At the same time, this method is also limited by manpower, time, and space, and is not efficient. At present, there are few practitioners in the field of psychology in my country, and the proportion of those who have received professional sleep training is low, which cannot meet the needs of the majority of visitors. Summary of the invention

[0006] In view of the above problems, the present invention provides a sleep collaborative mental health training device and an evaluation method based on human-computer interaction.

[0007] To solve the above problems, the technical solution adopted by the present invention is: A sleep collaborative mental health assessment and training method based on human-computer interaction, comprising the following steps: S1: Preset a user information storage module in the terminal device to obtain and store the user's account registration information; S2: The user information storage module has a built-in multimedia interaction system and a biometric recognition system, which is used to collect multi-modal, multi-dimensional, multi-scenario, and continuous data of the user in a resting state; S3: Process and analyze registration information and data; S4: Build a library of thematic training programs; S5: Recommend thematic training on sleep collaborative psychology to users based on thematic recommendation algorithm; S6: Pre-build language processing, visual processing and decision-making models through multiple information transmission methods, conduct human-computer interaction with users, and realize sleep collaborative psychological theme training; S7: After the user completes the recommended topic training, multi-modal, multi-dimensional, multi-scenario, and continuous data is collected again; S8: Process and analyze pre- and post-assessment data; S9: Sending the before-after comparison report to the terminal device.

[0008] Preferably, in step S3, part of all the collected data is treated as invalid and abnormal, invalid data includes data collected through multimedia interaction with too short or too long interaction intervals and irrelevant answers, and abnormal data includes data collected through biometrics with values ​​far below or above the normal range; different types of data are converted into data types that are convenient for analysis, voice data is split into text data and waveform data, and video data is split into text data, waveform data, and image data. The subject training program library constructed in step S4 includes a sleep collaborative psychology subject training database and an algorithm module.

[0009] Preferably, the subject training program library constructed in step S4 includes a sleep collaborative psychology subject training database and an algorithm module.

[0010] Preferably, the multiple information transmission modes in step S6 include text, voice, image, video, and virtual reality.

[0011] Preferably, the language processing in step S6 comprehensively analyzes the semantics, lexical structure, pragmatics and context according to the user characteristics; the visual processing obtains the scene image according to the user's practical scenario, predicts the posture set in the scene through the multimedia interaction system inside the device, and then captures the user's action characteristics.

[0012] Another object of the present invention is to provide an assessment and training device that operates according to the above method.

[0013] A sleep collaborative mental health assessment and training device based on human-computer interaction includes a terminal device, wherein the terminal device is electrically connected to a user information storage module, the user information storage module is electrically connected to a multimedia interaction system and a biometric recognition system, and the multimedia interaction system is electrically connected to a storage system, a multimedia playback system, an interactive information collection system, and an information processing system.

[0014] Preferably, the multimedia interaction system includes a storage system, a multimedia playback system, an interaction information acquisition system, and an information processing system. The storage system is used to store the interaction information required for interacting with the user, and the interaction information includes multimedia data on the playback mode and playback conditions.

[0015] Preferably, the interaction information collection system is used to collect and output the user's interaction information, and the interaction information includes the user's state information and the environment information of the user's environment.

[0016] Preferably, the information processing system is connected to the storage system, the multimedia playback system and the interactive information acquisition system respectively, and is used to retrieve multimedia data in the interactive information according to the interactive information, and output the retrieved multimedia data to the multimedia playback system.

[0017] Preferably, the user information storage module has built-in autocorrelation information, namely, modal autocorrelation information, dimensional autocorrelation information and scene autocorrelation information. The modal autocorrelation feature extraction module is configured to extract autocorrelation information of input speech preprocessing features, the dimensional autocorrelation feature extraction module is configured to extract autocorrelation information of input dimensional text preprocessing features, and the scene autocorrelation feature extraction module is configured to extract autocorrelation information of input scene video preprocessing features.

[0018] The beneficial effects of the present invention are: 1. Play multimedia data such as voice, video, and games through the multimedia playback system, and combine the interactive information collection system and the information processing system to complete the interaction with the user, reducing the impact of personal factors of professionals in the fields of sleep and psychology on the evaluation and training effects, while improving the efficiency of evaluation and training.

[0019] 2. By learning the dependencies between modalities and giving different adaptive weights during modal fusion, this method better takes into account the inherent differences between modalities compared to traditional methods, thereby improving the prediction effect of sleep. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of the present invention; Figure 2 It is a block diagram of the device system of the present invention. Implementation

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0023] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined. In addition, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0024] Reference Figure 1 , a sleep collaborative mental health assessment and training method based on human-computer interaction, comprising the following steps: S1: Preset a user information storage module in the terminal device to obtain and store the user's account registration information; Among them, user account registration information includes but is not limited to general demographic information such as age, gender, and occupation.

[0025] S2: The user information storage module has a built-in multimedia interaction system and a biometric recognition system, which is used to collect multi-modal, multi-dimensional, multi-scenario, and continuous data of users in a resting state; In the specific implementation, before S1, the user can view the data collection selection interface on the terminal device, and choose to interact by inputting text, voice, video, etc. according to needs, and collect data including but not limited to physiological signals, behavioral signals, language signals, etc. through multiple terminal devices.

[0026] S3: Process and analyze registration information and data; In the specific implementation, all the collected data are processed first.

[0027] First, user account registration information includes but is not limited to general demographic information such as age, gender, and occupation, which serves as a label for the collected data.

[0028] Secondly, invalid and abnormal data are processed; data collected through multimedia interaction, where the interaction interval is too short or too long, the answer content is irrelevant, etc., are all invalid data; data collected through biometrics, where the value is far below or above the normal range, or other abnormal conditions, are marked as abnormal data; invalid data are discarded during analysis, and abnormal data are additionally marked in the analysis.

[0029] Thirdly, different types of data are converted into data types that are convenient for analysis; for example, voice data is split into text data and waveform data, and video data is split into text data, waveform data, and image data.

[0030] When analyzing data, first analyze different types of data separately, then assign different weights to different types of data for integration, and finally complete the analysis.

[0031] S4: Construct a subject training program library, which includes a sleep collaborative psychology subject training database and an algorithm module; In the specific implementation, the theme training program library consists of two parts: one is the algorithm module, and the other is the sleep collaborative psychology theme training database.

[0032] The algorithm module is a hybrid recommendation algorithm, which performs weighted averaging of recommendation algorithms, deep learning algorithms, etc. to improve the accuracy of recommendations.

[0033] The sleep collaborative psychology theme training database refers to a collection of sleep collaborative psychology training programs, including different training program contents and the forms in which the training programs are presented on the terminal devices. For example, the content of training program A is mainly meditation, and the presentation form of training program A on the terminal device is audio + panoramic video; the content of training program B is mainly dialogue, and the presentation form of training program B on the terminal device is text + audio + dynamic picture.

[0034] S5: Recommend sleep collaborative psychology training to users based on the theme recommendation algorithm, with multiple information delivery methods including text, voice, image, video, and virtual reality; S5 matches the training program that best suits the current analysis result in the sleep collaborative psychological theme training database of S4 according to the analysis result of S3 and based on the algorithm of S4, and sends the training program to the terminal device.

[0035] S6: Pre-build language processing, visual processing and decision-making models through multiple information transmission methods, conduct human-computer interaction with users, and realize sleep collaborative psychological theme training. Language processing comprehensively analyzes its semantics, lexical structure, pragmatics and context according to user characteristics. Visual processing obtains scene images based on user practical scenarios, and predicts the pose set in the scene through the multimedia interaction system inside the device, thereby capturing the user's action characteristics. For example, in the current training plan, there are four branches in the interaction point No. 1, among which the training contents of 4-1 and 4-2 lead to the interaction point No. 2, and the training contents of 4-3 and 4-4 lead to the interaction point No. 3. The training contents of 4-1, 4-2, 4-3, and 4-3 are different. The user can input text, voice, video and other information on the terminal device. After analyzing the user input information, a decision model determines that the user enters the training content No. 4-1 to continue training.

[0036] S7: After the user completes the recommended topic training, multi-modal, multi-dimensional, multi-scenario, and continuous data is collected again; In the specific implementation, the form of data collected by S7 and the type of data finally collected are the same as those of S2.

[0037] S8: Process and analyze pre- and post-assessment data; In the specific implementation, the steps and methods of data processing and analysis in this step are the same as those in S3. After completing this data analysis, the analysis results of this step are subjected to secondary analysis with the analysis results of S3, such as performing paired sample t-test and chi-square test on the analysis results of this step and S3, to obtain comparative analysis results of this step and S3 analysis results.

[0038] S9: Send a before-and-after comparison report to the terminal device.

[0039] In the specific implementation, the comparative analysis results will be sent to the terminal device and presented in a visual form.

[0040] Reference Figure 1-Figure 2 A human-computer interaction-based sleep collaborative mental health assessment and training device using the above-mentioned human-computer interaction-based sleep collaborative mental health assessment and training method includes a terminal device, the terminal device is electrically connected to a user information storage module, the user information storage module is electrically connected to a multimedia interaction system and a biometric recognition system, and the multimedia interaction system is electrically connected to a storage system, a multimedia playback system, an interactive information collection system and an information processing system.

[0041] A multimedia interactive system includes a storage system, a multimedia playback system, an interactive information collection system, and an information processing system. The storage system is used to store interactive information required for interacting with users. The interactive information includes multimedia data of playback mode and playback conditions. The interactive information collection system is used to collect and output the user's interactive information. The interactive information includes user status information and environmental information of the user's environment. The information processing system is connected to the storage system, the multimedia playback system, and the interactive information collection system respectively, and is used to retrieve multimedia data in the interactive information according to the interactive information, and output the retrieved multimedia data to the multimedia playback system.

[0042] The information processing system can retrieve multimedia data in the interactive script from the storage system based on the user interaction information collected by the information collection system, and play multimedia data such as voice, video, and games through the multimedia playback system. The interactive information collection system and the information processing system are combined to complete the interaction with the user, reducing the impact of personal factors of professionals in the sleep and psychology fields on the evaluation results, and improving the stability and effectiveness of service results.

[0043] It should be noted that the user information storage module has built-in autocorrelation information, namely modal autocorrelation information, dimensional autocorrelation information and scene autocorrelation information. The modal autocorrelation feature extraction module is configured to extract autocorrelation information of the input speech preprocessing features, the dimensional autocorrelation feature extraction module is configured to extract autocorrelation information of the input dimensional text preprocessing features, and the scene autocorrelation feature extraction module is configured to extract autocorrelation information of the input scene video preprocessing features.

[0044] When performing modal fusion through an adaptive weighted fusion algorithm, the dependencies between modalities are learned and different adaptive weights are given during modal fusion. Compared with traditional methods, this method better takes into account the inherent differences between modalities, thereby improving the prediction effect of sleep.

[0045] To summarize, the information processing system can retrieve multimedia data in the interactive script from the storage system based on the user interaction information collected by the information collection system, play multimedia data such as voice, video, and games through the multimedia playback system, and combine the interactive information collection system with the information processing system to complete the interaction with the user, thereby reducing the impact of personal factors of professionals in the fields of sleep and psychology on the evaluation effect, and improving the stability and effectiveness of the service effect. When performing modal fusion through the adaptive weighted fusion algorithm, by learning the dependency between modalities, different adaptive weights are given during modal fusion. Compared with traditional methods, it better takes into account the inherent differences between modalities, thereby improving the prediction effect on sleep.

[0046] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A sleep collaborative mental health assessment and training method based on human-computer interaction, characterized in that: The following steps are involved: S1: Preset a user information storage module in the terminal device to obtain and store the user's account registration information; S2: The user information storage module has a built-in multimedia interaction system and a biometric recognition system, which is used to collect multi-modal, multi-dimensional, multi-scenario, and continuous data of the user in a resting state; S3: Process and analyze registration information and data; S4: Build a library of thematic training programs; S5: Recommend thematic training on sleep collaborative psychology to users based on thematic recommendation algorithm; S6: Pre-build language processing, visual processing and decision-making models through multiple information transmission methods, conduct human-computer interaction with users, and realize sleep collaborative psychological theme training; S7: After the user completes the recommended topic training, multi-modal, multi-dimensional, multi-scenario, and continuous data is collected again; S8: Process and analyze pre- and post-assessment data; S9: Sending the before-after comparison report to the terminal device.

2. The sleep collaborative mental health assessment and training method based on human-computer interaction according to claim 1 is characterized in that: In step S3, part of all the collected data is treated as invalid and abnormal. Invalid data includes data collected through multimedia interaction with interaction intervals that are too short or too long and answers that are irrelevant. Abnormal data includes data collected through biometrics with values ​​that are far below or above the normal range. Different types of data are converted into data types that are convenient for analysis. Voice data is split into text data and waveform data, and video data is split into text data, waveform data, and image data.

3. The sleep collaborative mental health assessment and training method based on human-computer interaction according to claim 1 is characterized in that: The subject training program library constructed in step S4 includes a sleep collaborative psychology subject training database and an algorithm module.

4. The sleep collaborative mental health assessment and training method based on human-computer interaction according to claim 1 is characterized in that: The multiple information transmission modes in step S6 include text, voice, image, video, and virtual reality.

5. The sleep collaborative mental health assessment and training method based on human-computer interaction according to claim 1 is characterized in that: In step S6, the language processing comprehensively analyzes the semantics, lexical structure, pragmatics and context according to the user's characteristics; the visual processing obtains the scene image according to the user's practical scene, predicts the posture set in the scene through the multimedia interaction system inside the device, and then captures the user's action characteristics.

6. A sleep collaborative mental health assessment and training device based on human-computer interaction, characterized in that: The method for sleep collaborative mental health assessment and training based on human-computer interaction is operated according to any one of claims 1 to 5, including a terminal device, wherein the terminal device is electrically connected to a user information storage module, the user information storage module is electrically connected to a multimedia interaction system and a biometric recognition system, and the multimedia interaction system is electrically connected to a storage system, a multimedia playback system, an interactive information collection system, and an information processing system.

7. The sleep-coordinated mental health assessment and training device based on human-computer interaction according to claim 6 is characterized in that: The multimedia interactive system includes a storage system, a multimedia playback system, an interactive information collection system, and an information processing system. The storage system is used to store the interactive information required for interacting with users. The interactive information includes multimedia data of playback mode and playback conditions.

8. The sleep-coordinated mental health assessment and training device based on human-computer interaction according to claim 6, characterized in that: The interaction information collection system is used to collect and output the user's interaction information, and the interaction information includes the user's state information and the environment information of the user's environment.

9. The sleep-coordinated mental health assessment and training device based on human-computer interaction according to claim 6, characterized in that: The information processing system is connected to the storage system, the multimedia playback system and the interactive information acquisition system respectively, and is used to retrieve multimedia data in the interactive information according to the interactive information, and output the retrieved multimedia data to the multimedia playback system.

10. The sleep-coordinated mental health assessment and training device based on human-computer interaction according to claim 6, characterized in that: The user information storage module has built-in autocorrelation information, namely, modal autocorrelation information, dimensional autocorrelation information and scene autocorrelation information. The modal autocorrelation feature extraction module is configured to extract autocorrelation information of input speech preprocessing features, the dimensional autocorrelation feature extraction module is configured to extract autocorrelation information of input dimensional text preprocessing features, and the scene autocorrelation feature extraction module is configured to extract autocorrelation information of input scene video preprocessing features.