Psychological health assessment method and system based on image information

Abstract

The invention provides a psychological health assessment method based on image information. The method comprises the following steps: acquiring facial image data and sent voice data of a target user in a process of participating in a reaction game; filtering the facial image data to obtain a filtered image; constructing an image enhancement window, and traversing the filtered image by using the image enhancement window for denoising to obtain a denoised image; sequentially performing histogram equalization processing and face segmentation on the denoised image to obtain a face image; performing signal denoising and text conversion on the voice data to obtain text data; performing feature extraction on the face image and the text data, and fusing the extracted features to obtain fused features; and inputting the fusion features into a trained mental health assessment model to obtain an assessment result. The emotion change and psychological state of the target user can be more comprehensively captured through the analysis of the comprehensive facial image, and the accuracy and reliability of psychological health assessment are improved.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a method and system for evaluating mental health based on image information. Background Art

[0002] Mental health is an important part of an individual's overall health, affecting his or her emotions, thinking, behavior and interpersonal relationships. With the increase of social pressure, mental health problems are becoming increasingly prominent, and effective assessment and intervention methods are urgently needed.

[0003] Traditional mental health assessment methods mainly rely on self-report questionnaires, clinical interviews, and behavioral observations. Although these methods are effective, they have disadvantages such as strong subjectivity, high time cost, and lack of objective assessment results. In addition, many mental health problems are difficult to detect in time at the early stages.

[0004] In recent years, with the development of computer vision and deep learning technology, the application of image information in mental health assessment has gradually attracted attention. By analyzing image data such as facial expressions, postures, and physiological signals of individuals, their mental state can be assessed more objectively and in real time.

[0005] Facial expressions are an important manifestation of emotions. Existing facial expression recognition technologies, such as convolutional neural networks (CNNs) based on deep learning, can accurately identify and classify different emotional states. These emotional states are closely related to mental health and can provide an important basis for mental health assessment.

[0006] In the invention patent with application number 202410271811.9, a first video of the user to be evaluated participating in a reaction game and including the facial image of the user to be evaluated is obtained; based on the first video, emotion recognition and concentration recognition are performed on the user to be evaluated to obtain the concentration parameter of the user to be evaluated and the first emotion parameter used to characterize the emotion; a second video of the user to be evaluated in the process of imitating facial expressions and movements and including the facial image of the user to be evaluated is obtained; based on the second video, emotion recognition is performed on the user to be evaluated to obtain the second emotion parameter characterizing the emotion; based on the concentration parameter, the first emotion parameter and the second emotion parameter, the health assessment score of the user to be evaluated is determined, so as to conduct a mental health assessment on the user to be evaluated based on the health assessment score. However, this method mainly relies on emotion and concentration parameters for evaluation, and may ignore other important dimensions of mental health, such as the emotional impact of an individual's social skills and communication skills. Summary of the invention

[0007] In order to overcome the deficiencies of the prior art, the object of the present invention is to provide a method and system for psychological health assessment based on image information, which improves the accuracy and reliability of psychological health assessment.

[0008] To achieve the above object, the present invention provides the following solutions:

[0009] A mental health assessment method based on image information, comprising:

[0010] Acquire facial image data and voice data of the target user during the reaction game;

[0011] Performing filtering processing on the facial image data to obtain a filtered image;

[0012] Constructing an image enhancement window, and using the image enhancement window to traverse the filtered image for denoising, to obtain a denoised image;

[0013] The denoised image is subjected to histogram equalization processing and facial segmentation in sequence to obtain a facial image;

[0014] Performing signal denoising and text conversion on the voice data to obtain text data;

[0015] Extracting features from the facial image and the text data, and fusing the extracted features to obtain fused features;

[0016] The fused features are input into the trained mental health assessment model to obtain the assessment results.

[0017] Preferably, filtering the facial image data to obtain a filtered image comprises:

[0018] Performing wavelet decomposition on the facial image data at multiple scales to obtain corresponding wavelet coefficients;

[0019] A filtering threshold is constructed according to the size and decomposition scale of the facial image data; the formula of the filtering threshold is: Among them, λ j represents the filtering threshold at the jth decomposition scale, σ j represents the standard deviation of the noise at the jth decomposition scale, N j represents the signal length at the jth decomposition scale, where j represents the decomposition scale;

[0020] constructing an image filtering model using the filtering threshold;

[0021] The facial image data is filtered using the image filtering model to obtain the filtered image.

[0022] Preferably, the formula of the image filtering model is:

[0023]

[0024] Wherein, sign is the sign function, a is the first preset coefficient, b is the second preset coefficient, ω j,k represents the kth wavelet coefficient at the jth decomposition scale, Represents the wavelet coefficients after filtering.

[0025] Preferably, the facial segmentation process includes:

[0026] Build the preset YOLOv7tiny model;

[0027] In the ELAN structure in the backbone network of the YOLOv7tiny model, the original CBL ordinary convolution module with a convolution kernel size of 3 is replaced with a deformable convolution DCNv2, and the outputs of different levels are subjected to feature fusion through the Concat module and then introduced into the SimAM attention mechanism to obtain the improved DCNv2-SimAM-ELAN structure;

[0028] The improved DCNv2-SimAM-ELAN structure replaces the last three ELAN structures in the backbone network of the YOLOv7tiny model, and adds a SimAM attention mechanism after the first ELAN structure;

[0029] Replace the loss function from CIoU to WIoU to obtain an improved YOLOv7tiny network recognition model;

[0030] The image after the histogram equalization processing is input into the improved YOLOv7tiny network recognition model to obtain the facial image.

[0031] Preferably, performing signal denoising and text conversion on the voice data to obtain text data includes:

[0032] Using Wiener filtering method to remove background noise of the speech data;

[0033] The data after background noise removal is converted into text to obtain the speech preprocessing data.

[0034] Preferably, the background noise of the speech data is removed using a Wiener filtering method, comprising:

[0035] Acquire noisy speech data;

[0036] Calculate the Fourier transform X(f) of x[n];

[0037] Estimate the power spectral density S(f) of speech data and the power spectral density N(f) of noise respectively;

[0038] The frequency response H(f) of the Wiener filter is calculated based on the power spectral density S(f) and N(f) of the signal and noise.

[0039] The background noise of the speech data is removed according to the frequency response H(f) and the Fourier transform X(f).

[0040] Preferably, extracting features from the facial image and the text data, and fusing the extracted features to obtain fused features, includes:

[0041] Extracting features from the facial image and the text data respectively to obtain facial image features and text data features; the text data features include MFCC feature information and emotional feature information;

[0042] Performing feature concatenation on the facial image features and the text data features to obtain a unified feature vector;

[0043] The unified feature vector is weightedly fused to obtain the fused feature.

[0044] Preferably, the method for constructing the mental health assessment model includes:

[0045] Obtain a preset facial feature dataset and a text feature dataset;

[0046] Construct an initial convolutional neural network;

[0047] The initial convolutional neural network is trained according to the facial feature data set and the text feature data set to obtain a trained first classifier and a second classifier;

[0048] Using the first classifier to identify the psychological state in the facial feature data set to obtain a facial confidence set, and using the second classifier to identify the psychological state in the text feature data set to obtain a text confidence set;

[0049] Summing the confidences of the psychological states in the face confidence set and the text confidence set to obtain the sums of the psychological state confidences of the face confidence set and the text confidence set respectively;

[0050] Selecting data with the largest sum of psychological state confidence as training threshold data, and retraining the classifiers corresponding to the other object confidence sums so that the classification confidence of the corresponding classifiers is greater than or equal to the training threshold data, and finally cascading the trained classifiers to obtain a classification network;

[0051] The trained LSTM neural network is connected to the classification network to obtain the mental health assessment model.

[0052] A mental health assessment system based on image information, comprising:

[0053] A data collection module, used to obtain facial image data and voice data of the target user during the reaction game;

[0054] A filtering module, used for filtering the facial image data to obtain a filtered image;

[0055] A denoising module is used to construct an image enhancement window, and use the image enhancement window to traverse the filtered image to perform denoising, so as to obtain a denoised image;

[0056] A segmentation module, used for performing histogram equalization processing and facial segmentation on the denoised image in sequence to obtain a facial image;

[0057] A text acquisition module, used for performing signal denoising and text conversion on the voice data to obtain text data;

[0058] A feature extraction module, used for extracting features from the facial image and the text data, and fusing the extracted features to obtain fused features;

[0059] The evaluation module is used to input the fusion features into the trained mental health evaluation model to obtain the evaluation results.

[0060] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0061] The present invention provides a mental health assessment method based on image information, comprising: obtaining facial image data and voice data emitted by a target user during a reaction game; filtering the facial image data to obtain a filtered image; constructing an image enhancement window, and using the image enhancement window to traverse the filtered image for denoising to obtain a denoised image; performing histogram equalization processing and facial segmentation on the denoised image in sequence to obtain a facial image; performing signal denoising and text conversion on the voice data to obtain text data; extracting features from the facial image and the text data, and fusing the extracted features to obtain fused features; inputting the fused features into a trained mental health assessment model to obtain an assessment result. The present invention can capture the emotional changes and psychological state of the target user more comprehensively by comprehensively analyzing the facial image and voice data. The denoising, enhancement and segmentation of the facial image can accurately extract the emotional features of the user, while the denoising and text conversion of the voice data effectively convert the emotional information in the verbal expression. By fusing these features, the present invention not only improves the accuracy and reliability of mental health assessment, but also provides real-time feedback, which helps to better identify and understand the psychological needs of the user, thereby providing a scientific basis for personalized intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0063] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0064] Figure 2 A schematic diagram of the system structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] The purpose of the present invention is to provide a psychological health assessment method based on image information, which can improve the accuracy and reliability of psychological health assessment.

[0067] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0068] Figure 1 A flow chart of a method provided by an embodiment of the present invention, such as Figure 1 As shown, the present invention provides a method for evaluating mental health based on image information, comprising:

[0069] Step 100: Obtain facial image data and voice data of the target user during the process of participating in the reaction game;

[0070] Step 200: filtering the facial image data to obtain a filtered image;

[0071] Step 300: construct an image enhancement window, and use the image enhancement window to traverse the filtered image for denoising, to obtain a denoised image;

[0072] Step 400: performing histogram equalization processing and facial segmentation on the denoised image in sequence to obtain a facial image;

[0073] Step 500: performing signal denoising and text conversion on the voice data to obtain text data;

[0074] Step 600: extracting features from the facial image and the text data, and fusing the extracted features to obtain fused features;

[0075] Step 700: Input the fused features into the trained mental health assessment model to obtain the assessment results.

[0076] Specifically, the steps of acquiring the facial image data and voice data of the target user during the reaction game in this embodiment are as follows:

[0077] 1. Use a high-resolution camera (such as 1080p or higher) and a high-quality microphone to ensure that the user's facial expressions and voice can be clearly captured. Make sure the device is well connected to the computer or mobile device and install the necessary drivers and software.

[0078] 2. Develop or use an existing reaction game application and integrate facial recognition and speech recognition capabilities. You can use libraries such as OpenCV for facial image capture and SpeechRecognition for speech data processing. Before the game starts, prompt the user to allow access to the camera and microphone.

[0079] 3. Capture facial image data in real time while the user is playing the game. Set a timer (e.g., one frame per second) or trigger the capture when the user reacts. At the same time, record the user's voice data. You can set an audio buffer to continue recording until the game ends or a certain condition is met (e.g., the user completes a round of reactions).

[0080] 4. Save the captured facial image data as image files (such as JPEG or PNG format) and name each frame for subsequent processing (for example, using a timestamp). Save the recorded voice data as audio files (such as WAV or MP3 format) and similarly name each recording file.

[0081] 5. After the game is over, the collected data is analyzed using image processing and speech processing algorithms. Image data can be filtered, denoised, and feature extracted, and speech data can be signal denoised and converted to text.

[0082] Through the above steps, this embodiment can effectively obtain the facial image data and voice data of the target user during the reaction game, providing basic data support for subsequent mental health assessment.

[0083] Further, the reaction game process of the present embodiment is an interactive activity designed to test and improve the reaction speed and attention of the user. Usually, the game presents a series of visual or auditory stimulations, such as flickering images, color changes or sound prompts, and the user needs to react in the shortest possible time, such as clicking the screen, pressing a button or making a sound. The design of the game of the present embodiment can include different difficulty levels and time limits to challenge the reaction ability of the user. By recording the reaction time, accuracy and decision-making process of the user, the reaction game can not only evaluate the cognitive ability of the user, but also provide valuable data about its psychological state and emotional response. This game is widely used in psychology research, cognitive training and entertainment fields.

[0084] Preferably, filtering the facial image data to obtain a filtered image comprises:

[0085] Performing wavelet decomposition on the facial image data at multiple scales to obtain corresponding wavelet coefficients;

[0086] A filtering threshold is constructed according to the size and decomposition scale of the facial image data; the formula of the filtering threshold is: Among them, λ j represents the filtering threshold at the jth decomposition scale, σ j represents the standard deviation of the noise at the jth decomposition scale, N j represents the signal length at the jth decomposition scale, where j represents the decomposition scale;

[0087] constructing an image filtering model using the filtering threshold;

[0088] The facial image data is filtered using the image filtering model to obtain the filtered image.

[0089] Preferably, the formula of the image filtering model is:

[0090]

[0091] Wherein, sign is the sign function, a is the first preset coefficient, b is the second preset coefficient, ω j,k represents the kth wavelet coefficient at the jth decomposition scale, Represents the wavelet coefficients after filtering.

[0092] Specifically, when wavelet denoising is used in this embodiment, the value of the wavelet threshold has an important influence on the final denoising effect. The wavelet threshold should be able to separate the wavelet coefficients of the noise and the original image signal to the greatest extent. As the number of wavelet decomposition layers increases, the wavelet coefficient area where the noise is located should gradually decrease, and most of the wavelet coefficients decomposed at a higher level come from the original image signal. Therefore, this embodiment constructs a self-transformed filtering threshold based on the decomposition scale.

[0093] Preferably, the facial segmentation process includes:

[0094] Build the preset YOLOv7tiny model;

[0095] In the ELAN structure in the backbone network of the YOLOv7tiny model, the original CBL ordinary convolution module with a convolution kernel size of 3 is replaced with a deformable convolution DCNv2, and the outputs of different levels are subjected to feature fusion through the Concat module and then introduced into the SimAM attention mechanism to obtain the improved DCNv2-SimAM-ELAN structure;

[0096] The improved DCNv2-SimAM-ELAN structure replaces the last three ELAN structures in the backbone network of the YOLOv7tiny model, and adds a SimAM attention mechanism after the first ELAN structure;

[0097] Replace the loss function from CIoU to WIoU to obtain an improved YOLOv7tiny network recognition model;

[0098] The image after the histogram equalization processing is input into the improved YOLOv7tiny network recognition model to obtain the facial image.

[0099] Specifically, this embodiment first obtains the definition and weight of the original model from the official code base of the YOLOv7tiny model. YOLOv7tiny is a lightweight target detection model suitable for real-time applications. In the backbone network, the ELAN (Efficient Layer Aggregation Network) structure is located, which is part of YOLOv7tiny and is responsible for feature extraction. In the ELAN structure, all CBL (Convolution-Batch Normalization-LeakyReLU) ordinary convolution modules with a convolution kernel size of 3 are found. Using deep learning frameworks such as PyTorch or TensorFlow, modify these modules and replace them with deformable convolution DCNv2. Deformable convolution can adaptively adjust the shape of the convolution kernel to better capture complex features. After the replacement is completed, the Concat module is used to fuse the features of the outputs of different levels, so that multi-scale information can be obtained and the expression ability of the model can be enhanced. Next, the SimAM (Simple Adaptive Modulation) attention mechanism is introduced. By weighting the feature map, the model pays more attention to important features when processing information, further improving the detection accuracy. The improved DCNv2-SimAM-ELAN structure replaces the last three ELAN structures in the backbone network of the YOLOv7tiny model to replace the original modules. After the first ELAN structure, add the SimAM attention mechanism to improve the feature selection ability of this layer. Replace the loss function from CIoU (Complete Intersection over Union) to WIoU (Weighted Intersection over Union). WIoU takes into account the confidence and attention of different categories, can better handle the problem of unbalanced categories, and helps to improve the generalization ability of the model. The image processed by histogram equalization is used as input and passed to the improved YOLOv7tiny network recognition model. Make sure that the size of the input image matches the model requirements and perform necessary preprocessing (such as normalization). Finally, the model will output the feature map, and after subsequent steps such as non-maximum suppression (NMS), the position and features of the facial image are obtained.

[0100] Through the above steps, this embodiment can achieve efficient and accurate facial segmentation based on the improved YOLOv7tiny model, thereby providing reliable image data support for subsequent mental health assessment.

[0101] Preferably, performing signal denoising and text conversion on the voice data to obtain text data includes:

[0102] Using Wiener filtering method to remove background noise of the speech data;

[0103] The data after background noise removal is converted into text to obtain the speech preprocessing data.

[0104] Preferably, the background noise of the speech data is removed using a Wiener filtering method, comprising:

[0105] Acquire noisy speech data;

[0106] Calculate the Fourier transform X(f) of x[n];

[0107] Estimate the power spectral density S(f) of speech data and the power spectral density N(f) of noise respectively;

[0108] The frequency response H(f) of the Wiener filter is calculated based on the power spectral density S(f) and N(f) of the signal and noise.

[0109] The background noise of the speech data is removed according to the frequency response H(f) and the Fourier transform X(f).

[0110] Furthermore, in this embodiment, x[n] is assumed to be an observed signal (a speech signal containing noise), s[n] is assumed to be an original signal, and d[n] is assumed to be a noise signal.

[0111] The output y[n] of the Wiener filter can be expressed as:

[0112] y[n]=H(f)·X(f)

[0113] in:

[0114] X(f) is the Fourier transform of the signal x[n].

[0115] H(f) is the frequency response of the Wiener filter and is defined as:

[0116]

[0117] in:

[0118] S(f) is the power spectral density of the signal s[n].

[0119] N(f) is the power spectral density of the noise d[n].

[0120] By estimating the power spectral density of the signal and noise, the Wiener filter can weight the signal in the frequency domain to suppress the noise. The value of H(f) is close to 1 when the signal power is greater than the noise power, and close to 0 otherwise, which can effectively retain the useful part of the signal while suppressing the noise.

[0121] In this embodiment, by performing inverse Fourier transform on y[n], a denoised time domain signal can be obtained.

[0122] Preferably, extracting features from the facial image and the text data, and fusing the extracted features to obtain fused features, includes:

[0123] Extracting features from the facial image and the text data respectively to obtain facial image features and text data features; the text data features include MFCC feature information and emotional feature information;

[0124] Performing feature concatenation on the facial image features and the text data features to obtain a unified feature vector;

[0125] The unified feature vector is weightedly fused to obtain the fused feature.

[0126] Specifically, this embodiment extracts MFCC coefficients to represent the audio features of speech. Emotional features are used to analyze the pitch, speech rate and emotional pitch (such as fundamental frequency change) of speech.

[0127] Preferably, the method for constructing the mental health assessment model includes:

[0128] Obtain a preset facial feature dataset and a text feature dataset;

[0129] Construct an initial convolutional neural network;

[0130] The initial convolutional neural network is trained according to the facial feature data set and the text feature data set to obtain a trained first classifier and a second classifier;

[0131] Using the first classifier to identify the psychological state in the facial feature data set to obtain a facial confidence set, and using the second classifier to identify the psychological state in the text feature data set to obtain a text confidence set;

[0132] Summing the confidences of the psychological states in the face confidence set and the text confidence set to obtain the sums of the psychological state confidences of the face confidence set and the text confidence set respectively;

[0133] Selecting data with the largest sum of psychological state confidence as training threshold data, and retraining the classifiers corresponding to the other object confidence sums so that the classification confidence of the corresponding classifiers is greater than or equal to the training threshold data, and finally cascading the trained classifiers to obtain a classification network;

[0134] The trained LSTM neural network is connected to the classification network to obtain the mental health assessment model.

[0135] Specifically, the process of building a mental health assessment model in this embodiment starts with the preparation of a data set. We need to collect and process two types of data: facial feature data sets and text feature data sets. The facial feature data set includes facial images in various psychological states, and reflects the psychological state by extracting features such as facial key points and expression intensity. The text feature data set consists of text data reflecting the psychological state, such as social media posts or self-described texts, and features are extracted through methods such as sentiment analysis and topic modeling.

[0136] Next, use a deep learning framework (such as Keras or PyTorch) to build an initial convolutional neural network (CNN), which is suitable for processing image features. We train CNN using the facial feature dataset and the text feature dataset respectively to obtain two classifiers: facial feature classifier and text feature classifier. Each classifier generates a confidence set by identifying the psychological state.

[0137] On this basis, we sum the confidence of each mental state to obtain the sum of the confidence of the face and text features. The value with the largest sum of confidence is selected as the training threshold, and the classifier with lower confidence is retrained to improve its classification confidence to above the threshold. Then, the trained face and text classifiers are cascaded to form a comprehensive classification network.

[0138] Finally, a LSTM (Long Short-Term Memory Network) is connected after the cascade classification network to capture changes in mental state over time. By training on serialized data, LSTM enhances the model's ability to capture changes in mental state. Ultimately, these network structures are combined to form a complete mental health assessment model that can use multimodal data to accurately estimate mental state.

[0139] Furthermore, the evaluation result of this embodiment is to evaluate the individual's mental state by combining facial features and text features. Specifically, the model uses trained convolutional neural networks (CNN) and long short-term memory networks (LSTM) to analyze the input facial images and text data to identify features and patterns related to mental health. Through the cascaded classification network, the model can integrate the confidence information of the face and text to provide an evaluation result about the individual's current mental state. The evaluation results include the recognition of the individual's emotional state (such as happiness, sadness, anxiety, etc.), as well as more complex mental health indicators (such as stress levels, depression tendencies, etc.). Ultimately, the psychological evaluation results output by the model can be used for mental health monitoring, diagnostic support, or personalized psychological intervention suggestions.

[0140] Corresponding to the above method, such as Figure 2 As shown, this embodiment provides a mental health assessment system based on image information, including:

[0141] A data collection module, used to obtain facial image data and voice data of the target user during the reaction game;

[0142] A filtering module, used for filtering the facial image data to obtain a filtered image;

[0143] A denoising module is used to construct an image enhancement window, and use the image enhancement window to traverse the filtered image to perform denoising, so as to obtain a denoised image;

[0144] A segmentation module, used for performing histogram equalization processing and facial segmentation on the denoised image in sequence to obtain a facial image;

[0145] A text acquisition module, used for performing signal denoising and text conversion on the voice data to obtain text data;

[0146] A feature extraction module, used for extracting features from the facial image and the text data, and fusing the extracted features to obtain fused features;

[0147] The evaluation module is used to input the fusion features into the trained mental health evaluation model to obtain the evaluation results.

[0148] The beneficial effects of the present invention are as follows:

[0149] (1) By integrating facial images and voice data, the present invention can assess mental health status from multiple perspectives. The combination of facial expressions and voice features provides rich emotional clues and improves the accuracy of the assessment.

[0150] (2) The present invention uses filtering and denoising techniques to improve data quality, ensuring that the facial images and voice data used are fully processed, thereby improving the reliability of subsequent analysis.

[0151] (3) The facial image data of the present invention is derived from a reaction game in which the user participates, potentially capturing the true psychological state in a relaxed and natural process, which may be more credible than data captured in a static or deliberate environment.

[0152] (4) The histogram equalization and facial segmentation techniques of the present invention improve the contrast and clarity of facial images, effectively highlighting important features to support more detailed expression analysis.

[0153] (5) The present invention integrates multiple features from images and texts, enabling the evaluation model to capture complex psychological feature patterns. This data fusion helps to more comprehensively describe the user's psychological state.

[0154] (6) Through automated data processing and analysis, the present invention enables users to obtain assessment results in a relatively short period of time, which is suitable for immediate mental health monitoring and feedback.

[0155] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0156] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for evaluating mental health based on image information, characterized in that: include: Acquire facial image data and voice data of the target user during the reaction game; Performing filtering processing on the facial image data to obtain a filtered image; Constructing an image enhancement window, and using the image enhancement window to traverse the filtered image for denoising, to obtain a denoised image; The denoised image is subjected to histogram equalization processing and facial segmentation in sequence to obtain a facial image; Performing signal denoising and text conversion on the voice data to obtain text data; Extracting features from the facial image and the text data, and fusing the extracted features to obtain fused features; The fused features are input into the trained mental health assessment model to obtain the assessment results.

2. The method for psychological health assessment based on image information according to claim 1, characterized in that: Performing filtering processing on the facial image data to obtain a filtered image includes: Performing wavelet decomposition on the facial image data at multiple scales to obtain corresponding wavelet coefficients; A filtering threshold is constructed according to the size and decomposition scale of the facial image data; the formula of the filtering threshold is: Among them, λ j represents the filtering threshold at the jth decomposition scale, σ j represents the standard deviation of the noise at the jth decomposition scale, N j represents the signal length at the jth decomposition scale, where j represents the decomposition scale; constructing an image filtering model using the filtering threshold; The facial image data is filtered using the image filtering model to obtain the filtered image.

3. The method for psychological health assessment based on image information according to claim 2, characterized in that: The formula of the image filtering model is: Wherein, sign is the sign function, a is the first preset coefficient, b is the second preset coefficient, ω j,k represents the kth wavelet coefficient at the jth decomposition scale, Represents the wavelet coefficients after filtering.

4. The method for psychological health assessment based on image information according to claim 1, characterized in that: The process of face segmentation includes: Build the preset YOLOv7tiny model; In the ELAN structure in the backbone network of the YOLOv7tiny model, the original CBL ordinary convolution module with a convolution kernel size of 3 is replaced with a deformable convolution DCNv2, and the outputs of different levels are subjected to feature fusion through the Concat module and then introduced into the SimAM attention mechanism to obtain the improved DCNv2-SimAM-ELAN structure; The improved DCNv2-SimAM-ELAN structure replaces the last three ELAN structures in the backbone network of the YOLOv7tiny model, and adds a SimAM attention mechanism after the first ELAN structure; Replace the loss function from CIoU to WIoU to obtain an improved YOLOv7tiny network recognition model; The image after the histogram equalization processing is input into the improved YOLOv7tiny network recognition model to obtain the facial image.

5. The method for psychological health assessment based on image information according to claim 1, characterized in that: The voice data is subjected to signal denoising and text conversion to obtain text data, including: Using Wiener filtering method to remove background noise of the speech data; The data after background noise removal is converted into text to obtain the speech preprocessing data.

6. The method for psychological health assessment based on image information according to claim 5, characterized in that: Using a Wiener filtering method to remove background noise from the speech data includes: Acquire noisy speech data; Calculate the Fourier transform X(f) of x[n]; Estimate the power spectral density S(f) of speech data and the power spectral density N(f) of noise respectively; Calculate the frequency response of the Wiener filter based on the power spectral densities S(f) and N(f) of the signal and noise The background noise of the speech data is removed according to the frequency response H(f) and the Fourier transform X(f).

7. The method for psychological health assessment based on image information according to claim 1, characterized in that: Extracting features from the facial image and the text data, and fusing the extracted features to obtain fused features, including: Extracting features from the facial image and the text data respectively to obtain facial image features and text data features; the text data features include MFCC feature information and emotional feature information; Performing feature concatenation on the facial image features and the text data features to obtain a unified feature vector; The unified feature vector is weightedly fused to obtain the fused feature.

8. The method for psychological health assessment based on image information according to claim 1, characterized in that: The method for constructing the mental health assessment model includes: Obtain a preset facial feature dataset and a text feature dataset; Construct an initial convolutional neural network; The initial convolutional neural network is trained according to the facial feature data set and the text feature data set to obtain a trained first classifier and a second classifier; Using the first classifier to identify the psychological state in the facial feature data set to obtain a facial confidence set, and using the second classifier to identify the psychological state in the text feature data set to obtain a text confidence set; Summing the confidences of the psychological states in the face confidence set and the text confidence set to obtain the sums of the psychological state confidences of the face confidence set and the text confidence set respectively; Selecting data with the largest sum of psychological state confidence as training threshold data, and retraining the classifiers corresponding to the other object confidence sums so that the classification confidence of the corresponding classifiers is greater than or equal to the training threshold data, and finally cascading the trained classifiers to obtain a classification network; The trained LSTM neural network is connected to the classification network to obtain the mental health assessment model.

9. A mental health assessment system based on image information, characterized in that: include: A data collection module, used to obtain facial image data and voice data emitted by the target user during the process of participating in the reaction game; A filtering module, used for filtering the facial image data to obtain a filtered image; A denoising module is used to construct an image enhancement window, and use the image enhancement window to traverse the filtered image to perform denoising, so as to obtain a denoised image; A segmentation module, used for performing histogram equalization processing and facial segmentation on the denoised image in sequence to obtain a facial image; A text acquisition module, used for performing signal denoising and text conversion on the voice data to obtain text data; A feature extraction module, used for extracting features from the facial image and the text data, and fusing the extracted features to obtain fused features; The evaluation module is used to input the fusion features into the trained mental health evaluation model to obtain the evaluation results.

Images