A mental health monitoring system based on large model

Abstract

The present invention provides a mental health monitoring system based on a large model, and relates to the technical field of data processing. The method includes: collecting multimodal data of a user, and preprocessing the data to obtain voice data, video data and text data; processing the voice data to obtain short-term high-frequency noise data, processing the video data to obtain image fuzzy data, and obtaining voice emotion features and behavior emotion features according to the frequency domain voice data and video data; processing the short-term high-frequency noise data to obtain voice repair emotion features, and repairing the image fuzzy data to obtain behavior repair emotion features; fusing the voice emotion features, behavior emotion features, voice repair emotion features and behavior repair emotion features to obtain emotion feature analysis data, and inputting the data into a preset emotion recognition large model to generate a mental health status assessment result. The present invention can obtain accurate data.

Description

Technical Field

[0001] The invention relates to the technical field of data processing, and in particular to a mental health monitoring system based on a large model. Background Art

[0002] When monitoring the mental health of athletes, data needs to be collected from three aspects: text, voice and video. However, since athletes' daily activities are mainly carried out in sports venues, the data collected in these venues may be inaccurate.

[0003] For example, during volleyball training or competition, the communication between players may be masked by the noise generated by the collision between the volleyball and the limbs. This noise is enough to cover the key emotional features in the speech signal. Existing noise suppression algorithms may often find it difficult to effectively filter these noises in this situation because they are mainly designed for background noise or constant ambient sound and cannot effectively process sudden, frequency-dispersed sounds. This situation not only affects the quality of the audio signal, but may also lead to a decrease in the accuracy of the emotion recognition system, thereby hindering the comprehensive assessment of the mental health of athletes. Summary of the invention

[0004] The purpose of the present invention is to provide a psychological health monitoring system based on a large model, aiming to solve the problems mentioned in the background technology.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] A mental health monitoring system based on a large model, the system comprising:

[0007] A data collection module, used to collect multimodal data of users;

[0008] A data preprocessing module is used to preprocess the multimodal data to obtain voice data, video data and text data;

[0009] The short-time high-frequency noise detection module is used to perform short-time Fourier transform on the voice data, convert the voice signal from the time domain to the frequency domain, obtain frequency-domain voice data, and monitor the frequency-domain voice data according to the short-time high-frequency energy peak and the short-time domain duration to obtain short-time high-frequency noise data;

[0010] The image blur detection module is used to obtain image blur data by calculating the clarity of the image in the video data and judging it;

[0011] The emotion feature recognition module is used to obtain speech emotion features and behavior emotion features based on frequency domain speech data and video data;

[0012] The emotion restoration module is used to obtain the timestamps of short-term high-frequency noise data, image blur data, speech emotion features and behavioral emotion features, and match them within a preset range according to the timestamps. When the timestamps of the short-term high-frequency noise data and the behavioral emotion features match, the short-term high-frequency noise data is processed and the speech restoration emotion features are obtained. When the timestamps of the image blur data and the speech emotion features match, the image blur data is restored and the behavior restoration emotion features are obtained.

[0013] The emotion feature fusion module is used to fuse the speech emotion features, behavior emotion features, speech repair emotion features and behavior repair emotion features of a preset time period to obtain emotion feature analysis data;

[0014] The large-model emotion recognition module is used to input emotion feature analysis data into the preset emotion recognition large model and generate mental health status assessment results.

[0015] Furthermore, the image blur detection module includes:

[0016] An image sampling unit is used to extract continuous frame images from the video data and perform sampling processing on each frame to obtain an image set;

[0017] The image feature extraction unit is used to extract the image features of the image in the image set, calculate the second-order derivative of the image features through the Laplace operator, and generate the Laplace eigenvalue matrix of the image. The calculation formula is:

[0018] ;

[0019] in, is the Laplacian eigenvalue matrix of the image, is the pixel intensity matrix of the input image, is the second-order derivative of the image in the x direction, reflecting the edge changes in the horizontal direction of the image. It is the second-order derivative of the image in the y direction, reflecting the edge changes in the vertical direction of the image;

[0020] Image blur discrimination unit, used to calculate the image clarity score based on the Laplace eigenvalue matrix , and compared with the preset clarity threshold T to determine whether the image is blurred. The clarity score calculation formula is:

[0021] ;

[0022] in, Score the clarity of the image, and reflect the overall sharpness and detail level of the image through the Laplace variance. is the total number of pixels in the image, is the Laplacian eigenvalue of the i-th pixel in the image, is the average value of the Laplacian eigenvalues of all pixels in the image, serving as a benchmark for reflecting the overall edge intensity;

[0023] When <T, mark this frame of image as picture blurry data and record the first timestamp of this image.

[0024] Furthermore, the image blurriness detection module further includes:

[0025] A dynamic threshold adjustment unit, used to adjust the clarity threshold T in real time according to the acquisition environment, lighting conditions, and motion state. The adjustment formula for the clarity threshold T is:

[0026] ;

[0027] Wherein, is the adjusted clarity threshold, is the initial clarity threshold, a, b, c are the weight coefficients of lighting, noise, and motion state, is the average brightness value of the current frame image, is the average brightness value of the reference lighting level, is the standard deviation of the brightness under the reference lighting conditions, is the standard deviation of the noise of the image, is the total number of pixels in the image, is the magnitude of the motion vector of the k-th pixel in the image.

[0028] Furthermore, analyze the transient high-frequency noise data, suppress the transient high-frequency noise in the transient high-frequency noise data, and obtain speech repair data, including:

[0029] Process the transient high-frequency noise data through an adaptive filter, identify and suppress the transient high-frequency noise within a preset frequency range and with a frequency change amplitude meeting the preset frequency change amplitude, and obtain the first speech repair data;

[0030] Perform a shielding operation on the transient high-frequency noise in the first speech repair data, and eliminate the distortion generated by the shielding operation through frequency interpolation to obtain the second speech repair data;

[0031] Process the second speech repair data in the time domain, repair the spectral structure damaged by the transient high-frequency noise in the frequency domain, and compare and optimize the original spectrum and the repaired spectrum to obtain the speech repair data.

[0032] Further, the short-term high-frequency noise data is processed by an adaptive filter to identify and suppress the short-term high-frequency noise within a preset frequency range and whose frequency variation amplitude satisfies the preset frequency variation amplitude, and obtain the first speech repair data, including:

[0033] Setting a preset frequency range and a preset frequency variation amplitude according to short-term high-frequency noise data;

[0034] Setting the adaptive filter parameters according to the preset frequency range and the preset frequency variation amplitude to obtain the first parameters;

[0035] According to the first parameter, identifying short-term high-frequency noise within a preset frequency range and with a frequency variation amplitude satisfying a preset frequency variation amplitude;

[0036] The frequency range and frequency variation amplitude of the short-term high-frequency noise are compared and calculated with the frequency range and frequency variation amplitude of the adjacent speech data to obtain an error value;

[0037] Dynamically adjust the first parameter to obtain a second parameter;

[0038] The frequency range and frequency variation amplitude of the short-term high-frequency noise are reduced by the second parameter to suppress the short-term high-frequency noise and obtain the first speech repair data.

[0039] Furthermore, the blurred image data is repaired and the behavior repair emotional features are obtained, including:

[0040] Repair the blurred data of the image to obtain image repair data;

[0041] By processing the image restoration data, analyzing the layout of key points of facial expressions, the placement of limbs, the layout of hand joints and the movement trajectory, the feature extraction of the image restoration data is performed to obtain the emotional features of behavioral restoration.

[0042] Furthermore, the blurred image data is repaired to obtain image repair data, including:

[0043] Classify the image blur data according to the image blur cause to obtain motion blur data and focal blur data;

[0044] Processing the motion blur data and the focal blur data to obtain a motion blur kernel and a focal blur kernel; performing a deconvolution operation on the motion blur image and the focal blur image by using the motion blur kernel and the focal blur kernel to obtain a noise blur image, and optimizing the noise blur image by using Wiener filtering to obtain a first repaired image;

[0045] The first repaired image is divided into multiple sub-regions, blurriness scores and similarity scores are performed on each sub-region and surrounding sub-regions, and then the sub-region with a low blurriness score is repaired, and the sub-region with a low blurriness score is filled by extracting texture samples from the surrounding sub-regions with high similarity scores to obtain a second repaired image;

[0046] The change of pixel values ​​in the second repaired image is calculated, edge information of the second repaired image is captured through the change of pixel values, and the edge information is filled to obtain image repair data.

[0047] Further, the motion blur data and the focal blur data are processed to obtain a motion blur kernel and a focal blur kernel, including:

[0048] Processing the motion blur data and the focus blur data to obtain a motion blur picture and a focus blur picture;

[0049] Analyze the motion blurred images according to the motion direction and length to obtain the directional peak value;

[0050] The blur direction is inferred based on the directional peak value to obtain the motion blur kernel;

[0051] The focus blur image is analyzed according to the frequency distribution to obtain the diffusion degree and radius;

[0052] According to the diffusion degree and radius, the Gaussian parameters are calculated to obtain the focal blur kernel.

[0053] Further, performing a deconvolution operation on the motion blurred image and the focus blurred image by using a motion blur kernel and a focus blur kernel to obtain a noise blurred image, and optimizing the noise blurred image by using a Wiener filter to obtain a first repaired image, including:

[0054] Convert the motion blurred image and the focus blurred image from the spatial domain to the frequency domain by Fourier transform to obtain a first motion blurred image and a first focus blurred image;

[0055] Processing the first motion blurred image and the first focal blurred image by using a motion blur kernel and a focal blur kernel to remove blur details and generate image noise, thereby obtaining a noise blurred image;

[0056] The image noise is processed by Wiener filtering to obtain a second blurred image. The calculation formula of the second blurred image is:

[0057] ;

[0058] in, represents the frequency domain of the second blurred image, represents the frequency domain of the noise-blurred image, represents the frequency domain of the motion blur kernel or the focal blur kernel, is the conjugate complex number of the motion blur kernel or the focal blur kernel, is the power spectral density of noise in the frequency domain, is the power spectral density of the noise blurred image, is the intensity of noise in the noise-blurred image;

[0059] Perform inverse Fourier transform on the second blurred image, converting it from frequency domain to spatial domain, to obtain a first restored image.

[0060] The above solution of the present invention includes at least the following beneficial effects:

[0061] The adaptive filter of the present invention can automatically adjust its parameters according to the change of the input signal to identify and suppress short-term high-frequency noise within a specific frequency range, and is suitable for processing noise that changes over time. For each sample, the error between the filter output and the expected output is calculated, and then the filter coefficient is adjusted according to the error. After multiple iterations, the filter coefficient gradually stabilizes and reaches the optimal state. In this way, the background noise is effectively removed, the speech is clearer and easier to understand, thereby improving the accuracy of emotional feature extraction and providing more natural and smooth speech data. The adaptive filter can cope with noise conditions in different environments, enhancing the breadth and practicality of data processing.

[0062] After shielding the residual noise, the present invention uses frequency interpolation to fill the shielded part, restore the continuity of the voice signal, and eliminate the distortion that may be introduced by the shielding operation through the interpolation method. The linear interpolation method is used to fill the vacant part after shielding, and the interpolated signal is smoothed to ensure a natural transition. The coherence and integrity of the voice signal are ensured to avoid discontinuity or mutation. The subtle changes in the voice are restored, which helps to more accurately capture and analyze the emotional state of the athletes. High-quality voice data can be better combined with other modal data to provide a comprehensive psychological state analysis.

[0063] The present invention utilizes a known blur kernel to perform inverse processing on the blurred image to restore the original clear image. It further optimizes on the basis of inverse convolution to reduce the noise introduced in the processing process, significantly improves the clarity of the blurred image, and makes behavioral features such as facial expressions and body postures easier to recognize. The optimization process is performed through Wiener filtering, which not only removes the blur but also retains the important details in the image as much as possible.

[0064] The present invention divides the image into multiple sub-regions, and performs blurriness and similarity scores on each sub-region and its surroundings, repairs the regions with high blurriness in a targeted manner, and then captures and strengthens the edge information in the image by minimizing the change in pixel values. The image after local repair looks more natural, without obvious splicing marks, and texture samples are extracted from the surrounding sub-regions with high similarity, ensuring the consistency of the repaired image with the surrounding areas. Clear and coherent images help to more accurately extract and analyze the user's behavioral characteristics, thereby achieving a more refined assessment of mental health status.

[0065] The present invention can comprehensively analyze the user's emotional state from multiple dimensions by extracting features of facial expressions, body postures, and gestures. The pitch, speech rate, and intonation changes in the voice signal are important acoustic features of emotional expression. Analyzing these features can more accurately capture the user's emotional fluctuations. By combining the feature extraction of images and voice, the user's emotional state can be fully understood from both visual and auditory aspects, providing more nuanced emotional insights. Through accurate emotion recognition and response, the user's psychological state can be better understood, and healthier and more effective suggestions can be provided. The high-quality image and voice features form a multimodal data set, which can be used to train more effective large models to further improve the assessment of mental health status. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a flowchart of a large model-based mental health monitoring system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0068] like Figure 1 As shown, an embodiment of the present invention proposes a mental health monitoring system based on a large model, the system comprising:

[0069] A data collection module, used to collect multimodal data of users;

[0070] A data preprocessing module is used to preprocess the multimodal data to obtain voice data, video data and text data;

[0071] The short-time high-frequency noise detection module is used to perform short-time Fourier transform on the voice data, convert the voice signal from the time domain to the frequency domain, obtain frequency-domain voice data, and monitor the frequency-domain voice data according to the short-time high-frequency energy peak and the short-time domain duration to obtain short-time high-frequency noise data;

[0072] The image blur detection module is used to obtain image blur data by calculating the clarity of the image in the video data and judging it;

[0073] The emotion feature recognition module is used to obtain speech emotion features and behavior emotion features based on frequency domain speech data and video data;

[0074] The emotion restoration module is used to obtain the timestamps of short-term high-frequency noise data, image blur data, speech emotion features and behavioral emotion features, and match them within a preset range according to the timestamps. When the timestamps of the short-term high-frequency noise data and the behavioral emotion features match, the short-term high-frequency noise data is processed and the speech restoration emotion features are obtained. When the timestamps of the image blur data and the speech emotion features match, the image blur data is restored and the behavior restoration emotion features are obtained.

[0075] The emotion feature fusion module is used to fuse the speech emotion features, behavior emotion features, speech repair emotion features and behavior repair emotion features of a preset time period to obtain emotion feature analysis data;

[0076] The large-model emotion recognition module is used to input emotion feature analysis data into the preset emotion recognition large model and generate mental health status assessment results.

[0077] In the embodiment of the present invention, multimodal data is collected from the user, and the multimodal data is preprocessed to obtain voice data, video data and text data; by collecting data in a variety of ways, the user's emotional state data can be more comprehensively understood. Standardization of data, and through preprocessing to ensure that all data meet unified standards, ensure time synchronization between different modal data, help accurate emotional analysis, and facilitate subsequent processing. The voice signal is converted from the time domain to the frequency domain through short-time Fourier transform, and the short-time high-frequency noise is detected and separated according to the short-time high-frequency energy peak and the short-time domain duration. The image blur data is obtained by calculating the clarity of the image in the video data and judging it.

[0078] According to the frequency domain voice data and video data, voice emotion features and behavioral emotion features are extracted. When the timestamp of short-term high-frequency noise data matches the timestamp of behavioral emotion features, noise suppression processing is prepared. When the timestamp of image blur data matches the voice emotion features, deblurring processing is prepared. The quality of voice and image collections is significantly improved through noise suppression and deblurring processing. High-quality data helps to extract emotion features more accurately, thereby improving the accuracy of emotion recognition. Combining emotion features of different modalities provides a more comprehensive and accurate emotion analysis. The fused feature vector contains more information, which helps to improve the accuracy of emotion recognition. Through time alignment and normalization processing, the consistency and comparability of data of different modalities are ensured. Through comprehensive analysis of multimodal data, users can be provided with more personalized and detailed mental health assessments. The large model can generate assessment results in real time, detect potential psychological problems in a timely manner, and provide corresponding suggestions and support.

[0079] Among them, the user's multimodal data mainly includes the user's spoken voice, conversation audio, input text, comments, photos, video call screen, etc.

[0080] Among them, the multimodal data is preprocessed, the voice files in the multimodal data are processed into the same format, the silent part in the voice signal is removed, the effective voice segment is retained, the part of the voice activity is identified and segmented, and the non-voice part is filtered out to obtain the voice data; the image files in the multimodal data are unified in image size and standardized in color, and the key frames are extracted and redundant frames are removed to retain the main information of the video to obtain the video data; the text files in the multimodal data are uniformly converted to lowercase to avoid the same word being recognized as different words due to different uppercase and lowercase letters, and the text length is standardized to obtain the text data.

[0081] Among them, for the short-term high-frequency noise detection module, by performing FFT (short-time Fourier transform) on the input voice signal, the voice signal can be decomposed into multiple frequency components to obtain the amplitude and phase information of each frequency component. For noise suppression, the role of short-time Fourier transform is to identify the main frequency components of the noise, which are usually manifested as the energy peak positions in the spectrum. By analyzing these peak positions and amplitudes, the frequency range and intensity of the noise can be determined. The frequency domain information obtained by the transformation mainly includes:

[0082] Frequency component: the frequency points in the noise signal.

[0083] Energy distribution: The energy value at each frequency point is used to judge the strength of the noise.

[0084] Time-frequency relationship: Observe the frequency characteristics of noise changing over time to capture the emergence and attenuation of short-term high-frequency noise.

[0085] A clarity threshold is set, and images below this threshold are considered blurred images; by calculating the clarity of each frame of the image through the Laplace operator, etc., blurred images can be identified quickly and accurately, providing a basis for subsequent deblurring processing. By identifying and processing blurred images, the quality of the entire video data is improved.

[0086] Among them, for the emotional feature recognition module, it can extract features from key information in voice data and image collections, combine voice and behavioral features, provide multi-dimensional emotional analysis, and enhance the accuracy of emotion recognition; analyze the fundamental frequency changes in the voice signal to reflect the speaker's emotional fluctuations, measure the speech speed changes in the voice signal to reflect the speaker's tension or relaxation, and analyze the energy distribution in the voice signal to reflect the speaker's emotional intensity; detect and track key points of the face, such as eyes, mouth, eyebrows, etc., analyze their layout changes, detect the main joint positions of the body, such as shoulders, elbows, wrists, etc., analyze the posture of the limbs, detect the key joints of the hands, and analyze their layout and movement trajectory. By analyzing voice and behavioral features, the user's emotional state can be captured more finely.

[0087] Among them, for the emotional feature fusion module, the voice emotional features, behavioral emotional features, voice repair emotional features and behavioral repair emotional features of the preset time period can be fused to obtain emotional feature analysis data. This process specifically adopts the weighted summation method; first, various emotional features are standardized to ensure that they are compared on the same scale, and each feature is assigned a corresponding weight according to its importance in emotional analysis. For example, the voice emotional feature may have a higher weight because it directly affects the emotional expression. Then, these standardized features are added by weighted summation to generate a comprehensive emotional feature vector to obtain the emotional feature analysis data; this method can effectively capture the comprehensive impact of multiple emotional features and provide strong data support for subsequent emotion recognition and mental health assessment.

[0088] Among them, for the large-model emotion recognition module, it can be structured based on the deep learning model in the existing technology and trained through historical data. It can process complex emotional features and identify potential emotional patterns. When the emotional feature analysis data is input, the model can understand the relationship between different emotional features and identify hidden emotional states and mental health indicators. After multi-level nonlinear mapping and processing, the model will output a mental health status assessment result. This result is usually presented in the form of scores, classification labels or probabilities, which can effectively reflect the individual's emotional health status and potential psychological problems.

[0089] Of course, it should be stated that the innovation of this application does not lie in this, and the existing emotion recognition model can also be directly adopted.

[0090] In a preferred embodiment of the present invention, the image blur detection module includes:

[0091] An image sampling unit, configured to extract consecutive frame images from video data and perform sampling processing on each frame to obtain an image set;

[0092] An image feature extraction unit, configured to extract image features of the image set, calculate the second-order derivative of the image features through a Laplacian operator, and generate a Laplacian eigenvalue matrix of the image. The calculation formula of the Laplacian eigenvalue matrix is:

[0093] ;

[0094] Wherein, is the Laplacian eigenvalue matrix of the image, is the pixel intensity matrix of the input image, is the second-order derivative of the image in the x direction, reflecting the edge change in the horizontal direction of the image, is the second-order derivative of the image in the y direction, reflecting the edge change in the vertical direction of the image;

[0095] An image blur discrimination unit, configured to calculate the clarity score of the image according to the Laplacian eigenvalue matrix, and compare it with a preset clarity threshold T to determine whether the image is blurred. The calculation formula of the clarity score is:

[0096] ;

[0097] Wherein, is the clarity score of the image, reflecting the overall sharpness and detail level of the image through the Laplacian variance, is the total number of pixels in the image, is the Laplacian eigenvalue of the i-th pixel in the image, is the average value of the Laplacian eigenvalues of all pixels in the image, reflecting the benchmark of the overall edge intensity;

[0098] When <T, mark the frame image as picture blur data and record the first timestamp of the image.

[0099] In an embodiment of the present invention, the image sampling unit and the image feature extraction unit together constitute a powerful image processing flow. The image sampling unit is responsible for extracting and preprocessing each frame of image from the video data to ensure the consistency and quality of the data; the image feature extraction unit generates rich image features by performing edge detection, texture feature extraction and application of Laplace operator on the image, thereby providing a solid foundation for subsequent clarity analysis and emotional feature extraction; this series of steps not only improves the accuracy and efficiency of image processing, but also enhances the robustness and reliability of the system, providing high-quality data support for the mental health monitoring system.

[0100] Among them, for the image sampling unit, the sampling process usually includes the following steps:

[0101] Adjust the frame image to a uniform resolution to ensure consistency in subsequent processing. This step can reduce the amount of calculation and improve processing efficiency.

[0102] Convert color images to grayscale images. Grayscale images retain brightness information but remove color information, simplifying subsequent processing.

[0103] Normalize the grayscale image so that its pixel values ​​are within a certain range, such as 0 to 1 or 0 to 255, to ensure consistency;

[0104] Apply filters to remove noise from images and improve image quality.

[0105] Among them, for the image feature extraction unit, the Laplace operator is a second-order differential operator used to detect local maximum and minimum values ​​in an image. It can be applied to the image through a convolution operation; the Laplace operator can detect zero-crossing points in the image, that is, changes from positive to negative or from negative to positive. These points usually correspond to edges in the image. The convolution kernel of the Laplace operator is applied to the image, the Laplace value of each pixel is calculated, and the zero-crossing points of the Laplace value are found. These points are edges.

[0106] By applying the Laplacian operator to an image and adding the result to the original image, the edges of the image can be enhanced to make the image look sharper. The convolution kernel of the Laplacian operator is applied to the image, the Laplacian value of each pixel is calculated, and the Laplacian value is added to the original image to obtain a sharpened image. The Laplacian operator can also be used to enhance the high-frequency components in the image, thereby improving the contrast and clarity of the image. The convolution kernel of the Laplacian operator is applied to the image, the Laplacian value of each pixel is calculated, the weight of the Laplacian value is adjusted as needed, and it is added to the original image to enhance the details of the image.

[0107] In a preferred embodiment of the present invention, the image blur detection module further includes:

[0108] The dynamic threshold adjustment unit is used to adjust the definition threshold T in real time according to the acquisition environment, lighting conditions and motion status. The adjustment formula of the definition threshold T is:

[0109] ;

[0110] in, is the adjusted clarity threshold, is the initial clarity threshold, a, b, c are the weight coefficients of illumination, noise and motion state, which respectively reflect the influence of each factor on the threshold adjustment. is the average brightness value of the current frame image, is the average brightness value of the reference light level, is the standard deviation of brightness under reference lighting conditions, reflecting the volatility of lighting changes. is the noise standard deviation of the image, which is used to measure the level of random noise in the image. is the total number of pixels in the image, For the image The magnitude of the motion vector for each pixel is used to measure the degree of motion between image frames.

[0111] In the embodiment of the present invention, this formula comprehensively considers three main factors, illumination, noise and motion state, which are represented by weight coefficients a, b and c respectively. When the environment changes, these weights will adjust the clarity threshold accordingly. ; In actual applications, the dynamic threshold adjustment unit will update the clarity threshold in real time according to environmental changes to ensure the best image processing results; for example, in low light conditions, will decrease to accommodate lower brightness; in the presence of a lot of noise, will also be reduced to avoid excessive filtering of noise; while in motion, The dynamic threshold adjustment unit adjusts the clarity threshold in real time. , ensuring the best image processing effect in various environments. This adaptive mechanism enables the system to maintain stable performance in complex environments and improve the accuracy and reliability of image processing.

[0112] in, Reflects the average brightness value of the current frame image With reference light level If the current frame is darker, then ,lead to <0, at this time decrease; on the contrary, if the current frame is brighter, Increase, so that the threshold is always within a reasonable range under different lighting conditions.

[0113] in, Reflects the random noise level in the image. A larger noise standard deviation lead to Decrease, because more noise will make the image blurry, and lowering the clarity threshold is beneficial to improving the image processing effect under noise.

[0114] in, Reflects the degree of motion between image frames. Larger motion vector amplitudes lead to Increase, because greater motion will cause image blur, and increasing the sharpness threshold helps improve image processing in motion situations.

[0115] In a preferred embodiment of the present invention, short-term high-frequency noise data is processed to obtain speech restoration emotion features, including:

[0116] Analyze the short-term high-frequency noise data, suppress the short-term high-frequency noise in the short-term high-frequency noise data, and obtain speech restoration data;

[0117] By analyzing the pitch frequency changes, speech speed numerical results and intonation energy changes in the speech in the frequency domain, the speech restoration data is feature extracted to obtain the speech restoration emotional features.

[0118] In an embodiment of the present invention, the process of processing short-term high-frequency noise data and extracting emotional features therefrom can be divided into two main stages, noise suppression and emotional feature extraction; the noise suppression stage removes noise from the speech signal and improves the speech quality through short-time Fourier transform, noise detection, noise suppression, frequency interpolation and inverse transformation steps; the emotional feature extraction stage generates an emotional feature vector that can reflect the user's emotional state by analyzing the pitch frequency changes, speech rate numerical results and intonation energy changes of the repaired speech data. This series of steps not only improves the clarity of the speech signal, but also provides a reliable data basis for accurate emotion recognition, thereby improving the performance and accuracy of the mental health monitoring system.

[0119] In a preferred embodiment of the present invention, the short-term high-frequency noise data is analyzed, and the short-term high-frequency noise in the short-term high-frequency noise data is suppressed to obtain the speech repair data, including:

[0120] Processing the short-term high-frequency noise data through an adaptive filter, identifying and suppressing the short-term high-frequency noise within a preset frequency range and whose frequency variation amplitude satisfies the preset frequency variation amplitude, and obtaining first speech repair data;

[0121] Performing a shielding operation on short-term high-frequency noise in the first voice repair data, and eliminating distortion caused by the shielding operation through frequency interpolation to obtain second voice repair data;

[0122] The second speech restoration data is processed in the time domain to restore the spectrum structure destroyed by short-term high-frequency noise in the frequency domain, and the original spectrum and the restored spectrum are compared and optimized to obtain speech restoration data.

[0123] In an embodiment of the present invention, speech data containing short-term high-frequency noise is processed. First, an adaptive filter is used to identify and suppress short-term high-frequency noise within a preset frequency range and whose variation amplitude meets specific conditions, so as to generate preliminary speech repair data; then, the remaining high-frequency noise in these preliminary repaired data is shielded, and a frequency interpolation technique is used to fill the spectrum gaps caused by the shielding operation, thereby eliminating distortion and obtaining further optimized speech data; finally, this optimized data is processed in the time domain to repair the spectrum structure destroyed by noise in the frequency domain, and by comparing and optimizing the spectrum structure, high-quality speech repair data is finally obtained, making the speech clearer and more natural; this process combines multiple technologies such as adaptive filtering, frequency shielding, and spectrum repair, aiming to effectively remove short-term high-frequency noise while maintaining the quality and integrity of the speech signal.

[0124] Among them, frequency interpolation restores the continuity of the signal by filling the gaps in the frequency domain caused by noise shielding; when processing short-term high-frequency noise, an adaptive filter is first used to identify and suppress the noise within a specific frequency range, and then the remaining noise is shielded. Although the shielding operation can remove noise, it will also leave blank areas in the spectrum, resulting in audio distortion or unnatural sound; frequency interpolation analyzes the surrounding unaffected spectrum information, reasonably estimates and fills these blank areas, thereby smoothing the spectrum and eliminating the negative effects of shielding, so that the final speech repair data not only removes noise interference, but also maintains the natural fluency and high quality of the speech.

[0125] Among them, by comparing the original spectrum and the repaired spectrum and making corresponding optimization adjustments, the spectral components of the speech signal are ensured to be accurately restored, so as to finally obtain high-quality, natural and clear speech repair data, making the listening experience more real and smooth. This process combines the processing advantages of frequency domain and time domain to comprehensively improve the overall quality of the speech signal.

[0126] In a preferred embodiment of the present invention, the short-term high-frequency noise data is processed by an adaptive filter, and the short-term high-frequency noise within a preset frequency range and whose frequency variation amplitude satisfies the preset frequency variation amplitude is identified and suppressed to obtain the first speech repair data, including:

[0127] Setting a preset frequency range and a preset frequency variation amplitude according to short-term high-frequency noise data;

[0128] Setting the adaptive filter parameters according to the preset frequency range and the preset frequency variation amplitude to obtain the first parameters;

[0129] According to the first parameter, identifying short-term high-frequency noise within a preset frequency range and with a frequency variation amplitude satisfying a preset frequency variation amplitude;

[0130] The frequency range and frequency variation amplitude of the short-term high-frequency noise are compared and calculated with the frequency range and frequency variation amplitude of the adjacent speech data to obtain an error value;

[0131] Dynamically adjust the first parameter to obtain a second parameter;

[0132] The frequency range and frequency variation amplitude of the short-term high-frequency noise are reduced by the second parameter to suppress the short-term high-frequency noise and obtain the first speech repair data.

[0133] In an embodiment of the present invention, when processing short-term high-frequency noise data through an adaptive filter, a preset frequency range and a frequency variation amplitude are first set according to the noise characteristics, and the first parameter of the adaptive filter is adjusted based on these settings, and the short-term high-frequency noise within the preset range and with a variation amplitude that meets the requirements is identified using the first parameter; in order to further improve the suppression effect, the identified noise is compared and calculated with adjacent voice data to obtain an error value, and then the filter parameters are dynamically adjusted to reduce the error value, and the identified noise is reduced using the optimized second parameter, thereby effectively suppressing the short-term high-frequency noise and generating the first voice repair data that is initially repaired; this process ensures that the noise is accurately identified and effectively removed, while retaining the quality of the original voice signal as much as possible.

[0134] Among them, when processing short-term high-frequency noise data, it is first necessary to set the preset frequency range and the preset frequency change amplitude according to the specific characteristics of the noise. The principle of this step is to determine which frequency components and change patterns are most likely to be noise based on the analysis of noise characteristics; by defining these parameters, a clear goal can be provided for the adaptive filter, so that it can focus on identifying and suppressing noise signals within a specific frequency range and with significant frequency change amplitudes. The purpose of this is to ensure that the filter can accurately locate and process those short-term high-frequency noises that interfere with speech quality, while avoiding unnecessary effects on normal speech components, thereby laying the foundation for subsequent noise suppression and speech repair processes.

[0135] Among them, the adaptive filter is a filter that can dynamically adjust its own parameters according to the input signal and the reference noise signal. Unlike the traditional fixed parameter filter, the core feature of the adaptive filter is to optimize the filtering effect by adjusting the filter coefficients in real time; the filter continuously learns and adjusts its frequency response and gain parameters according to the changes in environmental noise to achieve the best noise suppression effect.

[0136] Among them, in the process of the adaptive filter processing short-term high-frequency noise, the error value is reduced by dynamically adjusting the first parameter. The principle of this step is to continuously optimize the filter coefficients based on the adaptive algorithm to minimize the difference between the actual output and the expected output; specifically, the system will automatically adjust the parameter settings of the filter according to the error value calculated by comparing the identified noise with the adjacent speech data, and gradually reduce the error until a better set of parameters is found, such as the second parameter, which can more effectively suppress short-term high-frequency noise within a preset frequency range while maximizing the integrity of the speech signal; through this iterative optimization process, the adaptive filter can more accurately adapt to and remove noise under specific conditions, thereby improving the quality of speech restoration.

[0137] In a preferred embodiment of the present invention, the blurred image data is repaired and the behavior repair emotional features are obtained, including:

[0138] Repair the blurred data of the image to obtain image repair data;

[0139] By processing the image restoration data, analyzing the layout of key points of facial expressions, the placement of limbs, the layout of hand joints and the movement trajectory, the feature extraction of the image restoration data is performed to obtain the emotional features of behavioral restoration.

[0140] In an embodiment of the present invention, blurred image data is processed to first restore the clarity of the image and generate image restoration data; after obtaining the clear image restoration data, computer vision and machine learning techniques are used to analyze the facial expressions and body posture data in the image for comprehensive analysis, and the system can extract a series of behavioral restoration emotional features; these features include not only static expression information, but also dynamic body movement patterns, together forming a comprehensive emotional expression model, such multi-dimensional analysis helps to more accurately capture and understand the true emotional state of the characters in the picture, and provides a rich information basis for subsequent emotion recognition and behavior analysis, and the entire process reconstructs and parses high-quality emotional information from blurred pictures.

[0141] Among them, the blurred picture data is processed to first restore the clarity of the image and generate image restoration data; the blur is regarded as the convolution relationship between the clear image and the blur kernel, the motion direction and blur degree are identified to obtain the blur kernel, and the deconvolution algorithm is used to reversely process the blurred image. By minimizing the error between the blurred image and the clear image convolved with the blur kernel, the image blur caused by camera shake, motion blur or other reasons is reduced or eliminated, thereby making the details in the image more clearly visible.

[0142] Among them, the position and shape changes of the key points of facial expressions can reflect the individual's emotional state, such as happiness, sadness, surprise, etc. The placement of limbs is also an important part of emotional expression. For example, an open posture may indicate confidence or relaxation. When the hand joints are bent and the fingers are clenched into fists, it usually expresses anger and tension. When the fingers are fully extended and the palms are open, it often indicates openness, relaxation or confidence. It may also convey welcome and joy.

[0143] Among them, facial key point detection can be used to locate multiple key points on the face and the relative position changes of the key points. The geometric shape of the face can be calculated based on these changes, and features such as anger, joy, and surprise can be extracted through the amplitude and direction of expression changes; human posture estimation can be used to convert key parts of the body into two-dimensional or three-dimensional coordinates, and then the relative angles and position changes of various parts of the body can be analyzed to determine the movement state of the limbs and extract emotional features; hand key point detection can be used to extract the coordinate points of the finger joints, and the gesture movements can be detected by analyzing the bending degree, unfolding angle and movement trajectory of the finger joints to extract the key features.

[0144] In a preferred embodiment of the present invention, the blurred image data is repaired to obtain the image repair data, including:

[0145] Classify the image blur data according to the image blur cause to obtain motion blur data and focal blur data;

[0146] Processing the motion blur data and the focal blur data to obtain a motion blur kernel and a focal blur kernel;

[0147] Performing a deconvolution operation on the motion blurred image and the focus blurred image by using a motion blur kernel and a focus blur kernel to obtain a noise blurred image, and optimizing the noise blurred image by using a Wiener filter to obtain a first repaired image;

[0148] The first repaired image is divided into multiple sub-regions, blurriness scores and similarity scores are performed on each sub-region and surrounding sub-regions, and then the sub-region with a low blurriness score is repaired, and the sub-region with a low blurriness score is filled by extracting texture samples from the surrounding sub-regions with high similarity scores to obtain a second repaired image;

[0149] The change of pixel values ​​in the second repaired image is calculated, edge information of the second repaired image is captured through the change of pixel values, and the edge information is filled to obtain image repair data.

[0150] In an embodiment of the present invention, when processing blurred image data, the cause of image blur is first analyzed and a corresponding blur kernel is obtained. Then, based on the blur kernel, an inverse convolution operation is performed on the blurred image to preliminarily restore the image, and Wiener filtering is used to further optimize and reduce noise to obtain a first repaired image. The first repaired image is divided into multiple sub-regions, and blurriness and similarity scores are performed on each sub-region and its surrounding sub-regions. For sub-regions with lower blurriness scores, texture samples are extracted from surrounding sub-regions with higher similarity for filling, thereby generating a second repaired image. By minimizing the pixel value changes in the second repaired image, the edge information of the image is captured and enhanced, and finally clear and detail-rich image repair data is obtained.

[0151] Among them, the specific cause of image blur is determined by analyzing the blur pattern in the image, so as to obtain the blur kernel, and the inverse convolution technology is used to try to reverse the blur process, that is, to restore the original clear image by applying the inverse operation corresponding to the blur kernel to the blurred image; however, since the inverse convolution process tends to amplify noise, it is usually combined with noise reduction techniques such as Wiener filtering to suppress noise while maintaining image details as much as possible, so as to obtain a higher quality first repaired image.

[0152] Among them, for the first repaired image that has been preliminarily processed but may still have local blur problems, a region-based method is adopted for further optimization. This method divides the entire image into multiple sub-regions, and evaluates the blurriness and texture similarity of these sub-regions and their adjacent regions respectively; if a sub-region is judged to be relatively blurred, the surrounding sub-regions with higher similarity and clearer clarity will be selected as references, and appropriate texture information will be extracted from them to replace or supplement the content of the original blurred sub-region. The purpose of this is to improve the overall visual effect while retaining natural appearance features.

[0153] Among them, for calculating the change of pixel values ​​in the second repaired image, the edge information of the second repaired image is captured through the change of pixel values, and the edge information is filled to obtain image repair data:

[0154] The second repaired image is an image after preliminary repair, and its pixel values ​​have changed compared to the original image, which may show clearer details or traces of repair; the edges are identified by the gradient change of pixel intensity in the image, and the difference between each pixel in the second repaired image and its surrounding neighborhood pixels is calculated to detect areas with obvious intensity changes, which correspond to the edges or important features of the image; the edge information extracted in this way not only helps to understand the shape and outline of objects in the image, not only helps to enhance the overall contrast of the image, but also helps to highlight important features, so that the final image repair data is closer to the visual experience in the real scene.

[0155] Determine the size of a neighborhood window. Common window sizes are 3×3 or 5×5, and the center pixel position is , using absolute difference to quantify the intensity difference between the central pixel and all neighboring pixels, the change in pixel value is calculated as:

[0156] ;

[0157] in, is the sum of the intensity differences between the central pixel and all pixels in the neighborhood, is the center pixel, are other pixels in the neighborhood, and is the offset within the neighborhood window, is the defined neighborhood window pixel set;

[0158] The larger the value, the more significant the difference between the pixel and the surrounding neighborhood pixels, which is the edge information of the second repaired image.

[0159] The captured edge information can determine which parts of the repaired image can be filled with surrounding pixels to obtain more natural repair results and image repair data.

[0160] In a preferred embodiment of the present invention, the motion blur data and the focal blur data are processed to obtain the motion blur kernel and the focal blur kernel, including:

[0161] Processing the motion blur data and the focus blur data to obtain a motion blur image and a focus blur image;

[0162] The motion blurred image is analyzed according to the motion direction and length to obtain the directional peak value;

[0163] The blur direction is inferred based on the directional peak value to obtain the motion blur kernel;

[0164] The focus blurred image is analyzed according to the frequency distribution to obtain the diffusion degree and radius;

[0165] According to the diffusion degree and radius, the Gaussian parameters are calculated to obtain the focal blur kernel.

[0166] In an embodiment of the present invention, by processing the image blur data, the cause of blur in the image is first analyzed to distinguish motion blur and focal blur; for motion blur, the direction and length of the motion are detected by frequency domain analysis, and the direction of the blur is determined according to the directional peak, thereby constructing a motion blur kernel; and for focal blur, the frequency distribution of the blurred area is statistically analyzed to determine the degree of diffusion and radius of the blur, and based on this, the Gaussian parameters are calculated to generate the blur kernel.

[0167] Among them, the motion blur kernel is a function expression obtained by performing frequency domain analysis on the blurred image to detect the direction and length of the motion that causes the image to be blurred; specifically, the motion blur features in the image are identified, and then these features are analyzed in the frequency domain to determine the directional peak of the motion, thereby inferring the specific direction and degree of blur, and constructing a motion blur kernel that can describe this specific type of blur, which accurately reflects the image blur effect caused by object or camera movement.

[0168] The function expression of motion blur kernel is:

[0169] ;

[0170] in, represents the motion blur kernel, represents the spatial coordinates, is the length of the motion blur, for The value range of Indicates blur along the horizontal direction.

[0171] The focal blur kernel is calculated by analyzing the frequency distribution of the focal blur area, determining the diffusion degree and radius of the blur, and then calculating the corresponding Gaussian parameters. The function expression is obtained; specifically, by performing statistical analysis on the focal blur in the image, the diffusion range of the blur can be quantified, and then based on these diffusion characteristics and radius values, a blur kernel that conforms to the Gaussian distribution is constructed, which can accurately describe the image blur effect caused by focal length problems; this blur kernel effectively simulates the smooth and uniform blur phenomenon caused by lens defocus, which helps to remove such blur in a targeted manner during the image restoration process.

[0172] Gaussian parameters The calculation formula is:

[0173] ;

[0174] in, is the Gaussian parameter, is the blur radius.

[0175] The function expression of the focal blur kernel is:

[0176] q ;

[0177] Among them, q represents the focal blur kernel, represents the spatial coordinates, is the Gaussian parameter, express .

[0178] In a preferred embodiment of the present invention, a motion blur kernel and a focal blur kernel are used to perform a deconvolution operation on a motion blur image and a focal blur image to obtain a noise blur image, and the noise blur image is optimized by Wiener filtering to obtain a first repaired image, including:

[0179] Convert the motion blurred image and the focus blurred image from the spatial domain to the frequency domain by Fourier transform to obtain a first motion blurred image and a first focus blurred image;

[0180] Processing the first motion blurred image and the first focal blurred image by using a motion blur kernel and a focal blur kernel to remove blur details and generate image noise, thereby obtaining a noise blurred image;

[0181] The image noise is processed by Wiener filtering to obtain a second blurred image. The calculation formula of the second blurred image is:

[0182] ;

[0183] in, represents the frequency domain of the second blurred image, represents the frequency domain of the noise-blurred image, represents the frequency domain of the motion blur kernel or the focal blur kernel, is the conjugate complex number of the motion blur kernel or the focal blur kernel, is the power spectral density of noise in the frequency domain, is the power spectral density of the noise blurred image, is the intensity of noise in the noise-blurred image;

[0184] Perform inverse Fourier transform on the second blurred image, converting it from frequency domain to spatial domain, to obtain a first restored image.

[0185] In an embodiment of the present invention, a blurred image is converted to a frequency domain by using a Fourier transform, blurred details are removed and noise is generated by using convolution and inverse convolution operations, and then the noise is processed by using a Wiener filter formula to obtain a second blurred image; Wiener filtering is a commonly used noise reduction method, and its calculation formula takes into account factors such as the conjugate complex number of the blur kernel, the noise power spectral density, and the intensity of the noise in the noisy blurred image. The second blurred image is converted from the frequency domain back to the spatial domain by using an inverse Fourier transform to obtain a first restored image; this process aims to efficiently remove blur and noise in the image and achieve preliminary restoration of the image.

[0186] Among them, the deconvolution operation is to convolve the blurred image with the inverse of the blur kernel in the frequency domain in an attempt to reverse the blurring process and restore the original clear image, use Fourier transform to convert the blurred image and blur kernel to the frequency domain, and then apply the inverse of the blur kernel to the blurred image in the frequency domain, that is, the complex conjugate of the blur kernel divided by its squared modulus, to remove the blur effect. This process can be regarded as performing inverse filtering in the frequency domain, aiming to eliminate image degradation caused by the blur kernel, but due to the influence of noise in practice, direct deconvolution may amplify the noise, so it is usually combined with noise reduction techniques such as Wiener filtering to optimize the result, and finally obtain a clearer first repaired image.

[0187] Among them, the Wiener filtering operation is to process the noisy blurred image in the frequency domain by applying a filter based on the minimum mean square error criterion. The Wiener filtering formula combines the conjugate complex number of the blur kernel, the noise power spectral density and the proportional relationship between the original signal and the noise. By comprehensively calculating these factors, an optimized filter response is generated, which can suppress noise while retaining or restoring the useful information in the image as much as possible; the processed image is then converted back to the spatial domain through the inverse Fourier transform, so as to obtain the first repaired image with better noise reduction effect and clearer details. This process can balance the relationship between deblurring and noise reduction to obtain high-quality image restoration results.

[0188] Among them, in the formula of Wiener filtering, the core expression is the product of the conjugate of the blur kernel spectrum and the power spectrum of the noise blurred image divided by the square of the blur kernel spectrum modulus plus a regularization term, which is the ratio of the noise power spectrum to the noise blurred image power spectrum; the blur kernel spectrum describes the characteristics of the factors that cause image blur in the frequency domain; its conjugate is used to reverse the blurring process; the noise blurred image power spectrum reflects the energy distribution of the image signal when it is not affected by noise and blur; and the square of the blur kernel spectrum modulus is a measure of the degree of blur; the added regularization term takes into account the existence of noise, among which the noise power spectrum quantifies the noise level. By adjusting this term, the algorithm's sensitivity to noise can be controlled, thereby effectively suppressing the influence of noise while restoring image details; overall, this formula aims to calculate an optimal filter response based on the relative relationship between these parameters, so that the filtered image has both removed blur and reduced noise interference.

[0189] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A mental health monitoring system based on a large model, characterized in that: The system comprises: A data collection module, used to collect multimodal data of users; A data preprocessing module is used to preprocess the multimodal data to obtain voice data, video data and text data; The short-time high-frequency noise detection module is used to perform short-time Fourier transform on the voice data, convert the voice signal from the time domain to the frequency domain, obtain frequency-domain voice data, and monitor the frequency-domain voice data according to the short-time high-frequency energy peak and the short-time domain duration to obtain short-time high-frequency noise data; The image blur detection module is used to obtain image blur data by calculating the clarity of the image in the video data and judging it; The emotion feature recognition module is used to obtain speech emotion features and behavior emotion features based on frequency domain speech data and video data; The emotion restoration module is used to obtain the timestamps of short-term high-frequency noise data, image blur data, speech emotion features and behavioral emotion features, and match them within a preset range according to the timestamps. When the timestamps of the short-term high-frequency noise data and the behavioral emotion features match, the short-term high-frequency noise data is processed and the speech restoration emotion features are obtained. When the timestamps of the image blur data and the speech emotion features match, the image blur data is restored and the behavior restoration emotion features are obtained. The emotion feature fusion module is used to fuse the speech emotion features, behavior emotion features, speech repair emotion features and behavior repair emotion features of a preset time period to obtain emotion feature analysis data; The large-model emotion recognition module is used to input the emotion feature analysis data into the preset emotion recognition large model and generate the mental health status assessment results; Repair the blurred image data and obtain the behavior repair emotional features, including: Repair the blurred data of the image to obtain image repair data; By processing the image restoration data, analyzing the layout of key points of facial expressions, the placement of limbs, the layout of hand joints and the movement trajectory, the image restoration data is feature extracted to obtain the emotional features of behavioral restoration; Repair the blurred image data to obtain image repair data, including: Classify the image blur data according to the image blur cause to obtain motion blur data and focal blur data; Processing the motion blur data and the focal blur data to obtain a motion blur kernel and a focal blur kernel; performing a deconvolution operation on the motion blur image and the focal blur image by using the motion blur kernel and the focal blur kernel to obtain a noise blur image, and optimizing the noise blur image by using Wiener filtering to obtain a first repaired image; The first repaired image is divided into multiple sub-regions, blurriness scores and similarity scores are performed on each sub-region and surrounding sub-regions, and then the sub-region with a low blurriness score is repaired, and the sub-region with a low blurriness score is filled by extracting texture samples from the surrounding sub-regions with high similarity scores to obtain a second repaired image; Calculating the change of pixel values ​​in the second repaired image, capturing edge information of the second repaired image through the change of pixel values, and filling the edge information to obtain image repair data; The motion blur data and the focal blur data are processed to obtain a motion blur kernel and a focal blur kernel, including: Processing the motion blur data and the focus blur data to obtain a motion blur picture and a focus blur picture; Analyze the motion blurred images according to the motion direction and length to obtain the directional peak value; The blur direction is inferred based on the directional peak value to obtain the motion blur kernel; The focus blur image is analyzed according to the frequency distribution to obtain the diffusion degree and radius; According to the diffusion degree and radius, the Gaussian parameters are calculated to obtain the focal blur kernel; Performing a deconvolution operation on the motion blurred image and the focus blurred image by using a motion blur kernel and a focus blur kernel to obtain a noise blurred image, and optimizing the noise blurred image by using a Wiener filter to obtain a first repaired image, including: Convert the motion blurred image and the focus blurred image from the spatial domain to the frequency domain by Fourier transform to obtain a first motion blurred image and a first focus blurred image; Processing the first motion blurred image and the first focal blurred image by using a motion blur kernel and a focal blur kernel to remove blur details and generate image noise, thereby obtaining a noise blurred image; The noise blurred image is processed by Wiener filtering to obtain a second blurred image. The calculation formula of the second blurred image is: ； in, represents the frequency domain of the second blurred image, represents the frequency domain of the noise-blurred image, represents the frequency domain of the motion blur kernel or the focal blur kernel, is the conjugate complex number of the motion blur kernel or the focal blur kernel, is the power spectral density of noise in the frequency domain, is the power spectral density of the noise blurred image, is the intensity of noise in the noise-blurred image; Perform inverse Fourier transform on the second blurred image, converting it from frequency domain to spatial domain, to obtain a first restored image.

2. A mental health monitoring system based on a large model according to claim 1, characterized in that: The image blur detection module comprises: An image sampling unit is used to extract continuous frame images from the video data and perform sampling processing on each frame to obtain an image set; The image feature extraction unit is used to extract the image features of the image in the image set, calculate the second-order derivative of the image features through the Laplace operator, and generate the Laplace eigenvalue matrix of the image. The calculation formula is: ； in, is the Laplacian eigenvalue matrix of the image, is the pixel intensity matrix of the input image, is the second-order derivative of the image in the x direction, reflecting the edge changes in the horizontal direction of the image. It is the second-order derivative of the image in the y direction, reflecting the edge changes in the vertical direction of the image; Image blur discrimination unit, used to calculate the image clarity score based on the Laplace eigenvalue matrix , and compared with the preset clarity threshold T to determine whether the image is blurred. The clarity score calculation formula is: ； in, Score the clarity of the image, and reflect the overall sharpness and detail level of the image through the Laplace variance. is the total number of pixels in the image, is the Laplace eigenvalue of the i-th pixel in the image, It is the average value of the Laplace eigenvalues ​​of all pixels in the image, reflecting the benchmark of the overall edge strength; When <when T, mark the frame image as blurred data and record the first timestamp of the image.

3. A mental health monitoring system based on a large model according to claim 2, characterized in that: The image blur detection module also includes: The dynamic threshold adjustment unit is used to adjust the definition threshold T in real time according to the acquisition environment, lighting conditions and motion status. The adjustment formula of the definition threshold T is: ； in, is the adjusted clarity threshold, is the initial clarity threshold, a, b, c are the weight coefficients of illumination, noise and motion status, is the average brightness value of the current frame image, is the average brightness value of the reference light level, is the standard deviation of brightness under reference lighting conditions, is the noise standard deviation of the image, is the total number of pixels in the image, For the image The magnitude of the motion vector for each pixel.

4. A mental health monitoring system based on a large model according to claim 3, characterized in that: The short-term high-frequency noise data is processed to obtain the speech restoration emotional features, including: Analyze the short-term high-frequency noise data, suppress the short-term high-frequency noise in the short-term high-frequency noise data, and obtain speech restoration data; By analyzing the pitch frequency changes, speech speed numerical results and intonation energy changes in the speech in the frequency domain, the speech restoration data is feature extracted to obtain the speech restoration emotional features.

5. A mental health monitoring system based on a large model according to claim 4, characterized in that: Analyze the short-term high-frequency noise data, suppress the short-term high-frequency noise in the short-term high-frequency noise data, and obtain speech repair data, including: Processing the short-term high-frequency noise data through an adaptive filter, identifying and suppressing the short-term high-frequency noise within a preset frequency range and whose frequency variation amplitude satisfies the preset frequency variation amplitude, and obtaining first speech repair data; Performing a shielding operation on short-term high-frequency noise in the first voice repair data, and eliminating distortion caused by the shielding operation through frequency interpolation to obtain second voice repair data; The second speech restoration data is processed in the time domain to restore the spectrum structure destroyed by short-term high-frequency noise in the frequency domain, and the original spectrum and the restored spectrum are compared and optimized to obtain speech restoration data.

6. A mental health monitoring system based on a large model according to claim 5, characterized in that: The short-term high-frequency noise data is processed by an adaptive filter, and the short-term high-frequency noise within a preset frequency range and whose frequency variation amplitude satisfies the preset frequency variation amplitude is identified and suppressed to obtain the first speech repair data, including: Setting a preset frequency range and a preset frequency variation amplitude according to short-term high-frequency noise data; Setting the adaptive filter parameters according to the preset frequency range and the preset frequency variation amplitude to obtain the first parameters; According to the first parameter, identifying short-term high-frequency noise within a preset frequency range and with a frequency variation amplitude satisfying a preset frequency variation amplitude; The frequency range and frequency variation amplitude of the short-term high-frequency noise are compared and calculated with the frequency range and frequency variation amplitude of the adjacent speech data to obtain an error value; Dynamically adjust the first parameter to obtain a second parameter; The frequency range and frequency variation amplitude of the short-term high-frequency noise are reduced by the second parameter to suppress the short-term high-frequency noise and obtain the first speech repair data.

Images