Mental health evaluation system based on AI

Abstract

The invention relates to the field of intelligent evaluation, and particularly discloses an AI-based psychological health evaluation system, which uses an AI-based image recognition and analysis algorithm to carry out key frame sampling and face recognition on collected face data information (face video stream) of a target tested object, and then carries out face recognition on the face data information (face video stream) of the target tested object. Performing muscle unit partitioning and micro-motion state feature extraction on each recognized facial region image, and then performing core muscle displacement state time sequence transmission on a time sequence set of micro-motion state features of each extracted facial muscle point; therefore, the psychological health type is intelligently judged according to the comprehensive representation among the transmission characteristics of the micro-displacement state information of the transmitted facial muscle points. Operation such as key frame sampling can be rapidly completed, and efficient detection is achieved. Meanwhile, fine changes of facial muscles are accurately captured, and the evaluation accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent assessment, and more specifically, to an AI-based mental health assessment system. Background Art

[0002] At present, social and psychological health problems are becoming more and more prominent, and everyone from teenagers to adults is facing different degrees of distress. Due to academic and social pressure, teenagers are prone to mood swings, depression and other problems, and extreme events occur from time to time; while adults are increasingly suffering from mental illnesses such as anxiety and depression under the pressure of work and life. These problems not only affect the quality of life and work efficiency of individuals, but also pose a potential threat to family harmony and social stability.

[0003] However, traditional mental health assessment methods have many drawbacks. Specifically, when seeing a psychologist, patients have to go through a long process of queuing, registering, and waiting, and it is difficult for them to accurately express their feelings, resulting in the diagnosis results often relying on the doctor's personal experience, which cannot meet the needs of large-scale rapid testing. Scale analysis also faces challenges: the test cycle is long and it is not suitable for batch physical examinations; the questions are complex and ambiguous, which can easily lead to misjudgments; patients randomly tick answers due to impatience, further affecting accuracy. In addition, some people in important positions may conceal their symptoms for fear of affecting their future, and the scale cannot detect physiological indicator diseases, which significantly reduces the accuracy and comprehensiveness of the assessment.

[0004] Therefore, an AI-based mental health assessment solution is desired. Summary of the invention

[0005] In order to solve the above technical problems, this application is proposed.

[0006] According to one aspect of the present application, there is provided an AI-based mental health assessment system, which includes: A facial acquisition module is used to acquire facial data information of a target object through a camera, wherein the facial data information is a facial video stream; A recognition module, used for performing key-frame-based facial region recognition on the facial data information to obtain a time-series set of facial region images; A displacement state characterization module is used to perform a comprehensive analysis of facial muscle point micro-motion displacement on the time series set of the facial region image to obtain a comprehensive characterization of facial muscle point micro-motion displacement state, wherein the displacement state characterization module includes: a muscle point feature extraction unit, used to perform muscle point micro-motion feature extraction on the time series set of the facial region image to obtain a time series set of multiple facial muscle point micro-motion state features; a displacement comprehensive transmission unit, used to perform a core facial muscle point displacement time series comprehensive transmission on the time series set of the multiple facial muscle point micro-motion state features to obtain the comprehensive characterization of the facial muscle point micro-motion displacement state; The evaluation module is used to obtain a mental health evaluation result for indicating a mental health type label based on the comprehensive representation of the micro-displacement state of the facial muscle points.

[0007] Compared with the prior art, the present application provides an AI-based mental health assessment system, which collects the facial data information (facial video stream) of the target subject through a camera, uses an AI-based image recognition and analysis algorithm to perform key frame sampling and facial recognition on the facial data information, and then performs muscle unit partitioning and micro-motion state feature extraction on each facial area image after recognition, and then performs core muscle displacement state time series transmission on the extracted time series set of micro-motion state features of each facial muscle point, so as to intelligently judge the mental health type based on the comprehensive characterization between the micro-motion displacement state information transmission features of each transmitted facial muscle point. The present application uses a camera to collect data, which is processed by an AI algorithm, and can quickly complete key frame sampling and other operations to achieve efficient detection. At the same time, it accurately captures subtle changes in facial muscles, avoids subjective interference from patients, and improves the accuracy of assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0009] Figure 1 It is a block diagram of an AI-based mental health assessment system according to an embodiment of the present application.

[0010] Figure 2 It is a block diagram of the displacement state characterization module in the AI-based mental health assessment system according to an embodiment of the present application.

[0011] Figure 3 It is a block diagram of a muscle point feature extraction unit in an AI-based mental health assessment system according to an embodiment of the present application.

[0012] Figure 4 It is a block diagram of the displacement comprehensive transmission unit in the AI-based mental health assessment system according to an embodiment of the present application. DETAILED DESCRIPTION

[0013] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, but rather these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0014] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0015] It is worth noting that in this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0016] In response to the above technical problems, this application proposes an AI-based mental health assessment system, which collects facial data information (facial video stream) of the target subject through a camera, uses an AI-based image recognition and analysis algorithm to perform key frame sampling and facial recognition on the facial data information, and then performs muscle unit partitioning and micro-motion state feature extraction on each facial area image after recognition, and then performs core muscle displacement state time series transmission on the extracted time series set of micro-motion state features of each facial muscle point, so as to intelligently judge the mental health type based on the comprehensive characterization between the micro-motion displacement state information transmission features of each transmitted facial muscle point. This application uses a camera to collect data, and after processing by an AI algorithm, it can quickly complete key frame sampling and other operations to achieve efficient detection. At the same time, it accurately captures subtle changes in facial muscles, avoids subjective interference from patients, and improves assessment accuracy.

[0017] Figure 1 is a block diagram of an AI-based mental health assessment system according to an embodiment of the present application. Specifically, Figure 1As shown, according to the AI-based mental health assessment system 100 of the embodiment of the present application, it includes: a facial acquisition module 110, which is used to collect facial data information of the target object under test through a camera, and the facial data information is a facial video stream; an identification module 120, which is used to perform key frame-based facial area recognition on the facial data information to obtain a time series set of facial area images; a displacement state characterization module 130, which is used to perform a comprehensive analysis of the micro-movement displacement of facial muscle points on the time series set of facial area images to obtain a comprehensive characterization of the micro-movement displacement state of facial muscle points; an assessment module 140, which is used to obtain a mental health assessment result for representing a mental health type label based on the comprehensive characterization of the micro-movement displacement state of facial muscle points.

[0018] Specifically, the facial acquisition module 110 is used to collect facial data information of the target object through a camera, and the facial data information is a facial video stream. It should be understood that the face contains a lot of information related to the physiological and psychological state of the human body. When people are in different psychological and emotional states, facial muscles will make subtle movements. For example, they may frown unconsciously when they are anxious, and the corners of their mouths will move accordingly when they are happy. This information exists in a dynamic form, and the facial video stream can capture these rich physiological and psychological signals comprehensively and in real time, providing a sufficient data basis for subsequent analysis.

[0019] First of all, when choosing hardware equipment, you need to select a suitable camera. Determine the camera parameters according to the actual usage scenario and needs. For example, in scenarios with high image quality requirements and the need to accurately capture subtle facial muscle changes, choose a high-resolution camera to ensure that the collected facial video stream can clearly present every detail of the face, such as slight wrinkles at the corners of the eyes, subtle movements at the corners of the mouth, etc. These details are crucial for subsequent mental health assessments. At the same time, the frame rate of the camera cannot be ignored. A higher frame rate can ensure the smoothness of the video stream, fully record the dynamic process of facial muscle movement, avoid screen freezes or motion loss, and make the collected facial data more continuous and accurate.

[0020] When installing and deploying cameras, take full account of the shooting angle and lighting conditions. Install the camera in a suitable position to ensure that the face of the subject can be fully captured without obstruction. At the same time, avoid direct strong light or too dark light. Direct strong light may cause facial reflections, making some details unclear; too dark light will affect the image quality and increase the difficulty of subsequent image analysis. You can create a stable and suitable shooting environment by adjusting the position of the camera and using equipment such as a sunshade or fill light.

[0021] When collecting data, it is necessary to communicate with the subject appropriately, inform them of the collection process, keep them in a natural state, and avoid affecting the normal movement of facial muscles due to deliberate performance or tension. During the collection process, the camera works continuously, constantly capturing the facial images of the subject, forming a continuous facial video stream. These video streams are stored in specific formats and encoding methods. Common formats include MP4, AVI, etc., and encoding methods include H.264, H.265, etc. These formats and encodings can effectively compress the data size while ensuring video quality, facilitating subsequent data transmission and storage.

[0022] The collected facial video stream is not directly used for mental health assessment and analysis, and a series of preprocessing work is required. Video processing technology is used to remove noise in the video stream, such as noise and stripes caused by the camera device itself or environmental interference. At the same time, the video stream is stabilized to prevent the image from being unstable due to slight shaking of the object being measured or shaking of the camera, which affects the accuracy of subsequent image analysis. Through these steps, the quality of the collected facial video stream is ensured to be good, providing a reliable data foundation for subsequent AI-based image recognition and analysis algorithms, thereby achieving accurate assessment of mental health status.

[0023] Specifically, the recognition module 120 is used to perform keyframe-based facial region recognition on the facial data information to obtain a time series set of facial region images. More specifically, in an embodiment of the present application, the recognition module is used to: perform keyframe sampling on the facial data information to obtain a time series set of facial keyframes; perform facial region recognition on each facial keyframe in the time series set of facial keyframes to obtain a time series set of facial region images. Accordingly, considering that the amount of facial video stream data is huge, and that there are many frames in the video stream that have highly similar information, especially in static or slowly changing situations, if all video frames are processed, the amount of calculation will be greatly increased, and a lot of time and resources will also be consumed. However, it also contains the main features and significant change information of the face, and these key features play an important role in the judgment of mental health. Based on this, the present application obtains a time series set of facial keyframes by performing keyframe sampling on the facial data information to retain key information while effectively reducing the amount of data. It should be understood that key frames usually contain significant change information, such as changes in expression, muscle micro-movements, etc., so that key information related to mental health assessment can be obtained more accurately in the future, improving the accuracy of analysis. The obtained time series set of facial key frames retains the sequential information of facial state changes over time, and can reflect the dynamic change process of facial features and expressions of the subject, so as to help to understand the trend of changes in the psychological state of the subject more comprehensively and deeply, so as to more accurately judge the type of mental health. Then, considering that different areas of the face have different characteristics and sensitivities when reflecting the psychological state. For example, muscle movements around the eyes may be more related to emotional tension or relaxation, while movements at the corners of the mouth may be more closely related to the positive and negative expressions of emotions. Therefore, in order to focus on the facial information of key areas, such as key feature points such as eyes, eyebrows, mouth, nose, etc., in the technical solution of the present application, facial area recognition is performed on each facial key frame in the time series set of facial key frames to obtain a time series set of facial area images. In this way, by identifying the facial area, the system can extract more detailed expression features and muscle micro-movement information, thereby achieving high-precision analysis of individual emotions and psychological states.

[0024] In particular, the facial region recognition for each facial key frame in the time series set of facial key frames can be implemented in the following manner: Before performing facial region recognition, the system will first load a trained facial region recognition model. These models are usually built based on deep learning algorithms, such as convolutional neural networks (CNNs). CNNs have powerful feature extraction capabilities and can automatically learn various feature patterns in facial images. In order to enable the model to more accurately recognize facial regions, a large amount of image data containing different facial expressions, postures, and lighting conditions will be used for training. During the training process, the model continuously adjusts its own parameters to optimize the recognition of various facial region features.

[0025] When a facial keyframe enters the recognition process, the image must first be preprocessed. This includes adjusting the size and resolution of the image to meet the input requirements of the recognition model. At the same time, the image is normalized to map the pixel values ​​of the image to a uniform range to reduce the impact of factors such as lighting differences on the recognition results. For example, the pixel values ​​of the image are normalized to the interval [0, 1] or [-1, 1].

[0026] The preprocessed facial keyframe images are input into the facial region recognition model. The convolution layer in the model performs convolution operations on the image and extracts low-level features such as edges and textures in the image through different convolution kernels. As the convolution layer continues to go deeper, these low-level features will gradually be combined into higher-level and more abstract features, such as features of facial regions such as eyes, eyebrows, and mouth. The pooling layer downsamples the feature maps output by the convolution layer without losing key information, reducing the amount of data and computational complexity.

[0027] When the model processes images, it uses object detection algorithms to locate facial regions. Common object detection algorithms include YOLO and Faster R-CNN, which can quickly and accurately detect the location and category of target objects in images. In facial region recognition, these algorithms detect key regions such as eyes, eyebrows, mouth, nose, etc. in facial key frames and mark their locations and bounding boxes.

[0028] After identifying the facial region, the system will crop the images of each facial region from the facial keyframe image based on the detected bounding box information. These cropped images constitute the facial region image. According to the temporal sequence of the facial keyframes, the corresponding cropped facial region images are arranged in sequence to form a temporal set of facial region images. This set not only contains the image information of each facial region in each facial keyframe, but also retains the temporal sequence, providing an important data basis for the subsequent analysis of the changes in the micro-motion features of facial muscle points over time.

[0029] In order to improve the accuracy and stability of facial region recognition, the system also uses some post-processing techniques. For example, the non-maximum suppression (NMS) algorithm is used to remove overlapping bounding boxes to ensure that each facial region is accurately recognized only once. At the same time, some prior knowledge and rules are combined to verify and correct the recognition results. For example, based on the relative position relationship of the facial features, it is judged whether the recognized facial area is reasonable. If unreasonable situations are found, corresponding adjustments or re-identification will be made. In actual application scenarios, various complex situations may also be faced, such as facial occlusion, exaggerated expressions, etc. In response to these problems, researchers continue to improve and optimize facial region recognition technology. On the one hand, by increasing the diversity of training data, the model can learn more facial features in different situations; on the other hand, more advanced algorithms are developed to improve the model's ability to adapt to complex situations.

[0030] Specifically, the displacement state characterization module 130 is used to perform a comprehensive analysis of the micro-motion displacement of facial muscle points on the time series set of the facial region images to obtain a comprehensive characterization of the micro-motion displacement state of facial muscle points. Figure 2 FIG. 1 is a block diagram of a displacement state characterization module in an AI-based mental health assessment system according to an embodiment of the present application. Specifically, Figure 2 As shown, the displacement state characterization module 130 includes: a muscle point feature extraction unit 131, which is used to extract muscle point micro-motion features from the time series set of the facial area image to obtain a time series set of multiple facial muscle point micro-motion state features; a displacement comprehensive transfer unit 132, which is used to perform core facial muscle point displacement time series comprehensive transfer on the time series set of the multiple facial muscle point micro-motion state features to obtain the comprehensive characterization of the facial muscle point micro-motion displacement state.

[0031] Specifically, in the embodiment of the present application, the muscle point feature extraction unit 131 is used to extract muscle point micro-motion features from the time series set of the facial area image to obtain a time series set of multiple facial muscle point micro-motion state features. Figure 3 FIG. 1 is a block diagram of a muscle point feature extraction unit in an AI-based mental health assessment system according to an embodiment of the present application. Specifically, Figure 3As shown, more specifically, in the embodiment of the present application, the muscle point feature extraction unit 131 includes: a muscle unit partitioning subunit 1311, used to perform muscle unit partitioning on each facial area image in the time series set of the facial area images to obtain a time series set of multiple facial muscle unit area images; a muscle point micro-motion state feature extraction subunit 1312, used to pass each facial muscle unit area image in the time series set of the multiple facial muscle unit area images through a muscle point micro-motion feature extractor based on a void convolutional neural network model to obtain a time series set of multiple facial muscle point micro-motion state feature vectors as the time series set of the multiple facial muscle point micro-motion state features.

[0032] Specifically, the muscle unit partitioning subunit 1311 is used to perform muscle unit partitioning on each facial region image in the time series set of the facial region images to obtain a time series set of multiple facial muscle unit region images. It should be understood that different facial expressions (such as smiling, frowning, staring, etc.) are controlled by specific facial muscle groups. Different facial muscles have different functions and reaction patterns when expressing emotions and psychological states. For example, contraction of the corrugator supercilii muscle may indicate anxiety or thinking, and the movement of the orbicularis oculi muscle is related to emotional tension or relaxation. The facial region image contains a variety of muscle movement information, but it is difficult to accurately distinguish the specific activities of different muscles through overall analysis. Based on this, the present application performs muscle unit partitioning on each facial region image in the time series set of the facial region images to subdivide the complex facial muscle movement into small muscle units, so as to more clearly and accurately capture the movement details of each muscle and obtain a time series set of multiple facial muscle unit region images. That is, the movement of each muscle unit may contain specific psychological information. After partitioning, the micro-motion state features of each muscle unit area image can be extracted to obtain more accurate data such as muscle vibration amplitude, frequency, displacement, etc., thereby more keenly capturing the subtle manifestations of changes in psychological state in muscle movements.

[0033] In particular, muscle unit partitioning is performed on each facial region image in the time series set of facial region images respectively, which can be achieved in the following manner: After acquiring the facial region image, the facial muscle positioning algorithm based on computer vision technology is used to determine the approximate location of the facial muscles. These algorithms are trained based on facial anatomical knowledge and a large amount of sample data, and can identify the location and range of different facial muscles. For example, by learning a large number of facial images containing different expressions and different individuals, the algorithm can grasp the characteristics and location information of the main facial muscles such as the orbicularis oculi and orbicularis oris in different states. Using these trained algorithms, the location of each muscle is preliminarily marked in the facial region image. These algorithms can quickly and accurately identify the orbicularis oculi around the eyes, the corrugator and frontalis muscles that control the movement of the eyebrows, the orbicularis oris and zygomaticus major muscles around the mouth, etc. The algorithm extracts and analyzes the features in the image to determine which areas belong to specific muscles and mark their locations and approximate ranges.

[0034] After initially locating the facial muscle area, semantic segmentation technology is further used to achieve more refined muscle unit partitioning. Semantic segmentation aims to classify each pixel in the image into a specific category. For facial muscle partitioning, it is to classify each pixel into the corresponding muscle unit category. Commonly used semantic segmentation models, such as U-Net and DeepLab, learn the detailed features and boundary information of facial muscles by training a large number of facial images with precise muscle unit annotations. During the training process, the model continuously adjusts its own parameters to optimize the recognition ability of different muscle units. When processing new facial area images, the model will classify each pixel in the image according to the learned features, thereby accurately dividing the facial muscle area into individual muscle units. For example, for the orbicularis oculi muscle, subunits with different functions such as the orbital and palpebral parts can be subdivided; for the orbicularis oris muscle, muscle tissues in different parts can also be distinguished. These subdivided muscle units are crucial for accurately capturing the micro-movement characteristics of muscles.

[0035] In addition, considering the dynamics of facial muscle movement, the time series analysis method is used when processing the time series set of facial area images. The movement changes of muscle units between adjacent frames are analyzed to track the movement trajectory of muscles. For example, when a person smiles, the orbicularis oris and zygomaticus major muscles will move in coordination. By analyzing each frame of the time series set, the movement sequence and amplitude changes of these muscles can be observed. This not only helps to divide the muscle unit area more accurately, but also provides richer data for the subsequent analysis of the relationship between muscle movement and psychological state.

[0036] After completing the muscle unit partitioning, the divided muscle unit area images in each image are arranged in sequence according to the time sequence of the facial area images to form a time sequence set of multiple facial muscle unit area images. These sets record the state changes of facial muscles at different times, providing key data support for the subsequent extraction of facial muscle point micro-motion state features, thus playing an important role in the AI-based mental health assessment system and helping to more accurately assess the mental health status of individuals.

[0037] Specifically, the muscle point micro-motion state feature extraction subunit 1312 is used to pass each facial muscle unit area image in the time series set of the multiple facial muscle unit area images through a muscle point micro-motion feature extractor based on a hollow convolutional neural network model to obtain a time series set of multiple facial muscle point micro-motion state feature vectors as a time series set of the multiple facial muscle point micro-motion state features. Accordingly, considering that the muscle point micro-motion information contained in the facial muscle unit area image is very subtle and complex, these micro-motions often involve extremely small displacements and changes. Traditional feature extraction methods are difficult to accurately capture and effectively analyze. The hollow convolutional neural network model has a unique advantage. It can expand the receptive field of the convolution kernel without increasing the number of parameters and the amount of calculation, so that it can better capture the micro-motion features of facial muscle points at different scales. Therefore, in the technical solution of the present application, each facial muscle unit area image in the time series set of the plurality of facial muscle unit area images is respectively subjected to a muscle point micro-motion feature extractor based on a dilated convolutional neural network model to extract subtle change features in the facial muscle unit area image, thereby obtaining a time series set of multiple facial muscle point micro-motion state feature vectors. In particular, compared with ordinary convolution, dilated convolution can obtain richer contextual information, and the dilated convolutional neural network can more keenly perceive and extract the slight movement changes of muscle points in the facial muscle unit area image, without missing key subtle features, so as to better achieve high-precision recognition of individual emotional states.

[0038] Specifically, in the embodiment of the present application, the displacement comprehensive transmission unit 132 is used to perform a core facial muscle point displacement time series comprehensive transmission on the time series set of the micro-motion state characteristics of the multiple facial muscle points to obtain a comprehensive representation of the displacement state of the facial muscle point micro-motion. In particular, considering that the micro-motion of facial muscle points is a dynamic process, there is a close temporal correlation between the micro-motion characteristics of muscle points at different time points. That is, the micro-motion state of facial muscle points changes dynamically at different times, and the motion characteristics of each muscle point contain important information over time. Taking the corrugator supercilii as an example, the degree of its contraction and relaxation will change over time over a period of time. Therefore, in order to clarify the temporal trend of muscle point movement, such as whether it is gradually tense or gradually relaxed, and then analyze the potential connection between this change and the psychological state, for example, anxiety may start with slight tension and gradually develop into obvious frowning or clenching of teeth. The present application performs muscle displacement state time series information transmission based on core information focusing on each time series set of the micro-motion state feature vectors of the multiple facial muscle points in the time series set of the micro-motion state feature vectors to obtain multiple facial muscle points micro-motion displacement state information transmission feature vectors. In this way, it is possible to screen out information that is truly important for judging mental health, highlight key features, and eliminate interference, so as to better utilize the time series correlation between the micro-motion features of the muscle points at each time point and explore the laws of psychological state changes behind muscle movements.

[0039] Figure 4 FIG. 1 is a block diagram of a displacement comprehensive transmission unit in an AI-based mental health assessment system according to an embodiment of the present application. Specifically, Figure 4 As shown, the displacement comprehensive transmission unit 132 includes: a facial muscle point micro-motion displacement transmission subunit 1321, which is used to perform muscle displacement state timing information transmission based on core information focusing on each time series set of facial muscle point micro-motion state feature vectors in the time series set of the multiple facial muscle point micro-motion state feature vectors to obtain multiple facial muscle point micro-motion displacement state information transmission feature vectors; a facial muscle point micro-motion displacement global analysis subunit 1322, which is used to perform global facial muscle point micro-motion displacement comprehensive analysis on the multiple facial muscle point micro-motion displacement state information transmission feature vectors to obtain a facial muscle point micro-motion displacement state comprehensive representation vector as the facial muscle point micro-motion displacement state comprehensive representation.

[0040] Specifically, the facial muscle point micro-motion displacement transmission subunit 1321 is used to perform muscle displacement state time series information transmission based on core information focusing on each time series set of the facial muscle point micro-motion state feature vectors in the time series set of the multiple facial muscle point micro-motion state feature vectors to obtain multiple facial muscle point micro-motion displacement state information transmission feature vectors. In particular, considering that the micro-motion of facial muscle points is a dynamic process, there is a close time series correlation between the micro-motion features of muscle points at different time points. That is, the micro-motion state of facial muscle points changes dynamically at different times, and the motion features of each muscle point contain important information over time. Taking the corrugator muscle as an example, the degree of its contraction and relaxation will change over time over a period of time. Therefore, in order to clarify the time series trend of muscle point movement, such as whether it is gradually tense or gradually relaxed, and then analyze the potential connection between this change and psychological state, for example, anxiety may start from slight tension and gradually develop into obvious frowning or clenching of teeth. The present application performs muscle displacement state time series information transmission based on core information focusing on each time series set of the micro-motion state feature vectors of the multiple facial muscle points in the time series set of the micro-motion state feature vectors to obtain multiple facial muscle points micro-motion displacement state information transmission feature vectors. In this way, it is possible to screen out information that is truly important for judging mental health, highlight key features, and eliminate interference, so as to better utilize the time series correlation between the micro-motion features of the muscle points at each time point and explore the laws of psychological state changes behind muscle movements.

[0041] Specifically, in the embodiment of the present application, the facial muscle point micro-movement displacement transmission subunit 1321 includes: a facial muscle point micro-movement state coarse-grained encoding secondary subunit, which is used to perform coarse-grained aggregation of micro-movement state information kernels on the time series set of the facial muscle point micro-movement state feature vectors to obtain a coarse-grained aggregation coding vector of facial muscle point micro-movement state features; a facial muscle point micro-movement state feature fine-grained compensation secondary subunit, which is used to perform muscle point micro-movement state fine-grained weight compensation on the time series set of the facial muscle point micro-movement state feature vectors based on the coarse-grained aggregation coding vector of the facial muscle point micro-movement state features to obtain a facial muscle point micro-movement state feature fine-grained compensation aggregation coding vector; a coarse-fine granularity interactive analysis secondary subunit, which is used to interactively analyze the facial muscle point micro-movement state feature fine-grained compensation aggregation coding vector and the facial muscle point micro-movement state feature coarse-grained aggregation coding vector to obtain the facial muscle point micro-movement displacement state information transmission feature vector.

[0042] Specifically, the facial muscle point micro-motion state coarse-grained encoding secondary subunit is used to perform micro-motion state information kernel coarse-grained aggregation on the time series set of the facial muscle point micro-motion state feature vector to obtain the facial muscle point micro-motion state feature coarse-grained aggregation encoding vector, which can be expressed as follows:

[0043] in, is the time series set of the facial muscle point micro-motion state feature vectors, , , and are the first, second, and third facial muscle point micro-motion state feature vectors in the time series set. and facial muscle point micro-motion state feature vector, and Take The maximum and minimum values ​​of is the first in the time series set of the convergence values ​​of the facial muscle point micro-motion state features The convergence value of the facial muscle point micro-motion state features, is the normalization function, is the time series set of normalized facial muscle point micro-motion state feature aggregation values. Normalized facial muscle point micro-motion state feature convergence value, is the The number of vectors in It is the coarse-grained aggregation encoding vector of facial muscle point micro-motion state features.

[0044] It should be understood that the amount of data of the time series set of facial muscle point micro-motion state feature vectors is huge and complicated, and it is difficult to grasp the key information by direct processing. By performing coarse-grained aggregation of micro-motion state information kernels on the time series set of the facial muscle point micro-motion state feature vectors, the concept of information kernels can be utilized, just like building a macro framework in complex data, effectively integrating a large amount of facial muscle point micro-motion information in a high-dimensional space, and compressively modeling the global features, so as to quickly grasp the overall feature trend, and obtain the coarse-grained aggregation encoding vector of the facial muscle point micro-motion state feature, so that subsequent calculations and analysis can focus on key information, providing a basis for subsequent in-depth analysis.

[0045] More specifically, in an embodiment of the present application, the facial muscle point micro-motion state feature fine-grained compensation secondary subunit includes: a facial muscle point micro-motion state compensation factor calculation tertiary subunit, which is used to calculate the core convergence compensation factor of each facial muscle point micro-motion state feature vector in the time series set of the facial muscle point micro-motion state feature vector relative to the facial muscle point micro-motion state feature coarse-grained convergence coding vector to obtain a time series set of facial muscle point micro-motion state feature core convergence compensation factors; a weight factor calculation tertiary subunit, which is used to perform micro-motion state gating function explicit compensation on the time series set of the facial muscle point micro-motion state feature core convergence compensation factors to obtain a time series set of facial muscle point micro-motion state feature core convergence compensation weight factors; a compensation convergence tertiary subunit, which is used to perform muscle point micro-motion state fine-grained temporal dynamic compensation convergence on the time series set of the facial muscle point micro-motion state feature core convergence compensation weight factors, the facial muscle point micro-motion state feature coarse-grained convergence coding vector and the time series set of the facial muscle point micro-motion state feature vector to obtain the facial muscle point micro-motion state feature fine-grained compensation convergence coding vector.

[0046] More specifically, the facial muscle point micro-movement state compensation factor calculation three-level sub-unit is used to: perform deep extraction on the facial muscle point micro-movement state feature vector and the facial muscle point micro-movement state feature coarse-grained converged coding vector respectively to obtain the facial muscle point micro-movement state deep feature vector and the facial muscle point micro-movement state feature coarse-grained converged deep coding vector; calculate the facial muscle point micro-movement state feature differential compensation coding vector between the facial muscle point micro-movement state deep feature vector and the facial muscle point micro-movement state feature coarse-grained converged deep coding vector; multiply the facial muscle point micro-movement state feature differential compensation coding vector with the facial muscle point micro-movement state weight matrix and then add the facial muscle point micro-movement state bias value position by position to obtain the facial muscle point micro-movement state feature differential compensation correction vector; multiply the facial muscle point micro-movement state feature differential compensation correction vector with the scoring weight vector to obtain the facial muscle point micro-movement state feature core convergence compensation factor corresponding to the facial muscle point micro-movement state feature vector. The above process is expressed by the formula:

[0047] in, is the first in the time series set of the facial muscle point micro-motion state feature vector facial muscle point micro-motion state feature vector, is the coarse-grained aggregation encoding vector of facial muscle point micro-motion state features, is the point convolutional coding, is the activation function, and They are and The corresponding weight matrix is, yes The corresponding deep feature vector of the facial muscle point micro-motion state, It is the deep encoding vector of the coarse-grained aggregation of facial muscle point micro-motion state features, It is subtracted by position point. To take the absolute value operation, for and The facial muscle point micro-motion state characteristic difference compensation correction vector between yes The corresponding facial muscle point micro-motion state weight matrix, is matrix multiplication, for The corresponding facial muscle point micro-motion state bias value, for The corresponding scoring weight vector, It is the first in the time series set of facial muscle point micro-motion state feature kernel convergence compensation factors The characteristic kernel convergence compensation factor of the micro-motion state of facial muscle points.

[0048] Accordingly, considering that the coarse-grained converged coding vector can reflect the overall characteristics, the unique subtle change information of each muscle point may be weakened or lost in this process. By calculating the kernel convergence compensation factor of each facial muscle point micromotion state feature vector in the time series set of the facial muscle point micromotion state feature vector relative to the coarse-grained converged coding vector of the facial muscle point micromotion state feature, it is possible to measure the difference between the single muscle point micromotion state feature vector and the overall summary vector at each moment, and through this difference, retrieve the personalized information that is ignored in the global summary, and provide local supplement for the subsequent accurate characterization of the muscle point micromotion features.

[0049] In particular, in response to the fact that the binary norm of the deep feature vector of the facial muscle point micro-motion state is less than the binary norm of the deep coding vector of the coarse-grained convergence of the facial muscle point micro-motion state features, the ratio between the binary norm of the deep feature vector of the facial muscle point micro-motion state and the binary norm of the deep coding vector of the coarse-grained convergence of the facial muscle point micro-motion state features is added with a constant one, and the logarithmic function value obtained by taking the logarithm with base 2 is used as the bias value of the facial muscle point micro-motion state; in response to the fact that the binary norm of the deep feature vector of the facial muscle point micro-motion state is greater than or equal to the binary norm of the deep coding vector of the coarse-grained convergence of the facial muscle point micro-motion state features, the ratio of the binary norm of the deep feature vector of the facial muscle point micro-motion state and the binary norm of the deep coding vector of the coarse-grained convergence of the facial muscle point micro-motion state features is used as the bias value of the facial muscle point micro-motion state. This process is expressed by the formula:

[0050] in, yes The corresponding deep feature vector of the facial muscle point micro-motion state, It is the deep encoding vector of the coarse-grained aggregation of facial muscle point micro-motion state features, To calculate the second norm of a vector, is the logarithmic function value with base 2, for The corresponding facial muscle point micro-motion state bias value.

[0051] That is, here, for the deviation compensation between the facial muscle point micro-motion state feature vector and the facial muscle point micro-motion state feature coarse-grained converged encoding vector, the performance deviation of the kernel convergence strategy as a situational strategy can be measured by the regret metric based on the information kernel compression hypothesis in the quantified kernel convergence decision process, that is, the game counterfactual regret value. Specifically, the counterfactual regret value is provided with a normalized description of the decision point loss based on the strategy action, that is, the vector norm representation of the facial muscle point micro-motion state feature vector and the facial muscle point micro-motion state feature coarse-grained converged encoding vector through the vector norm representation, and then, for the possible vector distribution action game scene differences, the compensation rule of the node personalized information is corrected by the information distribution degree of the regret value and the relative distribution amplitude of the regret value, so that the personalized information of the facial muscle point micro-motion state feature is considered as the untaken action in the decision, and the biased compensation is performed based on the information kernel convergence hypothesis of its potential benefits.

[0052] Specifically, the weight factor calculation tertiary subunit is used to perform micro-motion state gating function compensation on the time series set of the facial muscle point micro-motion state characteristic core convergence compensation factor to obtain the time series set of facial muscle point micro-motion state characteristic core convergence compensation weight factors. The process is expressed by the formula:

[0053] in, It is the first in the time series set of facial muscle point micro-motion state feature kernel convergence compensation factors The facial muscle point micro-motion state characteristic kernel convergence compensation factor, For Perform gate compensation. is the preset threshold, is the first in the time series set of facial muscle point micro-motion state feature kernel convergence compensation weight factors The facial muscle point micro-motion state feature kernel converges the compensation weight factor.

[0054] It should be understood that not all information contained in the kernel convergence compensation factor is equally important, and there may be some redundant information caused by measurement errors or irrelevant factors. The time series set of the kernel convergence compensation factor of the facial muscle point micro-motion state feature is compensated by the micro-motion state gating function, and the kernel convergence compensation factor can be screened and weighted based on the ability of the gating function to dynamically select information under nonlinear constraints, highlighting the key difference information that can truly reflect the relationship between muscle point micro-motion and mental health status, and suppressing useless or interfering information, so that the model can grasp the effective information more accurately, and obtain the time series set of the kernel convergence compensation weight factor of the facial muscle point micro-motion state feature.

[0055] Specifically, the compensation convergence three-level subunit is used to perform muscle point micro-motion state fine-grained temporal dynamic compensation convergence on the time series set of the facial muscle point micro-motion state feature core convergence compensation weight factor, the facial muscle point micro-motion state feature coarse-grained convergence encoding vector and the facial muscle point micro-motion state feature vector to obtain the facial muscle point micro-motion state feature fine-grained compensation convergence encoding vector. The process is expressed by the formula:

[0056] in, is the first in the time series set of the facial muscle point micro-motion state feature vector facial muscle point micro-motion state feature vector, is the first in the time series set of facial muscle point micro-motion state feature kernel convergence compensation weight factors The facial muscle point micro-motion state feature kernel converges the compensation weight factor, is the coarse-grained aggregation encoding vector of facial muscle point micro-motion state features, It is the fine-grained compensation aggregation encoding vector of facial muscle point micro-motion state features.

[0057] Accordingly, in order to fully and accurately describe the micro-motion state of facial muscle points, it is necessary to comprehensively consider the overall features (coarse-grained converged encoding vectors), the original detailed features (a time series set of facial muscle point micro-motion state feature vectors), and the screened and weighted local difference features (a time series set of kernel convergence compensation weight factors). Through the fine-grained temporal dynamic compensation convergence of muscle point micro-motion states, the expression of muscle point micro-motion features can be dynamically adjusted according to the weight of this information at different times, thereby generating a fine-grained feature vector that is more in line with the actual muscle movement state.

[0058] Specifically, the coarse-grained and fine-grained interactive analysis secondary subunit is used to interactively analyze the fine-grained compensation convergence coding vector of the facial muscle point micro-motion state feature and the coarse-grained convergence coding vector of the facial muscle point micro-motion state feature to obtain the facial muscle point micro-motion displacement state information transmission feature vector. The process is expressed by the formula:

[0059] in, is the coarse-grained aggregation encoding vector of facial muscle point micro-motion state features, is the fine-grained compensation aggregation encoding vector of facial muscle point micro-motion state features, and is the weighted hyperparameter, It is the characteristic vector for transmitting the micro-displacement state information of the facial muscle points.

[0060] Finally, since the fine-grained compensation convergence encoding vector focuses on local details and fine-tuning, and the coarse-grained convergence encoding vector focuses on overall feature grasping, the interactive analysis of the two can comprehensively examine the micro-motion state of facial muscle points from both global and local dimensions, dig out the deep connection and coordinated change information between the whole and the local, and further improve the understanding of the micro-motion displacement state of muscle points, so as to obtain a more comprehensive and accurate feature vector reflecting the relationship between muscle movement and mental health.

[0061] Specifically, in the embodiment of the present application, the facial muscle point micro-displacement global analysis subunit 1322 is used to perform a global facial muscle point micro-displacement comprehensive analysis on the multiple facial muscle point micro-displacement state information transmission feature vectors to obtain a facial muscle point micro-displacement state comprehensive representation vector as the facial muscle point micro-displacement state comprehensive representation. More specifically, in the embodiment of the present application, the facial muscle point micro-displacement global analysis subunit is used to: pass the multiple facial muscle point micro-displacement state information transmission feature vectors through the facial muscle point micro-displacement comprehensive analysis module based on LSTM to obtain the facial muscle point micro-displacement state comprehensive representation vector. Accordingly, considering the actual situation of facial muscle movement, the movement of different muscle points does not occur in isolation, but cooperates and synergizes with each other to express various expressions and psychological states. For example, when a person is surprised, not only will the eyebrows rise, the eyes will open wide, and the mouth may also open slightly. The movements of these muscle points are interrelated in time and space, and together constitute the expression of "surprise". This association is not only reflected in the coordinated actions of different muscle points at the same time, but also in the sequence and change trend of muscle point movements at different times. Based on this, the present application obtains a comprehensive representation vector of the micro-displacement state of facial muscle points by passing the feature vectors of the micro-displacement state information of the multiple facial muscle points through a comprehensive analysis module of micro-displacement of facial muscle points based on LSTM. It can be understood that LSTM (Long Short-Term Memory Network) has a unique structure and gating mechanism, which enables it to effectively capture this contextual association influence. Its memory unit can store information for a long time, and the forget gate, input gate and output gate can control the inflow, outflow and retention of information. When processing the feature vectors of the micro-displacement state information transmission of multiple facial muscle points, LSTM can remember the motion state information of each muscle point at the previous moment, and update and adjust it according to the current input information. In this way, it can comprehensively consider the motion of different muscle points in the time dimension and capture the complex contextual associations between them. For example, LSTM can learn how the movement of eye muscle points affects the subsequent movement of the corner of the mouth muscle points during a certain emotional change, and the changing rules of this mutual influence in the time series. By capturing these contextual association influences, LSTM can effectively integrate the information of multiple facial muscle points, and ultimately obtain a comprehensive representation vector that can fully reflect the micro-displacement state of facial muscle points, providing strong support for subsequent accurate analysis and judgment of psychological states.

[0062] Specifically, the evaluation module 140 is used to obtain a mental health evaluation result for indicating a mental health type label based on the comprehensive representation of the facial muscle point micro-motion displacement state. Specifically, in the embodiment of the present application, the evaluation module is used to: pass the facial muscle point micro-motion displacement state comprehensive representation vector through a classifier-based mental health evaluator to obtain the mental health evaluation result, and the mental health evaluation result is used to indicate a mental health type label. That is, the facial muscle point micro-motion displacement state comprehensive representation vector obtained by performing a displacement comprehensive analysis using a plurality of facial muscle point micro-motion displacement state information transmission feature vectors is classified and processed to intelligently judge the mental health type. It should be understood that the classifier is an effective data classification tool, which can classify and judge these complex vector data based on existing training data and algorithms. Through a large amount of sample data training, the classifier can learn the mapping relationship between different facial muscle movement feature vectors and various mental health types. For example, during the training process, the classifier will analyze the facial muscle point micro-motion displacement state comprehensive representation vectors of a large number of people with known mental health conditions, thereby summarizing the corresponding rules between specific vector patterns and mental health types. In this way, when a new comprehensive representation vector is input, the classifier can quickly and accurately determine the mental health type it belongs to based on the learned rules, and convert complex data into intuitive and easy-to-understand mental health type labels. In particular, the mental health type labels here can be anxiety, inhibition, stress, etc. The mental health type labels can provide a preliminary assessment reference for professional psychologists and counselors, helping them to quickly understand the general mental health status of the subjects, so as to decide whether further in-depth assessment or appropriate intervention measures are needed. For schools, companies and other institutions, these labels can help them conduct preliminary screening of the mental health status of students or employees, and promptly identify individuals who may have psychological problems, so as to arrange subsequent counseling or treatment.

[0063] In summary, the AI-based mental health assessment system 100 based on the embodiment of the present application is explained, which collects the facial data information (facial video stream) of the target object through the camera, uses the AI-based image recognition and analysis algorithm to perform key frame sampling and facial recognition on the facial data information, and then performs muscle unit partitioning and micro-motion state feature extraction on each facial area image after recognition, and then performs core muscle displacement state time series transmission on the extracted time series set of micro-motion state features of each facial muscle point, so as to intelligently judge the mental health type according to the comprehensive characterization between the micro-motion displacement state information transmission features of each transmitted facial muscle point. The present application uses the camera to collect data, and after processing by the AI ​​algorithm, it can quickly complete key frame sampling and other operations to achieve efficient detection. At the same time, it accurately captures subtle changes in facial muscles, avoids subjective interference from patients, and improves the accuracy of evaluation.

[0064] In particular, in another specific embodiment of the present application, in addition to the analysis of facial muscle movements, optical imaging technology of blood flow information is also an important component. This technology captures blood flow changes under the facial skin in a non-invasive manner, providing additional physiological indicator data for mental health assessment. It should be understood that optical imaging technology (such as near-infrared spectral imaging, NIRS) uses the interaction between light and tissue to obtain physiological information in vivo. Specifically, the technology emits light of a specific wavelength (usually in the near-infrared range), which penetrates the skin and is absorbed and scattered by subcutaneous tissue. Different tissue components have different absorption and scattering properties of light, so the state of the tissue can be inferred by detecting the reflected or transmitted light signals. When an individual experiences emotional fluctuations, the blood flow under the facial skin changes. For example, in an anxious or tense state, the sympathetic nervous system is activated, resulting in vasoconstriction and reduced blood flow; while in a relaxed or happy state, the parasympathetic nervous system is dominant, vasodilation, and increased blood flow. Therefore, this application uses blood flow information optical imaging technology to collect different color signals reflected by the human facial cortex at different spectral wavelengths through a high-frame rate camera, studies the changing characteristics of facial blood information, uses image time filtering to decompose and amplify changes that the human eye cannot distinguish, and then uses a unique algorithm to calculate various human physiological indicators such as heart rate, blood oxygen, respiration, etc.

[0065] Specifically, the calculation of physiological indicators by optical imaging technology of blood flow information can be achieved in the following ways: The first is the selection and setting of equipment. High frame rate cameras are the key acquisition equipment of this technology. They need to have high frame rate shooting capabilities. Common frame rates can reach hundreds of frames per second or even higher to ensure that the rapid changes in facial blood information can be accurately captured. At the same time, its spectral response range must match the specific spectral wavelength used for detection, which is generally concentrated in the near-infrared spectrum region, because the light in this region can penetrate the skin tissue better, and the hemoglobin in the blood has obvious absorption and scattering characteristics of the light in this region, which is conducive to obtaining rich blood information. In actual application scenarios, it is also necessary to adjust the camera parameters such as aperture, shutter speed and gain according to the specific situation to ensure that the collected image has sufficient clarity and contrast. In addition, in order to accurately collect signals of different spectral wavelengths, corresponding optical filters will be equipped to filter out light in a specific wavelength range so that the camera can focus on the collection of the required spectrum.

[0066] Before collecting, make sure the person being tested is in a suitable state and environment. The person being tested needs to stay quiet and relaxed, and avoid factors such as strenuous exercise and emotional excitement that may cause abnormal fluctuations in facial blood flow and affect the accuracy of the test results. The testing environment should be kept as stable as possible, avoiding direct strong light and electromagnetic interference, to create good conditions for the stable operation of the equipment and signal collection.

[0067] During the acquisition process, the high-frame rate camera starts working and emits light of specific spectral wavelengths to the human face. These lights penetrate the facial skin and interact with the blood in the subcutaneous tissue. The hemoglobin in the blood has different absorption and scattering characteristics for light of different wavelengths. When the light shines on the blood, part of the light is absorbed by the hemoglobin, and part of the light is scattered and finally reflected back from the facial cortex. The high-frame rate camera continuously collects these reflected light signals at a set high frame rate, converts them into different color signals and records them. Since the intensity and color of light reflected back at different wavelengths are different, these differences contain information about the blood on the face, such as the hemoglobin content in the blood, blood flow rate, etc.

[0068] The collected raw image data contains a lot of complex information, among which the facial blood information change characteristics are relatively weak and mixed with various noises and interference factors. In order to extract useful blood information, it is necessary to use image time filtering technology for processing. Image time filtering is based on the continuity and regularity of blood information changes in the time dimension. By designing a specific filter, the continuously collected image sequence is filtered. For example, the use of temporal median filtering can effectively remove random noise in the image and retain the trend of blood information changes; and the time weighted filtering based on the Gaussian function can perform different degrees of weighted processing on the image according to the distance of time, and highlight the recent blood information changes. Through these filtering operations, the characteristics of blood information changes can be strengthened, other interference factors can be suppressed, and the originally weak and difficult to distinguish facial blood information changes can be made more obvious.

[0069] On the basis of filtering, the image is further processed using image decomposition and amplification technology. This technology is mainly based on mathematical algorithms. By analyzing and transforming the filtered image, the subtle changes hidden in the image can be amplified and displayed. For example, using wavelet transform technology, the image can be decomposed into different frequency sub-bands, and the sub-bands containing the characteristics of blood information changes can be extracted separately and amplified by adjusting relevant parameters. In this way, facial images that originally seemed almost unchanged to the human eye can clearly show the dynamic changes of blood information, such as the surging of blood flow, the expansion and contraction of blood vessels, etc. after image decomposition and amplification.

[0070] After filtering and decomposition and amplification, a relatively clear and prominent facial blood information change image sequence is obtained. Next, a unique algorithm is used to calculate various physiological indicators. Taking heart rate calculation as an example, it is mainly based on the relationship between blood volume changes and heart beats. By analyzing the blood volume change curve of specific facial areas in the image (such as the nose tip, cheeks and other vascular areas) over time, using signal processing algorithms such as Fourier transform, the blood volume change signal in the time domain is converted to the frequency domain, and the frequency components corresponding to the heart rate are extracted from it, and then the heart rate value is calculated.

[0071] The calculation of blood oxygen content is based on the difference in hemoglobin's absorption of light of different wavelengths. In the near-infrared spectral region, oxygenated hemoglobin and deoxygenated hemoglobin have different absorption coefficients for light of specific wavelengths. By collecting the intensity information of light reflected back at different wavelengths and using the Beer-Lambert law to establish a mathematical model, the relative content of oxygenated hemoglobin and deoxygenated hemoglobin is calculated, thereby obtaining the blood oxygen saturation index.

[0072] For breathing detection, the main focus is on the relationship between the tiny movements of the chest and face during breathing and the resulting changes in facial blood flow. By analyzing the subtle changes in overall brightness or specific areas in facial image sequences, combined with the hemodynamic changes caused by breathing movements, pattern recognition algorithms such as support vector machines (SVM) in machine learning or convolutional neural networks (CNN) in deep learning are used to classify and identify these change patterns, thereby inferring indicators such as breathing frequency and breathing depth.

[0073] During the entire implementation process, the calculated physiological indicators need to be calibrated and verified. By conducting comparative tests with traditional physiological detection equipment (such as ECG monitors, pulse oximeters, etc.), a large amount of experimental data is collected, and a calibration model is established to correct the physiological indicators calculated by blood flow information optical imaging technology to improve its accuracy and reliability.

[0074] In this way, the information provided by the two technologies can verify and complement each other. In an anxious state, not only will there be specific facial expressions (such as frowning), but there will also be changes in blood flow (such as facial vasoconstriction). By combining these two types of information, emotional states can be identified more comprehensively and accurately. Through parallel computing, this relationship can be used to cross-validate and comprehensively analyze the data. Finally, based on the analysis results of a variety of indicators related to mental health and emotions, the individual's professional psychological qualities, such as self-confidence, self-control, motivation, calmness, anxiety, etc., can be evaluated from all aspects and angles.

[0075] In particular, professional quality and ability assessment can be applied in the following scenarios: 1. Career guidance for college students: It can quickly analyze students' professional psychological quality indicators, evaluate their personality traits, and recommend suitable career directions accordingly.

[0076] 2. Enterprise recruitment and job matching: In the process of enterprise recruitment, a comprehensive professional psychological quality assessment can be conducted on applicants according to the specific psychological quality requirements of different positions, so as to select the most suitable talents. At the same time, the system can also be used in cadre promotion or political assessment to ensure that candidates meet the psychological quality requirements of the new position.

[0077] 3. Pre-job testing for high-risk special enterprises: For industries such as steelmaking, glass manufacturing, and chemicals, emotional health and stability tests are conducted on employees before they start work every day to ensure safe production and avoid accident risks caused by personal emotional safety and stability issues.

[0078] 4. Job assignment for new recruits: Conduct psychological assessments on new recruits, and assign those with strong self-discipline, high stress resistance and teamwork spirit to combat units with high requirements for discipline and teamwork based on the results of their professional psychological quality assessments; assign new recruits with high concentration and quick response to technical positions to achieve job matching and enhance the overall combat effectiveness of the army.

[0079] As described above, the AI-based mental health assessment system 100 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an AI-based mental health assessment algorithm. In one possible implementation, the AI-based mental health assessment system 100 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the AI-based mental health assessment system 100 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the AI-based mental health assessment system 100 can also be one of the many hardware modules of the wireless terminal.

[0080] Alternatively, in another example, the AI-based mental health assessment system 100 and the wireless terminal may also be separate devices, and the AI-based mental health assessment system 100 may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0081] The above descriptions of various implementations of the present disclosure are exemplary and not exhaustive. The disclosure is not limited to the disclosed implementations, and many modifications and variations are obvious to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The terms used herein are selected to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various practical methods disclosed herein.

Claims

1. A mental health assessment system based on AI, characterized in that: include: A facial acquisition module is used to acquire facial data information of a target object through a camera, wherein the facial data information is a facial video stream; A recognition module, used for performing key-frame-based facial region recognition on the facial data information to obtain a time-series set of facial region images; A displacement state characterization module is used to perform a comprehensive analysis of facial muscle point micro-motion displacement on the time series set of the facial region image to obtain a comprehensive characterization of facial muscle point micro-motion displacement state, wherein the displacement state characterization module includes: a muscle point feature extraction unit, used to perform muscle point micro-motion feature extraction on the time series set of the facial region image to obtain a time series set of multiple facial muscle point micro-motion state features; a displacement comprehensive transmission unit, used to perform a core facial muscle point displacement time series comprehensive transmission on the time series set of the multiple facial muscle point micro-motion state features to obtain the comprehensive characterization of the facial muscle point micro-motion displacement state; The evaluation module is used to obtain a mental health evaluation result for indicating a mental health type label based on the comprehensive representation of the micro-displacement state of the facial muscle points.

2. The AI-based mental health assessment system according to claim 1, characterized in that: The identification module is used to: Performing key frame sampling on the facial data information to obtain a time sequence set of facial key frames; Facial region recognition is performed on each facial key frame in the time series set of facial key frames to obtain the time series set of facial region images.

3. The AI-based mental health assessment system according to claim 2, characterized in that: The muscle point feature extraction unit comprises: A muscle unit partitioning subunit, used to perform muscle unit partitioning on each facial region image in the time series set of facial region images to obtain a time series set of multiple facial muscle unit region images; A muscle point micro-motion state feature extraction subunit is used to pass each facial muscle unit area image in the time series set of the multiple facial muscle unit area images through a muscle point micro-motion feature extractor based on a hole convolutional neural network model to obtain a time series set of multiple facial muscle point micro-motion state feature vectors as the time series set of the multiple facial muscle point micro-motion state features.

4. The AI-based mental health assessment system according to claim 1, characterized in that: The displacement integrated transmission unit comprises: A facial muscle point micro-motion displacement transmission subunit is used to perform muscle displacement state time series information transmission based on core information focusing on each time series set of facial muscle point micro-motion state feature vectors in the time series set of the plurality of facial muscle point micro-motion state feature vectors to obtain a plurality of facial muscle point micro-motion displacement state information transmission feature vectors; The facial muscle point micro-motion displacement global analysis subunit is used to perform a global facial muscle point micro-motion displacement comprehensive analysis on the multiple facial muscle point micro-motion displacement state information transmission feature vectors to obtain a facial muscle point micro-motion displacement state comprehensive representation vector as the facial muscle point micro-motion displacement state comprehensive representation.

5. The AI-based mental health assessment system according to claim 4, characterized in that: The facial muscle point micro-displacement transmission subunit comprises: A facial muscle point micro-motion state coarse-grained encoding secondary subunit is used to perform micro-motion state information kernel coarse-grained aggregation on the time series set of the facial muscle point micro-motion state feature vectors to obtain a facial muscle point micro-motion state feature coarse-grained aggregation encoding vector; A facial muscle point micro-motion state feature fine-grained compensation secondary subunit is used to perform muscle point micro-motion state fine-grained weight compensation on the time series set of the facial muscle point micro-motion state feature vectors based on the facial muscle point micro-motion state feature coarse-grained converged coding vector to obtain a facial muscle point micro-motion state feature fine-grained compensation converged coding vector; The coarse-grained and fine-grained interactive analysis secondary subunit is used to interactively analyze the fine-grained compensation aggregation coding vector of the facial muscle point micro-motion state characteristics and the coarse-grained aggregation coding vector of the facial muscle point micro-motion state characteristics to obtain the facial muscle point micro-motion displacement state information transmission feature vector.

6. The AI-based mental health assessment system according to claim 5, characterized in that: The facial muscle point micro-motion state feature fine-grained compensation secondary sub-unit includes: The facial muscle point micro-motion state compensation factor calculation tertiary subunit is used to calculate the kernel convergence compensation factor of each facial muscle point micro-motion state feature vector in the time series set of the facial muscle point micro-motion state feature vector relative to the facial muscle point micro-motion state feature coarse-grained convergence encoding vector to obtain the time series set of the facial muscle point micro-motion state feature kernel convergence compensation factor; The weight factor calculation tertiary subunit is used to perform micro-motion state gating function compensation on the time series set of the facial muscle point micro-motion state characteristic core convergence compensation factor to obtain the time series set of facial muscle point micro-motion state characteristic core convergence compensation weight factors; The compensation convergence tertiary sub-unit is used to perform muscle point micro-motion state fine-grained temporal dynamic compensation convergence on the time series set of the facial muscle point micro-motion state feature core convergence compensation weight factors, the facial muscle point micro-motion state feature coarse-grained convergence coding vector and the time series set of the facial muscle point micro-motion state feature vector to obtain the facial muscle point micro-motion state feature fine-grained compensation convergence coding vector.

7. The AI-based mental health assessment system according to claim 6, characterized in that: The facial muscle point micro-motion state compensation factor calculation three-level sub-unit is used for: Respectively performing deep extraction on the facial muscle point micro-motion state feature vector and the facial muscle point micro-motion state feature coarse-grained aggregation coding vector to obtain a facial muscle point micro-motion state deep feature vector and a facial muscle point micro-motion state feature coarse-grained aggregation deep coding vector; Calculate a facial muscle point micro-motion state feature differential compensation coding vector between the facial muscle point micro-motion state deep feature vector and the facial muscle point micro-motion state feature coarse-grained converged deep coding vector; The facial muscle point micro-motion state feature differential compensation coding vector is multiplied by the facial muscle point micro-motion state weight matrix, and then the resultant is added position by position to the facial muscle point micro-motion state bias value to obtain a facial muscle point micro-motion state feature differential compensation correction vector; The facial muscle point micro-motion state feature differential compensation correction vector is multiplied by the scoring weight vector to obtain the facial muscle point micro-motion state feature kernel convergence compensation factor corresponding to the facial muscle point micro-motion state feature vector.

8. The AI-based mental health assessment system according to claim 7, characterized in that: In response to the fact that the second norm of the deep feature vector of the facial muscle point micro-motion state is less than the second norm of the deep encoding vector of the coarse-grained aggregation of the facial muscle point micro-motion state features, a logarithmic function value obtained by taking the logarithm with base 2 as the logarithm of the ratio between the second norm of the deep feature vector of the facial muscle point micro-motion state and the second norm of the deep encoding vector of the coarse-grained aggregation of the facial muscle point micro-motion state features is added with a constant one as the bias value of the facial muscle point micro-motion state; In response to the binary norm of the deep feature vector of the facial muscle point micro-movement state being greater than or equal to the binary norm of the coarse-grained deep coding vector of the facial muscle point micro-movement state features, the ratio of the binary norm of the deep feature vector of the facial muscle point micro-movement state and the binary norm of the coarse-grained deep coding vector of the facial muscle point micro-movement state features is used as the facial muscle point micro-movement state bias value.

9. The AI-based mental health assessment system according to claim 8, characterized in that: The facial muscle point micro-motion displacement global analysis subunit is used to: pass the characteristic vectors of the micro-motion displacement state information of the multiple facial muscle points through the facial muscle point micro-motion displacement comprehensive analysis module based on LSTM to obtain the comprehensive representation vector of the facial muscle point micro-motion displacement state.

10. The AI-based mental health assessment system according to claim 9, characterized in that: The evaluation module is used to: pass the comprehensive representation vector of the micro-displacement state of the facial muscle points through a classifier-based mental health evaluator to obtain the mental health evaluation result, and the mental health evaluation result is used to represent the mental health type label.

Images