Auxiliary intelligent detection method for mental diseases through face recognition

Abstract

The invention provides an auxiliary intelligent detection method for mental diseases through face recognition, and relates to the technical field of mental disease detection. The detection method comprises the following steps: creating a plurality of virtual AR test environments based on an AR technology and forming an environment set; testing according to the corresponding environment in the age matching environment set of the detected person; real-time detection is carried out on the face based on testing, and key point extraction is carried out on the recognized face; analyzing facial features of the human face in combination with the key points; creating a face recognition mental disease detection model based on the convolutional neural network; and inputting the obtained facial action and expression features into a face recognition mental disease detection model to assist in judging whether a patient suffers from mental diseases or not. According to the method, the virtual AR test environment is created by adopting the AR technology, so that the problem that the test environment is difficult to match in reality is solved, the space limitation is greatly reduced, the possibility of multiple tests is provided, and the robustness of a test result is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of mental illness detection, and in particular to an auxiliary intelligent detection method for mental illness by face recognition. Background Art

[0002] Mental illness refers to a disease in which the brain is dysfunctional under the influence of various biological, psychological and social environmental factors, resulting in varying degrees of disorders in mental activities such as cognition, emotion, will and behavior as clinical manifestations. Mental health (psychiatric or psychological) diseases include abnormalities in thinking, emotion and behavior. When these fluctuations significantly affect daily life, they are called mental illness. The classification of mental illness is also relatively complex, mainly including organic mental disorders, mental disorders caused by psychoactive substances, mood disorders, neurotic disorders, stress-related disorders, somatoform disorders, personality disorders, etc. Before treating mental illness, it is necessary to test whether you have a mental illness.

[0003] Mental illness can affect the diversity of facial expressions. For example, people with depression may show fewer changes in expression, while people with schizophrenia may show inappropriate or exaggerated expressions. People with mental illness show inconsistency between their expressions and their inner experiences. They may appear happy on the outside, but feel sad or empty on the inside. Some mental illnesses may cause changes in the activity patterns of facial muscles. For example, people with Parkinson's disease may show reduced facial expressions, while facial paralysis may occur in certain mood disorders. Micro-expressions are short-lived facial expressions that usually occur unconsciously, and some people with mental illness cannot effectively control or hide these subtle expressions. Based on this, face recognition technology can be used to assist in determining whether a person has a mental illness.

[0004] Existing methods for detecting mental illness are diverse, such as clinical interviews: relying on the doctor's experience and the patient's subjective description; questionnaires and self-rating scales: assessing patients' symptoms by asking them to answer a series of questions; biomarker testing: looking for biological indicators related to mental illness, such as genes, brain imaging or blood tests; neuroimaging technology: such as functional magnetic resonance imaging (fMRI) and positron emission tomography (PET), used to observe brain structure and function.

[0005] Although the above-mentioned existing technical solutions can detect mental illness, the two testing methods of clinical interviews, questionnaires and self-assessment scales are usually highly subjective and easily affected by various external factors; while the technical costs of biomarker detection and neuroimaging technology are relatively high, and require high equipment and professional knowledge. Summary of the invention

[0006] The present invention provides an auxiliary intelligent detection method for mental illness by face recognition, which is used to solve the defects of the prior art, such as strong subjectivity, susceptibility to various external factors, high technical cost and high requirements for equipment and professional knowledge.

[0007] On the one hand, the present invention provides an auxiliary intelligent detection method for mental illness by face recognition, comprising:

[0008] S1: Create multiple virtual AR test environments based on AR technology and form an environment set;

[0009] S2: Test the corresponding environment in the environment set according to the age of the person being tested;

[0010] S3: Perform real-time face detection based on the test and extract key points of the recognized face;

[0011] S4: Analyze the facial features of the face in combination with key points, and record facial movements and expression features;

[0012] S5: Create a face recognition model for mental illness detection based on convolutional neural networks;

[0013] S6: Input the obtained facial movement and expression features into the face recognition mental illness detection model to assist in determining whether the patient suffers from a mental illness.

[0014] The above detection method uses AR technology to create a virtual AR test environment, which simplifies the test method, overcomes the difficulty of matching the test environment in reality, greatly reduces the limitation of space, provides the possibility of multiple tests, reduces external influencing factors, enhances the robustness of the test results, and can provide different test spaces for different mental illnesses; analyzes the facial features of the face through key points, and records facial movements and expression features, which improves the stability of the test and makes the results more accurate; and further improves the accuracy of the test results by creating a detection model for mental illnesses based on face recognition using convolutional neural networks.

[0015] In step S1, the method for creating a virtual AR test environment is as follows:

[0016] S10: Classify the people to be tested according to their age groups;

[0017] S11: Determine different situational modes, interactive items and virtual dialogue scenarios based on different age groups;

[0018] S12: Use AR development platform and IDE to model and develop test environment.

[0019] According to the auxiliary intelligent detection method for mental illness by face recognition provided by the present invention, in step S3, the specific method of extracting key points is as follows:

[0020] S30: Real-time facial monitoring of the inspector in the AR virtual environment test;

[0021] S31: converting the obtained real-time facial video image into a grayscale video image;

[0022] S32: Call the preloaded OpenCV face detector to find the face area on the video image and return the detection result;

[0023] S33: Use the MTCNN keypoint detection model to draw a bounding box around the detected face and draw markers at the keypoint locations;

[0024] S34: Calculate the key point coordinates and convert the key point coordinates into relative coordinates relative to the bounding box.

[0025] In step S34, the relative coordinates are converted as follows:

[0026]

[0027] In the formula, (r i ,r j ) are the relative coordinates of any key point, Δi and Δj are the horizontal and vertical offsets of each key point relative to the center of the bounding box, w is the width of the bounding box, and h is the height of the bounding box.

[0028] In step S4, the facial features are analyzed as follows:

[0029] S40: Calculate the distance and angle between the key points according to the facial key point information to analyze the geometric shape of the face;

[0030] S41: Analyze the positions of key points to determine the orientation and tilt angle of the face;

[0031] S42: Group key points to identify different areas of the face;

[0032] S43: Use optical flow to track the motion trajectory of key points in video sequences and analyze facial movements and expression features.

[0033] In step S40, the distance and angle between key points are calculated as follows:

[0034] S400: Calculate the Euclidean distance between facial key points. The formula is as follows:

[0035]

[0036] Where d is the distance between any two key points, (a1, b1) and (a2, b2) are the relative coordinate positions of the two key points;

[0037] S401: Calculate the angle θ between the facial key point solutions using the coordinates of any three key points. The formula is as follows:

[0038]

[0039] In the formula, and are two vectors formed between the three key points, and is the magnitude of two vectors;

[0040] S402: Obtaining a geometric shape formed by the key points through the distance and angle between the key points of the face.

[0041] In step S43, the specific method of tracking the motion trajectory of the key point using the optical flow method is as follows:

[0042] S430: Calculating the motion vector of the key point in the continuous video according to the relative coordinates of the key point;

[0043] S431: Obtain the motion trajectory of the key points by accumulating the optical flow vectors of the key points in the continuous frames. The calculation method of the optical flow vector is as follows:

[0044] X m u+X z v=-X s

[0045] Where, X m and X z are the partial derivatives of image brightness with respect to the m and z directions, u and v are the vectors of optical flow components, X s is the rate of change of brightness over time.

[0046] In step S5, the training method of the detection model is as follows:

[0047] S50: Collect facial images of people with mental illness and healthy individuals;

[0048] S51: Label each face image with its emotional state and mark whether it has a mental illness;

[0049] S52: filtering and noise processing the face image to enhance the data set;

[0050] S53: Divide the dataset into training set, validation set and test set, and use cross-validation to evaluate model performance.

[0051] In step S52, the filtering noise processing method is to convolve each pixel in the image with a Gaussian kernel to calculate a new pixel value; the specific calculation formula is as follows:

[0052]

[0053] Where I(x,y) is the pixel value of the original image at the coordinate point (x,y), I'(x,y) is the new pixel value of the image at the coordinate point (x,y) after Gaussian filtering, k is the radius of the Gaussian kernel, and G'(s,t) is the value of the normalized Gaussian kernel at the coordinate point (s,t).

[0054] In step S53, the cross-validation accuracy formula is as follows:

[0055]

[0056] Where, Accuracy n is the accuracy of the nth cross-validation, and N is the total number of folds.

[0057] The invention provides an auxiliary intelligent detection method for mental illness by face recognition. By using AR technology to create a virtual AR test environment, the problem of difficulty in matching the test environment in reality is solved, the test method is simplified, the space limitation is greatly reduced, the possibility of multiple tests is provided, and the robustness of the test results is enhanced. By filtering and noise processing the face image, the problems of low image quality and low recognizability are solved, the performance of face recognition is improved, and the adaptability under different acquisition conditions is strong. By analyzing the facial features of the face through key points and recording facial movements and expression features, the problem of influence of other uncertain factors in traditional detection is solved, the stability of the test is improved, and the accuracy of the results is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 This is a step diagram of an auxiliary intelligent detection method for mental illness by face recognition provided by an embodiment of the present invention;

[0060] Figure 2 This is a diagram of key point extraction steps of a method for auxiliary intelligent detection of mental illness by face recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The facial data and other related data collection processes of the present invention are authorized by the person being tested or the guardian, and comply with relevant laws and regulations.

[0062] Combine the following Figure 1-Figure 2 The present invention describes an auxiliary intelligent detection method for mental illness by face recognition.

[0063] Figure 1 This is a step diagram of an auxiliary intelligent detection method for mental illness by face recognition provided by an embodiment of the present invention.

[0064] Figure 2 This is a diagram of key point extraction steps of a method for auxiliary intelligent detection of mental illness by face recognition provided by an embodiment of the present invention.

[0065] like Figure 1 As shown, an embodiment of the present invention provides an auxiliary intelligent detection method for mental illness by face recognition, and the method mainly includes the following steps:

[0066] S1: Create multiple virtual AR test environments based on AR technology and form an environment set. The method is as follows:

[0067] S10: The people to be tested are classified according to their age groups, specifically into four categories: teenagers, young people, middle-aged people and the elderly. The age groups corresponding to the three categories are: teenagers under 18 years old, young people between 18 and 40 years old, middle-aged people between 41 and 60 years old, and the elderly over 61 years old.

[0068] S11: Determine different situational patterns, interactive objects, and virtual dialogue scenarios by combining people of different age groups. This enhances the robustness and rigor of mental illness detection.

[0069] S12: Use the AR development platform and a suitable IDE to model and develop the test environment. Use 3D modeling software (such as Blender, Maya, or 3ds Max) to create or import 3D models, and set up scenes in the AR development platform, including lights, cameras, and 3D objects. Write and debug code in the IDE to ensure that the AR test environment runs smoothly and without errors. IDE, or Integrated Development Environment, is a software application that provides programmers with various functions required for software development. It usually includes tools such as code editors, compilers, debuggers, and graphical user interfaces, which work together to help developers improve programming efficiency and simplify the development process.

[0070] S2: Test in the corresponding environment in the environment set according to the age of the person being tested. Conduct separate tests for people in different age groups. People of different ages may show different symptoms of mental illness. Testing in scenarios that are appropriate for their age and social environment can improve the accuracy of symptom identification. By observing people of different age groups in different scenarios, we can have a more comprehensive understanding of how mental illness develops with age and environmental influences. Understanding the disease characteristics of specific age groups in specific scenarios will help develop more effective customized treatment and intervention strategies. Detecting mental illness in specific scenarios, especially for children and adolescents, can promote early identification and intervention, thereby improving long-term prognosis.

[0071] S3: Based on the test, real-time detection of human faces is performed, and key points of the recognized faces are extracted. Figure 2 As shown, the specific method of key point extraction is as follows:

[0072] S30: Real-time facial monitoring of the test person in the AR virtual environment test. This method can be combined with a camera to capture the two-dimensional data of the test person's face. The test person's information is matched by data transmission with the background computer, and the facial video is imported into the video image processing program for the next step of processing.

[0073] S31: Convert the obtained real-time facial video image into a grayscale video image. The video image conversion into a grayscale video image is the process of converting each frame in the color video into a grayscale image. The grayscale image only contains grayscale information, not color information. The brightness of each pixel is represented by a grayscale value, usually ranging from 0 to 255, where 0 represents black and 255 represents white. Using grayscale video images has the advantages of simplifying data processing, reducing computational complexity, reducing the risk of overfitting, and reducing storage requirements.

[0074] S32: Call the preloaded OpenCV face detector to find the face area on the video image and return the detection result. When detecting the face of the video image, it is necessary to perform cyclic detection on the face that is always moving to return different results in different video frames, so as to achieve the purpose of real-time face tracking.

[0075] S33: Use the MTCNN (Multi-task Convolutional Neural Network) key point detection model to draw bounding boxes around the detected face and draw markers at the key point locations. MTCNN is a deep learning model for face detection and facial key point location. It consists of three cascaded networks, namely P-Net (Proposal Network) that quickly generates candidate windows, R-Net (Refinement Network) that refines candidate windows, and O-Net (Output Network) that finally outputs face detection and key point locations. MTCNN learns two tasks, face detection and face alignment, at the same time, which improves the generalization ability and efficiency of the model. Through the three cascaded network structures, MTCNN is able to process images quickly while ensuring high accuracy. This automatically detects faces in the image and accurately marks the location of each face.

[0076] S34: Calculate the coordinates of the key points and convert them into relative coordinates relative to the bounding box. The advantages of using relative coordinates are: scale invariance, that is, relative coordinates can eliminate the influence of different face sizes, so that the key point coordinates do not change with the face size, thereby improving the generalization ability of the model; translation invariance, by converting the key point coordinates into relative positions, it can ensure that the coordinates do not change with the position of the face in the image, making the model insensitive to the position of the face in the image; simplify model training, using relative coordinates can simplify the model training process, because the model does not need to learn facial features of different scales, positions and directions, but only needs to focus on the relative relationship between key points; improve robustness, relative coordinates can improve the robustness of the model to interference factors such as occlusion, lighting changes, and facial expression changes, because these factors usually affect the absolute position of the key points, but have little effect on their relative positions.

[0077] The relative coordinate conversion method is as follows:

[0078] Determine the coordinates of the bounding box, expressed as the coordinates of the upper left corner of the bounding box (i min ,j min ) and the coordinates of the lower right corner of the bounding box (i max ,j max ).

[0079] Calculate the center coordinates of the bounding box using the following formula:

[0080]

[0081] In the formula, (c i ,c j ) are the center coordinates of the bounding box.

[0082] For each key point, calculate its horizontal and vertical offset relative to the center of the bounding box. The formula is as follows:

[0083] Δi=p i -c i

[0084] Δj=p j -c j

[0085] Where Δi is the horizontal offset, Δj is the vertical offset, (p i ,p j ) is the original coordinate of the key point. The original coordinate of the key point here can be expressed as the coordinate position of the key point in the video window, and this position changes with the state of the characters in the picture in different picture frames, but the relative position between the key points does not change, so the original coordinates between the key points can be converted into the relative coordinates between the key points to facilitate the subsequent analysis of the key point position.

[0086] The relative coordinates of the key points are calculated based on their offsets. The formula is as follows:

[0087]

[0088] In the formula, (r i ,r j ) are the relative coordinates of any key point, Δi and Δj are the horizontal and vertical offsets of each key point relative to the center of the bounding box, w is the width of the bounding box, and h is the height of the bounding box.

[0089] S4: Analyze the facial features of the face in combination with key points, and record facial movements and expression features. This is used for subsequent model judgment and matching. The method is as follows:

[0090] S40: Based on the facial key point information, the distance and angle between the key points are calculated to analyze the geometric shape of the face. By converting the image information into digital information for processing, the changes in facial features can be analyzed more simply and conveniently, reducing the difficulty of calculation and improving the accuracy of calculation. The steps for calculating the distance and angle between the key points are as follows:

[0091] S400: Calculate the Euclidean distance between facial key points. The formula is as follows:

[0092]

[0093] Where d is the distance between any two key points, (a1, b1) and (a2, b2) are the relative coordinates of the two key points. Euclidean distance is an intuitive measurement method that is based on geometric intuition and is easy to understand. In two-dimensional or three-dimensional space, it is equivalent to the straight-line distance between two points.

[0094] S401: Calculate the angle θ between the facial key point solutions using the coordinates of any three key points in the following manner:

[0095] According to the three key points P1(x1,y1), P2(x2,y2), P3(x3,y3), calculate the vector from vector P1 to P2.

[0096]

[0097] The vector from P1 to P3,

[0098]

[0099] and The angle θ between these two vectors is calculated as follows:

[0100]

[0101] In the formula, and is the magnitude of two vectors.

[0102]

[0103] The formula for calculating the magnitude of a vector is:

[0104]

[0105] Combining the above calculation formula with the coordinates of the facial key points, the angle formed between any three facial key points can be calculated.

[0106] S402: Obtaining the geometric shape formed by the key points through the distance and angle between the key points of the face. The geometric shape formed by the key points can simplify the facial features and reduce the difficulty of calculation.

[0107] S41: Analyze the position of key points to determine the orientation and tilt angle of the face. Face recognition systems typically rely on the analysis of facial feature points, and if the face is not oriented correctly, the position of these feature points may change significantly, resulting in recognition errors. By first determining the facial orientation, the system can adjust the way the feature points are extracted and analyzed to improve the accuracy of recognition. In multi-angle or complex environments, if the system cannot accurately determine the facial orientation, it may mistakenly identify non-target objects, especially from the side or back view. Determining the facial orientation can help reduce such misidentifications. When processing large amounts of image or video data, determining the facial orientation can help the system prioritize those faces that are frontal or near-frontal, because these types of images typically contain more recognition information. This helps optimize computing resources and increase processing speed.

[0108] S42: Grouping key points to identify different areas of the face. According to the coordinates of the key points, the key point groups corresponding to the facial features are grouped together and matched with the facial areas. For example, the key points corresponding to the eyes are grouped together and marked as eyes. The same is true for the key point classification of other facial features.

[0109] S43: Use the optical flow method to track the motion trajectory of key points in the video sequence and analyze facial movements and expression features. Optical flow is a method in computer vision for tracking the motion of objects in image sequences. It is based on the visual changes of objects or scenes between consecutive frames, and determines the direction and speed of the object's movement by calculating the changes in time for each pixel in the image. This method has the characteristics of continuity, locality, and robustness, and can estimate the motion vector of each pixel in the image, thereby tracking the position changes of key points. The specific method of tracking the motion trajectory of key points using the optical flow method is as follows:

[0110] S430: Calculate the motion vector of the key point in the continuous video according to the relative coordinates of the key point, and its components are represented as u and v.

[0111] S431: Obtain the motion trajectory of the key points by accumulating the optical flow vectors of the key points in the continuous frames. The calculation method of the optical flow vector is as follows:

[0112] X m u+X z v=-X s

[0113] Where, X m and X z are the partial derivatives of image brightness with respect to the m and z directions, X s is the rate of change of brightness over time.

[0114] S5: Create a face recognition mental illness detection model based on convolutional neural networks. This model analyzes facial images and identifies subtle facial features associated with specific mental illnesses. This feature is added to the detection set for the next detection, thereby achieving the purpose of self-learning. The detection model is created as follows:

[0115] S50: Collect facial images of patients with mental illness and healthy individuals. Faces can be collected from big data, and the collected facial information can be classified into healthy individuals and patients to form a data set for subsequent model training.

[0116] S51: Label the emotional state of each face image and mark whether the face has a mental illness to facilitate feature matching during detection.

[0117] S52: Perform filtering noise processing on the face image to enhance the data set. The filtering noise processing method is to convolve each pixel in the image with a Gaussian kernel to calculate a new pixel value. The specific calculation method is as follows:

[0118] The mathematical expression of the two-dimensional Gaussian function is:

[0119]

[0120] Where (x, y) is the coordinate on the image plane, and σ is the standard deviation of the Gaussian function, which controls the width and flatness of the Gaussian function.

[0121] Generate Gaussian kernel: In image processing, Gaussian kernel makes a discrete approximation of a two-dimensional Gaussian function. The size of the Gaussian kernel is usually an odd number to ensure that the center of the kernel is located at the center. The values ​​of the Gaussian kernel are obtained by sampling on the two-dimensional Gaussian function. The size and standard deviation of the kernel determine the shape and coverage of the kernel.

[0122] The Gaussian kernel is normalized to ensure that the sum of all weights of the kernel is 1, which is a requirement of the convolution operation. The calculation formula is as follows:

[0123]

[0124] Where G' is the normalized Gaussian kernel.

[0125] The convolution operation of Gaussian filtering is designed to perform point-by-point multiplication and summation of the normalized Gaussian kernel with the image. For each pixel in the image, the Gaussian kernel is convolved with the pixels around it to calculate the new pixel value. The calculation formula is as follows:

[0126]

[0127] Where I(x,y) is the pixel value of the original image at the coordinate point (x,y), I'(x,y) is the new pixel value of the image at the coordinate point (x,y) after Gaussian filtering, k is the radius of the Gaussian kernel, and G'(s,t) is the value of the normalized Gaussian kernel at the coordinate point (s,t).

[0128] S53: Divide the data set into training set, validation set and test set, and use cross-validation to evaluate the model performance. The calculation method is as follows:

[0129]

[0130] Where, Accuracy n is the accuracy of the nth cross-validation, and N is the total number of folds. “Fold” refers to the number of subsets into which the dataset is split. Each fold is a unique part of the dataset that is used as the validation set in one iteration of cross-validation. The remaining folds are combined as the training set.

[0131] S6: The obtained facial movements and expression features are input into the face recognition mental illness detection model to assist in determining whether a person has a mental illness. Mental illness can affect the diversity of facial expressions. For example, patients with depression may show fewer changes in expression, while patients with schizophrenia may show inappropriate or exaggerated expressions. Patients with mental illness show inconsistency between expression and inner experience. They may appear happy on the outside, but feel sad or empty on the inside. Certain mental illnesses may cause changes in facial muscle activity patterns. For example, patients with Parkinson's disease may show reduced facial expressions, while facial paralysis may occur in certain emotional disorders. Micro-expressions are short-lived facial expressions that usually occur unconsciously, and some patients with mental illness cannot effectively control or hide these subtle expressions. Based on this, face recognition technology can be used to assist in determining whether a person has a mental illness.

[0132] In summary, by using AR technology to create a virtual AR test environment, the problem of difficulty in matching the actual test environment is solved, the test method is simplified, the space limitation is greatly reduced, the possibility of multiple tests is provided, and the robustness of the test results is enhanced; by filtering the noise of the face image, the problems of low image quality and low recognizability are solved, the performance of face recognition is improved, and the adaptability under different acquisition conditions is strong; by analyzing the facial features of the face through key points and recording facial movements and expression features, the problem of the influence of other uncertain factors in traditional detection is solved, the stability of the test is improved, and the accuracy of the results is higher.

[0133] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent face recognition method for detecting mental illness, characterized in that: include: S1: Create multiple virtual AR test environments based on AR technology and form an environment set; S2: Test the corresponding environment in the environment set according to the age of the person being tested; S3: Perform real-time face detection based on the test and extract key points of the recognized face; S4: Analyze the facial features of the face in combination with key points, and record facial movements and expression features; S5: Create a face recognition model for mental illness detection based on convolutional neural networks; S6: Input the obtained facial movement and expression features into the face recognition mental illness detection model to assist in determining whether the patient suffers from a mental illness.

2. The method for intelligently detecting mental illness by face recognition according to claim 1, characterized in that: In step S1, the method for creating a virtual AR test environment is as follows: S10: Classify the people to be tested according to their age groups; S11: Determine different situational modes, interactive items and virtual dialogue scenarios based on different age groups; S12: Use AR development platform and IDE to model and develop test environment.

3. The method for intelligently detecting mental illness by face recognition according to claim 1, characterized in that: In step S3, the specific method of key point extraction is as follows: S30: Real-time facial monitoring of the inspector in the AR virtual environment test; S31: converting the obtained real-time facial video image into a grayscale video image; S32: Call the preloaded OpenCV face detector to find the face area on the video image and return the detection result; S33: Use the MTCNN keypoint detection model to draw a bounding box around the detected face and draw markers at the keypoint locations; S34: Calculate the key point coordinates and convert the key point coordinates into relative coordinates relative to the bounding box.

4. The method for intelligently detecting mental illness by face recognition according to claim 3, characterized in that: In step S33, the relative coordinates are converted as follows: In the formula, (r i ,r j ) are the relative coordinates of any key point, Δi and Δj are the horizontal and vertical offsets of each key point relative to the center of the bounding box, w is the width of the bounding box, and h is the height of the bounding box.

5. The method for intelligently detecting mental illness by face recognition according to claim 1, characterized in that: In step S4, the facial features are analyzed as follows: S40: Calculate the distance and angle between the key points according to the facial key point information to analyze the geometric shape of the face; S41: Analyze the positions of key points to determine the orientation and tilt angle of the face; S42: Group key points to identify different areas of the face; S43: Use optical flow to track the motion trajectory of key points in video sequences and analyze facial movements and expression features.

6. The method for intelligently detecting mental illness by face recognition according to claim 5, characterized in that: In step S40, the distance and angle between key points are calculated as follows: S400: Calculate the Euclidean distance between facial key points. The formula is as follows: Where d is the distance between any two key points, (a1, b1) and (a2, b2) are the relative coordinate positions of the two key points; S401: Calculate the angle θ between the facial key point solutions using the coordinates of any three key points. The formula is as follows: In the formula, and are two vectors formed between the three key points, and is the magnitude of two vectors; S402: Obtaining a geometric shape formed by the key points through the distance and angle between the key points of the face.

7. The method for intelligently detecting mental illness by face recognition according to claim 5, characterized in that: In step S43, the specific method of tracking the motion trajectory of the key point using the optical flow method is as follows: S430: Calculating the motion vector of the key point in the continuous video according to the relative coordinates of the key point; S431: Obtain the motion trajectory of the key points by accumulating the optical flow vectors of the key points in the continuous frames; the calculation method of the optical flow vector is as follows: X m u+X z v=-X s Where, X m and X z are the partial derivatives of image brightness with respect to the m and z directions, u and v are the vectors of optical flow components, X s is the rate of change of brightness over time.

8. The method for intelligently detecting mental illness by face recognition according to claim 1, characterized in that: In step S5, the training method of the detection model is as follows: S50: Collect facial images of people with mental illness and healthy individuals; S51: Label each face image with its emotional state and mark whether it has a mental illness; S52: filtering and noise processing the face image to enhance the data set; S53: Divide the dataset into training set, validation set and test set, and use cross-validation to evaluate model performance.

9. The method for intelligently detecting mental illness by face recognition according to claim 8, characterized in that: In step S52, the filtering noise processing method is to convolve each pixel in the image with a Gaussian kernel to calculate a new pixel value; the specific calculation formula is as follows: Where I(x,y) is the pixel value of the original image at the coordinate point (x,y), I ’ (x, y) is the new pixel value of the image at the coordinate point (x, y) after Gaussian filtering, k is the radius of the Gaussian kernel, G ’ (s, t) is the value of the normalized Gaussian kernel at the coordinate point (s, t).

10. The method for intelligently detecting mental illness by face recognition according to claim 8, characterized in that: In step S53, the cross-validation accuracy formula is as follows: Where Accuracy n is the accuracy of the nth cross-validation, and N is the total number of folds.

Images