Construction of Detection Model, Detection Method and Related Device for Physical and Mental State Detection

Abstract

The present invention relates to the technical field of health detection, and is a detection model construction, detection method and related device for physical and mental state detection. The method obtains a training set and a test set, wherein each sample in the training set and the test set includes a face model feature and a true detection index; uses the training set to train a preset detection model, and outputs the trained detection model. The detection model includes an XYZ deep learning model for physiological index detection and an XYZ+RGB deep learning model for physiological and psychological index detection; uses the test set to evaluate the detection model, and reversely adjusts the detection model according to the evaluation result to output the final detection model. The present invention uses three-dimensional and RGB cameras to sample the user's face, and then combines machine learning and deep learning to detect the user's health index and psychological index, simplifies the basic physical and mental index detection process, and provides convenience for expanding the regular detection of basic physical and mental indexes.

Description

Technical Field

[0001] The present invention relates to the technical field of health detection, and is concerned with the construction of a detection model, a detection method and related devices for detecting physical and mental states. Background Art

[0002] Traditional health and psychological checkups require appointments with doctors and medical centers. Even simple health checkups (such as blood pressure and blood sugar) and psychological checkups (such as anxiety and happiness) still require these procedures, making the process cumbersome and costly.

[0003] With the continuous development of computer technology, the application of facial recognition is becoming increasingly widespread. Facial recognition technology is a biometric identification technology that uses facial features to identify people. It uses a camera or video camera to capture images or video streams containing faces, automatically detect and track faces, and perform facial recognition. Existing facial recognition systems are mainly used to determine physical attributes such as face matching and document recognition. They are unable to further predict user health or psychological indicators based on facial recognition. Summary of the Invention

[0004] Purpose of the invention: The present invention provides a detection model construction, detection method and related devices for physical and mental state detection, which are used to detect the physiological and psychological indicators of people through facial data.

[0005] The technical solution adopted by the present invention to solve this technical problem is: In view of this, the first aspect of the present application provides a method for constructing a detection model for physical and mental state detection, comprising:

[0006] Obtain training and test sets, where each sample in the training and test sets includes face model features and real detection indicators. Face model features include UV-map and face VTX data, and real detection indicators include real physiological indicators and real psychological indicators.

[0007] Use the training set to train the preset detection model and output the trained detection model, where the detection model includes an XYZ deep learning model for physiological indicator detection and an XYZ+RGB deep learning model for physiological and psychological indicator detection;

[0008] The detection model is evaluated using the test set, and the detection model is reversely adjusted based on the evaluation results to output the final detection model.

[0009] The face model features of each sample in the above training set and test set include:

[0010] Obtain the 3D information obj file and 2D information jpg file of the sample face;

[0011] Process the 3D information obj file and 2D information jpg file using the face3d algorithm, and output the uv-map and face vtx data;

[0012] Preprocess the face vtx data, where the preprocessing includes normalization and dimensionality reduction.

[0013] The above processing of the 3D information obj file and 2D information jpg file using the face3d algorithm includes:

[0014] Use the 2D information jpg file to find the corresponding 6 key feature points and 11 secondary feature points, and reconstruct to obtain a reference face, where the 6 key feature points are four eye corners plus two mouth corners, and the 11 secondary feature points are two nasal wing points, one nose tip point; one nasal bridge point, the center point of the lips, the center point of the upper edge of the lips, the center point of the lower edge of the lips, the center point of the chin, and two earlobe points;

[0015] Use the 3D information obj file to find the corresponding 6 key feature points and 11 secondary feature points, and determine the target face, where the 6 key feature points are four eye corners plus two mouth corners, and the 11 secondary feature points are two nasal wing points, one nose tip point; one nasal bridge point, the center point of the lips, the center point of the upper edge of the lips, the center point of the lower edge of the lips, the center point of the chin, and two earlobe points;

[0016] Register the target face with the reference face to obtain the target face after registration with the reference face;

[0017] Use the GPA algorithm to normalize the target face after registration with the reference face;

[0018] Output the uv-map and face vtx data of the target face after normalization processing.

[0019] The above XYZ deep learning model is a model constructed based on the MLP multi-layer neural network.

[0020] The above XYZ+RGB deep learning model is a model constructed based on the Inception-Resnet neural network.

[0021] In view of this, the second aspect of this application provides a method for detecting physical and mental states, including:

[0022] Collect the face model features of the user to be tested, where the face model features include uv-map and face vtx data;

[0023] Input the face model features of the user to be tested into the detection model to obtain detection indicators, where the detection indicators include true physiological indicators and true psychological indicators; among them, the detection model is a detection model constructed using the detection model construction method for physical and mental state detection.

[0024] In view of this, a third aspect of the present application provides a detection model construction device, including:

[0025] A sample establishment unit, which obtains a training set and a test set. Each sample in the training set and the test set includes face model features and true detection indicators. The face model features include uv-map and face vtx data, and the true detection indicators include true physiological indicators and true psychological indicators;

[0026] A model training unit, which trains a preset detection model using the training set and outputs the trained detection model. The detection model includes an XYZ deep learning model for physiological index detection and an XYZ+RGB deep learning model for physiological and psychological index detection;

[0027] A model evaluation unit, which evaluates the detection model using the test set, adjusts the detection model in reverse according to the evaluation result, and outputs the final detection model.

[0028] In view of this, a fourth aspect of the present application provides a physical and mental state detection device, including:

[0029] A preprocessing unit, which collects the face model features of the user to be measured and displays the detection results. The face model features include uv-map and face vtx data;

[0030] A Tensorflow serving service unit, which inputs the face model features of the user to be measured into the detection model to obtain detection indicators, including true physiological indicators and true psychological indicators. The detection model is the detection model constructed by using the detection model construction device described in claim 8.

[0031] The above preprocessing unit includes:

[0032] A front-end unit, which obtains the three-dimensional information obj file and two-dimensional information jpg file of the sample face and displays the detection results;

[0033] A Flask service unit, which processes the three-dimensional information obj file and two-dimensional information jpg file using the face3d algorithm, outputs uv-map and face vtx data, preprocesses the face vtx data, and receives and outputs the detection results. The preprocessing includes normalization and dimensionality reduction, and receives and outputs the detection results.

[0034] Beneficial effects: The present invention uses three-dimensional and RGB cameras to sample the user's face, and combines machine learning and deep learning to detect the user's health indicators (such as blood pressure, blood sugar, etc.) and psychological indicators (such as anxiety, happiness, etc.). This enables users to conveniently detect basic health indicators (such as blood pressure, blood sugar, etc.) and psychological indicators (such as anxiety, happiness, etc.) through terminal devices, thus simplifying the process of detecting basic physical and mental indicators and providing convenience for expanding the regular detection of basic physical and mental indicators. Brief Description of the Drawings

[0035] Attached Figure 1 is a flowchart for constructing a detection model according to an embodiment of the present invention.

[0036] Attached Figure 2 is a schematic diagram of two-dimensional affine transformation of the GPA algorithm in an embodiment of the present invention.

[0037] Attached Figure 3 is a flowchart of a detection method according to an embodiment of the present invention.

[0038] Attached Figure 4 is a schematic structural diagram of a device for constructing a detection model according to an embodiment of the present invention.

[0039] Attached Figure 5 is a schematic structural diagram of a device for detecting physical and mental states according to an embodiment of the present invention. Detailed Embodiments

[0040] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0041] The present application provides a detection model construction, detection method and related device for physical and mental state detection, which are used to solve the problem that existing face recognition is mainly applied to the judgment of physical attributes such as face matching and document recognition, and cannot further predict the user's health indicators or psychological indicators based on face recognition.

[0042] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0043] The embodiments of the present application will be introduced in detail below.

[0044] Embodiment 1: As shown in the appendix Figure 1 This embodiment discloses a method for constructing a detection model for physical and mental state detection, including:

[0045] Step S101, obtaining a training set and a test set, where each sample in the training set and the test set includes a face model feature and a true detection index. The face model feature includes uv-map and face vtx data, and the true detection index includes a true physiological index and a true psychological index.

[0046] In the embodiments of the present application, the face model features of each sample in the training set and the test set include:

[0047] 1. Obtaining a 3D information obj file and a 2D information jpg file of the sample face;

[0048] In this step, the Bellus3D arc4 3D face imaging system can be used to obtain a 3D image of the sample face and output the corresponding 3D information obj file and 2D information jpg file. The 3D information obj file includes 4 types of data, which start with the following letters: v vertex, vt texture coordinate, vn vertex normal vector, f face. The 2D information jpg file is the complete face image.

[0049] 2. Using the face3d algorithm to process the 3D information obj file and the 2D information jpg file, and outputting uv-map and face vtx data;

[0050] In this step, using the face3d algorithm to process the 3D information obj file and the 2D information jpg file includes:

[0051] (1) Using the 2D information jpg file to find the corresponding 6 key feature points and 11 secondary feature points, and reconstructing to obtain a reference face. The 6 key feature points are four eye corners plus two mouth corners, and the 11 secondary feature points are two nose wing points, one nose tip point, one nose bridge point, the center point of the lips, the center point of the upper edge of the lips, the center point of the lower edge of the lips, the center point of the chin, and two earlobe points;

[0052] After finding the corresponding 6 key feature points and 11 secondary feature points in this step, a reference face is obtained through spherical resampling and triangular mesh surface reconstruction. The role of spherical resampling and triangular mesh surface reconstruction here is to solve the problem that the ears are blocked by the face when obtaining the face image through the z-direction projection method during sampling. If the spherical sampling method is used, it can be not used.

[0053] The specific implementation steps of spherical resampling are as follows: spherically parameterize the three-dimensional vertex coordinates of the mesh surface of the complete face, and then use the aforementioned uniform sampling method to obtain a quadrilateral mesh surface. The steps for reconstructing the triangular mesh surface are: reconstruct the quadrilateral mesh obtained by spherical resampling into a triangular mesh, and then inverse spherically parameterize all the sampling points to return to the original three-dimensional coordinate system.

[0054] (2) Use the 3D information obj file to find the corresponding 6 key feature points and 11 secondary feature points to determine the target face. Among them, the 6 key feature points are the four eye corners plus the two mouth corners, and the 11 secondary feature points are the two nasal wing points, one nasal tip point, one nasal bridge point, the center point of the lips, the central point of the upper edge of the lips, the central point of the lower edge of the lips, the central point of the chin, and the two earlobe points;

[0055] In this step, use the PCA algorithm to establish a feature space for the 6 key feature points, automatically find the 6 key feature points using the PCA algorithm, and then automatically find the 11 secondary feature points using a heuristic method to obtain the target face. When using the PCA algorithm to automatically find the 6 key feature points here, the reference face can be combined, and using the characteristic that the neighborhood of the nasal tip point of the human face is approximately spherical, use the spherical fitting method to accurately find the position of the nasal tip point on the reference face, which is the position of the nasal tip point of the target face.

[0056] (3) Register the target face using the reference face to obtain the target face after registration with the reference face;

[0057] This step is to use the TPS method to bend and deform the reference face to fit the target face, and find the nearest point on the target face corresponding to each vertex of the reference face, so as to establish a dense point-to-point correspondence between the reference face and the target face, and complete the registration between the target face and the reference face.

[0058] (4) Normalize the target face after registration with the reference face using the GPA algorithm;

[0059] The GPA algorithm is the Procrustes analysis method, which is a graphical statistical analysis method. Since there are large deviations in the shape and position of the target object in the image, the data obtained does not have affine invariance and needs to be normalized. Its purpose is to obtain a better local change model as the basis for subsequent model learning. After normalization, the structure of the human face gets closer and closer to their average position; after a series of iterative scale and rotation normalization operations, different human faces become more and more similar to the pose and scale of the average face in terms of pose and scale.

[0060] Here, the Procrustes analysis method is to continuously iterate to find the standard shape and use the least squares method to find the affine transformation method from each sample shape to this standard shape.

[0061] For example, as shown in the appendixFigure 2 The two-dimensional affine transformation shown. The normalization process for two shapes (one is the canonical shape and the other is the sample shape) is as follows:

[0062] 1. Calculate the mean of each sample point i (i = 1, 2.., n) in N images:

[0063]

[0064] 2. Normalize the sizes of all shapes, that is, subtract the corresponding mean from each sample point:

[0065]

[0066] 3. Calculate the centroid of the shape in each image based on the decentralized data. For the i-th image, its centroid is:

[0067]

[0068] 4. Align the standard and sample shapes together according to the centroid and angle so that the Procrustes distance between the two shapes is minimized. The following is the definition of the Procrustes distance:

[0069]

[0070] Among them, step 4 needs to be implemented through continuous iteration. The iteration process includes:

[0071] A. Obtain the standard shape of each image by calculating the average of all normalized sample points in each image.

[0072] B. Use the least squares method to find the rotation angle of the sample shape to the standard shape in each image. Here, according to the definition of the Procrustes distance, it can be obtained through the following formula:

[0073]

[0074] Among them, a and b represent the parameters of the rotation change in the affine transformation and are obtained through the following steps:

[0075]

[0076] Take the partial derivatives of the above formula to obtain the required a and b;

[0077]

[0078] C. Perform a rotation transformation on the sample shape according to the rotation parameters to obtain a new shape aligned with the standard shape

[0079]

[0080] D. Repeat the above steps until the specified number of loops is reached or the absolute norm of the standard shape between two consecutive iterations meets a certain threshold.

[0081] (5) Output the uv-map of the target face and the face vtx data after normalization. Here, the face vtx data is the model vertices of the target face, and the uv-map is the model texture map of the target face.

[0082] 3. Preprocess the face vtx data, where the preprocessing includes normalization and dimensionality reduction.

[0083] Here, the normalization can be achieved using the existing min-max normalization algorithm. The dimensionality reduction can be achieved through the PCA algorithm. The specific processing process of the PCA algorithm is as follows:

[0084] For example, given m pieces of n-dimensional data, the specific steps are as follows:

[0085] 1. Organize the original data by columns to form an n-row m-column matrix X;

[0086] 2. Zero-mean each row of X (representing an attribute field), that is, subtract the mean of this row;

[0087] 3. Calculate the covariance matrix:

[0088]

[0089] 4. Calculate the eigenvalues and corresponding eigenvectors of the covariance matrix;

[0090] 5. Arrange the eigenvectors in a matrix by rows from top to bottom according to the corresponding eigenvalue magnitudes, and take the first k rows to form a matrix P;

[0091] 6. Y = PX is the data after dimensionality reduction to k dimensions.

[0092] Step S102. Use the training set to train a preset detection model, and output the trained detection model. The detection model includes an XYZ deep learning model for physiological index detection and an XYZ+RGB deep learning model for physiological and psychological index detection.

[0093] In the embodiment of the present application, the XYZ deep learning model (XYZ is three-dimensional coordinates) is a model constructed based on the MLP multi-layer neural network. Among them, the MLP multi-layer neural network uses a variety of algorithms such as residual neural network, attention mechanism, normalization algorithm, and ordinal classification algorithm. In the embodiment of the present application, the preprocessed face vtx data is input into the XYZ deep learning model, and the physiological index detection results are output, specifically including: age, body mass index (BMI), gender, systolic blood pressure (SBP), diastolic blood pressure (DBP), blood glucose (GLU), high-density lipoprotein cholesterol (HDL), low-density lipoprotein cholesterol (LDL), total cholesterol (TC), triglyceride (TG).

[0094] In the embodiment of the present application, the XYZ+RGB deep learning model is a model constructed based on the Inception-Resnet neural network. It can take the UV-Map that combines RGB two-dimensional data and three-dimensional information as the input, use multiple CNN networks for prediction, and uses a collection of multiple technologies such as ResNet, InceptionNet, and SENet. The physiological index detection results are output, specifically including: age, body mass index (BMI), gender, systolic blood pressure (SBP), diastolic blood pressure (DBP), blood glucose (GLU), high-density lipoprotein cholesterol (HDL), low-density lipoprotein cholesterol (LDL), total cholesterol (TC), triglyceride (TG). Then, the face is detected by the CNN neural network using the facial expression recognition algorithm, and the detected face is cut out and input into another CNN network to detect the facial expression of the face: angry, nauseous, afraid, happy, sad, surprised, etc.

[0095] Step S103, evaluate the detection model using the test set, and inversely adjust the detection model according to the evaluation result to output the final detection model.

[0096] In summary, the present application uses the three-dimensional information of the face to obtain the face model features, and trains the detection model through the algorithm combining deep learning and machine learning to obtain a detection model for physical and mental state detection, realizing the physical and mental state detection.

[0097] Embodiment 2: As shown in the appendix Figure 3 This embodiment discloses a method for detecting physical and mental states, including:

[0098] Step S201, collect the face model features of the user to be measured, where the face model features include uv-map and face vtx data;

[0099] Step S202: Input the face model features of the user to be tested into the detection model to obtain detection metrics, which include real physiological metrics and real psychological metrics. Among them, the detection model is constructed using the detection model construction method for physical and mental state detection. The detection metrics obtained through the model can be saved in JSON.

[0100] Embodiment 3: As shown in the appendix Figure 4 This embodiment discloses a detection model construction device, including:

[0101] A sample establishment unit that obtains a training set and a test set. Each sample in the training set and the test set includes face model features and real detection metrics. The face model features include uv-map and face vtx data, and the real detection metrics include real physiological metrics and real psychological metrics;

[0102] A model training unit that trains a preset detection model using the training set and outputs the trained detection model. Among them, the detection model includes an XYZ deep learning model for physiological metric detection and an XYZ+RGB deep learning model for physiological and psychological metric detection;

[0103] A model evaluation unit that evaluates the detection model using the test set, adjusts the detection model in reverse according to the evaluation results, and outputs the final detection model.

[0104] Embodiment 4: As shown in the appendix Figure 5 This embodiment discloses a physical and mental state detection device, including:

[0105] A preprocessing unit that collects the face model features of the user to be tested and displays the detection results. The face model features include uv-map and face vtx data. Specifically, it includes:

[0106] A front-end unit that obtains the 3D information obj file and 2D information jpg file of the sample face and displays the detection results. Here, the front-end acquisition unit can be a Bellus3D arc4 3D face imaging system, which uses 3D and RGB cameras to obtain the 3D information obj file and 2D information jpg file of the sample face.

[0107] A Flask service unit that processes the 3D information obj file and 2D information jpg file using the face3d algorithm, outputs uv-map and face vtx data, and preprocesses the face vtx data. The preprocessing includes normalization and dimensionality reduction, and receives and outputs the detection results.

[0108] Here, the Flask service unit is containerized and deployed using Docker, which makes it convenient to deploy the algorithm across platforms and also for local deployment. The front-end acquisition unit accesses the Flask service unit at the back-end through a network interface and communicates with Flask.

[0109] The Flask service unit can also open the following series of interfaces according to the actual situation: a face detection interface, which determines whether the faces in the files are wearing glasses, masks or other items that block facial information by using the 3D information obj file and 2D information jpg file transmitted by the front-end acquisition unit; a face expression interface, which judges the expressions and micro-expressions of the users in the videos or pictures transmitted by the front-end; a heart rate interface, which measures the heart rate of the users in the videos or video streams transmitted by the front-end; and a health interface, which measures the health indicators of the users by using the 3D information obj file and 2D information jpg file transmitted by the front-end.

[0110] An SQL database is set up in the Flask service unit, which can receive and store the detection results sent by the TensorFlow serving service unit and forward them to the front-end acquisition unit.

[0111] The TensorFlow serving service unit inputs the face model features of the user to be tested into the detection model to obtain detection indicators, including real physiological indicators and real psychological indicators; among them, the detection model is a detection model constructed by a detection model construction device.

[0112] Here, TensorFlow serving is a deep learning inference service that schedules server resources, calculates the data processed by Flask, uses pre-trained weights to predict results, and then returns the results to the Flask service. All deep learning models will be loaded by the TensorFlow serving service.

[0113] It should be noted that: for the facial prediction model construction device and the facial prediction device provided in the above embodiments, only the above division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0114] Finally, specific embodiments are applied in the present invention to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for constructing a detection model for physical and mental state detection, characterized in that, Including: Obtain a training set and a test set, where each sample in the training set and the test set includes a face model feature and a true detection index. The face model feature includes a uv-map and face vtx data, and the true detection index includes a true physiological index and a true psychological index; Use the training set to train a preset detection model, and output the trained detection model, where the detection model includes an XYZ deep learning model for physiological index detection and an XYZ+RGB deep learning model for physiological and psychological index detection; Use the test set to evaluate the detection model, and inversely adjust the detection model according to the evaluation result to output the final detection model; Among them, the face model feature of each sample in the training set and the test set includes: Obtain the 3D information obj file and 2D information jpg file of the sample face; Use the face3d algorithm to process the 3D information obj file and 2D information jpg file, and output the uv-map and face vtx data, including: Use the 2D information jpg file to find the corresponding 6 key feature points and 11 secondary feature points, and reconstruct to obtain a reference face. The 6 key feature points are four eye corners plus two mouth corners, and the 11 secondary feature points are two nose wing points, one nose tip point, one nose bridge point, the center point of the lips, the center point of the upper edge of the lips, the center point of the lower edge of the lips, the center point of the chin, and two earlobe points; Use the 3D information obj file to find the corresponding 6 key feature points and 11 secondary feature points, and determine the target face. The 6 key feature points are four eye corners plus two mouth corners, and the 11 secondary feature points are two nose wing points, one nose tip point, one nose bridge point, the center point of the lips, the center point of the upper edge of the lips, the center point of the lower edge of the lips, the center point of the chin, and two earlobe points; Register the target face with the reference face to obtain the target face after registration with the reference face; Use the GPA algorithm to normalize the target face after registration with the reference face; The model vertices of the target face are used as face vtx data, and the model texture map of the target face is used as the uv-map; Preprocess the face vtx data, where the preprocessing includes normalization and dimensionality reduction; Among them, the XYZ deep learning model is a model constructed based on an MLP multi-layer neural network, and the model constructed based on the MLP multi-layer neural network is trained using the face vtx data. The XYZ+RGB deep learning model is a model constructed based on an Inception-Resnet neural network, and the model constructed based on the Inception-Resnet neural network is trained using the uv-map.

2. A method for detecting physical and mental states, characterized in that, Including: Collect the face model features of the user to be tested, where the face model features include a uv-map and face vtx data; Input the face model features of the user to be tested into the detection model to obtain detection indexes, and the detection indexes include a true physiological index and a true psychological index; among them, the detection model is a detection model constructed by using the detection model construction method for physical and mental state detection as described in claim 1.

3. A detection model construction device, characterized in that, Including: A sample building unit acquires a training set and a test set. Each sample in the training set and the test set includes face model features and real detection metrics. The face model features include uv-map and face vtx data, and the real detection metrics include real physiological metrics and real psychological metrics; A model training unit trains a preset detection model using the training set and outputs the trained detection model. The detection model includes an XYZ deep learning model for physiological metric detection and an XYZ+RGB deep learning model for physiological and psychological metric detection; A model evaluation unit evaluates the detection model using the test set, adjusts the detection model reversely according to the evaluation result, and outputs the final detection model; Among them, the face model features of each sample in the training set and the test set include: Obtain the 3D information obj file and 2D information jpg file of the sample face; Use the face3d algorithm to process the 3D information obj file and 2D information jpg file, and output uv-map and face vtx data, including: Use the 2D information jpg file to find the corresponding 6 key feature points and 11 secondary feature points, and reconstruct to obtain a reference face. The 6 key feature points are four eye corners plus two mouth corners, and the 11 secondary feature points are two nose wing points, one nose tip point, one nose bridge point, the center point of the lips, the center point of the upper edge of the lips, the center point of the lower edge of the lips, the center point of the chin, and two earlobe points; Use the 3D information obj file to find the corresponding 6 key feature points and 11 secondary feature points, and determine the target face. The 6 key feature points are four eye corners plus two mouth corners, and the 11 secondary feature points are two nose wing points, one nose tip point, one nose bridge point, the center point of the lips, the center point of the upper edge of the lips, the center point of the lower edge of the lips, the center point of the chin, and two earlobe points; Register the target face with the reference face to obtain the target face after registration with the reference face; Use the GPA algorithm to normalize the target face after registration with the reference face; The model vertices of the target face are used as face vtx data, and the model texture map of the target face is used as uv-map; Preprocess the face vtx data, where the preprocessing includes normalization and dimensionality reduction; Among them, the XYZ deep learning model is a model constructed based on an MLP multi-layer neural network, and the model constructed based on the MLP multi-layer neural network is trained using face vtx data. The XYZ+RGB deep learning model is a model constructed based on an Inception-Resnet neural network, and the model constructed based on the Inception-Resnet neural network is trained using uv-map.

4. A physical and mental state detection device, characterized in that, Include: A preprocessing unit collects the face model features of the user to be measured and displays the detection results. The face model features include uv-map and face vtx data; The TensorFlow Serving service unit inputs the face model features of the user to be tested into the detection model to obtain detection metrics, including real physiological metrics and real psychological metrics; wherein, the detection model is constructed by the detection model construction device described in claim 3.

5. The physical and mental state detection device according to claim 4, characterized in that The preprocessing unit includes: The front-end unit obtains the 3D information obj file and 2D information jpg file of the sample face and displays the detection results; The Flask service unit processes the 3D information obj file and 2D information jpg file using the face3d algorithm, outputs the uv-map and face vtx data, preprocesses the face vtx data, and receives and outputs the detection results, where the preprocessing includes normalization and dimensionality reduction, and receives and outputs the detection results.

Images