A psychological state detection method, system, device and storage medium

Abstract

The application relates to a psychological state detection method, system, device and storage medium, and belongs to the technical field of psychological measurement. The method comprises the following steps: acquiring image features and sound features, wherein the image features represent behaviors and / or expressions of a to-be-tested person in a detection process, and the sound features represent sounds and / or languages spoken by the to-be-tested person in the detection process; inputting the image features and the sound features into a preset psychological detection model to obtain psychological detection data; and determining a psychological detection result of the to-be-tested person according to a preset result comparison table and the psychological detection data. The application has the effect of improving the accuracy of psychological analysis conclusions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of psychological measurement, and in particular to a psychological state detection method, system, device and storage medium. BACKGROUND

[0002] Analysis of psychological state has great value in many fields such as commercial, civil, military or scientific research. For the psychological analysis method relying on artificial, the psychological state analysis method has high psychological professional requirements for the analyst. Due to the difference in professional ability of the analyst, the analysis conclusion may be inaccurate. For the psychological detection method based on machine learning model or deep learning model, a large amount of labeled data is needed for model training. However, the performance of psychological activities, such as micro-expression, is usually short and subtle, and the sample size of the data is small and difficult to accurately label, so it may be challenging to obtain sufficient training data.

[0003] The related technical solutions in the above have the following defects: the accuracy of the psychological analysis conclusion is low. SUMMARY

[0004] In order to improve the problem of low accuracy of psychological analysis conclusion, the present application provides a psychological state detection method, system, device and storage medium.

[0005] In a first aspect of the present application, a psychological state detection method is provided. The method comprises:

[0006] Obtaining image features and sound features, the image features representing the behavior and / or expression of the testee in the detection process, and the sound features representing the sound and / or language spoken by the testee in the detection process;

[0007] Inputting the image features and the sound features into a preset psychological detection model to obtain psychological detection data;

[0008] According to the preset result comparison table and the psychological detection data, determining the psychological detection result of the testee.

[0009] From the above technical solution, by obtaining the behavior, expression and sound of the testee, the psychological state of the testee is evaluated, and multiple factors can reflect the psychological state of the testee to different degrees, reducing the phenomenon that the psychological state evaluation result is wrong due to considering a single factor, and improving the accuracy of psychological state evaluation.

[0010] In one possible implementation, the image features include expression features and behavior features, and the image features are obtained by the following way:

[0011] acquire a plurality of facial images and a plurality of behavior images of the to-be-tested person in the detection process;

[0012] input the plurality of facial images and the acquisition time corresponding to the facial images into a preset expression recognition model to determine expression features;

[0013] extract action data in the behavior images;

[0014] determine behavior features according to the long short-term memory network model and the action data.

[0015] According to the above technical solution, the facial images and the behavior images are considered, and the acquisition time corresponding to the facial images is also acquired. The sequence between different facial images can be obtained according to the acquisition time, that is, the time sequence of the expression is considered. The time sequence of the action data is also considered by analyzing the action data through the long short-term memory network model. The psychological state reflected by the sequence of the expression or the action is not the same. The time sequence of the expression and the action is considered to further improve the accuracy of the psychological state detection.

[0016] In a possible implementation, the sound features include a voiceprint feature and a semantic feature, and the sound features are obtained in the following manner:

[0017] acquire audio data of the to-be-tested person in the detection process;

[0018] determine a voiceprint feature corresponding to the audio data, and the voiceprint feature reflects fluctuation of the audio data;

[0019] extract text information in the audio data, and the text information is used to reflect the narrative content of the to-be-tested person in the audio data;

[0020] determine a semantic feature according to the text information.

[0021] According to the above technical solution, the voice fluctuation of the to-be-tested person and the text information contained in the audio data are analyzed. The voice fluctuation can reflect the emotional state of the to-be-tested person to a certain extent, and the text information represents the actual content conveyed by the to-be-tested person. Considering both of them can make the result of the psychological state evaluation closer to the real psychological state and improve the evaluation accuracy of the psychological state.

[0022] In a possible implementation, the psychological detection model is determined in the following manner:

[0023] acquire a training data set, and the training data set includes a plurality of groups of image features and sound features in different psychological states;

[0024] establish a logic function and a logarithmic loss function according to the relationship between the image features and the sound features in the training data set and the psychological state;

[0025] determine the image regression coefficient corresponding to the image feature and the sound regression coefficient corresponding to the sound feature according to the maximum likelihood estimation method, the logistic function and the log loss function;

[0026] determine the psychological detection model according to the preset factor loading matrix, the image regression coefficient and the sound regression coefficient.

[0027] According to the above technical solutions, the relationship between the psychological state and each feature is analyzed and the corresponding regression coefficient is calculated according to the maximum likelihood estimation method, the logistic function and the log loss function based on the image features and the sound features under multiple different psychological states. The psychological detection model is constructed according to each regression coefficient and the factor loading matrix.

[0028] In a possible implementation, the factor loading matrix is calculated according to the independent variable matrix and the factor matrix, and the elements in the independent variable matrix represent different psychological states, and the factor matrix is calculated by the expectation maximization algorithm.

[0029] In a possible implementation, the psychological detection model is determined according to the preset factor loading matrix, the image regression coefficient and the sound regression coefficient, including:

[0030] The regression coefficient matrix and the factor loading matrix are multiplied to obtain a target matrix, and the regression coefficient matrix includes the image regression coefficient and the sound regression coefficient.

[0031] The target matrix is normalized to obtain the psychological detection model.

[0032] In the second aspect of the present application, a psychological state detection system is provided. The system includes:

[0033] The data acquisition module is configured to acquire image features and sound features, wherein the image features represent behaviors and / or expressions exhibited by the testee during the detection process, and the sound features represent sounds and / or languages spoken by the testee during the detection process.

[0034] The data calculation module is configured to input the image features and the sound features into a preset psychological detection model to obtain psychological detection data.

[0035] The result determination module is configured to determine a psychological detection result of the testee according to a preset result reference table and the psychological detection data.

[0036] In the third aspect of the present application, a psychological state detection system is provided. The system includes: AR glasses and a server;

[0037] The AR glasses are configured to acquire image features and sound features.

[0038] The server is configured to perform the method of the first aspect of the application.

[0039] From the above technical solutions, by using the AR glasses to obtain the image features and sound features of the to-be-tested person, the obtained image features are more consistent with the information actually seen by the eyes, and meanwhile wearing the AR glasses can timely adjust the shooting angle to avoid the occurrence of visual occlusion and other phenomena in the process of obtaining data, thereby affecting the data collection of the to-be-tested person and improving the efficiency of data collection.

[0040] In a fourth aspect of the application, an electronic device is provided. The electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method as described above.

[0041] In a fifth aspect of the application, a computer readable storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the method according to the first aspect of the application.

[0042] In summary, the present application includes at least one beneficial technical effect:

[0043] By obtaining the behavior, expression and sound of the to-be-tested person, the psychological state of the to-be-tested person is evaluated, and multiple factors can reflect the psychological state of the to-be-tested person to different degrees, thereby reducing the phenomenon of incorrect psychological state evaluation results caused by considering a single factor and improving the accuracy of psychological state evaluation. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a structural schematic diagram of a psychological state detection system provided by the present application.

[0045] Figure 2 is a flowchart of a psychological state detection method provided by an embodiment of the present application.

[0046] Figure 3 is a flowchart of a psychological state detection method provided by an embodiment of the present application.

[0047] Figure 4 is a structural schematic diagram of a psychological state detection system provided by the present application.

[0048] Figure 5 is a structural schematic diagram of an electronic device provided by the present application.

[0049] In the figure, 1, AR glasses; 2, server; 201, data acquisition module; 202, data calculation module; 203, result determination module; 301, CPU; 302, ROM; 303, RAM; 304, I / O interface; 305, input part; 306, output part; 307, storage part; 308, communication part; 309, driver; 310, detachable medium. DETAILED DESCRIPTION

[0050] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0051] In addition, the term "and / or" in this paper is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in this paper generally represents an "or" relationship between the associated objects unless otherwise specified.

[0052] Currently, there are mainly two ways to analyze the psychological state, one is a scale, and the other is to analyze through a machine learning model or a deep learning model. For the first analysis method, it is used more in a clinical environment to measure the anxiety of outpatients or inpatients. The most widely used scales at present are Hamilton Anxiety Scale (HAMA) and Hamilton Depression Scale (HAMD). Both of the above two scales need a professional doctor to evaluate the patient's condition, which is time-consuming and laborious. Moreover, due to the difference in professional ability of doctors, there will be a certain deviation in the evaluation results. For the second analysis method, the accuracy of the machine learning model and the deep learning model depends largely on the training data set itself. The larger the amount of data in the data set for training the model, the higher the accuracy of the machine learning model and the deep learning model. The higher the quality of the data in the data set for training the model, the higher the accuracy of the machine learning model and the deep learning model. It can be understood that the analysis of the psychological state is mostly made by some subtle actions or expressions, which are difficult to capture and difficult to label the data in the training data set. Therefore, the accuracy of the machine learning model and the deep learning model in analyzing the psychological state is low, and misjudgment is easy to occur. Moreover, the machine learning model and the deep learning model cannot consider the time sequence relationship between micro-expressions or micro-motions, which will also lead to a decrease in the accuracy of the psychological state evaluation.

[0053] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0054] The embodiments of the present application provide a psychological state detection system, which refers to Figure 1 The psychological state detection system includes AR glasses 1 and a server 2, and the AR glasses 1 and the server 2 are communicatively connected. In one specific use scenario, a person to be tested wears the AR glasses 1, and the audio data and image data of the person to be tested are obtained through the downward wide-angle camera and the camera for detecting the face in the AR glasses 1, and the obtained audio data and image data are sent to the server 2. The server 2 analyzes the psychological state of the person to be tested according to the obtained audio data and image data, and then obtains a psychological detection result. In another specific use scenario, there are a person to be tested and a professional doctor, the professional doctor wears the AR glasses 1, the professional doctor guides the person to be tested to make corresponding behaviors or answers, and the audio data and image data of the person to be tested are obtained by the professional doctor through the gaze of the person to be tested. The AR glasses 1 send the obtained audio data and image data to the server 2, and the server 2 analyzes the psychological detection result according to the audio data and image data. The process of obtaining the psychological detection result by the server 2 according to the audio data and image data can refer to the specific process of the psychological state detection method, which will not be described here.

[0055] By using the AR glasses 1 to acquire the audio data and the image data of the to-be-tested person, the information actually seen by the eyes is more fitted, and meanwhile the shooting angle can be adjusted in time while wearing the AR glasses 1, so as to avoid the phenomenon of visual occlusion and other phenomena in the process of acquiring data, thereby affecting the data acquisition of the to-be-tested person, improving the efficiency of data acquisition, and ensuring the real-time performance of data acquisition to a certain extent.

[0056] The embodiment of the application provides a psychological state detection method, and a main process of the method is described as follows.

[0057] As shown in Figure 2

[0058] Step S101: acquiring image features and sound features.

[0059] Specifically, the image features represent behaviors and / or expressions of the to-be-tested person in the detection process, and the sound features represent sounds and / or languages spoken by the to-be-tested person in the detection process.

[0060] The image features include expression features and behavior features, and the process of acquiring the image features is as follows:

[0061] After obtaining the image data sent by the AR glasses 1, the image data is further processed. In a specific embodiment, the image data includes a face image and a behavior image, the face image is a face image of the to-be-tested person, and the behavior image is a body image of the to-be-tested person. In a specific embodiment, the behavior image is an image of the to-be-tested person. The face image has a plurality of face images, which are acquired in the detection process of the to-be-tested person. Similarly, the behavior image also has a plurality of behavior images, which are acquired in the detection process of the to-be-tested person. The plurality of face images are continuous multiple frames of images, and the plurality of behavior images can be continuous multiple frames of images or discontinuous multiple frames of images.

[0062] ​The plurality of facial images and the acquisition time corresponding to the facial images are input into a preset expression recognition model to determine expression features. The acquisition time corresponding to each facial image can reflect the order of each face, and the order of expression changes is different, and the psychological state is also different. The expression recognition model includes a deep convolutional neural network (DCNN) and a transformer model. First, the facial images are detected and aligned to make the faces in each frame of facial image located at the same position. The aligned facial images are input into the DCNN model to extract static features, each facial image corresponds to a static feature, and a plurality of facial images can obtain a group of static features. Then, the group of static features is input into the transformer model to obtain the micro-expression category corresponding to the group of static features, that is, the expression features. The transformer model mentioned in the embodiments provided in the application can also use chatGPT and SwissArmyTransformer to realize feature extraction, semantic analysis and other functions, which are not limited here.

[0063] A plurality of skeleton points in the behavior image are extracted, and according to the relative positions of the skeleton points in the behavior image and the preset corresponding relationship between the relative positions of the skeleton points and the actions, the action data is determined. In a specific implementation manner, yolo-pose is used to extract the skeleton points in the behavior image, and one action data can be determined according to each behavior image, a group of action data can be obtained from a plurality of behavior images, the group of action data is sorted according to the time sequence relationship between the behavior images, and then the group of action data is input into a long short-term memory (LSTM) network model. The model can perform sequential behavior analysis according to the front and back association of the action data to obtain behavior features. In other implementation manners, openpose and alphapose can also be used to extract the skeleton points.

[0064] The sound features include voiceprint features and semantic features, and the process of obtaining the sound features is as follows:

[0065] Audio data of the to-be-tested personnel in the detection process is obtained, and the audio data corresponding to the audio data can be obtained according to the audio data, the fluctuation of the sound is obtained, that is, the voiceprint features corresponding to the audio data are determined, and the voiceprint features reflect the fluctuation of the audio data. At the same time, the text information in the audio data is extracted, and the text information is used to reflect the narrative content of the to-be-tested personnel in the audio data. Then the text information is input into the transformer model to determine the semantic features. The transformer model can perform semantic understanding and emotion analysis on the input text information to obtain the semantic features corresponding to the text information.

[0066] Step S102: input the image features and the sound features into a preset psychological detection model to obtain psychological detection data.

[0067] Before step S102, the psychological detection model needs to be constructed, and the specific process is as follows:

[0068] Firstly, a training data set is obtained, and the training data set includes multiple groups of image features and sound features under different psychological states; the image features and the sound features in the training data set are obtained by the same method as in step S101, and are not described herein. Each group of image features and sound features corresponds to a psychological state.

[0069] According to the relationship between the image features and the sound features in the training data set and the psychological state, a logistic function and a log loss function are established. In a specific embodiment, the logistic function is P(Y=1|X)=sigmoid(βX), where β is a regression coefficient, X is one of the image features or the sound features, that is, X is one of the expression features, the behavior features, the voiceprint features and the semantic features, and Y represents one of the multiple psychological states. When X is the expression feature, β is the expression regression coefficient; when X is the behavior feature, β is the behavior regression coefficient; when X is the voiceprint feature, β is the voiceprint regression coefficient; and when X is the semantic feature, β is the semantic regression coefficient. The log loss function is L(β)=-sum(y*log(p)+(1-y)*log(1-p)), where y in the log loss function and Y in the logistic function have the same meaning, and p represents the logistic function P(Y=1|X).

[0070] According to the maximum likelihood estimation method, the logistic function and the log loss function, the image regression coefficient corresponding to the image features and the sound regression coefficient corresponding to the sound features are determined.

[0071] In a specific embodiment, according to the maximum likelihood estimation method, β=argmax(L(β)) can be obtained, and in combination with the logistic function and the log loss function, β=argmax(-sum(y*log(sigmoid(βX))+(1-y)*log(1-sigmoid(βX))) can be obtained. According to the above description, when X is different, β also has different meanings. Since X can be one of the expression features, the behavior features, the voiceprint features and the semantic features, the calculated β also has four values of the expression regression coefficient, the behavior regression coefficient, the voiceprint regression coefficient and the semantic regression coefficient. It can be understood that the image regression coefficient includes the expression regression coefficient and the behavior regression coefficient, and the sound regression coefficient includes the voiceprint regression coefficient and the semantic regression coefficient.

[0072] According to the preset factor loading matrix, the image regression coefficient and the sound regression coefficient, the psychological detection model is determined.

[0073] Specifically, the factor loading matrix is calculated according to an independent variable matrix and a factor matrix, elements in the independent variable matrix represent different psychological states, and the factor matrix is calculated by an expectation maximization algorithm. A target matrix is obtained by multiplying a regression coefficient matrix and the factor loading matrix, the regression coefficient matrix includes an image regression coefficient and a sound regression coefficient; the target matrix is normalized to obtain the psychological detection model.

[0074] In a specific embodiment, Q = LF + ε; wherein Q is an independent variable matrix, F is a factor matrix, L is a factor loading matrix, and ε is an error term. Given the independent variable matrix and the factor matrix, the factor matrix can be rotated, and then the independent variable matrix and the rotated factor matrix are multiplied to obtain the factor loading matrix. The expression regression coefficient, the behavior regression coefficient, the voiceprint regression coefficient and the semantic regression coefficient are combined to form a regression coefficient matrix with four rows and one column, the factor loading matrix and the regression coefficient matrix are multiplied to obtain a target matrix, and then the target matrix is normalized to obtain the psychological detection model, that is, W = normalize(β*L), wherein β represents the regression coefficient matrix in the psychological detection model. The normalize function is used to make the norm or numerical range of the input matrix, that is, the target matrix, within a certain range. According to different actual situations, the target matrix can be normalized by maximum and minimum value, zero mean value or standard deviation, or the target matrix can be normalized in other ways, so that the normalized target matrix meets a specific proportional relationship.

[0075] Each element in the factor loading matrix can represent the correlation between a certain psychological state and a certain feature. When the psychological detection model is used for psychological detection, a certain feature of the testee is placed in the position corresponding to the feature in the factor loading matrix, and the calculated W value is the psychological detection data.

[0076] Step S103: According to the preset result comparison table and the psychological detection data, the psychological detection result of the testee is determined.

[0077] Specifically, the result comparison table includes the range of the psychological detection data and the corresponding psychological detection result of the range, the calculated psychological detection data is matched with the range in the result comparison table to obtain the psychological detection result of the testee.

[0078] Reference Figure 3, the image and audio sent by the AR glasses 1 are analyzed respectively. According to the image, the face and the skeleton points on the body of the to-be-tested personnel are extracted respectively, the expression feature is obtained by performing expression recognition on the face, the action data corresponding to the skeleton points is determined according to the skeleton points, and the behavior feature is obtained by performing analysis on the action data using an LSTM. According to the audio, the text information in the audio and the sound fluctuation corresponding to the audio are obtained respectively to obtain the voiceprint feature, the semantic feature is obtained by performing semantic analysis on the text information using a Transformer model, and the expression feature, the behavior feature, the voiceprint feature and the semantic feature are input into the psychological detection model to obtain the psychological detection result.

[0079] The embodiments provided in the application evaluate the psychological state of the to-be-tested personnel by considering the factors such as the expression, behavior, voice and speaking content of the to-be-tested personnel, and improve the accuracy of the psychological state evaluation.

[0080] At the same time, the correlation between the expression and the behavior is considered, the group of static features of the facial expression and the group of action data of the body behavior are analyzed by using the Transformer model, the changes of the expression and the behavior are more easily captured, and therefore more accurate psychological state evaluation is realized.

[0081] In other embodiments, any one of the expression feature and the behavior feature can be used as the image feature to analyze the psychological state, any one of the voiceprint feature and the semantic feature can be used as the sound feature to analyze the psychological state, and factors such as the gesture of the to-be-tested personnel and the environment in which the to-be-tested personnel is located can be added to the image feature to evaluate the psychological state of the to-be-tested personnel, and the like are not limited.

[0082] The embodiment of the application provides a psychological state detection system, referring to Figure 4 , the psychological state detection system comprises:

[0083] The data acquisition module 201 is configured to acquire image features and sound features, the image features representing the behavior and / or expression of the to-be-tested personnel in the detection process, and the sound features representing the voice and / or language of the to-be-tested personnel in the detection process.

[0084] The data calculation module 202 is configured to input the image features and the sound features into a preset psychological detection model to obtain psychological detection data.

[0085] The result determination module 203 is configured to determine the psychological detection result of the to-be-tested personnel according to a preset result reference table and the psychological detection data.

[0086] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the foregoing method embodiment, and will not be described here.

[0087] Embodiments of the present application disclose an electronic device. Referring to Figure 5 , the electronic device includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage portion 307 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for system operation are also stored. The CPU 301, the ROM 302, and the RAM 303 are connected to each other through a bus. An input / output (I / O) interface 304 is also connected to the bus.

[0088] The following components are connected to the I / O interface 304: an input portion 305 including a keyboard, a mouse, and the like; an output portion 306 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage portion 307 including a hard disk, and the like; and a communication portion 308 including a network interface card such as a local area network (LAN) card, a modem, and the like. The communication portion 308 performs communication processing via a network such as the Internet. A drive 309 is also connected to the I / O interface 304 as necessary. A removable medium 310 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 309 as necessary, so that a computer program read therefrom is installed in the storage portion 307 as necessary.

[0089] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts Figure 2 may be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication portion 308, and / or installed from the removable medium 310. When the computer program is executed by the central processing unit (CPU) 301, the above-described functions defined in the apparatus of the present application are performed.

[0090] It should be noted that the computer readable medium shown in the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a carrier wave in a carrier wave part, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.

[0091] The above description is only the preferred embodiment of the present application and the explanation of the technical principles used. Those skilled in the art should understand that the scope of the application involved in the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above application concept. For example, the above features can be replaced with technical features with similar functions applied in the present application (but not limited to) to form technical solutions.

Claims

1. A method for detecting psychological states, characterized in that, include: Acquire image features and sound features, wherein the image features represent the behavior and / or facial expressions exhibited by the test subject during the detection process, and the sound features represent the sounds and / or language spoken by the test subject during the detection process; The image features and sound features are input into a preset psychological detection model to obtain psychological detection data; Based on the preset result comparison table and the psychological test data, the psychological test results of the test subject are determined; The psychological testing model was determined in the following way: Obtain a training dataset, which includes multiple sets of image and sound features under different psychological states; Based on the relationship between image features and sound features and psychological states in the training dataset, a logistic function and a log loss function are established. Based on the maximum likelihood estimation method, the logistic function, and the log loss function, the image regression coefficients corresponding to the image features and the sound regression coefficients corresponding to the sound features are determined respectively. The target matrix is ​​obtained by multiplying the regression coefficient matrix and the factor loading matrix, wherein the regression coefficient matrix includes image regression coefficients and sound regression coefficients; The factor loading matrix is ​​calculated based on the independent variable matrix and the factor matrix; the elements in the independent variable matrix represent different psychological states, and the factor matrix is ​​calculated using the expectation-maximization algorithm. The target matrix is ​​normalized to obtain the psychological detection model, which specifically includes: Using Q = LF + ε, where Q is the independent variable matrix, F is the factor matrix, L is the factor loading matrix, and ε is the error term; given the independent variable matrix and factor matrix, the factor matrix is ​​rotated, and then the independent variable matrix and the rotated factor matrix are multiplied to obtain the factor loading matrix; the facial expression regression coefficient, behavioral regression coefficient, voiceprint regression coefficient, and semantic regression coefficient are combined into a 4x1 regression coefficient matrix, and the factor loading matrix and regression coefficient matrix are multiplied to obtain the target matrix, and then the target matrix is ​​normalized to obtain the psychological testing model, i.e., W = normalize(β*L), where β represents the regression coefficient matrix, and the normalize function is used to make the norm or numerical range of the input matrix, i.e., the target matrix, within a certain range.

2. The psychological state detection method according to claim 1, characterized in that, The image features include facial expression features and behavioral features, and the image features are obtained through the following methods: Acquire multiple facial images and multiple behavioral images of the person to be tested during the detection process; The multiple facial images and their corresponding acquisition times are input into a preset facial expression recognition model to determine facial expression features; Extract motion data from the behavior image; Behavioral characteristics are determined based on the long short-term memory network model and the action data.

3. The psychological state detection method according to claim 1, characterized in that, The voice features include voiceprint features and semantic features, and the voice features are obtained through the following methods: Acquire the audio data of the person being tested during the testing process; Determine the voiceprint features corresponding to the audio data, wherein the voiceprint features reflect the fluctuation of the audio data; Extract text information from the audio data; the text information is used to reflect the content of the test subject's narration in the audio data. Based on the textual information, semantic features are determined.

4. A psychological state detection system, characterized in that, The method described by any one of claims 1-3 includes: The data acquisition module is used to acquire image features and sound features, wherein the image features represent the behavior and / or facial expressions exhibited by the test subject during the detection process, and the sound features represent the voices and / or language spoken by the test subject during the detection process; The data calculation module is used to input the image features and the sound features into a preset psychological testing model to obtain psychological testing data; The result determination module is used to determine the psychological test results of the test subject based on a preset result comparison table and the psychological test data.

5. A psychological state detection system, characterized in that, Includes AR glasses (1) and a server (2); The AR glasses (1) are used to acquire image features and sound features; The server (2) is used to execute the mental state detection method as described in any one of claims 1 to 3.

6. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1 to 3.

Images