Personality prediction method and system based on millimeter wave radar and facial expression analysis

Abstract

The invention relates to the technical field of personality prediction, in particular to a personality prediction method and system based on millimeter wave radar and facial expression analysis, and the method comprises the steps: obtaining a millimeter wave signal of the face of a user; generating a facial expression feature map according to the millimeter wave signal; extracting a time-frequency feature and a facial feature at each time point in the facial expression feature map, and generating a multi-scale feature map according to the time-frequency feature and the facial feature at each time point; and identifying one or more emotional features of the user according to the multi-scale feature map, and predicting personality traits of the user based on the one or more emotional features of the user. Therefore, the problems that the risk of privacy disclosure is easily caused, the influence of light rays and angles is easily caused and the accuracy is poor due to the fact that character features are analyzed in a mode of capturing facial expressions through a camera in the related technology are solved.

Description

Technical Field

[0001] This application relates to the technical field of personality prediction, and particularly to a personality prediction method and system based on millimeter-wave radar and facial expression analysis. Background Art

[0002] In the research of psychology and behavior, the assessment of personality traits usually relies on traditional methods such as direct observation, self-report questionnaires, and interviews. However, these methods have certain limitations, including but not limited to privacy issues, subjective biases, and ethical challenges. With the increasing awareness of personal privacy protection in society and the development of technology, there is a need for a more objective, non-invasive, and privacy-respecting way to assess an individual's personality traits.

[0003] In related technologies, a camera is usually used to capture facial expressions, and then image processing and machine learning algorithms are used to analyze emotional states and personality characteristics. However, this method is prone to the risk of privacy leakage and may affect the recognition accuracy due to factors such as light and angle. Summary of the Invention

[0004] This application provides a personality prediction method and system based on millimeter-wave radar and facial expression analysis to solve the problems in related technologies that analyzing personality characteristics by capturing facial expressions through a camera is prone to the risk of privacy leakage, is easily affected by light and angle, and has poor accuracy.

[0005] The first aspect of the embodiments of this application provides a personality prediction method based on millimeter-wave radar and facial expression analysis, including the following steps: obtaining millimeter-wave signals of a user's face; generating a facial expression feature map according to the millimeter-wave signals; extracting time-frequency features and facial features at each time point in the facial expression feature map, and generating a multi-scale feature map according to the time-frequency features and facial features at each time point; identifying one or more emotional features of the user according to the multi-scale feature map, and predicting the user's personality traits based on the one or more emotional features of the user.

[0006] Optionally, obtaining millimeter-wave signals of a user's face includes: transmitting a collection signal and receiving a reflected signal of the user's face; mixing the collection signal and the reflected signal to obtain millimeter-wave signals of the user's face.

[0007] Optionally, generating a facial expression feature map according to the millimeter-wave signals includes: decomposing the millimeter-wave signals into high-frequency components and low-frequency components through continuous wavelet transform, where the high-frequency components include environmental noise or other irrelevant information, and the low-frequency components include facial muscle movement information; removing the corresponding millimeter-wave signals of the high-frequency components, and reconstructing the millimeter-wave signals corresponding to the low-frequency components by using inverse wavelet transform; generating a facial expression feature map according to the reconstructed facial expression signals.

[0008] Optionally, one or more emotional characteristics of the user are identified based on the multi-scale feature map, including: inputting the multi-scale feature map into a pre-trained network model, and the network model outputs one or more emotional characteristics of the user. Among them, the network model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The input-output process of the network model includes: taking the multi-scale feature map as high-dimensional image data and inputting it into the network model through the input layer. Among them, hierarchical features from basic edges to complex patterns in the high-dimensional image data are extracted through multiple convolutional layers, the pooling layer identifies key features in the hierarchical features, the fully connected layer integrates the key features, and determines one or more emotional characteristics of the user based on the key features, and the output layer outputs one or more emotional characteristics of the user.

[0009] Optionally, based on one or more emotional characteristics of the user, the personality traits of the user are predicted, including: converting one or more emotional characteristics of the user into score values corresponding to the personality traits; determining the personality traits of the user according to the score values and the self-test data of the user.

[0010] Optionally, the facial expression feature map includes: a static feature map and a dynamic feature map; among them, the static feature map represents the bone structure, facial symmetry, and the ratio between facial features, and the dynamic feature map represents instantaneous facial movements.

[0011] An embodiment of the second aspect of the present application provides a personality prediction system based on millimeter-wave radar and facial expression analysis. The system is used to implement the personality prediction method based on millimeter-wave radar and facial expression analysis in the above embodiment, including: a millimeter-wave radar module for transmitting and receiving millimeter-wave signals of the face; a signal processing module for processing the millimeter-wave signals to generate a facial expression feature map, and extracting the time-frequency features and facial features at each time point in the facial expression feature map; generating a multi-scale feature map according to the time-frequency features and facial features at each time point; a machine learning module for identifying one or more emotional characteristics of the user according to the multi-scale feature map; a personality prediction engine for predicting the personality traits of the user based on one or more emotional characteristics of the user.

[0012] An embodiment of the third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor executes the program to implement the personality prediction method based on millimeter-wave radar and facial expression analysis in the above embodiment.

[0013] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium, on which a computer program or instruction is stored. When the computer program or instruction is executed, it is used to implement the personality prediction method based on millimeter-wave radar and facial expression analysis in the above embodiment.

[0014] The fifth aspect of the present application provides a computer program product, including: a computer program or instructions, which when executed, can implement the personality prediction method based on millimeter-wave radar and facial expression analysis as described in the above embodiments.

[0015] Therefore, the present application has at least the following beneficial effects:

[0016] In the embodiments of the present application, by acquiring the millimeter-wave signal of the user's face, non-contact capture of the user's facial expression is realized, and by processing the radar signal, a detailed facial expression feature map is generated. According to the time-frequency features and facial features at each time point in the facial expression feature map, a multi-scale feature map is generated. By processing the multi-scale feature map, one or more emotional features of the user are recognized, and accurate prediction of the user's personality traits can be realized without involving taking pictures of the user's facial images, greatly improving the privacy and security of the user. Thus, it solves the problems in the related art that analyzing personality traits by capturing facial expressions through a camera is prone to the risk of privacy leakage, is easily affected by light and angle, and has poor accuracy.

[0017] The additional aspects and advantages of the present application will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present application will become obvious and easy to understand from the following description of the embodiments in conjunction with the drawings, where:

[0019] Figure 1 is a flowchart of the personality prediction method based on millimeter-wave radar and facial expression analysis according to the embodiments of the present application;

[0020] Figure 2 is a schematic diagram of signal transmission and processing according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of the network model according to an embodiment of the present application;

[0022] Figure 4 is a distribution diagram of emotions under the principal component analysis method according to an embodiment of the present application;

[0023] Figure 5 is an example diagram of the personality prediction method based on millimeter-wave radar and facial expression analysis according to an embodiment of the present application;

[0024] Figure 6 is a block diagram of the personality prediction system based on millimeter-wave radar and facial expression analysis according to the embodiments of the present application;

[0025] Figure 7 Schematic diagram of the working principle of a personality prediction system based on millimeter-wave radar and facial expression analysis provided according to an embodiment of the present application;

[0026] Figure 8 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0027] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals indicate the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application and should not be construed as limiting the present application.

[0028] A personality prediction method and system based on millimeter-wave radar and facial expression analysis according to an embodiment of the present application will be described below with reference to the accompanying drawings. In view of the problems mentioned in the above background art, the present application provides a personality prediction method based on millimeter-wave radar and facial expression analysis. In this method, by acquiring the millimeter-wave signal of the user's face, non-contact capture of the user's facial expression is achieved, and by processing the radar signal, a detailed facial expression feature map is generated. According to the time-frequency features and facial features at each time point in the facial expression feature map, a multi-scale feature map is generated, and by processing the multi-scale feature map, one or more emotional features of the user are identified, which can accurately predict the user's personality traits without involving taking pictures of the user's facial image, greatly improving the privacy security of the user.

[0029] Specifically, Figure 1 Schematic flow chart of a personality prediction method based on millimeter-wave radar and facial expression analysis provided according to an embodiment of the present application.

[0030] As Figure 1 shown, the personality prediction method based on millimeter-wave radar and facial expression analysis includes the following steps:

[0031] In step S101, a millimeter-wave signal of the user's face is acquired.

[0032] In an embodiment of the present application, acquiring the millimeter-wave signal of the user's face includes: transmitting an acquisition signal and receiving the reflected signal of the user's face; mixing the acquisition signal and the reflected signal to obtain the millimeter-wave signal of the user's face.

[0033] It can be understood that, as Figure 2As shown, in the embodiment of the present application, a millimeter-wave radar transmits a signal of a specific frequency and receives the signal reflected from the user's face. After mixing processing, a millimeter-wave signal containing rich facial muscle movement information is obtained, laying a foundation for subsequent emotion recognition and personality trait prediction. This not only improves the accuracy of data collection but also effectively protects user privacy without actually touching or photographing the user's facial image.

[0034] Specifically, the embodiment of the present application can use a frequency-modulated continuous-wave radar to capture signals that change over time. These signals contain information about facial movements. The transmitted acquisition signal interacts with the facial surface, and the reflected signal is mixed with the original signal to generate a millimeter-wave signal, i.e., an intermediate-frequency signal, which can include: distance: representing the distance between the facial surface and the radar, speed: representing the movement of facial muscles, and time variation: representing the changes caused by expressions over time.

[0035] Among them, the transmitted signal can be expressed as:

[0036]

[0037] Among them, fcs represents the starting frequency, B w represents the bandwidth, and Tsw represents the chip duration.

[0038] After the reflected signal is mixed with the original acquisition signal, an intermediate-frequency (IF) signal is generated. This signal contains key data related to distance, movement, and facial expression changes, which are specifically represented as follows:

[0039]

[0040] In step S102, a facial expression feature map is generated based on the millimeter-wave signal.

[0041] Among them, the facial expression feature map includes: a static feature map and a dynamic feature map; among them, the static feature map represents the skeletal structure, facial symmetry, and the proportions between facial features, which remain consistent over time (for example, the distance between the two eyes, the width of the mouth), and the dynamic feature map represents instantaneous facial movements, including instantaneous facial movements such as micro-expressions, blinks, lip movements, and other muscle activities that change over time. These features can reflect an individual's emotional state and provide real-time insights into their behavior.

[0042] In one embodiment of the present application, generating a facial expression feature map based on millimeter-wave signals includes: decomposing millimeter-wave signals into high-frequency components and low-frequency components through continuous wavelet transform, where the high-frequency components include environmental noise or other irrelevant information, and the low-frequency components include facial muscle movement information; removing the corresponding millimeter-wave signals of the high-frequency components, and reconstructing the millimeter-wave signals corresponding to the low-frequency components using inverse wavelet transform; generating a facial expression feature map based on the reconstructed facial expression signals.

[0043] To reduce environmental noise and isolate relevant features of facial expressions, embodiments of the present application can effectively decompose the frequency components of signals using continuous wavelet transform, identify and isolate meaningful frequency bands related to facial expressions, thereby retaining key facial muscle movement features.

[0044] Specifically, the continuous wavelet transform (CWT) is used to decompose the original millimeter-wave signal into different frequency components. Wavelet transform is an effective time-frequency analysis tool that can provide localized information of the signal at different scales. The specific formula is as follows:

[0045] W(a,b)=∫R(t)ψ * (a,b)dt

[0046] where ψ*(a,b) is the complex conjugate of the wavelet function. This transform enables the radar signal to be decomposed into its frequency components. Particular attention is paid to distinguishing high-frequency components (usually containing environmental noise or other irrelevant information) and low-frequency components (containing signals related to facial muscle movement).

[0047] Furthermore, the continuous wavelet transform (CWT) is applied again to filter out high-frequency noise components and then reconstruct the signal, only retaining important data related to facial muscle activity. The denoised signal is reconstructed using inverse wavelet transform:

[0048]

[0049] where ψ(a,b) is the complex conjugate of the wavelet function. This transform can decompose the radar signal into its frequency components. Finally, embodiments of the present application can generate a facial expression feature map based on the reconstructed millimeter-wave signal. Thus, the key signal features representing facial muscle movement are retained, thereby improving the accuracy of emotion and personality prediction.

[0050] In step S103, extract the time-frequency features and facial features at each time point in the facial expression feature map, and generate a multi-scale feature map based on the time-frequency features and facial features at each time point.

[0051] It can be understood that the embodiments of the present application can extract the time-frequency features and facial features at each time point in the facial expression feature map through the continuous wavelet transform method. These features can provide information about the changes of expressions over time and frequency, which helps to capture the dynamic change process of expressions, so as to more accurately capture subtle expression changes.

[0052] Furthermore, the embodiments of the present application can generate multi-scale feature maps according to the time-frequency features and facial features at each time point, which can effectively represent the time evolution process and spatial structure features of the original facial expressions. Among them, the resolution of the generated multi-scale feature maps can be 400×400 pixels, and the Jet128 color scheme is used to enhance the clarity.

[0053] In step S104, one or more emotion features of the user are recognized according to the multi-scale feature map, and based on the one or more emotion features of the user, the personality traits of the user are predicted.

[0054] In an embodiment of the present application, recognizing one or more emotion features of the user according to the multi-scale feature map includes: inputting the multi-scale feature map into a pre-trained network model, and the network model outputs one or more emotion features of the user. Among them, the network model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The input-output process of the network model includes: inputting the multi-scale feature map as high-dimensional image data into the network model through the input layer. Among them, hierarchical features from basic edges to complex patterns in the high-dimensional image data are extracted through multiple convolutional layers, the pooling layer identifies key features in the hierarchical features, the fully connected layer integrates the key features, and one or more emotion features of the user are determined based on the key features, and the output layer outputs one or more emotion features of the user.

[0055] Among them, the network model can be a convolutional neural network. In the embodiments of this application, the network model can be trained first. First, five core emotion features are defined, namely neutral, smiling, sad, angry, and surprised. These emotion categories are selected based on psychological research because they are generally representative in different cultures and have a strong correlation with personality traits. Among them, neutral is used as a baseline to allow comparison with other expressions. Smiling is related to positive emotions and personality traits (such as approachability and extroversion). Sadness reflects a negative emotional state and is related to neuroticism. Anger represents a high level of excitement and is often related to characteristics such as impulsiveness or low approachability. Surprise represents curiosity and openness to new experiences and is related to characteristics such as openness. In the embodiments of this application, a total of 2,500 training samples can be used for training and testing. These samples can cover the above five emotions and contain as diverse facial features and expression changes as possible to improve the generalization ability of the model. During actual execution, approximately 2,000 samples can be divided into a training data set to train the network model. After training, the remaining 500 samples are used as a test data set to verify the network model. The trained model can effectively identify and classify different emotional states.

[0056] In the embodiments of this application, the multi-scale feature map is input into the pre-trained network model, and the network model outputs one or more emotion features of the user, such as Figure 3 shown. Specifically:

[0057] Input layer: The multi-scale feature map enters the network model through the input layer as high-dimensional image data.

[0058] Convolutional layer: The input high-dimensional image data is processed through a series of convolutional layers. Each convolutional layer applies a set of filters to scan the entire image and extract various hierarchical features from basic edges to more complex patterns. These filters can automatically learn and identify important structures in the image, such as lines, textures, and shapes, and gradually construct a representation useful for emotion classification.

[0059] Pooling layer: Reduces the spatial size of the feature map while retaining the most important feature information. This step helps reduce the computational cost, reduce the risk of overfitting, and focus on the key features that best represent different emotional states.

[0060] Fully connected layer: The key features obtained after convolution and pooling are fed into the fully connected layer for integration. The fully connected layer usually contains a large number of neurons and is used to learn complex non-linear mappings. Finally, the input data is mapped to relevant emotional states;

[0061] Output layer: Outputs one or more emotion features of the user.

[0062] In one embodiment of the present application, predicting the personality traits of a user based on one or more emotional characteristics of the user includes: converting the one or more emotional characteristics of the user into scoring values corresponding to the personality traits; and determining the personality traits of the user according to the scoring values and the self-test data of the user.

[0063] Based on the above embodiments, the embodiments of the present application can identify the current emotional characteristics of the user. These emotional characteristics may include, but are not limited to, neutral, smiling, sad, angry, and surprised. According to psychological research, different facial expressions are associated with specific personality traits. For example: Openness is associated with the expression of surprise. Conscientiousness is associated with neutral or controlled expressions. Extraversion can be reflected in smiling and energetic expressions. Agreeableness is manifested through calm and approachable facial expressions. Emotional stability (or neuroticism): manifested as sad or angry expressions.

[0064] Based on this association, the embodiments of the present application can convert the identified emotional characteristics into scoring values corresponding to the personality traits. For example, for each identified emotion, a personality trait score is assigned. If "smiling" is detected, the "extraversion" score is increased; if "sad" is detected, the "emotional instability" score is increased. Or different weights are assigned to different emotions based on their degree of influence on personality traits. For example, "smiling" may affect "extraversion" more strongly than "surprise".

[0065] Further, in order to further improve the prediction accuracy, the embodiments of the present application can also combine and collect the data of the user's personality report questionnaire. As Figure 4 shown, using statistical methods such as principal component analysis, combining the scoring values with the self-test data of the user to determine the personality traits of the user, using objective radar data and subjective questionnaire input to improve the comprehensiveness and accuracy of the prediction, so that the results obtained can accurately reflect the true personality traits of the user.

[0066] The following combines a specific example to detail the personality prediction method based on millimeter-wave radar and facial expression analysis in the embodiments of the present application. As Figure 5 shown, it includes the following steps:

[0067] Step 1: The millimeter-wave radar captures the reflected signals in the user's facial area by transmitting and receiving signals. Among them, Tx represents the transmitter and Rx represents the receiver. The millimeter-wave radar transmits and receives signals through multiple antenna arrays (as shown by the rectangular frames in Figure 5 ).

[0068] Step 2: The signals received by the millimeter-wave radar usually contain noise and other interferences. The embodiments of the present application can use wavelet decomposition technology to preliminarily process the original signals and extract the features related to facial expressions.

[0069] Step 3: Remove high-frequency noise through a low-pass filter, retain the low-frequency signal, further improve the signal quality, and apply thresholding to convert the signal into a binary form. Use continuous wavelet transform to further extract the time-frequency features in the signal and generate a multi-scale feature map.

[0070] Step 4: After the above signal processing steps, the embodiments of the present application can extract detailed facial expression features from millimeter-wave signals. Based on the extracted facial expression features, use a network model (such as a convolutional neural network CNN) to identify the user's emotional features, and combine the user's self-test data (such as a questionnaire survey) to further verify and adjust the accuracy of the emotional features. Finally, according to the Big Five personality theory (openness, conscientiousness, extraversion, agreeableness, and neuroticism), output the user's personality traits.

[0071] According to the personality prediction method based on millimeter-wave radar and facial expression analysis proposed by the embodiments of the present application, by acquiring the millimeter-wave signal of the user's face, non-contact capture of the user's facial expression is realized, and by processing the radar signal, a detailed facial expression feature map is generated. According to the time-frequency features and facial features at each time point in the facial expression feature map, a multi-scale feature map is generated. By processing the multi-scale feature map, one or more emotional features of the user are identified, and accurate prediction of the user's personality traits can be realized without involving taking pictures of the user's facial images, greatly improving the privacy and security of the user.

[0072] Next, a personality prediction system based on millimeter-wave radar and facial expression analysis proposed by the embodiments of the present application is described with reference to the accompanying drawings. This system is used to implement the personality prediction method based on millimeter-wave radar and facial expression analysis in the above embodiments.

[0073] As Figure 6 shown, the personality prediction system 10 based on millimeter-wave radar and facial expression analysis includes: a millimeter-wave radar module 100, a signal processing module 200, a machine learning module 300, and a personality prediction engine 400.

[0074] Among them, the millimeter-wave radar module 100 is used to transmit and receive the millimeter-wave signal of the face; the signal processing module 200 is used to process the millimeter-wave signal to generate a facial expression feature map and extract the time-frequency features and facial features at each time point in the facial expression feature map; the machine learning module 300 is used to generate a multi-scale feature map according to the time-frequency features and facial features at each time point; the personality prediction engine 400 is used to identify one or more emotional features of the user according to the multi-scale feature map and predict the user's personality traits based on the one or more emotional features of the user.

[0075] Specifically, the millimeter-wave radar module 100 in the embodiments of the present application may include a signal source, a transmitting (TX) antenna, a receiving (RX) antenna, a mixer, a low-pass filter (LPF), and an analog-to-digital converter (A / D). The radar module captures facial expression data through a frequency-modulated continuous wave (FMCW) signal, and each signal contains 256 data samples. The frequency increase expression of the FMCW signal is as follows:

[0076] f(t) = f_0 + (B / T) * t

[0077] where f_0 is the initial frequency, B is the sweep bandwidth, and T is the sweep duration.

[0078] It should be noted that the initial frequency of the millimeter-wave radar module 100 in the embodiments of the present application is set to 77 GHz, the sweep bandwidth is 4 GHz, and it supports an intermediate frequency bandwidth of up to 15 MHz. It can be a TI IWR1443 millimeter-wave radar, which has excellent ability to capture high-resolution facial muscle movements, while also maintaining non-invasive and privacy-conscious data collection. The radar operates in the 77 - 81 GHz frequency range and emits millimeter-wave signals towards the participant's face. After these signals are reflected, they can provide detailed data on facial muscle movements. During actual execution, the radar emits chirp signals whose frequency increases linearly with time, enabling precise tracking of the time delay of the reflections from the facial surface, and the reflected signals carry key information about range, speed, and facial movements.

[0079] Specifically, as Figure 7As shown, the millimeter-wave radar 100 is used to transmit and receive millimeter-wave signals of the face. The signal processing module 200 is responsible for preprocessing the received millimeter-wave signals, including steps such as filtering, denoising, and feature extraction, to generate a high-quality facial expression feature map. In the actual execution process, the CWT (Continuous Wavelet Transform) is used to convert the millimeter-wave signal into a multi-scale feature map in the time-frequency domain. The CWT can effectively extract the time and frequency characteristics of the signal, providing rich input data for the subsequent training of the machine learning module. The machine learning module 300 uses deep learning technology, especially CNN (Convolutional Neural Network), to analyze the generated multi-scale feature map and identify complex patterns related to specific emotions and personality traits. In the actual execution process, an efficient CNN model is trained in the embodiment of the present application. This model can automatically learn the features in the scalar map and correspond them to predefined emotion labels (such as neutral, smiling, sad, angry, and surprised, etc.). The personality prediction engine 400 can convert the emotion features identified from the facial expression into specific personality trait scores to achieve personality prediction. In the actual execution process, the embodiment of the present application can use objective radar data and subjective questionnaire inputs, not only considering the objective physiological indicators extracted from the radar signals, but also integrating the information of the participants' self-assessment questionnaires to ensure the comprehensiveness and accuracy of the prediction results.

[0080] Figure 8 It is a schematic structural diagram of the electronic device provided by the embodiment of the present application. The electronic device may include:

[0081] A memory 801, a processor 802, and a computer program stored on the memory 801 and executable on the processor 802.

[0082] When the processor 802 executes the program, it implements the personality prediction method based on millimeter-wave radar and facial expression analysis provided in the above embodiment.

[0083] Furthermore, the electronic device further includes:

[0084] A communication interface 803 for communication between the memory 801 and the processor 802.

[0085] The memory 801 is used to store a computer program executable on the processor 802.

[0086] The memory 801 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0087] If the memory 801, the processor 802, and the communication interface 803 are implemented independently, the communication interface 803, the memory 801, and the processor 802 can be interconnected through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 8 only a thick line is used in Figure 8 , but it does not mean that there is only one bus or one type of bus.

[0088] Optionally, in a specific implementation, if the memory 801, the processor 802, and the communication interface 803 are integrated on a single chip, the memory 801, the processor 802, and the communication interface 803 can communicate with each other through an internal interface.

[0089] The processor 802 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0090] The embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned personality prediction method based on millimeter-wave radar and facial expression analysis is implemented.

[0091] The embodiments of the present application provide a computer program product, including: a computer program or instruction, and when the computer program or instruction is executed, the above-mentioned personality prediction method based on millimeter-wave radar and facial expression analysis is implemented.

[0092] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0093] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of this application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0094] Any process or method description shown in a flowchart or described in other ways herein can be understood to represent a module, segment, or portion of code including one or more N executable instructions for implementing a customized logical function or process, and the scope of the preferred embodiments of this application includes additional implementations, where the functions can be executed in a manner that is not in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the involved functions, which should be understood by those skilled in the art to which the embodiments of this application pertain.

[0095] It should be understood that each part of this application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware as in another embodiment, any one or a combination of the following technologies well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays, field programmable gate arrays, etc.

[0096] Those of ordinary skill in the technical field of this application can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0097] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A personality prediction method based on millimeter-wave radar and facial expression analysis, characterized in that, Including the following steps: Obtain the millimeter-wave signal of the user's face; Generate a facial expression feature map according to the millimeter-wave signal; Extract the time-frequency features and facial features at each time point in the facial expression feature map, and generate a multi-scale feature map according to the time-frequency features and facial features at each time point; Identify one or more emotional features of the user according to the multi-scale feature map, and predict the personality traits of the user based on the one or more emotional features of the user.

2. The personality prediction method based on millimeter-wave radar and facial expression analysis according to claim 1, wherein The obtaining of the millimeter-wave signal of the user's face includes: Transmit the acquisition signal and receive the reflected signal of the user's face; Mix the acquisition signal and the reflected signal to obtain the millimeter-wave signal of the user's face.

3. The personality prediction method based on millimeter-wave radar and facial expression analysis according to claim 1 or 2, characterized in that Generating a facial expression feature map according to the millimeter-wave signal includes: Decompose the millimeter-wave signal into high-frequency components and low-frequency components through continuous wavelet transform, where the high-frequency components include environmental noise or other irrelevant information, and the low-frequency components include facial muscle movement information; Remove the millimeter-wave signal corresponding to the high-frequency components, and reconstruct the millimeter-wave signal corresponding to the low-frequency components by inverse wavelet transform; Generate the facial expression feature map according to the reconstructed facial expression signal.

4. The personality prediction method based on millimeter-wave radar and facial expression analysis according to claim 1, characterized in that, Identifying one or more emotional features of the user according to the multi-scale feature map includes: Input the multi-scale feature map into a pre-trained network model, and the network model outputs one or more emotional features of the user, where the network model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer, and the input-output process of the network model includes: Input the multi-scale feature map as high-dimensional image data into the network model through the input layer, where hierarchical features from basic edges to complex patterns are extracted from the high-dimensional image data through multiple convolutional layers, the pooling layer identifies key features in the hierarchical features, the fully connected layer integrates the key features, and determines one or more emotional features of the user based on the key features, and the output layer outputs one or more emotional features of the user.

5. The personality prediction method based on millimeter-wave radar and facial expression analysis according to claim 1 or 4, characterized in that, Predicting the personality traits of the user based on the one or more emotional features of the user includes: Convert the one or more emotional features of the user into a scoring value corresponding to the personality traits; Determine the personality traits of the user according to the scoring value and the self-test data of the user.

6. The personality prediction method based on millimeter-wave radar and facial expression analysis according to claim 1, characterized in that The facial expression feature map includes: a static feature map and a dynamic feature map; where the static feature map represents the bone structure, facial symmetry, and the ratio between facial features, and the dynamic feature map represents instantaneous facial movements.

7. A personality prediction system based on millimeter-wave radar and facial expression analysis, characterized in that, The system for implementing the personality prediction method based on millimeter-wave radar and facial expression analysis according to any one of claims 1-6 includes: A millimeter-wave radar module for transmitting and receiving millimeter-wave signals of the face; A signal processing module for processing the millimeter-wave signal to generate a facial expression feature map, and extracting time-frequency features and facial features at each time point in the facial expression feature map; A machine learning module for generating a multi-scale feature map according to the time-frequency features and facial features at each time point; A personality prediction engine for identifying one or more emotional characteristics of a user based on the multi-scale feature map and predicting the personality traits of the user based on the one or more emotional characteristics of the user.

8. An electronic device, characterized in that, Comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the personality prediction method based on millimeter-wave radar and facial expression analysis according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instruction is executed, the personality prediction method based on millimeter-wave radar and facial expression analysis according to any one of claims 1-6 is implemented.

10. A computer program product, comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed, the personality prediction method based on millimeter-wave radar and facial expression analysis according to any one of claims 1-6 is implemented.

Images