Emotion perception system and method based on WiFi channel state analysis

Abstract

The invention belongs to the field of WiFi perception and emotion recognition, provides an emotion perception system and method based on WiFi channel state analysis, solves the problem that an existing system is high in cost, and facilitates laying of the system. The system adopts mobile equipment as a signal processing module, so that the system is more convenient to use; the judgment model uses the relationship between the human body breathing mode and the human emotion, and the limitation of the existing emotion perception system based on the human body action posture is improved; in addition, the system also adopts a multi-channel contrastive analysis method to eliminate sudden factors during signal acquisition as much as possible, so that the emotion judgment result of the system is more pleasant, and the anti-interference performance of the system is improved; and a database updating mechanism and a privacy security mechanism are also added, so that the data privacy of the user is ensured, the database is continuously expanded along with the increase of the user and the feedback of the user on the judgment result, and the data collection scene is continuously enriched, thereby enabling the judgment accuracy of the system to be higher and higher.

Description

Technical Field

[0001] The present invention belongs to the field of WiFi perception and emotion recognition, and in particular relates to an emotion perception system and method based on WiFi channel state analysis. Background Art

[0002] With the development of robotics, especially humanoid robots, human-computer interaction technology has attracted widespread attention from industry and academia.

[0003] In human-computer interaction technology, emotion recognition technology refers to how to make machines perceive human emotions and communicate with them in the same way as humans do in daily interactions. How to make machines recognize emotions accurately and quickly has become an urgent problem to be solved.

[0004] Emotion recognition based on wireless signals is a new emotion recognition technology that can improve the robustness of the emotion recognition system to environmental influences, reduce the use of dedicated equipment, and enable continuous emotion recognition.

[0005] There are two main types of existing emotion recognition systems based on wireless signals: 1) Using dedicated equipment to transmit wireless signals and analyzing the changes in wireless signals to analyze people's emotions. However, this solution requires the use of specific devices, such as radar transmitting and receiving devices, and the system cost is high, which is not conducive to popularization. 2) Using the already widely available WiFi signals to analyze and judge people's emotional state by detecting human body movements and postures. However, there are limitations to the correlation between human movements and emotional states, and specific emotions do not correspond one-to-one with specific movements and postures.

[0006] In addition, due to computing power limitations, the discrimination models of existing emotion recognition systems all run on hosts with strong computing power, which is not convenient enough. In addition, the existing emotion recognition system database has a small capacity and limited data collection scenarios, which will lead to problems such as inaccurate emotion recognition results, subjective performance comparisons between different emotion recognition systems, difficulty in reproducing experimental results, and difficulty in accumulating technological progress. Summary of the invention

[0007] The problem to be solved by the present invention is: on the one hand, to improve the portability of the existing emotion recognition system based on human body motion posture while improving the limitations of the existing system; on the other hand, to enrich the existing emotion recognition-related database and reduce users' concerns about the privacy security of their information.

[0008] This system adopts the following technical solutions to solve the above problems:

[0009] An emotion perception system based on WiFi channel status analysis, including a signal transmission module, a signal processing module, and an emotion discrimination module that deploys a discrimination model and a database;

[0010] The signal transmission module includes a WiFi signal generating unit and a signal transmission unit; the WiFi signal generating unit generates a WiFi signal and transmits it to the signal processing module through the signal transmission unit;

[0011] The signal processing module includes a signal receiving unit, a signal filtering unit, an information extraction unit, an information uploading unit, a result receiving unit, a result display unit, a result distinguishing unit, and a result feedback unit; the signal receiving unit receives the WiFi signal emitted by the signal transmitting module, and after being processed by the signal filtering unit and the information extraction unit, the information uploading unit transmits the signal to the emotion distinguishing module; the result receiving unit receives the distinguishing result output by the emotion distinguishing module, and displays the result through the result display unit, and the result distinguishing unit collects user evaluations, and then transmits the evaluations to the emotion distinguishing module through the result feedback unit;

[0012] The emotion discrimination module includes an information receiving unit, a data storage unit, a model training unit, a discrimination model unit, a result transmission unit, and a feedback receiving unit; the information receiving unit receives the data transmitted by the signal processing module and stores it in the data storage unit, the model training unit performs model training based on the received data, and then the discrimination model unit judges the result based on the data, and the judgment result is transmitted to the signal processing module through the result transmission unit, and the feedback receiving unit receives feedback information output by the signal processing module.

[0013] Furthermore, in the signal transmission module, the WiFi signal generation unit is supported by the capabilities of multiple routers and is used to generate multiple specific WiFi signals; the signal transmission unit, that is, the physical channel for signal transmission, the specific WiFi signal transmitted in the physical channel is affected by environmental objects and human bodies in the channel and then reaches the signal processing module.

[0014] Furthermore, in the signal processing module, the signal receiving unit is used to receive WiFi signals on each channel affected by the signal transmission unit in the environment; the signal filtering unit is used to remove noise superimposed on the WiFi signals on each channel; the information extraction unit is used to extract the channel state information CSI of the corresponding WiFi signal on each channel; the information uploading unit is used to package and encrypt the CSI obtained by the information extraction unit, and further send it to the emotion discrimination module; the result receiving unit is used to receive the discrimination result from the emotion discrimination module; the result display unit is used to prompt the user with the discrimination result of the current emotion obtained by the emotion discrimination module; the result discrimination unit is used to collect the user's evaluation on the correctness of the emotion discrimination result; the result feedback unit is used to feedback the user's evaluation information on the correctness of the emotion discrimination result to the emotion discrimination module.

[0015] Furthermore, in the emotion discrimination module, the information receiving unit is used to receive CSI from the information uploading unit in the signal processing module; the data storage unit is used to store CSI in different scenarios to improve the problem of limited data collection scenarios in the current relevant database; the model training unit is used to regularly train the emotion discrimination model according to the update of the data storage unit; the discrimination model unit is used to make a judgment on the state of the user in the current scenario according to the received CSI; the result downloading unit is used to send the judgment result made by the discrimination model according to the received CSI to the signal processing module; the feedback receiving unit is used to receive feedback from the user of the signal module on the correctness of the emotion discrimination result.

[0016] The emotion perception system method based on WiFi channel state analysis includes the following steps:

[0017] Step 1: With the user's knowledge and consent, the system starts running. The signal transmission module first generates a specific WiFi signal and transmits it on multiple channels. The signal reaches the signal processing module through the signal transmission unit. The signal transmission unit is the environment and the human body. The periodic ups and downs of the chest cavity when a person breathes affect the channel state information CSI of the signal, so that the WiFi signal carries information related to human activities.

[0018] Step 2: After receiving the WiFi signal in the environment, the signal processing module processes it to obtain the CSI on each channel and sends them to the emotion discrimination module;

[0019] Step 3: After receiving the CSI uploaded by the signal processing module, the emotion discrimination module makes a judgment on the current emotional state of the user according to the current model and transmits it back to the signal processing module;

[0020] Step 4: After receiving the emotion discrimination result, the signal processing module informs the user of the emotion discrimination result in the form of a message, and gives corresponding prompt information according to the emotion discrimination result to make the user feel comfortable, and at the same time prompts the user to evaluate the accuracy of the emotion discrimination result, and feeds back the evaluation result to the emotion discrimination module;

[0021] Step 5: The emotion discrimination module adjusts the database and updates the discrimination model based on the user feedback.

[0022] Furthermore, in step 1, the human body is in the physical channel of signal propagation. The periodic ups and downs of the chest during breathing affect the CSI of the signal, causing the amplitude |H| and phase angle ∠H of the channel matrix to change periodically. These periodic changes are enhanced and analyzed to recover the corresponding chest movement, that is, the breathing pattern.

[0023] Furthermore, in step 2, after receiving the WiFi signal in the environment, the signal processing module performs filtering, denoising and other processing on it, thereby obtaining the channel frequency response CFR information on multiple channels in the current environment, and further obtains the channel state information CSI on multiple channels according to the CFR; thereafter, the signal processing module packages and encrypts the CSI on multiple channels to protect the privacy and security of the user; finally, the encrypted packaged data is sent to the emotion discrimination module to discriminate the user's emotions;

[0024] CFR is the response of the signal characteristics in different frequency ranges, generally including amplitude / frequency and phase / frequency responses. In the case of a single transmitting antenna and a single receiving antenna, it is expressed by the following formula:

[0025]

[0026] Where f is the carrier frequency, which refers to the transmission frequency of the signal; t represents time; H(f,t) represents the channel frequency response, which describes the channel effect at frequency f and time t; N represents the number of multipaths, that is, the number of different paths the signal passes through; a n (t) is the amplitude attenuation factor, which represents the attenuation of the signal amplitude of each path over time; τ n (t) is the propagation delay, which represents the signal propagation delay of each path; is the phase factor, which indicates the phase shift of the signal on each path;

[0027] For the received CFR information, f in formula (1) is fixed to the carrier frequency within the frequency band of each of the multiple routers, thereby extracting multiple CSIs corresponding to different routers;

[0028] CSI is the expression of the wireless channel in the frequency domain using the channel state information matrix H under a specific carrier frequency f:

[0029] Y=H*X+N (2)

[0030] Among them, Y represents the vector of received signals; H represents the matrix of channel state information; X represents the vector of transmitted signals; N represents Gaussian white noise, which represents the random noise added during the communication process;

[0031] This channel matrix H representing CSI is expressed in the form of amplitude and phase;

[0032] H=|H|e j∠H (3)

[0033] Where |H| represents the amplitude of the CSI matrix; e j∠H represents the phase part of the CSI matrix; j represents the imaginary unit (i.e. ), and ∠H represents the phase angle of each element in the CSI matrix. The phase angle ∠H is a real number that represents the phase change of the signal after passing through the channel on the subcarrier. j∠H It is convenient to use complex numbers to represent the phase information of the channel.

[0034] Furthermore, in step 3, after receiving the data packet from the signal processing module, the emotion discrimination module first decrypts and splits it to obtain the original CSI on multiple channels in the user's current environment; the information receiving unit of the emotion discrimination module interpolates the signal on each channel to obtain CSI uniformly distributed in time, and independently stores the interpolated CSI in the data storage unit; the emotion discrimination module makes a comprehensive judgment on the user's current emotional state based on the CSI on multiple channels obtained after interpolation and the current discrimination model: happy, sad, angry, fearful, and returns the judgment result to the signal processing module.

[0035] Furthermore, in step 4, after receiving the judgment result from the emotion discrimination module, the signal processing module makes different speech prompts to the user to adjust the mood according to the different emotional states of the user; after the speech prompt, the signal processing module prompts the user to feedback whether the current emotion judgment result is accurate, as well as the current environment information, and feeds back the collected information to the emotion discrimination module, so that the emotion discrimination module can adjust its database according to the feedback information.

[0036] Furthermore, in step 5, if the emotion discrimination module receives feedback information that the emotion judgment result is correct, it will judge the validity of the CSI on each channel temporarily stored in the data storage unit in combination with the comprehensive judgment result obtained in step 3, and confirm the CSI that leads to the correct result as valid data, label them with the corresponding label of the correct judgment result, and add the current environment information fed back by the user, and store them in the data storage unit; at the same time, the emotion discrimination module will discard the CSI that leads to other erroneous results temporarily stored in the data storage unit; after the data storage unit is updated to a certain extent, the emotion discrimination module will re-train the machine learning model used for emotion judgment based on the data in the current data storage unit to update the parameters of the machine learning model to make the judgment result more accurate. This function stops when the system reaches a certain accuracy and stability. At this time, the machine learning model is directly sent to the signal processing for offline local emotion perception.

[0037] The present invention adopts the above technical solution to compare with the existing system solution, and has the following beneficial effects:

[0038] (1) An emotion recognition system based on WiFi breathing sensing is proposed, which reduces the high cost of the existing system and facilitates the deployment of the system. The system uses mobile devices as signal processing modules, making the system more convenient to use.

[0039] (2) The discriminant model uses the relationship between human breathing patterns and human emotions to improve the limitations of existing emotion perception systems based on human body movements and postures. In addition, the system also uses a multi-channel comparative analysis method to try its best to eliminate sudden factors during signal acquisition, making the system's emotion judgment results more convincing and improving the system's anti-interference performance.

[0040] (3) A database update mechanism and a privacy security mechanism have also been added. While ensuring the privacy of user data, as the number of users increases and their feedback on the judgment results increases, the database will continue to expand and the data collection scenarios will continue to be enriched, making the system's judgment more and more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of an emotion perception system based on WiFi channel status analysis in an embodiment of the present invention.

[0042] Figure 2 Schematic diagram of a signal transmission module in an embodiment of the present invention.

[0043] Figure 3 Schematic diagram of a signal processing module in an embodiment of the present invention.

[0044] Figure 4 Schematic diagram of an emotion discrimination module in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0046] like Figure 1 As shown, the present invention proposes an emotion perception system based on WiFi channel state analysis, which consists of three modules: a signal transmission module, a signal processing module, and an emotion discrimination module.

[0047] The specific implementation process of the signal transmission module is as follows Figure 2 As shown, it includes a WiFi signal generating unit and a signal transmitting unit:

[0048] The WiFi signal generation unit is supported by multiple commercial routers (a single commercial router is also acceptable, but it is not as accurate as comparing multiple results). These routers generate specific WiFi signals on multiple channels. Various activities of the human body and environmental objects together constitute the signal transmission unit. After passing through the signal transmission unit, the specific WiFi signal emitted by the WiFi signal generation unit will be affected by the human body and the environment, thereby carrying relevant information about various human activities, including human chest movements and breathing patterns.

[0049] The specific implementation process of the signal processing module is as follows Figure 3 As shown, it includes a signal receiving unit, a signal filtering unit, an information extraction unit, an information uploading unit, a result receiving unit, a result display unit, a result judging unit, and a result feedback unit:

[0050] The signal receiving unit receives the WiFi signal corresponding to the channel of the WiFi signal generating unit from the signal transmission unit by adjusting the center frequency of its own receiver. This process will last for a period of time to ensure that the collected signal contains information related to the human breathing pattern. The signal filtering unit is used to remove the noise superimposed on the WiFi signal to facilitate the processing of subsequent steps. Its input is the WiFi signal on multiple channels received. In the information extraction unit, since the WiFi signal generating unit generates a specific WiFi signal, the CFR on the corresponding channel contained in the WiFi signal on each channel can be obtained by comparing the received signal with the original specific signal. Subsequently, the CSI on each channel is obtained based on the received CFR information, and the corresponding CSI on multiple channels is further transmitted to the information uploading unit. The information uploading unit receives the WiFi signals on multiple channels from the information extraction unit. After receiving the CSI, multiple CSIs are packaged, and then the packaged data is encrypted, and finally the encrypted data is transmitted to the emotion discrimination module; after receiving the emotion judgment result from the emotion discrimination module, the result receiving unit transmits the result to the result display unit; the result display unit prompts different words according to the different emotion judgment results received (for example: it is detected that you are in a low mood, tell a joke..., etc.) to make the user feel good; after the result display unit prompts the emotion judgment result, the result discrimination unit prompts the user to feedback whether the current emotion judgment result is accurate, as well as the current environment information (this function can be turned off after the system judgment accuracy reaches a certain level, and when the system judges that the person is in an angry or sad emotional state, it can wait for a period of time before prompting the user to feedback information), and transmits the collected feedback information to the result feedback unit; the function of the result feedback unit is to transmit the received feedback information back to the emotion discrimination module.

[0051] The specific implementation process of the emotion discrimination module is as follows: Figure 4As shown, it includes an information receiving unit, a data storage unit, a model training unit, a discriminant model unit, a result transmission unit, and a feedback receiving unit:

[0052] After receiving the encrypted data from the signal processing module, the information receiving unit first decrypts it to obtain the packaged data of CSI on multiple channels, and then splits the packaged data to obtain the CSI on each channel; in the process of collecting signals, due to the discontinuity of receiving packets or the delay of decoding, sampling, etc., the collected signals are unevenly distributed in time, and the evenly spaced distribution of signals can better reflect the continuity of human breathing. Therefore, the information receiving unit also needs to interpolate the signals on each channel to obtain CSI evenly spaced in time; finally, the information receiving unit transmits the CSI on multiple channels to the data storage unit and the discrimination model unit; the data storage unit initially stores the original model training data, and after receiving the CSI on multiple channels from the information receiving unit, the data storage unit temporarily stores them in its database and waits for the next confirmation operation; the discrimination model unit After receiving CSI on multiple channels from the information receiving unit, the discrimination model unit performs the following processing according to the current latest model: first, the human breathing pattern information (such as breathing frequency and breathing depth) carried by the CSI is estimated based on the CSI, and the obtained breathing pattern information is used as the input of the machine learning classifier in the discrimination model unit. After judgment by the classifier, the emotional state represented by the breathing pattern is finally obtained, that is, the emotional state represented by the CSI on the channel (emotional states include happiness, sadness, anger, and fear); secondly, by comparing the judgment results on multiple channels, the discrimination model unit selects the result that it thinks is most likely to be correct as the output, and sends the emotional judgment result to the result transmission unit, and at the same time sends the judgment result on each channel and the comprehensive judgment result to the data storage unit, so as to facilitate the subsequent data storage information to confirm which CSI on which channels is valid CSI. After receiving the result given by the discrimination model unit, the result transmission unit sends the emotion judgment result to the signal processing module; after receiving the feedback information from the information processing module, the feedback receiving unit sends the evaluation of whether the emotion judgment result is correct and the user's current environmental information to the data storage unit; if the data storage unit receives feedback information that the emotion judgment result is correct, it will judge the validity of the CSI on each channel in combination with the comprehensive judgment result given by the discrimination model unit, confirm the CSI that leads to the correct result as valid data, mark them with the corresponding label of the correct judgment result, and add the current environmental information fed back by the user, and store them in the database; at the same time, the emotion discrimination module will discard the CSI that causes other erroneous results temporarily stored in the database; after the database is updated to a certain extent (or at regular intervals), the model training unit will re-train the machine learning model in the discrimination model unit based on the data in the current data storage unit to update the parameters of the machine learning model and make the judgment result more accurate.

[0053] The discriminant model is a convolutional neural network, which consists of three key units: feature encoder, emotion discrimination classifier, and result output layer;

[0054] Feature encoder: The feature encoder consists of multiple convolutional layers and pooling layers. After the CSI data passes through the feature encoder, the discriminant model can focus more on the discriminant features that distinguish different emotions. At the same time, the feature encoder can also reduce the amount of CSI data input to the emotion discrimination classifier. In the emotion discrimination model, these discriminant features will be flattened into a one-dimensional array before being passed to the emotion discrimination classifier;

[0055] Emotion Discriminant Classifier: This classifier consists of two fully connected layers (FCLs) that map the extracted discriminant features to the relevant classes (i.e. emotional states: happy, sad, angry, fearful);

[0056] In addition, through more convolutional layers and fully connected layers, the CNN model can extract more discriminative features for breathing detection, but at the cost of larger training data and longer training time;

[0057] Result output layer: Since the emotion discrimination classifier obtains the correlation between the discriminant features extracted from the input CSI data and each related class (that is, the result obtained by the emotion discrimination classifier is the probability that the discriminant features correspond to various emotional states); therefore, in order to obtain the final emotion discrimination result, it is necessary to pass the result vector obtained by the emotion discrimination classifier through a Softmax function to obtain the related class with the highest probability (that is, the final emotion discrimination result); in practical applications, this process is usually completed automatically, especially in deep learning frameworks (such as PyTorch, TensorFlow), using .argmax() or similar functions to directly obtain the related class with the highest probability.

[0058] The steps of the emotion perception method using the above system are as follows:

[0059] Step 1: Before the system starts running, the signal processing module will prompt the user and ask the user whether to agree to collect information, so that the user has the right to know about the signal processing module's collection of information. In the signal transmission module, multiple routers first transmit the original WiFi signal. During the signal transmission process, due to the influence of the environment and the human body, the signal received at the WiFi signal receiving end will carry information related to human activities. Further, using this information, the system can analyze the user's current emotional state.

[0060] In step 1, the impact of human motion on WiFi signals is reflected in the following aspects: the human body is in the physical channel for signal propagation. Since the chest cavity of a person experiences periodic ups and downs when breathing, this motion will affect the CSI of the signal: the amplitude |H| and phase angle ∠H of the channel matrix will have periodic changes. By enhancing these periodic changes through certain algorithms and analyzing them, the corresponding chest cavity movement, that is, the breathing pattern, can be recovered.

[0061] Step 2: After receiving the WiFi signal in the environment, the signal processing module performs filtering and denoising processes on it, and then obtains the channel frequency response (CFR) information on multiple channels in the current environment, and further obtains the channel state information (CSI) on multiple channels based on CFR; thereafter, the signal processing module packages and encrypts the CSI on multiple channels to protect the user's privacy and security; finally, the encrypted packaged data is sent to the emotion discrimination module to discriminate the user's emotions.

[0062] Furthermore, in step 2, the physical principles of CFR and CSI are as follows:

[0063] CFR: The response of the signal characteristics in different frequency ranges, generally including amplitude / frequency and phase / frequency responses, which can be expressed by a formula (single transmitting antenna and single receiving antenna):

[0064]

[0065] Where f is the carrier frequency, which refers to the transmission frequency of the signal; t represents time; H(f,t) represents the channel frequency response, which describes the channel effect at frequency f and time t; n represents the number of multipaths, that is, the number of different paths through which the signal passes; a n (t) is the amplitude attenuation factor, which represents the attenuation of the signal amplitude of each path over time; τ n (t) is the propagation delay, which represents the signal propagation delay of each path; is the phase factor, which represents the phase shift of the signal on each path.

[0066] For the received CFR information, f in formula (1) is fixed to the carrier frequency within the frequency band of each of the multiple routers, thereby extracting multiple CSIs corresponding to different routers;

[0067] CSI: Under a specific carrier frequency (i.e., the frequency f in equation (1) is a fixed value), we can use H to express the wireless channel (frequency domain expression):

[0068] Y=H*X+N (2)

[0069] Among them, Y represents the vector of the received signal; H represents the matrix of the channel state information (CSI Matrix); X represents the vector of the transmitted signal; N represents Gaussian white noise, which represents the random noise added during the communication process.

[0070] This channel matrix H representing CSI can also be expressed in the form of amplitude and phase;

[0071] H=|H|e j∠H (3)

[0072] Where |H| represents the amplitude of the CSI matrix; e j∠H represents the phase part of the CSI matrix; j represents the imaginary unit (i.e. ), and ∠H represents the phase angle of each element in the CSI matrix. The phase angle ∠H is a real number that represents the phase change of the signal after passing through the channel on the subcarrier. j∠H It is convenient to use complex numbers to represent the phase information of the channel.

[0073] Step 3: After receiving the data packet from the signal processing module, the emotion discrimination module first decrypts and splits it to obtain the original CSI on multiple channels in the user's current environment; in the process of collecting signals, due to the discontinuity of receiving packets or the delay of decoding, sampling, etc., the collected signals are unevenly distributed in time, and the evenly spaced distribution of signals can better reflect the continuity of human breathing. Therefore, the information receiving unit also needs to interpolate the signals on each channel to obtain CSI that is evenly distributed in time; then, the emotion discrimination module makes a comprehensive judgment on the user's current emotional state based on the CSI on multiple channels obtained after interpolation, combined with the current discrimination model, including happy, sad, angry, and fearful, and returns the judgment result to the signal processing module.

[0074] In step 3, the interpolated CSI on each channel will also be independently temporarily stored in the database of the emotion discrimination module, waiting for further confirmation as valid CSI data, so as to achieve the purpose of continuously enriching the database and further continuously updating the judgment model.

[0075] Step 4: After receiving the judgment result from the emotion discrimination module, the signal processing module makes different speech prompts to the user according to the different emotional states of the user (for example: it is detected that you are in a low mood, tell a joke..., etc.) to make the user feel good; after the speech prompt, the signal processing module prompts the user to feedback whether the current emotion judgment result is accurate, as well as the current environment information (this function can be turned off after the system judgment accuracy reaches a certain level, and when the system judges that the person is in an angry or sad emotional state, it can wait for a period of time before prompting the user to feedback information), and feeds back the collected information to the emotion discrimination module, so that the emotion discrimination module can adjust its database according to the feedback information.

[0076] Step 5: If the emotion discrimination module receives feedback information that the emotion judgment result is correct, it will combine the comprehensive judgment result obtained in step 3 to determine the validity of the CSI on each channel temporarily stored in the database, confirm the CSI that leads to the correct result as valid data, label them with the corresponding label of the correct judgment result, and add the current environment information fed back by the user, and store them in the database; at the same time, the emotion discrimination module will discard the CSI that leads to other erroneous results temporarily stored in the database.

[0077] After the database is updated to a certain extent (or at regular intervals), the emotion discrimination module will re-train the machine learning model used for emotion judgment based on the data in the current database to update the parameters of the machine learning model and make the judgment results more accurate. This function can be stopped when the system reaches a certain level of accuracy and stability. At this time, the machine learning model can be directly sent to the signal processing for offline local emotion perception.

[0078] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. Emotion perception system based on WiFi channel status analysis, characterized by: The system includes a signal transmission module, a signal processing module, and an emotion discrimination module that deploys a discrimination model and a database; The signal transmission module includes a WiFi signal generating unit and a signal transmission unit; the WiFi signal generating unit generates a WiFi signal and transmits it to the signal processing module through the signal transmission unit; The signal processing module includes a signal receiving unit, a signal filtering unit, an information extraction unit, an information uploading unit, a result receiving unit, a result display unit, a result distinguishing unit, and a result feedback unit; the signal receiving unit receives the WiFi signal sent by the signal transmitting module, and after being processed by the signal filtering unit and the information extraction unit, the WiFi signal is transmitted to the emotion distinguishing module by the information uploading unit; The result receiving unit receives the discrimination result output by the emotion discrimination module, displays it through the result display unit, collects user evaluations through the result discrimination unit, and transmits them to the emotion discrimination module through the result feedback unit; The emotion discrimination module includes an information receiving unit, a data storage unit, a model training unit, a discrimination model unit, a result transmission unit, and a feedback receiving unit; The information receiving unit receives the data transmitted by the signal processing module and stores it in the data storage unit. The model training unit performs model training based on the received data, and then the discrimination model unit judges the result. The judgment result is transmitted to the signal processing module through the result transmission unit. The feedback receiving unit receives the feedback information output by the signal processing module.

2. The emotion perception system based on WiFi channel state analysis according to claim 1, characterized in that: In the signal transmission module, the WiFi signal generation unit is supported by multiple routers and is used to generate multiple specific WiFi signals; the signal transmission unit, that is, the physical channel for signal transmission, the specific WiFi signal transmitted in the physical channel is affected by environmental objects and human bodies in the channel and then reaches the signal processing module.

3. The emotion perception system based on WiFi channel state analysis according to claim 1, characterized in that: In the signal processing module, the signal receiving unit is used to receive WiFi signals on each channel affected by the signal transmission unit in the environment; the signal filtering unit is used to remove noise superimposed on the WiFi signals on each channel; the information extraction unit is used to extract the channel state information CSI of the corresponding WiFi signal on each channel; the information uploading unit is used to package and encrypt the CSI obtained by the information extraction unit, and further send it to the emotion discrimination module; The result receiving unit is used to receive the discrimination result from the emotion discrimination module; The result display unit is used to prompt the user with the current emotion identification result obtained by the emotion identification module; The result determination unit is used to collect evaluations from users on the correctness of the emotion determination results; The result feedback unit is used to feed back the user's evaluation information on the correctness of the emotion discrimination result to the emotion discrimination module.

4. The emotion perception system based on WiFi channel state analysis according to claim 1, characterized in that: In the emotion discrimination module, the information receiving unit is used to receive CSI from the information uploading unit in the signal processing module; the data storage unit is used to store CSI in different scenarios to improve the problem of limited data collection scenarios in the current related database; the model training unit is used to regularly train the emotion discrimination model according to the update of the data storage unit; the discrimination model unit is used to make a judgment on the state of the user in the current scenario according to the received CSI; The result transmission unit is used to send the discrimination result made by the discrimination model according to the received CSI to the signal processing module; The feedback receiving unit is used to receive feedback from the user of the signal module on the correctness of the emotion discrimination result.

5. An emotion perception method using the emotion perception system based on WiFi channel state analysis as described in any one of claims 1 to 4, characterized in that: The method comprises the following steps: Step 1: With the user's knowledge and consent, the system starts running. The signal transmission module first generates a specific WiFi signal and transmits it on multiple channels. The signal passes through the signal transmission unit and reaches the signal processing module. The signal transmission unit is the environment and the human body. The periodic ups and downs of the chest cavity when a person breathes affect the channel state information (CSI) of the signal, so that the WiFi signal carries information related to human activities. Step 2: After receiving the WiFi signal in the environment, the signal processing module processes it to obtain the CSI on each channel and sends them to the emotion discrimination module; Step 3: After receiving the CSI uploaded by the signal processing module, the emotion discrimination module makes a judgment on the current emotional state of the user according to the current model and transmits it back to the signal processing module; Step 4: After receiving the emotion discrimination result, the signal processing module informs the user of the emotion discrimination result in the form of a message, and gives corresponding prompt information according to the emotion discrimination result to make the user feel comfortable, and at the same time prompts the user to evaluate the accuracy of the emotion discrimination result, and feeds back the evaluation result to the emotion discrimination module; Step 5: The emotion discrimination module adjusts the database and updates the discrimination model based on the user feedback.

6. The emotion perception method based on WiFi channel state analysis according to claim 5, characterized in that: In step 1, the human body is in the physical channel of signal propagation. The periodic ups and downs of the chest during breathing affect the CSI of the signal, causing the amplitude |H| and phase angle ∠H of the channel matrix to change periodically. These periodic changes are enhanced and analyzed to recover the corresponding chest movement, that is, the breathing pattern.

7. The emotion perception method based on WiFi channel state analysis according to claim 5, characterized in that: In step 2, after receiving the WiFi signal in the environment, the signal processing module performs filtering and denoising on it, thereby obtaining the channel frequency response CFR information on multiple channels in the current environment, and further obtains the channel state information CSI on multiple channels according to the CFR; After that, the signal processing module packages and encrypts the CSI on multiple channels to protect the user's privacy and security; finally, the encrypted packaged data is sent to the emotion discrimination module to discriminate the user's emotions; CFR is the response of the signal characteristics in different frequency ranges, generally including amplitude / frequency and phase / frequency responses. In the case of a single transmitting antenna and a single receiving antenna, it is expressed by the following formula: Where f is the carrier frequency, which refers to the transmission frequency of the signal; t represents time; H(f, t) represents the channel frequency response, which describes the channel effect at frequency f and time t; N represents the number of multipaths, that is, the number of different paths through which the signal passes; a n (t) is the amplitude attenuation factor, which represents the attenuation of the signal amplitude of each path over time; τ n (t) is the propagation delay, which represents the signal propagation delay of each path; is the phase factor, which indicates the phase shift of the signal on each path; For the received CFR information, f in formula (1) is fixed to the carrier frequency within the frequency band of each of the multiple routers, thereby extracting multiple CSIs corresponding to different routers; CSI is the expression of the wireless channel in the frequency domain using the channel state information matrix H under a specific carrier frequency f: Y=H*X+N (2) Among them, / represents the vector of received signals; H represents the matrix of channel state information; X represents the vector of transmitted signals; N represents Gaussian white noise, which represents the random noise added during the communication process; This channel matrix H representing CSI is expressed in the form of amplitude and phase; H=|H|e j∠H (3) Where |H| represents the amplitude of the CSI matrix; e j∠H represents the phase part of the CSI matrix; j represents the imaginary unit (i.e. ), and ∠H represents the phase angle of each element in the CSI matrix. The phase angle ∠H is a real number that represents the phase change of the signal after passing through the channel on the subcarrier. j∠H It is convenient to use complex numbers to represent the phase information of the channel.

8. The emotion perception method based on WiFi channel state analysis according to claim 5, characterized in that: In step 3, after receiving the data packet from the signal processing module, the emotion discrimination module first decrypts and splits it to obtain the original CSI on multiple channels in the user's current environment; the information receiving unit of the emotion discrimination module interpolates the signal on each channel to obtain the CSI evenly distributed in time, and independently stores the interpolated CSI in the data storage unit; The emotion discrimination module makes a comprehensive judgment on the user's current emotional state based on the CSI on multiple channels obtained after interpolation and the current discrimination model: happy, sad, angry, fearful, and returns the judgment result to the signal processing module.

9. The emotion perception method based on WiFi channel state analysis according to claim 5, characterized in that: In step 4, after receiving the discrimination result from the emotion discrimination module, the signal processing module gives different speech prompts to the user to adjust the mood according to the different emotional states of the user; After the speech prompt, the signal processing module prompts the user to provide feedback on whether the current emotion judgment result is accurate, as well as information about the current environment, and feeds back the collected information to the emotion discrimination module so that the emotion discrimination module can adjust its database based on the feedback information.

10. The emotion perception method based on WiFi channel state analysis according to claim 5, characterized in that: In step 5, if the emotion discrimination module receives feedback information that the emotion judgment result is correct, it will combine the comprehensive judgment result obtained in step 3 to judge the validity of the CSI on each channel temporarily stored in the data storage unit, and confirm the CSI that leads to the correct result as valid data, mark them with labels corresponding to the correct judgment results, and add the current environment information fed back by the user, and store them in the data storage unit; at the same time, the emotion discrimination module will discard the CSI that leads to other erroneous results temporarily stored in the data storage unit; after the data storage unit is updated to a certain extent, the emotion discrimination module will re-train the machine learning model used for emotion judgment based on the data in the current data storage unit to update the parameters of the machine learning model to make the judgment result more accurate. This function stops when the system reaches a certain accuracy and stability. At this time, the machine learning model is directly sent to the signal processing for offline local emotion perception.

Images