A CSI in-vehicle personnel left detection method based on ESP32

Abstract

The application discloses a CSI in-vehicle personnel left detection method based on ESP32 and relates to the technical field of in-vehicle personnel safety protection, which comprises an ESP32 device and a cloud processing platform, wherein the ESP32 device further comprises an antenna switch, an RF balun, a power amplifier, a receiving low-noise amplifier, a filter and a temperature sensor, characterized in that the ESP32 device directly accesses CSI and other RF (Radio Frequency) signals from a microcontroller, the ESP32 device is arranged at front and rear positions in the vehicle in pairs, and can establish Wi-Fi and start working after being powered on. In view of the in-vehicle personnel retention problem detection and the limited computing capacity of the intelligent terminal device ESP32 itself, it is determined whether children or pets are retained in the vehicle according to the number of persons and the positions of the persons, a threshold value is set in combination with the temperature sensor parameter to assist in alarm, the effect of preventing child safety is achieved, and meanwhile, the picture recording of the private space in the vehicle is avoided, so that good user privacy is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle interior personnel safety protection, and particularly relates to a CSI vehicle interior personnel left detection method based on ESP32. BACKGROUND

[0002] With the continuous growth of the number of automobiles in recent years, the intelligent development of the automobile industry has not completely solved some safety problems we are facing. According to public reports, every year in the world, there are safety accidents caused by children left in the car, and some parents still do not pay attention to it. In fact, in the closed space in the car, it is easy to cause the problem of too high carbon dioxide concentration or too high temperature in the car. In such an environment, the brain and kidneys of children will be damaged for ten minutes or so, and when the body temperature exceeds 40°C, life safety is threatened.

[0003] The existing solutions mostly rely on a single Internet of Things detection method: for example, using GPS, three-axis sensor and other multi-sensor to judge whether the vehicle is completely stopped, using a camera to detect whether there are children left in the car, and at the same time setting a dangerous threshold through a temperature sensor to avoid danger, but since the camera uses a face recognition method, the sensing performance is poor, it cannot work in a lightless environment, and privacy cannot be guaranteed. Or use infrared sensors, temperature sensors, carbon dioxide sensors and other devices to realize the judgment of children left in the car, the devices used are more, the deployment cost is higher, and the infrared sensor has a large sensing dead angle. Compared with the traditional Internet of Things sensor, using the CSI signal of Wi-Fi for wireless sensing overcomes the defects of the above methods, where CSI is Channel State Information, which is a PHY layer, and comes from the decoded subcarriers in the OFDM system. CSI is a fine-grained physical information, which has high sensitivity to the environment. SUMMARY

[0004] The present application aims to provide a CSI vehicle interior personnel left detection method based on ESP32 to solve the problems in the background art.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0006] A CSI in-vehicle personnel left detection method based on ESP32, 1. comprising ESP32 devices and a cloud platform, the ESP32 device also contains an antenna switch, an RF balun, a power amplifier, a receiving low noise amplifier, a filter, a temperature sensor and the like, characterized in that the ESP32 device directly accesses CSI and other RF (Radio Frequency) signals from the microcontroller, and the ESP32 device is arranged in pairs at the front and rear positions inside the vehicle.

[0007] Further, the ESP32 devices are respectively used as RX (receive) and TX (transport).

[0008] Further, the RX ESP32 device establishes a WLAN local area network, the TX ESP32 device is connected to the local area network established by the RX ESP32 device to send probe frames to the RX at a constant rate, and part of the calculation content is shared locally, which greatly reduces the server calculation pressure and speeds up the response speed.

[0009] A CSI in-vehicle personnel left detection method based on ESP32, characterized in that it comprises the following steps:

[0010] S1, an ESP32-WiFi module is established in the vehicle to acquire CSI, RSSI and other data generated by different numbers of people staying in the vehicle in real time;

[0011] S2, the original CSI data acquired by the ESP32 is preprocessed locally, a total of 114 subcarriers, after Hanning filter and least mean square error filter processing, the optimal subcarrier is extracted through the PCA algorithm. Then the preprocessed optimal subcarrier data is extracted to extract the characteristics of signal amplitude fluctuation and the corresponding characteristics through signal distribution change, including mean, variance, range, etc.

[0012] S3, RX aggregates all feature data (including mean, variance, range, RSSI, temperature data), and sends the data to the cloud server in 20 seconds as a cycle;

[0013] S4, different data models are established by adjusting parameters according to the data collected in the early stage for person number recognition and position recognition. In the recognition stage, first, the CSI feature data is obtained according to the pre-set KNN algorithm model to obtain the in-vehicle person number recognition result. Secondly, the fingerprint positioning model is used to predict the corresponding RSSI signal to obtain the personnel position result. Finally, the threshold is set combined with the temperature sensor data on the ESP32 to assist detection.

[0014] Compared with the prior art, the beneficial effects of the present application are:

[0015] 1. The innovative use of ESP32 devices to analyze the real-time status of personnel through wireless signals is in line with the current era of Internet of Things, is cost-effective and accurate, and does not infringe on user privacy or record images in private spaces.

[0016] 2. For the detection of the problem of personnel retention in the vehicle mentioned in this method, considering the limited computing power of the intelligent terminal device ESP32 itself, it is determined whether there is a child or pet retention situation in the vehicle by comprehensively determining the number of personnel and the location of the personnel. In addition, combined with the temperature sensor parameters, a threshold value is set to assist in the alarm, which plays a role in preventing child safety.

[0017] 3. Traditional devices use infrared sensors, temperature sensors, carbon dioxide sensors, and other devices to determine whether children are retained in the vehicle. The devices used are complex, have high deployment costs, and have dead angles, and the accuracy is not good. This technology can further detect the retention of living beings in the vehicle while reducing costs. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The vehicle deployment scheme and the general working content of the present application are described.

[0019] Figure 2 The flowchart of the detection method of the present application is shown. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0021] Please refer to Figures 1-2 In this embodiment, two ESP32 microcontrollers are arranged at the front and rear positions inside the vehicle as RX and TX. First, RX establishes a WLAN local area network, and TX is connected to the local area network and sends data packets to the gateway (RX) at a certain rate. RX parses the required CSI, RSSI, and other wireless signal data from the received data packets, performs real-time data noise reduction processing, and extracts the amplitude characteristics of the optimal subcarriers. The content is identified by the cloud platform with a machine learning model, and the identification result is fed back to TX to obtain the result in real time.

[0022] Please refer to Figure 2 In this example, data collection and data preprocessing are implemented on the ESP32 device, and model testing and evaluation are performed on the cloud.

[0023] When they are active, the device can automatically start communication, and can collect CSI, RSSI, temperature and other data for detection, and then make a timely judgment on whether there is a person staying in the vehicle.

[0024] Specifically as follows:

[0025] 1. Obtain CSI from ESP32

[0026] In the field of wireless communication, CSI is an index to describe the signal propagation in a multipath environment, which contains information about the change of channel environment in wireless signal propagation. ESP32 contains a Wi-Fi link with a channel link bandwidth of 40MHz, which transmits 128 subcarriers. Since there are 6 pilot signals in the subcarriers-53, -25, -11, 11, 25, 53, the signal is transmitted on subcarriers-58 ~-2 and 2 ~ 58, a total of 114 valid subcarriers, and the amplitude of each subcarrier is different. When the human body moves in the Wi-Fi signal, it will affect the CSI, causing the amplitude of the CSI to change accordingly. Different number of human bodies in different positions will cause different changes in the amplitude of the CSI. ESP32 has been verified to be used as an intelligent terminal node to provide intelligent services to users. ESP32 uses the WiFi module integrated by the chip to obtain WiFi CSI data, and uses the data processing module written in the chip in advance to analyze the signal information through program calling, so as to provide long-term and stable state perception.

[0027] In this method, we exclude other redundant data that are invalid for this method based on the open source software method ESP32-CSI-Tool, and develop the data acquisition method again to meet the requirement of only packaging CSI, RSSI, temperature and other data to the next data processing stage.

[0028] 2. Data preprocessing

[0029] When obtaining personnel data, due to the influence of the multipath effect of the complex environment inside the vehicle, there is a lot of noise in the original CSI data. The information fed back by such time domain signals will have a large deviation from the true data content, so it is necessary to choose a certain noise reduction method for pre-processing of the data. In this method, we choose Hanning filter and parameter Wiener filter (also known as minimum mean square error filter) for filtering. After filtering, it can be found that the abnormal values of the data are reduced, and the characteristics of the data are more obvious.

[0030] For the non-real-time data used in the early offline training model, in order to the integrity of the model, we select all effective 114 subcarriers in the data processing stage, and perform corresponding noise reduction and smoothing processing. For online real-time data, after corresponding noise reduction and smoothing processing of CSI original data, in order to balance the computing power of nodes, the optimal subcarrier in the time domain signal is selected by using PCA, and the eigenvalue of the optimal subcarrier is extracted and the model is called for analysis.

[0031] According to the change of CSI amplitude fluctuation, the mean, variance and standard deviation of subcarrier CSI amplitude are used as features to represent the fluctuation of CSI amplitude. In this paper, data features are extracted according to each fixed size time window to form a feature vector, as follows:

[0032]

[0033] Where m is the mean of all 114 subcarrier data, v is the variance of all subcarrier data, and s is the standard deviation of all subcarrier data.

[0034] 3. Classification data model

[0035] For the detection of the problem of people staying in the car mentioned in this method, and considering the limited computing power of the intelligent terminal device ESP32 itself, it is determined whether there is a child or pet staying in the car by comprehensively determining the number of people and the location of the people. In addition, combined with the temperature sensor parameters, the threshold value is set to assist the alarm.

[0036] For the number of people model, the activity of people in the car will have a certain impact on the propagation of WiFi signal. When the number of people in the car increases, the impact on WiFi signal becomes more and more obvious, and then a specific pattern is generated in the channel state information. By analyzing the fluctuation of channel state information, the number of people in the car can be effectively estimated. Therefore, the key to using CSI to identify the number of people is to find the relationship between the fluctuation of CSI amplitude and the number of people in the car. We use the machine learning algorithm KNN, use the mean, variance and standard deviation of CSI amplitude obtained in the two stages as feature values, do machine learning classification, and finally get the three-dimensional feature value corresponding to the number of people.

[0037] For the personnel location positioning model, we divide the internal space of the vehicle into a 2*2 grid. Next, in order to extract the features of the signals used for classification, we measure the frequency signal RSSI in each grid position, and also use clustering algorithm to realize a classifier. In the online stage, the target position is located, that is, the RSSI signal of the target position is classified by using the classifier and further positioning results are obtained.

[0038] 4. Model optimization and deployment

[0039] For the landing practice of the method, we recruit volunteers to collect offline data, and based on part of the offline data, we first train a preliminary classification model, and adjust the parameters of the model according to the true labels of the offline data, and optimize the model.

[0040] On this basis, online real-time experiments are carried out. And more sample data collected can be used to optimize the model, and achieve more accurate personnel detection results.

[0041] It is apparent to those skilled in the art that the present application is not limited to the details of the foregoing exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and not restrictive, and the scope of the present application is defined by the appended claims rather than the foregoing description, and it is intended to include all changes falling within the meaning and scope of the equivalent elements of the claims, and any drawing reference in the claims should not be considered as limiting the claims involved.

[0042] In addition, it should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description manner of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be properly combined to form other embodiments that those skilled in the art can understand.

Claims

1. A CSI in-vehicle personnel left detection method based on ESP32, characterized in that, The method comprises the following steps: S1, an ESP32-WiFi module with one transmitter and one receiver is established in the vehicle to obtain CSI and RSSI generated by different numbers of people staying in the vehicle in real time; S2, the CSI data obtained by the original ESP32 is preprocessed locally, a total of 114 subcarriers, after Hanning filtering and least mean square error filtering, the optimal subcarrier is extracted by PCA algorithm, then the characteristic of signal amplitude fluctuation of the preprocessed optimal subcarrier data is extracted, and the corresponding characteristic is extracted through the change of signal distribution, the corresponding characteristic includes mean, variance, range; S3, RX collects all the characteristic data, including mean, variance, range, RSSI, temperature data, and sends the data to the cloud server in 20 seconds as a cycle; S4, different data models are established by adjusting parameters according to the data collected in the early stage for people recognition and position recognition, in the recognition stage, first, the CSI characteristic data is obtained according to the pre-set KNN algorithm model to obtain the recognition result of the number of people in the vehicle, second, the corresponding RSSI signal is predicted by using the fingerprint positioning model to obtain the position result of the personnel, and finally, the threshold is set combined with the temperature sensor data on the ESP32 to assist detection.

Images