Classroom activity situation awareness method and system based on WIFI

Abstract

The invention belongs to the technical field of educational informatization, and discloses a WIFI-based classroom activity situation sensing method, which comprises the following steps of: sensing and calculating a classroom situation region by region, acquiring a WIFI signal reflected by a human body in a classroom by using a multi-input multi-output WIFI device, converting an original signal into a frequency domain image through methods such as noise reduction, filtering and STFT (Short Time Fourier Transform) and the like; and extracting time-space features of each channel by using a deep learning technology, distributing weights and performing fusion calculation on different channel features through an attention mechanism, outputting fusion features of each region of the classroom, and further calculating a situation category of the corresponding region. According to the invention, the activity intensity of different areas in a classroom is identified through multiple-input multiple-output WIFI signals, and a low-invasion non-perception classroom dynamic perception service is provided. The advantages of the WIFI technology and the support of the artificial intelligence algorithm are utilized, intelligent and automatic monitoring of the classroom activity situation is achieved, and powerful support is provided for education management and teaching improvement.

Description

Technical Field

[0001] The present invention belongs to the field of educational information technology, and in particular relates to a WIFI-based classroom activity situation awareness method and system. Background Art

[0002] With the rapid development of information technology and the continuous advancement of educational informatization, the monitoring and management of classroom activities has gradually changed from traditional manual methods to intelligent and automated methods. Traditional classroom activity monitoring methods often rely on manual observation or wearable devices, which may interfere with students' normal learning and even cause privacy leaks.

[0003] Therefore, the industry began to explore classroom activity situation awareness methods based on WIFI technology. As a widely used wireless communication technology, WIFI technology has the advantages of wide coverage, fast transmission speed, and low cost. Using WIFI signals to perceive classroom activity situation can achieve non-invasive monitoring, that is, without interfering with students' normal learning, the classroom activity situation can be perceived by collecting and analyzing WIFI signals.

[0004] Specifically, the non-intrusive perception method of classroom activity status based on WIFI mainly relies on the transmission characteristics and signal processing technology of WIFI signals. WIFI signals are affected by many factors during transmission, such as human activities, obstacles, etc. These factors will cause changes in the propagation path, propagation speed, signal strength, etc. of WIFI signals. By collecting and analyzing these changes, feature information related to classroom activities can be extracted, such as students' location, movements, behavior patterns, etc.

[0005] In addition, with the continuous development of artificial intelligence technology, especially the widespread application of machine learning, deep learning and other algorithms, the accuracy and reliability of WIFI-based classroom activity situation awareness methods have been significantly improved. By training appropriate models, the system can automatically identify and analyze classroom activity situations and provide teachers with real-time classroom management information, such as student participation, classroom activity, etc.

[0006] At present, there is a recognition method based on WiFi channel status information using CSI-PSD spectrogram to study the recognition of student classroom behavior. This technology has indeed achieved good results in the recognition of individual student behavior. However, when this technology is placed in a real classroom environment and faces a multi-student scene, its recognition accuracy is greatly reduced, and it is difficult to meet the needs of actual teaching management.

[0007] There are also existing studies on gesture recognition based on WiFi. However, the gesture categories collected in this method are relatively small and fail to cover all gesture combinations required in the classroom. This greatly limits its application scope in the actual classroom environment. At the same time, relying solely on gesture recognition to evaluate students' activities in the classroom is also obviously insufficient, because students' classroom activities include not only gestures, but also postures, position movements, interactive behaviors and other dimensions.

[0008] Through the above analysis, the problems and defects of the prior art are as follows:

[0009] (1) Traditional classroom activity monitoring methods often rely on manual observation or wearable devices, which may interfere with students' normal learning and even cause privacy leaks.

[0010] (2) Currently, CSI-based behavior recognition is focused on single-person behavior recognition, and no research has been conducted on the behavior of multiple people.

[0011] (3) Research on gesture recognition based on Wi-Fi is not sufficient to evaluate students’ activities in the classroom. Summary of the invention

[0012] In view of the problems existing in the prior art, the present invention provides a classroom activity situation awareness method based on WIFI.

[0013] The present invention is implemented in this way: a WIFI-based classroom activity situation awareness method comprises:

[0014] Step 1: Classroom situation perception and calculation by region;

[0015] Use multi-transmitter and multi-receiver WIFI devices to obtain WIFI signals reflected by the human body in the classroom. Convert the original signals into frequency domain images through noise reduction, filtering, and STFT methods. Then use deep learning technology to extract the spatiotemporal features of each channel. Use the attention mechanism to assign weights and fusion calculations to different channel features, output the fusion features of each area of ​​the classroom, and then calculate the situation category of the corresponding area.

[0016] Step 2: Optimize the resolution of classroom situation;

[0017] Compare and evaluate the regional prediction value with the true value, calculate the prediction accuracy at the current resolution, adjust the regional resolution on this basis, calculate the prediction accuracy at different resolutions, and then calculate the maximum resolution and optimal resolution of the classroom;

[0018] Step 3: Optimize the layout of WIFI devices;

[0019] On the basis of fixed resolution, remove the data of a certain channel of the WIFI receiving end, calculate the predicted value of the regional situation again and compare and evaluate it with the actual value, calculate the recognition accuracy under the current WIFI combination, and then adjust the removed data channel, calculate the situation recognition accuracy under different combinations, and obtain the best WIFI combination by comparing the results, and optimize the layout of the equipment.

[0020] Furthermore, the classroom situation is perceived and calculated by region:

[0021] It includes data collection and preprocessing, spatiotemporal feature extraction, multi-channel feature fusion and regional situation classification processes;

[0022] The multi-transmit and multi-receive Wi-Fi antenna layout system is composed of c signal transmitters and n signal receivers. The frequency response function H(f) of the low-pass filter can be expressed in the frequency domain as:

[0023]

[0024] where f c is the cutoff frequency, that is, signals above this frequency will be completely blocked, and signals below this frequency will pass without attenuation; from the perspective of the relationship between time domain and frequency domain, the input and output signals are connected through Fourier transform and its inverse transform; if the Fourier transform of the input signal x(t) is X(f), and the Fourier transform of the output signal y(t) is Y(f), then:

[0025] Y(f)=H(f)X(f)

[0026] Then the output signal is obtained through inverse Fourier transform:

[0027] y(t)=F -1 {H(f)X(f)}

[0028] This step can convert the raw data into a spectrum, which shows the distribution of the signal at different frequencies;

[0029] VGG and BiGRU are used to extract spatial features and temporal features respectively. This method realizes the comprehensive analysis of the signal in the time domain and frequency domain, and finally outputs the feature combination [h1; h2; ...; h n ]; the details are as follows:

[0030] For a single-channel spectrum graph, VGG is used for deep spatial feature extraction. The calculation method can be expressed by the following formula:

[0031] V=F(Z;θ)=F out (F N (...F2(F1(Z))))

[0032] Among them, Z is the digital matrix of the spectrum map conversion, as the input of VGG, F1 to F N Represents the operations of each layer in the VGG network, including convolutional layer, pooling layer, activation function, F out is the output layer of the VGG network, V is the feature vector sequence output by the VGG network, and for channel i, V i =[v i1 ,v i2 ,...,v il ], l is the maximum number of features in a single sequence;

[0033] The feature sequence extracted by the VGG network is input into the BiGRU network to further extract the temporal features. The calculation process is as follows:

[0034] h i =H([V i ; V′ i ])

[0035] Among them, H represents the feature operation of BiGRU, V i is the characteristic sequence of channel i, V' i The reverse sequence of channel i, h i is the feature vector output by BiGRU; for n signal receiving channels, the feature vectors of all channels [h1; h2; ...; h n ];

[0036] The gating mechanism is used to dynamically learn the weights of different channel features. The calculation formula is as follows:

[0037] [g1;g2;...;gn]=softmax([W1·h1;W2·h2;...;W n ·h n ])

[0038] Among them, g n represents the feature weight of the nth channel, W n Represents the weight matrix of the channel. The weight value of each channel is calculated by the softmax function. The value is a decimal between 0 and 1, and the sum of all weight values ​​is 1.

[0039] On this basis, the multi-channel fusion features are calculated, and the calculation formula is:

[0040]

[0041] Among them, i represents the i-th channel, and h is the fusion feature after weighted calculation of different channels;

[0042] The classification features of each target region are calculated based on the fusion feature h, and the enhanced features of each target region are obtained using the Transferformer mechanism. The calculation formula for a single region is:

[0043] a ij = Transformer(h q ,h k ,h v )

[0044] where a ij Represents the enhanced features of the target area. The values ​​of i and j are constrained by the resolution. For m×m resolution, 1≤i,j≤m;

[0045] On this basis, the predicted value of the regional situation is calculated using the following formula:

[0046]

[0047] Where W ij is the weight matrix, b ij is the deviation, is the forecast category of regional situation;

[0048] According to the training principle of deep learning, the loss is calculated using the predicted value and the true value, and then the weight parameters are optimized using back propagation. After multiple rounds of iterative training, the weight parameters with the best accuracy are obtained.

[0049] For a single region, cross entropy is used as the loss function, which is expressed as:

[0050]

[0051] where y ij is the true value of the area, and N is the number of samples used in a single training;

[0052] For the entire classroom area, its loss function is the sum of the loss values ​​of all areas, expressed as:

[0053]

[0054] where γ ij It is a hyperparameter, which indicates the calculation weight of the loss function in different regions. It is set to γ ​​by default. 11 =γ 12 =... =γ mm =1.

[0055] Further, the classroom situation resolution is optimized:

[0056] Includes fixed resolution accuracy calculation and dynamic resolution tuning;

[0057] Multi-resolution calculation and tuning strategy for classroom situation categories; resolution represents the number of regional divisions, and m*m resolution means that the classroom is divided into m*m regions, and each region is represented by A ij Indicates, i,j∈[1,m]; for region A ij , through the method proposed in module 1, the predicted situation category can be calculated Further, the area A can be calculated ij The prediction accuracy can be expressed as:

[0058]

[0059] where y ij For area A ij The true situation category, N is the total number of samples;

[0060] The average accuracy of all regions at m*m resolution can be calculated, and the calculation formula can be expressed as:

[0061]

[0062] By changing the resolution m, the average accuracy of the region under different resolutions can be calculated; setting the threshold η as the minimum acceptable accuracy, 0<η<1, the resolution combination greater than the threshold η can be calculated, where max(m) is the maximum resolution allowed, and the maximum accuracy in the combination is the optimal resolution, that is:

[0063] P best =max([p m >η])

[0064] P best The corresponding resolution is the optimal resolution.

[0065] Furthermore, the WIFI device layout is optimized:

[0066] WiFi layout optimization strategy for classroom activity situation;

[0067] In the initial state, all devices are in the activation stage, and all data are collected to obtain the data of the c-channel transmitter and the n-channel receiver. Initially, the best accuracy rate p best is the prediction accuracy when all data are input;

[0068] Reduce one or a group of receiver data (or transmitter data) in turn, and calculate the predicted situation category p under this layout according to the process of module one. ij ; Calculation area A ij The prediction accuracy is calculated as follows:

[0069]

[0070] Among them, y ij For area A ij The real situation category, is the predicted situation category, N is the total number of samples; the method of module 2 is also used to obtain the average accuracy p of all regions com ,but

[0071]

[0072] Among them, p best The best accuracy after this step, and record the device layout under the best accuracy, p com is the accuracy under the current layout, is the best accuracy before this step;

[0073] If the accuracy of the WiFi signal decreases after reducing the number of devices, we use triangular relationship prediction technology to analyze and predict the best solution for adding new devices or adjusting the location of existing devices, and then repeat the above process;

[0074] The above steps will be executed repeatedly until an optimal configuration state is reached. best This is the resolution of the current optimal layout.

[0075] Another object of the present invention is to provide a WIFI-based classroom activity situation awareness system comprising:

[0076] The perception computing module is used for classroom situation perception computing in different regions. It uses a multi-transmitter and multi-receiver WIFI device to obtain the WIFI signal reflected by the human body in the classroom, and converts the original signal into a frequency domain image through noise reduction, filtering, and STFT methods. Then, deep learning technology is used to extract the spatiotemporal features of each channel. The attention mechanism is used to assign weights and fusion calculations to different channel features, and the fusion features of each area of ​​the classroom are output, and then the situation category of the corresponding area is calculated.

[0077] Resolution optimization module, used for optimizing the resolution of classroom situations; compare and evaluate the regional prediction value with the actual value, calculate the prediction accuracy at the current resolution, adjust the regional resolution on this basis, calculate the prediction accuracy at different resolutions, and then calculate the maximum resolution and optimal resolution of the classroom;

[0078] The layout optimization module is used to optimize the layout of WIFI devices. On the basis of fixed resolution, the data of a certain channel of the WIFI receiving end is removed, and the predicted value of the regional situation is calculated again and compared with the actual value for evaluation. The recognition accuracy under the current WIFI combination is calculated, and then the removed data channel is adjusted to calculate the situation recognition accuracy under different combinations. The best WIFI combination is obtained by comparing the results, and the layout of the device is adjusted.

[0079] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the WIFI-based classroom activity situation awareness method.

[0080] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the WIFI-based classroom activity situation awareness method.

[0081] Another object of the present invention is to provide an information data processing terminal, which is used to implement the WIFI-based classroom activity situation awareness system.

[0082] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0083] First, the present invention provides a WIFI-based classroom activity situation awareness method and system, which identifies the activity intensity of different areas in the classroom through multiple WIFI signals, and provides low-intrusion and non-perceptual classroom dynamic perception services. The present invention uses the advantages of WIFI technology and the support of artificial intelligence algorithms to achieve intelligent and automated monitoring of classroom activity situations, providing strong support for education management and teaching improvement.

[0084] Second, the technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0085] Most existing studies are limited to the behavior analysis of a single student, capturing and analyzing their movements or position changes in the classroom through WiFi signals. However, in a real classroom environment, the interaction and cooperation between students and the overall classroom atmosphere are important factors affecting the teaching effect. Therefore, it is particularly important to evaluate the activities and behaviors of multiple students and the overall situation of the classroom.

[0086] In addition, the current research on the multi-region classroom activity situation is still insufficient, and this model is of great significance in real teaching scenarios. Specifically, it can promote the level of interaction in the classroom, allowing students to exchange ideas more actively. At the same time, this model also plays a positive role in optimizing classroom management and understanding student status.

[0087] Furthermore, previous studies have mostly focused on the single-transmit-multiple-receive layout, while this study adopts a layout that combines multiple transmit and multiple receive and can be adjusted dynamically. This method requires sophisticated processing of complex signals received by multiple antennas, covering multiple links such as signal separation, decoding, merging, and noise reduction. Although this move undoubtedly increases the complexity and computational load of signal processing, it provides strong support for more accurate capture of signal characteristics.

[0088] The technical solution of the present invention solves a technical problem that people have been eager to solve but have never succeeded in solving: in WiFi signal acquisition, the signal coverage of a single transmitter is relatively limited, especially in complex environments or large areas, it may not be able to provide stable and comprehensive signal coverage, and the layout and configuration are relatively fixed, and it is difficult to flexibly adjust according to actual needs. In this regard, this study adopts a multi-transmit and multi-receive layout, and can introduce deep learning algorithms according to different classroom layouts to flexibly adjust the number and position of transmitters and receivers. In addition, we can use deep learning algorithms to realize the extraction of regional features, thereby meeting diverse teaching needs.

[0089] Third, the WIFI-based classroom activity situation awareness method proposed in the present invention solves the key problems in the existing classroom environment situation awareness technology in industrial applications and achieves significant technical progress in many aspects, as follows:

[0090] 1. Solve the problems of low accuracy and insufficient resolution of classroom situation awareness in existing technologies

[0091] Traditional classroom situation awareness technologies mostly rely on video surveillance or single sensors (such as infrared, sound sensors, etc.). These methods are usually difficult to obtain accurate and comprehensive classroom activity situation information in complex classroom environments, and are greatly limited in terms of resolution and monitoring range. The present invention uses WIFI signals to perceive the activities of different areas in the classroom through the reflection of the signal by the human body, overcoming the limitations of traditional technologies that rely on image or point sensing. At the same time, the regional perception calculation process proposed by the present invention realizes the accurate identification of the situation of each area through the extraction of spatiotemporal features and the fusion calculation of the attention mechanism, effectively improving the accuracy and resolution of classroom situation awareness.

[0092] 2. Significantly reduce hardware costs and improve system adaptability

[0093] Traditional methods require the deployment of a large number of cameras or sensors, which not only has high hardware costs, but also faces problems such as difficult equipment installation and complex maintenance. The present invention uses multi-transmitter and multi-receiver WIFI devices as a sensing method, which can achieve low-cost, non-invasive classroom situation awareness. Compared with traditional hardware-intensive solutions, WIFI signal acquisition equipment is easy to deploy and does not rely on video acquisition, thus protecting students' privacy. In addition, WIFI-based solutions are easy to deploy and highly adaptable in environments with existing wireless networks such as schools, greatly reducing the hardware cost and subsequent maintenance difficulty of the system.

[0094] 3. Real-time resolution and equipment layout optimization to improve the intelligence level of situation monitoring

[0095] The present invention improves the intelligence level of the system through situation resolution optimization and WIFI device layout optimization. The resolution optimization module adaptively adjusts and selects the optimal monitoring resolution by adjusting the regional resolution and comparing the prediction accuracy, ensuring situation awareness at the optimal resolution, avoiding the accuracy loss caused by the fixed resolution in the traditional monitoring system. The device layout optimization module intelligently adjusts the WIFI channel combination to determine the channel combination that has the greatest impact on the recognition accuracy, thereby optimizing the WIFI device layout, maximizing the device utilization and optimizing the monitoring accuracy.

[0096] 4. Significantly improve the value of classroom management and educational data analysis

[0097] The present invention not only provides high-precision and high-resolution classroom situation awareness, but also can use the detected data for classroom management and educational analysis. For example, by analyzing data such as the activity level and static areas of different areas, students' classroom participation can be better understood, helping teachers to adjust teaching methods in real time and improve the quality of classroom interaction. At the same time, the collected situation data can also be stored and analyzed for a long time, providing teaching managers with visual data support for student behavior patterns, participation, etc., and providing data support for schools to formulate personalized teaching strategies and improve teaching quality.

[0098] Compared with the prior art, the present invention has achieved the following significant technical advances:

[0099] Double improvement in accuracy and resolution: Through deep learning feature extraction, attention mechanism and adaptive resolution optimization, the accuracy and resolution of classroom situation awareness are greatly improved, overcoming the problems of insufficient accuracy and monitoring range of traditional methods.

[0100] Dual advantages of cost and privacy protection: Using low-cost WIFI devices to achieve situational awareness avoids the high cost and privacy issues of video surveillance, and has the advantages of non-intrusiveness and low maintenance requirements.

[0101] Enhanced system intelligence and adaptability: Through resolution optimization and equipment layout optimization, the system can adaptively adjust the resolution and layout according to the actual monitoring situation, improving the intelligence level of the system.

[0102] Discovering the value of educational data: Through real-time and long-term analysis of classroom situation data, key data support for teaching feedback, student engagement and classroom management is provided to teachers and school administrators, thereby enhancing the application value of educational data.

[0103] The present invention has achieved significant technological progress in the accuracy, cost efficiency and intelligence level of classroom situation awareness, providing schools and educational institutions with an innovative and efficient classroom management and teaching feedback tool, which has broad industrial application prospects and commercial value. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] Figure 1 It is a flow chart of a WIFI-based classroom activity situation awareness method provided by an embodiment of the present invention.

[0105] Figure 2 It is a structural block diagram of a WIFI-based classroom activity situation awareness system provided by an embodiment of the present invention.

[0106] Figure 3 It is a detailed structural block diagram of a WIFI-based classroom activity situation awareness system provided by an embodiment of the present invention.

[0107] Figure 4 It is an experimental configuration diagram provided by an embodiment of the present invention.

[0108] Figure 5 It is an actual scene graph provided by an embodiment of the present invention.

[0109] Figure 6 It is a before and after comparison diagram of the low-pass Butterworth filter noise reduction provided by an embodiment of the present invention.

[0110] Figure 7 It is a comparison diagram before and after wavelet transform denoising provided by an embodiment of the present invention.

[0111] Figure 8 It is a schematic diagram of the formation of the CSI feature graph provided by an embodiment of the present invention.

[0112] Fig. 9 The results of five experiments provided in the embodiments of the present invention are made into a bar chart. DETAILED DESCRIPTION

[0113] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0114] like Figure 1 As shown, a WIFI-based classroom activity situation awareness method provided by an embodiment of the present invention includes the following steps:

[0115] S101, classroom situation perception computing by region;

[0116] Use multi-transmitter and multi-receiver WIFI devices to obtain WIFI signals reflected by the human body in the classroom. Convert the original signals into frequency domain images through noise reduction, filtering, and STFT methods. Then use deep learning technology to extract the spatiotemporal features of each channel. Use the attention mechanism to assign weights and fusion calculations to different channel features, output the fusion features of each area of ​​the classroom, and then calculate the situation category of the corresponding area.

[0117] S102, classroom situation resolution optimization;

[0118] Compare and evaluate the regional prediction value with the true value, calculate the prediction accuracy at the current resolution, adjust the regional resolution on this basis, calculate the prediction accuracy at different resolutions, and then calculate the maximum resolution and optimal resolution of the classroom;

[0119] S103, WIFI equipment layout optimization;

[0120] On the basis of fixed resolution, remove the data of a certain channel of the WIFI receiving end, calculate the predicted value of the regional situation again and compare and evaluate it with the actual value, calculate the recognition accuracy under the current WIFI combination, and then adjust the removed data channel, calculate the situation recognition accuracy under different combinations, and obtain the best WIFI combination by comparing the results, and optimize the layout of the equipment.

[0121] The classroom situation perception calculation by region provided by the embodiment of the present invention:

[0122] It includes data collection and preprocessing, spatiotemporal feature extraction, multi-channel feature fusion and regional situation classification processes.

[0123] The WIFI-based classroom activity situation awareness method provided in the embodiment of the present invention gradually realizes accurate awareness of the activity situation in the classroom, resolution optimization, and optimization of the WIFI device layout through multiple steps. The specific working principle is as follows:

[0124] First, in step S101 (classroom situation regional perception calculation), the system uses multi-transmitter and multi-receiver WIFI devices to obtain WIFI signals reflected by the human body in the classroom. The original signal obtained undergoes data acquisition and preprocessing, including noise reduction and filtering, to improve signal quality. Next, the time domain signal is converted into a frequency domain image through short-time Fourier transform (STFT). This frequency domain image uses deep learning technology to extract spatiotemporal features, extracts the spatiotemporal features of each channel, and combines the attention mechanism to assign weights to different channels, and finally obtains the fusion features of each area. This fusion feature is further calculated and classified to output the activity situation category of each area, such as static, active, etc., to achieve regional situation awareness.

[0125] In step S102 (classroom situation resolution optimization), the system optimizes the situation prediction accuracy of each region. The prediction results are compared and evaluated with the actual situation to calculate the prediction accuracy at the current resolution. On this basis, the system adjusts the resolution of each region and calculates and evaluates the prediction accuracy at multiple resolutions. Through this adaptive adjustment, the system obtains the optimal resolution and maximum resolution of the classroom to achieve more accurate situation awareness. The optimization of this step not only improves the accuracy of overall recognition, but also ensures the refined monitoring of situation changes in each region.

[0126] Next, in step S103 (WIFI device layout optimization), the system optimizes the channel combination of the WIFI receiving end based on a fixed resolution. The specific method is to remove a certain channel data of the receiving end one by one, calculate the situation recognition accuracy under different channel combinations, and compare and evaluate the predicted value under each combination with the true value. This method effectively finds the channel combination that has the greatest impact on the recognition accuracy. After comprehensively considering the recognition effects of each combination, the system determines the best WIFI channel combination and finally optimizes the layout of the WIFI device to ensure the highest recognition accuracy with the least equipment and the best channel configuration.

[0127] The embodiment of the present invention gradually realizes efficient and accurate monitoring of activity status in the classroom environment through three main processes: classroom situation regional perception calculation, resolution optimization, and equipment layout optimization. By using the fusion calculation of WIFI signals, deep learning feature extraction, and attention mechanism, high-resolution situation classification of different areas in the classroom is achieved. At the same time, through the optimization of resolution and channel combination, equipment redundancy and resource consumption are reduced, and the overall performance and adaptability of the system are significantly improved. This innovative WIFI perception solution has a wide range of application value in application scenarios such as education management and classroom activity analysis.

[0128] The present invention designs a multi-transmit and multi-receive Wi-Fi antenna layout system, which is composed of c signal transmitters and n signal receivers; both the transmitter and the receiver continuously transmit and receive Wi-Fi signals at a stable frequency of 80 Hz, so as to obtain the original Wi-Fi signal data; and then implement a series of noise elimination strategies to ensure the accuracy and reliability of the data; for example, a low-pass filter is used to filter the noise generated by the air-conditioning equipment; the frequency response function H(f) of the low-pass filter can be expressed in the frequency domain as:

[0129]

[0130] where f c is the cutoff frequency, that is, signals above this frequency will be completely blocked, and signals below this frequency will pass without attenuation; from the perspective of the relationship between time domain and frequency domain, the input and output signals are connected through Fourier transform and its inverse transform; if the Fourier transform of the input signal x(t) is X(f), and the Fourier transform of the output signal y(t) is Y(f), then:

[0131] Y(f)=H(f)X(f)

[0132] Then the output signal is obtained through inverse Fourier transform:

[0133] y(t)=F -1 {H(f)X(f)}

[0134] This step can convert the raw data into a spectrum, which shows the distribution of the signal at different frequencies;

[0135] VGG and BiGRU are used to extract spatial features and temporal features respectively. This method realizes the comprehensive analysis of the signal in the time domain and frequency domain, and finally outputs the feature combination [h1; h2; ...; h n ]; the details are as follows:

[0136] For a single-channel spectrum graph, VGG is used for deep spatial feature extraction. The calculation method can be expressed by the following formula:

[0137] V=F(Z;θ)=F out (F N (...F2(F1(Z))))

[0138] Among them, Z is the digital matrix of the spectrum map conversion, as the input of VGG, F1 to F N Represents the operations of each layer in the VGG network, including convolutional layer, pooling layer, activation function, F out is the output layer of the VGG network, V is the feature vector sequence output by the VGG network, and for channel i, Vi =[v i1 ,v i2 ,...,v il ], l is the maximum number of features in a single sequence;

[0139] The feature sequence extracted by the VGG network is input into the BiGRU network to further extract the temporal features. The calculation process is as follows:

[0140] h i =H([V i ; V′ i ])

[0141] Among them, H represents the feature operation of BiGRU, V i is the characteristic sequence of channel i, V' i The reverse sequence of channel i, h i is the feature vector output by BiGRU; for n signal receiving channels, the feature vectors of all channels [h1; h2; ...; h n ];

[0142] The gating mechanism is used to dynamically learn the weights of different channel features. The calculation formula is as follows:

[0143] [g1;g2;...;gn]=softmax([W1·h1;W2·h2;...;W n ·h n ])

[0144] Among them, g n represents the feature weight of the nth channel, W n Represents the weight matrix of the channel. The weight value of each channel is calculated by the softmax function. The value is a decimal between 0 and 1, and the sum of all weight values ​​is 1.

[0145] On this basis, the multi-channel fusion features are calculated, and the calculation formula is:

[0146]

[0147] Among them, i represents the i-th channel, and h is the fusion feature after weighted calculation of different channels;

[0148] The classification features of each target region are calculated based on the fusion feature h, and the enhanced features of each target region are obtained using the Transferformer mechanism. The calculation formula for a single region is:

[0149] a ij = Transformer(h q ,h k ,h v)

[0150] where a ij Represents the enhanced features of the target area. The values ​​of i and j are constrained by the resolution. For m×m resolution, 1≤i,j≤m;

[0151] On this basis, the predicted value of the regional situation is calculated using the following formula:

[0152]

[0153] Where W ij is the weight matrix, b ij is the deviation, is the forecast category of regional situation;

[0154] According to the training principle of deep learning, the loss is calculated using the predicted value and the true value, and then the weight parameters are optimized using back propagation. After multiple rounds of iterative training, the weight parameters with the best accuracy are obtained.

[0155] For a single region, cross entropy is used as the loss function, which is expressed as:

[0156]

[0157] where y ij is the true value of the area, and N is the number of samples used in a single training;

[0158] For the entire classroom area, its loss function is the sum of the loss values ​​of all areas, expressed as:

[0159]

[0160] where γ ij It is a hyperparameter, which indicates the calculation weight of the loss function in different regions. It is set to γ ​​by default. 11 =γ 12 =... =γ mm =1.

[0161] The classroom situation resolution optimization provided by the embodiment of the present invention:

[0162] Includes fixed resolution accuracy calculation and dynamic resolution tuning;

[0163] The present invention proposes a multi-resolution calculation and tuning strategy for classroom situation categories; the resolution represents the number of regional divisions, and the m*m resolution means that the classroom is divided into m*m regions, and each region is represented by A ij Indicates, i,j∈[1,m]; for region A ij , through the method proposed in module 1, the predicted situation category can be calculated Further, the area A can be calculated ij The prediction accuracy can be expressed as:

[0164]

[0165] where y ij For area A ij The true situation category, N is the total number of samples;

[0166] The average accuracy of all regions at m*m resolution can be calculated, and the calculation formula can be expressed as:

[0167]

[0168] By changing the resolution m, the average accuracy of the region under different resolutions can be calculated; setting the threshold η as the minimum acceptable accuracy, 0<η<1, the resolution combination greater than the threshold η can be calculated, where max(m) is the maximum resolution allowed, and the maximum accuracy in the combination is the optimal resolution, that is:

[0169] P best =max([p m >η])

[0170] P best The corresponding resolution is the optimal resolution.

[0171] Optimization of WIFI device layout provided by the embodiment of the present invention:

[0172] The present invention proposes a WiFi layout optimization strategy for classroom activity status;

[0173] In the initial state, all devices are in the activation stage, and all data are collected to obtain the data of the c-channel transmitter and the n-channel receiver. Initially, the best accuracy p best is the prediction accuracy when all data are input;

[0174] Reduce one or a group of receiver data (or transmitter data) in turn, and calculate the predicted situation category p under this layout according to the process of module one. ij ; Calculation area A ij The prediction accuracy is calculated as follows:

[0175]

[0176] Among them, y ij For area A ij The real situation category, is the predicted situation category, N is the total number of samples; the method of module 2 is also used to obtain the average accuracy p of all regions com,but

[0177]

[0178] Among them, p best The best accuracy after this step, and record the device layout under the best accuracy, p com is the accuracy under the current layout, is the best accuracy before this step;

[0179] If the accuracy of the WiFi signal decreases after reducing the number of devices, we use triangular relationship prediction technology to analyze and predict the best solution for adding new devices or adjusting the location of existing devices, and then repeat the above process;

[0180] The above steps will be executed repeatedly until an optimal configuration state is reached. best This is the resolution of the current optimal layout.

[0181] like Figure 2 , 3 As shown, a WIFI-based classroom activity situation awareness system provided by an embodiment of the present invention includes:

[0182] The perception computing module is used for classroom situation perception computing in different regions. It uses a multi-transmitter and multi-receiver WIFI device to obtain the WIFI signal reflected by the human body in the classroom, and converts the original signal into a frequency domain image through noise reduction, filtering, and STFT methods. Then, deep learning technology is used to extract the spatiotemporal features of each channel. The attention mechanism is used to assign weights and fusion calculations to different channel features, and the fusion features of each area of ​​the classroom are output, and then the situation category of the corresponding area is calculated.

[0183] Resolution optimization module, used for optimizing the resolution of classroom situations; compare and evaluate the regional prediction value with the actual value, calculate the prediction accuracy at the current resolution, adjust the regional resolution on this basis, calculate the prediction accuracy at different resolutions, and then calculate the maximum resolution and optimal resolution of the classroom;

[0184] The layout optimization module is used to optimize the layout of WIFI devices. On the basis of fixed resolution, the data of a certain channel of the WIFI receiving end is removed, and the predicted value of the regional situation is calculated again and compared with the actual value for evaluation. The recognition accuracy under the current WIFI combination is calculated, and then the removed data channel is adjusted to calculate the situation recognition accuracy under different combinations. The best WIFI combination is obtained by comparing the results, and the layout of the device is adjusted.

[0185] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the WIFI-based classroom activity situation awareness method.

[0186] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the WIFI-based classroom activity situation awareness method.

[0187] Another object of the present invention is to provide an information data processing terminal, which is used to implement the WIFI-based classroom activity situation awareness system.

[0188] 1. Specific application fields or related products of the present invention.

[0189] 2. Relevant evidence of the technical effects obtained by the embodiments of the present invention.

[0190] like Figure 4 Experimental configuration: This article uses an Intel 5300-based network card and a small computer with an Ubuntu 16.04 kernel operating system. The 802.s11nCSI Tool acquisition tool provided by Daniel Halperin et al. is installed and configured in monitor mode.

[0191] Actual scenario Figure 5 :The classroom for this experiment recruited 16 volunteers, including 10 boys and 6 girls, with an age range of 23 to 25 years old, a height range of 1.60 to 1.75 meters, and a weight range of 50 to 80 kg. Three classroom activity postures were set up, namely "standing up", "discussing", "pushing", and "sitting quietly".

[0192] Data preprocessing:

[0193] Before and after noise reduction using low-pass Butterworth filter Figure 6 :

[0194] Comparison before and after wavelet transform denoising Figure 7 :

[0195] The feature extraction steps are based on VGG, BIGRU, gating mechanism, and attention mechanism as mentioned above.

[0196] Schematic diagram of CSI feature map formation Figure 8 。

[0197] Analysis of experimental results: In order to analyze the impact of feature type selection on experimental results, this paper analyzes the detection of 5 different feature types.

[0198]

[0199] Analysis of the results of different classification algorithms.

[0200] According to the design of the experimental plan, this paper compares the effective information, selects CNN model, CNN+LSTM model, CNN+GRU model, BiLSTM+Attention, deep neural network model based on model training and experimental comparison, and compares the accuracy, recall rate and F1 value of the classification prediction of students' classroom activity status recognition by each model after parameter optimization. The experimental comparison results of each model are shown in Table 3.6 below.

[0201]

[0202] In order to facilitate the comparative analysis of the effects of various algorithm models on classroom activity situation analysis, the results of the five experiments were made into bar statistics. Fig. 9 :

[0203] It can be clearly seen from the above data and visualization results that the deep neural network model based on CNN+BiGRU+Att selected in this paper performs well in the results of the overall activity situation in the classroom, which shows the superiority of the algorithm in processing CSI data.

[0204] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0205] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for classroom activity situation awareness based on WIFI, characterized in that: The following steps are involved: Step 1: Classroom situation perception and calculation by region; Use multi-transmitter and multi-receiver WIFI devices to obtain WIFI signals reflected by the human body in the classroom. Convert the original signals into frequency domain images through noise reduction, filtering, and STFT methods. Then use deep learning technology to extract the spatiotemporal features of each channel. Use the attention mechanism to assign weights and fusion calculations to different channel features, output the fusion features of each area of ​​the classroom, and then calculate the situation category of the corresponding area. Step 2: Optimize the resolution of classroom situation; Compare and evaluate the regional prediction value with the true value, calculate the prediction accuracy at the current resolution, adjust the regional resolution on this basis, calculate the prediction accuracy at different resolutions, and then calculate the maximum resolution and optimal resolution of the classroom; Step 3: Optimize the layout of WIFI devices; On the basis of fixed resolution, remove the data of a certain channel of the WIFI receiving end, calculate the predicted value of the regional situation again and compare and evaluate it with the actual value, calculate the recognition accuracy under the current WIFI combination, and then adjust the removed data channel, calculate the situation recognition accuracy under different combinations, and obtain the best WIFI combination by comparing the results, and optimize the layout of the equipment.

2. The WIFI-based classroom activity situation awareness method as claimed in claim 1, characterized in that: The classroom situation is divided into regional perception calculations: It includes data collection and preprocessing, spatiotemporal feature extraction, multi-channel feature fusion and regional situation classification processes; The multi-transmit and multi-receive Wi-Fi antenna layout system is composed of c signal transmitters and n signal receivers. The frequency response function H(f) of the low-pass filter can be expressed in the frequency domain as: where f c is the cutoff frequency, that is, signals above this frequency will be completely blocked, and signals below this frequency will pass without attenuation; from the perspective of the relationship between time domain and frequency domain, the input and output signals are connected through Fourier transform and its inverse transform; if the Fourier transform of the input signal x(t) is X(f), and the Fourier transform of the output signal y(t) is Y(f), then: Y(f)=H(f)X(f) Then the output signal is obtained through inverse Fourier transform: y(t)=F -1 {H(f)X(f)} This step can convert the raw data into a spectrum, which shows the distribution of the signal at different frequencies; VGG and BiGRU are used to extract spatial features and temporal features respectively. This method realizes the comprehensive analysis of the signal in the time domain and frequency domain, and finally outputs the feature combination [h1; h2; ...; h n ]; the details are as follows: For a single-channel spectrum graph, VGG is used for deep spatial feature extraction. The calculation method can be expressed by the following formula: V=F(Z;θ)=F out (F N (...F2(F1(Z)))) Among them, Z is the digital matrix of the spectrum map conversion, as the input of VGG, F1 to F N Represents the operations of each layer in the VGG network, including convolutional layer, pooling layer, activation function, F out is the output layer of the VGG network, V is the feature vector sequence output by the VGG network, and for channel i, V i =[v i1 ,v i2 ,...,v il ], l is the maximum number of features in a single sequence; The feature sequence extracted by the VGG network is input into the BiGRU network to further extract the temporal features. The calculation process is as follows: h i =H([V i ;V′ i ]) Among them, H represents the feature operation of BiGRU, V i is the characteristic sequence of channel i, V' i The reverse sequence of channel i, h i is the feature vector output by BiGRU; for n signal receiving channels, the feature vectors of all channels [h1; h2; ...; h n ]; The gating mechanism is used to dynamically learn the weights of different channel features. The calculation formula is as follows: [g1;g2;...;gn]=softmax([W1·h1;W2·h2;...;W n ·h n ]) Among them, g n represents the feature weight of the nth channel, W n Represents the weight matrix of the channel. The weight value of each channel is calculated by the softmax function. The value is a decimal between 0 and 1, and the sum of all weight values ​​is 1. On this basis, the multi-channel fusion features are calculated, and the calculation formula is: Among them, i represents the i-th channel, and h is the fusion feature after weighted calculation of different channels; The classification features of each target region are calculated based on the fusion feature h, and the enhanced features of each target region are obtained using the Transferformer mechanism. The calculation formula for a single region is: a ij =Transformer(h q ,h k ,h v ) where a ij Represents the enhanced features of the target area. The values ​​of i and j are constrained by the resolution. For m×m resolution, 1≤i,j≤m; On this basis, the predicted value of the regional situation is calculated using the following formula: Where W ij is the weight matrix, b ij is the deviation, is the forecast category of regional situation; According to the training principle of deep learning, the loss is calculated using the predicted value and the true value, and then the weight parameters are optimized using back propagation. After multiple rounds of iterative training, the weight parameters with the best accuracy are obtained. For a single region, cross entropy is used as the loss function, which is expressed as: where y ij is the true value of the area, and N is the number of samples used in a single training; For the entire classroom area, its loss function is the sum of the loss values ​​of all areas, expressed as: where γ ij It is a hyperparameter, which indicates the calculation weight of the loss function in different regions. It is set to γ ​​by default. 11 =γ 12 =... =γ mm =1.

3. The WIFI-based classroom activity situation awareness method as claimed in claim 1, characterized in that: The classroom situation resolution optimization: Includes fixed resolution accuracy calculation and dynamic resolution tuning; Multi-resolution calculation and tuning strategy for classroom situation categories; resolution represents the number of regional divisions, and m*m resolution means that the classroom is divided into m*m regions, and each region is represented by A ij Indicates, i,j∈[1,m]; for region A ij , through the method proposed in module 1, the predicted situation category can be calculated Further, the area A can be calculated ij The prediction accuracy can be expressed as: where y ij For area A ij The true situation category, N is the total number of samples; The average accuracy of all regions at m*m resolution can be calculated, and the calculation formula can be expressed as: By changing the resolution m, the average accuracy of the region at different resolutions can be calculated; The threshold η is set as the minimum acceptable accuracy, 0<η<1, and the resolution combination greater than the threshold η can be calculated, where max(m) is the maximum resolution allowed, and the maximum accuracy in the combination is the optimal resolution, that is: P best =max([p m >h]) P best The corresponding resolution is the optimal resolution.

4. The method for classroom activity situation awareness based on WIFI as claimed in claim 1, characterized in that: The WIFI equipment layout is optimized: WiFi layout optimization strategy for classroom activity situation; In the initial state, all devices are in the activation stage, and all data are collected to obtain the data of the c-channel transmitter and the n-channel receiver. Initially, the best accuracy p best is the prediction accuracy when all data are input; Reduce one or a group of receiver data (or transmitter data) in turn, and calculate the predicted situation category p under this layout according to the process of module one. ij ; Calculation area A ij The prediction accuracy is calculated as follows: Among them, y ij For area A ij The real situation category, is the predicted situation category, N is the total number of samples; the method of module 2 is also used to obtain the average accuracy p of all regions com ,but Among them, p best The best accuracy after this step, and record the device layout under the best accuracy, p com is the accuracy under the current layout, is the best accuracy before this step; If the accuracy of the WiFi signal decreases after reducing the number of devices, we use triangular relationship prediction technology to analyze and predict the best solution for adding new devices or adjusting the location of existing devices, and then repeat the above process; The above steps will be executed repeatedly until an optimal configuration state is reached. best This is the resolution of the current optimal layout.

5. A WIFI-based classroom activity situation awareness system implementing the WIFI-based classroom activity situation awareness method as described in any one of claims 1 to 4, characterized in that: The WIFI-based classroom activity situation awareness system includes: The perception computing module is used for classroom situation perception computing in different regions. It uses a multi-transmitter and multi-receiver WIFI device to obtain the WIFI signal reflected by the human body in the classroom, and converts the original signal into a frequency domain image through noise reduction, filtering, and STFT methods. Then, deep learning technology is used to extract the spatiotemporal features of each channel. The attention mechanism is used to assign weights and fusion calculations to different channel features, and the fusion features of each area of ​​the classroom are output, and then the situation category of the corresponding area is calculated. Resolution optimization module, used for optimizing the resolution of classroom situations; compare and evaluate the regional prediction value with the actual value, calculate the prediction accuracy at the current resolution, adjust the regional resolution on this basis, calculate the prediction accuracy at different resolutions, and then calculate the maximum resolution and optimal resolution of the classroom; The layout optimization module is used to optimize the layout of WIFI devices. On the basis of fixed resolution, the data of a certain channel of the WIFI receiving end is removed, and the predicted value of the regional situation is calculated again and compared with the actual value for evaluation. The recognition accuracy under the current WIFI combination is calculated, and then the removed data channel is adjusted to calculate the situation recognition accuracy under different combinations. The best WIFI combination is obtained by comparing the results, and the layout of the device is adjusted.

6. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the WIFI-based classroom activity situation awareness method as described in any one of claims 1-4.

7. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the WIFI-based classroom activity situation awareness method as described in any one of claims 1 to 4.

8. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the WIFI-based classroom activity situation awareness system as described in claim 5.

Images