CSI (Channel State Information) indoor position classification and behavior identification method based on double networks and related equipment

Abstract

The invention discloses a CSI (Channel State Information) indoor position classification and behavior identification method based on double networks and related equipment, and belongs to the technical field of indoor perception. The method comprises the following steps: firstly, dividing an indoor position data set into a first indoor position data set containing a plurality of grid position point CSI data corresponding to a first room and a second indoor position data set containing a plurality of grid position point CSI data corresponding to other rooms; the first indoor position data set is used for training a Resnet-Densenet parallel network, and according to the second indoor position data set and the behavior posture data set, the trained Resnet-Densenet parallel network is used for achieving a behavior posture recognition result and outputting position classification results of other rooms; according to the method, a learning framework of cross-domain migration is used, a neural network model fully trained by a large amount of data can be reused, when classification and recognition tasks are achieved on a new target domain, the network does not need to be trained from the beginning, the training cost is saved, and the efficiency and the result precision are improved.

Description

Technical Field

[0001] The present invention belongs to the field of indoor perception technology, and specifically relates to a dual-network-based CSI indoor position classification and behavior recognition method and related equipment. Background Art

[0002] With the continuous advancement of location-based service technology, smart indoor applications have shown explosive growth, bringing broad application prospects for the development of indoor sensing technology. Among them, accurately identifying the user's location and behavior is the most critical link in realizing smart indoor applications. For example, service robots need to accurately perceive the user's location and movement to provide personalized services; checkout-free supermarkets need to track customer dynamics in real time and optimize the layout of shopping malls; smart homes need to perceive the status of residents and automatically adjust environmental parameters. Therefore, paying attention to and improving the classification and positioning capabilities of indoor environmental perception has become a top priority in the current development of indoor sensing technology.

[0003] At present, with the development of machine learning and deep learning, traditional technical means have gradually failed to meet the increasingly complex needs of users. Location classification and behavioral posture recognition tasks have also put forward higher requirements for the field of indoor perception. Many deep learning studies have made significant progress in location classification and human posture recognition based on channel state information (CSI) based on WIFI. However, although there are certain similarities in extracting CSI features to achieve location classification and posture recognition tasks respectively, since the CSI pattern of the location classification task will change in different domains (i.e., room, environment), and relying on CSI features to achieve two completely different classification tasks is extremely challenging, using a single model often has weak generalization ability and cannot be well generalized to new scenarios or data distributions. At this time, repeatedly training the model from scratch in the new domain will consume a lot of time and computing resources; and due to the differences between feature spaces or tasks, directly migrating the model from one domain to another often has poor results; in addition, the current wireless cross-domain solution is limited to cross-single-factor tasks, requiring a large amount of target domain data to participate in training or using multiple transceivers for location-independent feature extraction.

[0004] Most existing position classification and behavior recognition technologies are based on several public gesture recognition data sets, or require a large number of training samples, complex network models, and cumbersome feature transformation methods to achieve high-precision position classification and gesture recognition tasks. Once the task domain or experimental scenario changes, the model must be retrained, which is time-consuming, labor-intensive, inefficient, and the task accuracy will also degrade significantly.

[0005] It can be seen that the existing location classification and behavior recognition technologies, due to domain differences, need to retrain the model when implementing classification and recognition tasks for new target domains, resulting in low efficiency and poor accuracy. Summary of the invention

[0006] The present invention provides a dual-network based CSI indoor location classification and behavior recognition method and related equipment to solve the technical problems of low efficiency and poor accuracy caused by the need to retrain the model when implementing classification and recognition tasks for new target domains due to domain differences in existing location classification and behavior recognition technologies.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A dual-network based CSI indoor location classification and behavior recognition method, comprising:

[0009] Acquire an indoor location dataset and a behavior posture dataset, and divide the acquired indoor location dataset into a first indoor location dataset and a second indoor location dataset; wherein the first indoor location dataset includes CSI data of multiple grid location points corresponding to the first room, and the second indoor location dataset includes CSI data of multiple grid location points corresponding to other rooms; the behavior posture dataset includes multiple behavior posture CSI data collected at multiple grid locations corresponding to the first room;

[0010] Based on the first indoor location data set, a pre-built Resnet-Densenet parallel network is trained, and the trained Resnet-Densenet parallel network and the first location classification result are output;

[0011] The behavior posture dataset and the second indoor location dataset are respectively input into the trained Resnet-Densenet parallel network, and the behavior posture recognition results and the location classification results of other rooms are output; wherein, before the behavior posture dataset and the second indoor location dataset are input into the Resnet-Densenet parallel network, the parameter weights of the fully connected layer of the Resnet-Densenet parallel network are unfrozen and the parameters of other layers remain unchanged, so as to realize the cross-domain migration of the behavior posture dataset and the second indoor location dataset under the Resnet-Densenet parallel network.

[0012] Furthermore, after obtaining the indoor location data set and the corresponding behavior posture data set, Hampel filtering is used to remove outliers and noise from all data sets, and sliding window filtering is used to dynamically repeat reading for data enhancement processing to achieve data set expansion.

[0013] Furthermore, the Resnet-Densenet parallel network includes parallel Resnet34 and Densenet121 networks; the first indoor position data set is input into the Resnet-Densenet parallel network for feature extraction, and the contact features are spliced ​​and fused to obtain the first position probability as input data of the Resnet-Densenet parallel network.

[0014] Furthermore, 20% to 30% of the samples in the behavior posture dataset are input into the trained Resnet-Densenet parallel network to realize network training and feature extraction, and the probability of a certain posture is obtained, thereby outputting the behavior posture recognition result; wherein, before network training and feature extraction, the parameters of the fully connected layer of the trained Resnet-Densenet parallel network have been unfrozen and the parameters of other layers remain unchanged.

[0015] Furthermore, 20% to 30% of the samples in the second indoor location data are input into the trained Resnet-Densenet parallel network to realize network training and feature extraction, and the probability of a certain location is obtained, thereby outputting the location classification results of other rooms; wherein, before network training and feature extraction, the parameters of the trained Resnet-Densenet parallel network have been unfrozen in the fully connected layer and the parameters of other layers remain unchanged.

[0016] Furthermore, the first room is a living room, and the other rooms include a bedroom, a kitchen, a bathroom and a dining room; the location points of the living room and the location points of the other rooms are collected separately according to a grid; the behavior posture data set includes standing, squatting, bending over, marching in place, jumping in place and sitting down behavior posture CSI data.

[0017] Further, the CSI data is represented as:

[0018]

[0019] In the formula, H i,j represents the CSI of the jth subcarrier of the i-th receiving antenna, N C Indicates the number of subcarriers.

[0020] A dual-network-based CSI indoor location classification and behavior recognition system, used to implement the steps of the dual-network-based CSI indoor location classification and behavior recognition method, including:

[0021] A data acquisition module is used to obtain an indoor position data set and a behavior posture data set, and divide the obtained indoor position data set into a first indoor position data set and a second indoor position data set; wherein the first indoor position data set includes CSI data of multiple grid position points corresponding to the first room, and the second indoor position data set includes CSI data of multiple grid position points corresponding to other rooms; the behavior posture data set includes multiple behavior posture CSI data collected at multiple grid positions corresponding to the first room;

[0022] A model training module, used for training a pre-built Resnet-Densenet parallel network based on a first indoor location data set, outputting the trained Resnet-Densenet parallel network and outputting a first location classification result;

[0023] The target result output module is used to input the behavior posture dataset and the second indoor location dataset into the trained Resnet-Densenet parallel network respectively, output the behavior posture recognition results and output the location classification results of other rooms; wherein, before inputting the behavior posture dataset and the second indoor location dataset into the Resnet-Densenet parallel network, the parameter weights of the fully connected layer of the Resnet-Densenet parallel network are unfrozen and the parameters of other layers remain unchanged, so as to realize the cross-domain migration of the behavior posture dataset and the second indoor location dataset under the Resnet-Densenet parallel network.

[0024] A device comprising:

[0025] Memory for storing computer programs;

[0026] The processor is used to implement the steps of the above-mentioned dual-network-based CSI indoor location classification and behavior recognition method when executing the computer program.

[0027] A computer-readable storage medium stores a computer program, which, when executed by a processor, is used to implement the steps of the dual-network-based CSI indoor location classification and behavior recognition method.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention also provides a dual-network-based CSI indoor location classification and behavior recognition method. The method first divides an indoor location data set into a first indoor location data set including CSI data of multiple grid location points corresponding to a first room and a second indoor location data set including CSI data of multiple grid location points corresponding to other rooms; the Resnet-Densenet parallel network is trained using the first indoor location data set, and the behavior posture recognition results and the location classification results of other rooms are output using the trained Resnet-Densenet parallel network according to the second indoor location data set and the behavior posture data set; the method uses a cross-domain migration learning framework, which can reuse a neural network model that has been fully trained with a large amount of data. When implementing classification and recognition tasks in a new target domain, there is no need to train the network from scratch, which saves training costs and improves efficiency and result accuracy.

[0030] In addition, the present invention also has the following advantages:

[0031] First, Hampel filtering is used to remove outliers and noise from all data sets to ensure the accuracy and effectiveness of the data; sliding window filtering is used for dynamic repeated reading for data enhancement processing, which greatly expands the data set and improves the flexibility and adaptability of this method.

[0032] Second, by connecting the traditional Resnet and Densenet in parallel to form a dual network structure, the width of the neural network is increased, which greatly improves the accuracy and effectiveness of feature extraction compared to the traditional single network model.

[0033] Third, by using only a small number of samples to fine-tune the model parameters in the new target domain, good classification and recognition effects can be achieved. This does not require a large amount of data resources, improves task efficiency, and does not waste computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A system framework diagram corresponding to the dual-network-based CSI indoor location classification and behavior recognition method provided in an embodiment of the present invention;

[0035] Figure 2 An experimental scene diagram of the dual-network-based CSI indoor location classification and behavior recognition method provided in an embodiment of the present invention;

[0036] Figure 3 A dual network feature extraction and fusion module diagram provided in an embodiment of the present invention;

[0037] Figure 4 A cross-domain migration framework diagram provided for an embodiment of the present invention;

[0038] Figure 5A diagram showing the effect of cross-domain migration on the convergence speed of the loss function provided by an embodiment of the present invention; wherein (a) is direct training of a small sample data set; (b) is using a migration learning framework;

[0039] Figure 6 The effect diagram of the dual network feature fusion module provided in the embodiment of the present invention on the source domain training accuracy and loss function; wherein (a) is the effect of the dual network model on the accuracy; (b) is the effect of the dual network model on the loss function;

[0040] Figure 7 A comparison diagram of classification results of a single and double network for a target domain indoor location classification dataset provided by an embodiment of the present invention;

[0041] Figure 8 A comparison chart of classification results of the single and double networks provided in the embodiment of the present invention for target domain behavior posture recognition;

[0042] Fig. 9 A flow chart of the dual-network-based CSI indoor location classification and behavior recognition method provided by the present invention;

[0043] Fig.10 A schematic diagram of the structure of the dual-network-based CSI indoor location classification and behavior recognition system provided by the present invention. DETAILED DESCRIPTION

[0044] The present invention provides a dual-network based CSI indoor location classification and behavior recognition method, such as Fig. 9 As shown, the following steps are included:

[0045] S1: Acquire an indoor location dataset and a behavior posture dataset, and divide the acquired indoor location dataset into a first indoor location dataset and a second indoor location dataset; wherein the first indoor location dataset includes CSI data of multiple grid location points corresponding to the first room, and the second indoor location dataset includes CSI data of multiple grid location points corresponding to other rooms; the behavior posture dataset includes multiple behavior posture CSI data collected at multiple grid locations corresponding to the first room;

[0046] After obtaining the indoor location data set and the corresponding behavior posture data set, Hampel filtering is used to remove outliers and noise from all data sets. At the same time, sliding window filtering is used to dynamically repeat reading for data enhancement processing to achieve data set expansion.

[0047] S2: Train the pre-built Resnet-Densenet parallel network based on the first indoor location data set, output the trained Resnet-Densenet parallel network and output the first location classification result;

[0048] The Resnet-Densenet parallel network includes parallel Resnet34 and Densenet121 networks; the first indoor location data set is input into the Resnet-Densenet parallel network for feature extraction, and the contact features are spliced ​​and fused to obtain the first location probability as input data of the Resnet-Densenet parallel network.

[0049] S3: Input the behavior posture dataset and the second indoor location dataset into the trained Resnet-Densenet parallel network respectively, output the behavior posture recognition results and output the location classification results of other rooms.

[0050] Specifically, some samples in the behavior posture dataset are input into the trained Resnet-Densenet parallel network to realize network training and feature extraction, and the probability of a certain posture is obtained as input data, thereby outputting the behavior posture recognition result; wherein, before network training and feature extraction, the parameters of the fully connected layer of the trained Resnet-Densenet parallel network have been unfrozen, and the parameters of other layers remain unchanged, which is used to realize the cross-domain migration of the behavior posture dataset and the second indoor position dataset under the Resnet-Densenet parallel network.

[0051] Some samples in the second indoor location data set are input into the trained Resnet-Densenet parallel network to realize network training and feature extraction, and the probability of a certain location is obtained as input data, thereby outputting the location classification results of other rooms; wherein, before network training and feature extraction, the parameters of the fully connected layer of the trained Resnet-Densenet parallel network have been unfrozen.

[0052] Among them, the first room is the living room, and the other rooms include bedrooms, kitchens, bathrooms and dining rooms; the location points of the living room and the location points of other rooms are collected separately according to the grid; the behavior posture data set includes standing, squatting, bending over, stepping on the spot, jumping on the spot and sitting down behavior posture CSI data.

[0053] Specifically, the CSI data is represented as:

[0054]

[0055] In the formula, H i,j represents the CSI of the jth subcarrier of the i-th receiving antenna, N C Indicates the number of subcarriers.

[0056] like Fig.10As shown, the present invention also provides a dual-network-based CSI indoor location classification and behavior recognition system, including: a data acquisition module, used to obtain an indoor location data set and a behavior posture data set, and divide the obtained indoor location data set into a first indoor location data set and a second indoor location data set; wherein the first indoor location data set includes CSI data of multiple grid location points corresponding to the first room, and the second indoor location data set includes CSI data of multiple grid location points corresponding to other rooms; the behavior posture data set includes multiple behavior posture CSI data collected at multiple grid locations corresponding to the first room; a model training module, used to train a pre-constructed Resnet-Densenet parallel network based on the first indoor location data set, and output the training A good Resnet-Densenet parallel network and output the first position classification result; a target result output module is used to input the behavior posture dataset and the second indoor position dataset into the trained Resnet-Densenet parallel network respectively, output the behavior posture recognition result and output the position classification results of other rooms; wherein, before the behavior posture dataset and the second indoor position dataset are input into the Resnet-Densenet parallel network, the parameter weights of the fully connected layer of the Resnet-Densenet parallel network are unfrozen and the parameters of other layers remain unchanged, so as to realize the cross-domain migration of the behavior posture dataset and the second indoor position dataset under the Resnet-Densenet parallel network.

[0057] The present invention also provides a device, comprising: a memory for storing a computer program; a processor for implementing the steps of the dual-network-based CSI indoor location classification and behavior recognition method when executing the computer program.

[0058] When the processor executes the computer program, the above-mentioned dual-network-based CSI indoor position classification and behavior recognition steps are implemented, for example: an indoor position data set and a behavior posture data set are obtained, and the obtained indoor position data set is divided into a first indoor position data set and a second indoor position data set; wherein the first indoor position data set includes CSI data of multiple grid position points corresponding to the first room, and the second indoor position data set includes CSI data of multiple grid position points corresponding to other rooms; the behavior posture data set includes multiple behavior posture CSI data collected at multiple grid positions corresponding to the first room; based on the first indoor position data set, a pre-constructed Resnet-Densenet parallel network is trained, and the trained Resnet-Densenet parallel network and the first position classification result are output; the behavior posture data set and the second indoor position data set are respectively input into the trained Resnet-Densenet parallel network, and the behavior posture recognition result and the position classification result of other rooms are output.

[0059] Alternatively, when the processor executes the computer program, the functions of each module in the above system are implemented, for example: a data acquisition module is used to obtain an indoor position data set and a behavior posture data set, and divide the obtained indoor position data set into a first indoor position data set and a second indoor position data set; wherein the first indoor position data set includes CSI data of multiple grid position points corresponding to the first room, and the second indoor position data set includes CSI data of multiple grid position points corresponding to other rooms; the behavior posture data set includes multiple behavior posture CSI data collected at multiple grid positions corresponding to the first room; a model training module is used to train a pre-constructed Resnet-Densenet parallel network based on the first indoor position data set, and output the trained model. The target result output module is used to input the behavior posture dataset and the second indoor location dataset into the trained Resnet-Densenet parallel network respectively, output the behavior posture recognition result and output the location classification results of other rooms; wherein, before inputting the behavior posture dataset and the second indoor location dataset into the Resnet-Densenet parallel network, the parameter weights of the fully connected layer of the Resnet-Densenet parallel network are unfrozen and the parameters of other layers remain unchanged, so as to realize the cross-domain migration of the behavior posture dataset and the second indoor location dataset under the Resnet-Densenet parallel network.

[0060] Exemplarily, the computer program may be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing preset functions, and the instruction segments are used to describe the execution process of the computer program in the dual-network-based CSI indoor location classification and behavior recognition device. For example, the computer program may be divided into a data acquisition module, a model training module, and a target result output module; the specific functions of each module are as follows: a data acquisition module, used to acquire an indoor location data set and a behavior posture data set, and divide the acquired indoor location data set into a first indoor location data set and a second indoor location data set; wherein the first indoor location data set contains CSI data of multiple grid location points corresponding to the first room, and the second indoor location data set contains CSI data of multiple grid location points corresponding to other rooms; the behavior posture data set contains multiple behavior posture CSI data collected at multiple grid locations corresponding to the first room; a model training module, used to train a pre-built Resnet-Densenet parallel network based on the first indoor location data set The training module outputs the trained Resnet-Densenet parallel network and the first position classification result; the target result output module is used to input the behavior posture dataset and the second indoor position dataset into the trained Resnet-Densenet parallel network respectively, output the behavior posture recognition result and output the position classification result of other rooms; wherein, before the behavior posture dataset and the second indoor position dataset are input into the Resnet-Densenet parallel network, the parameter weights of the fully connected layer of the Resnet-Densenet parallel network are unfrozen and the parameters of other layers remain unchanged, so as to realize the cross-domain migration of the behavior posture dataset and the second indoor position dataset under the Resnet-Densenet parallel network.

[0061] The dual-network-based CSI indoor location classification and behavior recognition device can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The dual-network-based CSI indoor location classification and behavior recognition device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above is an example of a dual-network-based CSI indoor location classification and behavior recognition device, and does not constitute a limitation on the dual-network-based CSI indoor location classification and behavior recognition device, and may include more components than the above, or a combination of certain components, or different components. For example, the dual-network-based CSI indoor location classification and behavior recognition device may also include input and output devices, network access devices, buses, and the like.

[0062] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The processor is the control center of the dual-network-based CSI indoor location classification and behavior recognition, and uses various interfaces and lines to connect various parts of the entire dual-network-based CSI indoor location classification and behavior recognition device.

[0063] The memory can be used to store the computer program and / or module, and the processor implements various functions of the dual-network-based CSI indoor location classification and behavior recognition device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory.

[0064] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0065] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the CSI indoor location classification and behavior recognition method based on dual networks are implemented.

[0066] If the module / unit integrated in the dual-network-based CSI indoor location classification and behavior recognition system is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0067] Based on such understanding, the present invention implements all or part of the processes in the above-mentioned dual-network-based CSI indoor location classification and behavior identification method, and can also be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned dual-network-based CSI indoor location classification and behavior identification method can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or preset intermediate form, etc.

[0068] The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0069] It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electrical carrier signals and telecommunication signals.

[0070] The present invention will be further described below in conjunction with the embodiments and drawings:

[0071] Example

[0072] As described in the background technology, researchers have conducted extensive research on current indoor perception tasks. Among them, sensor-based methods mainly rely on wearable sensors, requiring users to carry sensor devices with them, which is easy to cause inconvenience to users' lives; some researchers have proposed vision-based human behavior recognition technology, using cameras and other devices to capture human motion data, and combining computer vision algorithms to accurately analyze user postures, but camera-based technology requires users to appear in a well-lit environment, which is easy to infringe on user privacy. Unlike other methods, the deployment of a large number of WIFI hotspots has led to an increase in research on using WIFI signals to identify user locations and behavior postures. It does not require additional hardware equipment, does not require wearing sensors, and can achieve relatively accurate activity recognition, with broad application prospects. In traditional WIFI-based indoor location classification and behavior posture recognition tasks, using received signal strength indication (RSSI) is a common practice, but due to the complex and changeable indoor environment and the serious multipath effect, the RSSI signal no longer decreases monotonically with distance, but presents complex fluctuation characteristics, which greatly limits the perception ability of RSSI, making it more suitable for some coarse-grained perception tasks, and unable to achieve fine location classification and posture recognition. In order to overcome the limitations of RSSI, researchers began to focus on using channel state information (CSI) as a perception feature. CSI is a fine-grained physical layer that is more sensitive to the environment and reflects the propagation characteristics of wireless signals in complex indoor environments. At the same time, CSI data has a higher dimension and can provide stronger representation capabilities for machine learning and deep learning algorithms, improving the accuracy of classification and recognition. In addition, with the development of wireless communication technology, the acquisition cost and complexity of CSI information are gradually decreasing. Using the frequency response information of multiple subcarriers of the channel as features to train the classifier can achieve relatively accurate classification and recognition. Most of the existing location classification and behavior recognition technologies are based on several public gesture recognition data sets, or require a large number of training samples, complex network models, and cumbersome feature transformation methods to achieve high-precision location classification and gesture recognition tasks. Once the task domain or experimental scenario changes, the model must be retrained, which is time-consuming and labor-intensive, inefficient, and the task accuracy will also be significantly degraded. In addition, there is currently no work that can use the data from training indoor location classification to realize user behavior posture recognition and realize a system that spans two different tasks. Instead, the classification and recognition of the two are often implemented separately, or they are regarded as a joint task, resulting in the model requiring multiple branches, being more complex, having high training costs, and lacking generalization and robustness.

[0073] In order to solve the above problems, the present invention provides a CSI indoor location classification and behavior recognition method based on dual networks. This method refers to the human learning mechanism. When humans learn new tasks, they often use previously learned knowledge and experience. By simulating this human learning behavior, a more intelligent and efficient machine learning model is constructed. Therefore, it is considered to train the model in the first indoor location data set, i.e., the source domain, and enhance the feature extraction ability of the neural network based on the transfer learning framework, and use a few samples in the target domain to fine-tune the network. In the indoor smart home environment, this idea is applied to realize low-cost classification and recognition functions between different rooms and different locations.

[0074] like Figure 1 As shown, this embodiment provides a CSI indoor location classification and behavior recognition method based on dual networks, combined with Figure 1 The system framework in the steps include:

[0075] Step 1: Collect CSI data. Collect data sets in a furniture apartment with multiple rooms to obtain two data sets for indoor location classification and one data set for behavior and posture recognition. The indoor location classification data set is divided into data set 1 (the first indoor location data set) containing 30 grid locations in the living room and data set 2 (the second indoor location data set) containing 16 location points in the other 4 rooms.

[0076] Step 2: Use hampel filtering to remove data noise, use sliding window filtering with custom starting coordinates for data enhancement, split the data group and expand the data set by repeated sliding;

[0077] Step 3: For the indoor location classification dataset 1, use the designed Resnet-Densenet parallel dual network to extract features, concatenate and fuse the extracted features, and obtain the probability that the input data is a certain location;

[0078] Step 4: Output the position classification results and save the parameters and weights of each layer of the neural network trained in step 3;

[0079] Step 5: For the behavioral posture recognition dataset, use the network model with the parameters of each layer frozen in the previous step, unfreeze the parameters of the fully connected layer, input a small number of samples (20% to 30%) into the network for training and feature extraction, obtain the probability that the input data is a certain posture, and output the posture classification result;

[0080] Step 6: For indoor location classification dataset 2, use the network model saved in step 4, only unfreeze the parameters of the fully connected layer, input a small number of samples (20% to 30%) into the network, and output the location classification results of several other rooms.

[0081] Step 1: Data Collection:

[0082] In the field of wireless communications, CSI is the channel property of a communication link. The channel impact response (CIR) is used to describe the multipath effect of the channel. Under the linear time-invariant assumption, the channel impulse response can be given by the following formula:

[0083]

[0084] Where α i ,θ i and τ i They represent the amplitude attenuation, phase shift and time delay of the i-th path respectively; N represents the total number of propagation paths; δ(τ) represents the Dirichlet pulse function.

[0085] On the other hand, the frequency response (CFR) of a wireless channel can also be used to describe the multipath propagation of a signal using amplitude-frequency characteristics and phase-frequency characteristics. The frequency response of a channel can be expressed as follows.

[0086] H(k)=||H(k)||e j∠H(k)

[0087] Where H(k) represents the channel state information (CSI) of the kth subcarrier; ||H(k)|| and ∠H(k) represent the amplitude and phase of the kth subcarrier.

[0088] With the development of hardware technology, commercial WIFI infrastructure such as Inter 5300 wireless network interface card can obtain the physical layer channel frequency response information of multiple channels (subcarriers). If the transmitting antenna is 1 and the receiving antenna is N, the CSI extracted for each data packet is:

[0089]

[0090] Where H i,j represents the CSI of the jth subcarrier of the i-th receiving antenna, N C Indicates the number of subcarriers. For the Intel 5300 network card used in this article, Nc=30.

[0091] During the collection, the experimental equipment includes two computers equipped with Intel 5300 network cards, on which the Ubuntu operating system and CSItool are installed. The receiver uses Ubuntu commands to receive data, and the received signals are saved in files for CSI extraction. One transmitting antenna and three receiving antennas are used to collect CSI information. The sampling frequency of the receiving end is 100HZ, that is, 100 CSI packets are received per second. Figure 2As shown in the figure, 30 positions are collected in the living room according to the grid, and 1000 sets of CS I data are collected at each position for 10 seconds. In addition, six different postures are tested at these 30 positions: stand, squat, bend, walk, jump and sit. The first two columns are completed by tester 1, and the last three columns are performed by tester 2. In the remaining 4 rooms, 4 different positions are collected according to the size of the room. The collection method is the same as that of the living room, and all are completed by tester 1. The corresponding relationship between the labels and positions of different rooms is shown in Table 1, and the corresponding relationship between the labels of different actions is shown in Table 2.

[0092] Table 1 shows the correspondence between different room labels and locations

[0093]

[0094] Table 2 shows the correspondence between different action labels.

[0095]

[0096] Step 2: Data preprocessing and data enhancement:

[0097] After the CSI data set is collected, the original CSI measurement values ​​of the receiver are preprocessed by Hampel filtering to remove outliers and noise. In the data enhancement part, dynamic repeated reading is performed by sliding window filtering.

[0098] The Hampel filter is a filter based on the median and median absolute deviation (MAD), which is designed to identify and remove outliers in time series data. Compared with the traditional mean and standard deviation method, this filter is more robust to outliers, and it helps to remove outliers in the signal without over-smoothing the data. The core of the Hampel filter lies in the calculation of the median and the solution of the MAD. The median represents the middle value of the data, while the MAD measures the degree of dispersion between the data points and the median.

[0099] Median: For a data set X containing N data points, the median calculation formula is as follows:

[0100] If N is odd: If N is even:

[0101] Median Absolute Deviation (MAD): MAD is the median of the absolute differences between each data point Xi and the median of the data set X, and is calculated as:

[0102] MAD=median(|Xi-median(X)|), where Xi represents each data point.

[0103] In summary, the Hampel filter outlier judgment standard is: if the data point Xi is identified as an outlier (that is, its absolute deviation divided by MAD exceeds the threshold), then it is replaced with the median of the data set X; otherwise, the original value of the data point is retained. The mathematical expression is:

[0104]

[0105] The collected CSI data is filtered with a sliding window of a custom starting coordinate, where the sliding window size is set to 100 and the step length stride is 10. Because the format of the CSI data is (datalength, 1, 3, 30), a loop is designed so that the starting coordinate of the sliding window is i*stride, i starts from 1 and the maximum value is 90. In the actual data reading process, the size of i can be set as needed. In this experiment, i is set from 1 to 30 in the training phase and from 1 to 10 in the fine-tuning test phase. For the 1*3*30 CSI complex matrix, one column is extracted each time and made into the j column. The loop is also designed so that j is from 1 to 3.

[0106] Through data enhancement, the data set is greatly expanded. At this time, the CSI data is divided into several sequences of (100, 1, 1, 30).

[0107] Step 3: Dual network feature extraction and fusion:

[0108] Studies have shown that deepening the network depth and widening the network width can improve the performance of convolutional neural networks. Using a single network to extract features for different tasks is obviously not enough. Therefore, based on the existing convolutional neural networks, we consider increasing the network width by combining different convolutional neural network feature layers, learning complementary features, and extracting more accurate feature representations for recognition.

[0109] The Resnet34 model is a lightweight residual network. Compared with the traditional CNN model, it solves the gradient vanishing and representation bottleneck problems in deep networks by introducing residual blocks. The residual block adds a direct connection (short circuit) between the input and output, allowing the network to learn the residual mapping between the input and output, thereby effectively alleviating the training difficulties of deep networks.

[0110] On this basis, using Densenet as the second network is conducive to feature reuse. The dense connection mechanism of Densene can effectively reuse the features generated by the previous layer, reducing the repeated calculation of features and improving the parameter utilization of the network. This feature reuse capability can form a good complement to the underlying features extracted by Resnet.

[0111] At the same time, the network structure and regularization methods of Densenet can effectively suppress overfitting and achieve good generalization performance on small data sets. In addition, the multi-scale feature representations extracted at different depth layers are well integrated with the features extracted by Resnet to obtain richer representation capabilities; and the dense connection structure of Densenet can better promote the flow of gradients in the network, which helps to solve the gradient diffusion problem in deep networks.

[0112] In general, Densenet as the second network complements Resnet well and can improve the training stability and convergence of the network. Therefore, the present invention uses Resnet and Densenet networks as feature extractors, such as Figure 3 As shown, the input data passes through the feature layers of the two networks respectively, and the obtained features are first flattened and then fused through contact splicing. The splicing refers to the splicing of feature dimensions, rather than the addition of feature vectors. A fully connected layer is then added at the end of the algorithm for position or posture classification.

[0113] Step 4: Cross-domain migration:

[0114] For cross-domain transfer learning, the cross-tasks in the present invention are different. The training data in the source domain is dataset 1 for indoor room location classification, that is, 30 location points collected in the living room, and the target domains are dataset 2 for indoor room location classification, that is, a total of 16 different location data in the other four rooms and a behavioral posture dataset, that is, six different posture data collected at different locations in the living room.

[0115] In addition, the present invention spans different locations, and experiments test the recognition results of the trained model for six behaviors at different locations;

[0116] In this embodiment, the situation of different users is also studied, that is, two different testers are invited to perform the same action at the corresponding grid locations. Since different testers often have different behavioral habits, even the same action will produce different disturbances on the signal between the transmitter and the receiver. By comparing the results of different testers, the generalization of the network is verified.

[0117] Therefore, after expanding the source domain dataset through data augmentation, the dataset is used to train the designed Resnet-Densenet parallel dual neural network model, and the trained model parameters are completely preserved and frozen. In the target domain, the network is fine-tuned with a small amount of target domain data, and only the parameter weights of the fully connected layer are unfrozen, such as Figure 4 As shown, the final classification result of the target domain across tasks is obtained.

[0118] Step 5: Output of classification and recognition results:

[0119] The proposed dual network model determines the predicted label by outputting the probabilities corresponding to different positions and different behaviors, and compares it with the true label value to obtain the accuracy result. The following table shows the recognition results of the target domain in 10 different positions in the living room, with two testers performing six actions of standing (stand), squatting (squat), bending (bend), walking (walk), jumping (jump) and sitting (sit) respectively. This test result also proves that the proposed system has strong generalization, as shown in Table 3.

[0120] Table 3 shows the recognition results of six actions

[0121]

[0122] This embodiment also provides a dual-network-based CSI indoor location classification and behavior recognition system, which includes:

[0123] The data acquisition module collects test CSI data at the experimental site and obtains dat files containing the amplitude and phase of different CSI subcarriers to form three different data sets, which are used for training the source domain and testing the target domain respectively;

[0124] The data preprocessing and data enhancement module receives the collected indoor location classification data set 1 as the source domain, wherein the indoor location classification data set 1 refers to 30 location points collected in the living room, uses hampel filtering to remove noise, and obtains the expanded CSI data set through sliding window processing;

[0125] The dual network feature extraction and fusion module uses the Resnet-Densenet parallel network structure. The Resnet is a lightweight residual network Resnet34, and the Densenet is a densely connected neural network Densenet121. The processed data set is sent to the two networks in parallel, that is, the feature extraction effect of the two networks is connected in parallel, and then the two parts of the features are fused by contact splicing.

[0126] The cross-domain migration module uses the dual-network parallel model trained on dataset 1 to complete the cross-domain task, which is to achieve the indoor location classification of dataset 2 and the behavior recognition of the behavior posture dataset in two target domains, freeze the parameters of the trained network model, and use a small number of samples in the target domain to fine-tune the parameters of the fully connected layer;

[0127] The output module outputs the results of location classification and behavior recognition in the target domain.

[0128] The following is a further analysis and explanation of the experimental results with specific examples:

[0129] See also Figure 5 , the target domain is now the data of Room 2. Compare the results of directly using a small sample dataset for training and using the trained model to migrate to new data, and observe the convergence of the accuracy and loss function. From the graph, we can see that using the cross-domain migration framework can make the loss function of the training model converge quickly.

[0130] See also Figure 6 For the basic source domain training model, we first compare the changes in training and verification accuracy and loss function with the increase of training rounds when using the dual network parallel model or not. It can be seen that compared with using only the traditional Resnet network, the training and testing accuracy of the parallel dual network of Resnet-Densenet proposed in the present invention is significantly improved; at the same time, when the dual network model is used for training and verification, the loss function can converge faster and be reduced to a smaller level.

[0131] See also Figure 7 For the indoor location classification dataset 2 in the target domain data, that is, the location point data of the other 4 rooms, comparing the difference between using a single network model and a dual network parallel model, for rooms 2, 3, 4, and 5, the use of the dual network model has an improvement of about 5% in the classification accuracy results.

[0132] See also Figure 8 For the behavioral posture recognition dataset in the target domain data, by comparing the differences between using a single network model and a dual network parallel model, there is a significant improvement in the recognition accuracy for squatting and jumping on the spot, two actions that are easily confused as other behaviors.

[0133] In summary, the present invention provides a dual-network-based CSI indoor location classification and behavior recognition method, which reuses the CSI model trained in the source domain through a cross-domain migration framework, uses a sliding filter with a custom starting coordinate to achieve data enhancement, and uses the Resnet-Densenet parallel dual network to perform feature extraction and splicing fusion to improve the performance of the network. Through experimental comparison, the method is improved by 5% to 10% compared with the basic network model, and achieves a maximum accuracy of 98.15% for location classification tasks across location domains and test personnel domains; the accuracy of posture classification across task domains is as high as 92.27%.

[0134] The above embodiment is only one of the implementation methods that can realize the technical solution of the present invention. The scope of protection claimed by the present invention is not limited only to this embodiment, but also includes changes, replacements and other implementation methods that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed by the present invention.

Claims

1. A dual-network based CSI indoor location classification and behavior recognition method, characterized in that: include: Acquire an indoor location dataset and a behavior posture dataset, and divide the acquired indoor location dataset into a first indoor location dataset and a second indoor location dataset; wherein the first indoor location dataset includes CSI data of multiple grid location points corresponding to the first room, and the second indoor location dataset includes CSI data of multiple grid location points corresponding to other rooms; the behavior posture dataset includes multiple behavior posture CSI data collected at multiple grid locations corresponding to the first room; Based on the first indoor location data set, a pre-built Resnet-Densenet parallel network is trained, and the trained Resnet-Densenet parallel network and the first location classification result are output; The behavior posture dataset and the second indoor location dataset are respectively input into the trained Resnet-Densenet parallel network, and the behavior posture recognition results and the location classification results of other rooms are output; wherein, before the behavior posture dataset and the second indoor location dataset are input into the Resnet-Densenet parallel network, the parameter weights of the fully connected layer of the Resnet-Densenet parallel network are unfrozen and the parameters of other layers remain unchanged, so as to realize the cross-domain migration of the behavior posture dataset and the second indoor location dataset under the Resnet-Densenet parallel network.

2. The dual-network-based CSI indoor location classification and behavior recognition method according to claim 1 is characterized in that: After obtaining the indoor location data set and the corresponding behavior posture data set, Hampel filtering is used to remove outliers and noise from all data sets. At the same time, sliding window filtering is used to dynamically repeat reading for data enhancement processing to achieve data set expansion.

3. The dual-network-based CSI indoor location classification and behavior recognition method according to claim 1 is characterized in that: The Resnet-Densenet parallel network includes parallel Resnet34 and Densenet121 networks; the first indoor position data set is input into the Resnet-Densenet parallel network for feature extraction, and the contact features are spliced ​​and fused to obtain the first position probability as input data of the Resnet-Densenet parallel network.

4. The dual-network-based CSI indoor location classification and behavior recognition method according to claim 1 is characterized in that: 20% to 30% of the samples in the behavior posture dataset are input into the trained Resnet-Densenet parallel network to realize network training and feature extraction, obtain the probability of a certain posture, and thus output the behavior posture recognition result; wherein, before network training and feature extraction, the parameters of the trained Resnet-Densenet parallel network have been unfrozen in the fully connected layer and the parameters of other layers remain unchanged.

5. The dual-network-based CSI indoor location classification and behavior recognition method according to claim 1 is characterized in that: 20% to 30% of the samples in the second indoor location data are input into the trained Resnet-Densenet parallel network to realize network training and feature extraction, obtain the probability of a certain location, and thus output the location classification results of other rooms; wherein, before network training and feature extraction, the parameters of the trained Resnet-Densenet parallel network have been unfrozen in the fully connected layer and the parameters of other layers remain unchanged.

6. The dual-network-based CSI indoor location classification and behavior recognition method according to claim 1 is characterized in that: The first room is a living room, and the other rooms include a bedroom, a kitchen, a bathroom and a dining room; the location points of the living room and the location points of the other rooms are collected separately according to a grid; the behavior posture data set includes standing, squatting, bending over, stepping on the spot, jumping on the spot and sitting down behavior posture CSI data.

7. The dual-network-based CSI indoor location classification and behavior recognition method according to claim 1, characterized in that: in, CSI data is represented as: In the formula, H i,j represents the CSI of the jth subcarrier of the i-th receiving antenna, N C Indicates the number of subcarriers.

8. A dual-network based CSI indoor location classification and behavior recognition system, used to implement the steps of the dual-network based CSI indoor location classification and behavior recognition method according to any one of claims 1 to 7, characterized in that: include: A data acquisition module is used to obtain an indoor position data set and a behavior posture data set, and divide the obtained indoor position data set into a first indoor position data set and a second indoor position data set; wherein the first indoor position data set includes CSI data of multiple grid position points corresponding to the first room, and the second indoor position data set includes CSI data of multiple grid position points corresponding to other rooms; the behavior posture data set includes multiple behavior posture CSI data collected at multiple grid positions corresponding to the first room; A model training module, used for training a pre-built Resnet-Densenet parallel network based on a first indoor location data set, outputting the trained Resnet-Densenet parallel network and outputting a first location classification result; The target result output module is used to input the behavior posture dataset and the second indoor location dataset into the trained Resnet-Densenet parallel network respectively, output the behavior posture recognition results and output the location classification results of other rooms; wherein, before inputting the behavior posture dataset and the second indoor location dataset into the Resnet-Densenet parallel network, the parameter weights of the fully connected layer of the Resnet-Densenet parallel network are unfrozen and the parameters of other layers remain unchanged, so as to realize the cross-domain migration of the behavior posture dataset and the second indoor location dataset under the Resnet-Densenet parallel network.

9. A device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the dual-network-based CSI indoor location classification and behavior recognition method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it is used to implement the steps of the dual-network-based CSI indoor location classification and behavior recognition method described in any one of claims 1 to 7.

Images