CSI-based open set domain adaptation behavior identification method and system

Abstract

The invention provides an open set domain adaptation behavior recognition method and system based on CSI, and relates to the technical field of behavior recognition, and the method comprises the steps: inputting a to-be-recognized CS I into an open set domain adaptation behavior recognition model, and recognizing and classifying different behavior actions; wherein the open set domain adaptive behavior recognition model adopts a generative adversarial network structure, and a loss function constraint training process is constructed by taking minimization of a domain difference between a source domain and a target domain as a target; centroids of the same category between the source domain and the target domain are continuously aligned, the distance between the sample in the target domain and the centroid of the category of the source domain is adjusted, and the sample in the target domain and the sample of the known category in the source domain are aligned until the training stop condition is met; according to the method, the CS I is extracted from the Wi-Fi signal, the source domain sample and the target domain sample are constructed, the target domain sample is well identified under the open set condition, and the problem of negative migration caused by the fact that the source domain label space is a subset of the target domain label space is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of behavior recognition, and in particular, relates to an open set domain adaptive behavior recognition method and system based on CSI. Background Art

[0002] In recent years, wireless communication technology and Internet technology have developed rapidly, and various artificial intelligence devices have been widely used in daily life. Human behavior recognition has always attracted much attention due to its wide range of application scenarios. Human behavior recognition technology can be applied to danger protection, smart home, daily work, and human-computer interactive somatosensory games. For example, human behavior recognition technology can detect and identify people's behavior and actions, and can give timely attention when people take dangerous actions; by monitoring the movement status of children and the elderly, they can be provided with better health care and rehabilitation training; in the field of smart home, human behavior recognition technology can use virtual behavior to control the switches of curtains and lights, making the home environment more intelligent and convenient.

[0003] There are three main human behavior recognition technologies commonly used at present: one is the method of using specific sensors to perform behavior recognition. This method requires users to carry special sensor devices with them to collect behavior data. However, this method has obvious inconveniences because users must always remember to wear these devices. If users do not cooperate or forget to wear them, passive behavior recognition will not be possible. At the same time, many studies focus on applying radio target detection technology to the field of human behavior recognition, such as continuous wave radar and ultra-wideband technology. However, the high transmission cost and complex operation of radio equipment have hindered its widespread popularization in daily life.

[0004] The second is the technology of behavior recognition through computer vision. This method uses a camera to capture the user's behavior images or videos, and then extracts the user's behavior characteristics through preliminary data processing and algorithm analysis to perform behavior recognition. Although this technology has achieved good recognition results, it has high requirements for hardware equipment, requires high-resolution images or videos, and may be affected by environmental factors such as light. In addition, for home users, there is a risk of privacy leakage because the technology requires direct photography of family members, which may cause privacy issues.

[0005] Third, human behavior recognition technology based on WiFi signals has achieved remarkable results in a single environment. However, in actual applications, the distribution differences of the same behavior in different environments often lead to a decline in the performance of the perception model and impaired recognition accuracy. In addition, retraining the model in a new environment usually requires a large number of re-collected and labeled samples, which is time-consuming and laborious.

[0006] Therefore, how to effectively utilize data resources in different environments to achieve accurate and efficient cross-domain human behavior recognition has become the focus of current research. In the process of cross-domain human behavior recognition, a common challenge is that the label spaces of the two environments are not always consistent; sometimes, the label space of the target domain may be just a subset of the source domain label space, or vice versa; this mismatch in label space brings great difficulty to cross-domain recognition. Therefore, the existing technology has the problem of cross-domain recognition domain differences when the source domain label category is a subset of the target domain label category. Summary of the invention

[0007] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides an open set domain adaptive behavior recognition method and system based on CSI, which extracts CSI information from WiFi signals, constructs source domain samples and target domain samples, and realizes good recognition of target domain samples under open set conditions, thereby solving the negative migration problem caused by the source domain label space being a subset of the target domain label space.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] A first aspect of the present invention provides an open set domain adaptive behavior recognition method based on CSI.

[0010] An open set domain adaptive behavior recognition method based on CSI, comprising:

[0011] Obtain the WiFi signal to be identified and extract the channel state information CSI therein;

[0012] The channel state information CSI is input into the trained open set domain adaptive behavior recognition model to identify and classify different behavior actions;

[0013] Among them, the open set domain adaptation behavior recognition model adopts the structure of a generative adversarial network, with the goal of minimizing the domain difference between the source domain and the target domain, constructs a loss function to constrain the training process, continuously aligns the centroids of the same category between the source domain and the target domain, adjusts the distance between the samples in the target domain and the centroid of the source domain category, and aligns the samples in the target domain with the samples of known categories in the source domain until the conditions for stopping training are met.

[0014] Furthermore, the open set domain adaptation behavior recognition model includes a feature extractor, a semantic category alignment module, a semantic contrast mapping module, and an adversarial domain adaptation module;

[0015] The feature extractor adopts a CNN network to extract features from the channel state information CSI;

[0016] The semantic category alignment module is used to align the centroids of the same category between the source domain and the target domain;

[0017] The semantic contrast mapping module is used to adjust the distance between the samples in the target domain and the centroid of the source domain categories;

[0018] The adversarial domain adaptation module is used to align samples in the target domain with samples of known categories in the source domain.

[0019] Furthermore, the loss function includes a semantic category alignment loss, a semantic contrast mapping loss, and an adversarial domain adaptation loss, which correspond to a semantic category alignment module, a semantic contrast mapping module, and an adversarial domain adaptation module, respectively;

[0020] The semantic category alignment loss includes contrastive center loss and cross-domain based center loss, which are used to compress source domain samples belonging to the same category and minimize the distance between the centroids of the same category between the source domain and the target domain respectively;

[0021] The semantic contrast mapping loss is used to minimize the distance between the known category samples in the target domain and the corresponding source domain category centroids, and maximize the distance between the unknown category samples in the target domain and the centroids of all known categories in the source domain;

[0022] The adversarial domain adaptation loss includes a binary cross entropy loss and a classifier loss, which are respectively used to strengthen the similarity between samples of the same category and to enhance the distinction between samples of different categories.

[0023] Furthermore, the adversarial domain adaptation module includes a domain discriminator, a classifier, and a gradient reversal layer;

[0024] The domain discriminator is used to distinguish whether the sample is from the source domain or the target domain;

[0025] The classifier is used to classify the behavior according to the features extracted by the feature extractor;

[0026] The gradient reversal layer is used to flip the sign of the gradient in the reverse process.

[0027] Furthermore, the classifier uses a spherical logistic regression layer to replace the fully connected layer in the traditional classifier, and regards the feature as a sphere, where a circle on the sphere corresponds to a hyperplane in the feature space.

[0028] Furthermore, it also includes introducing the idea of ​​knowledge graph in distance calculation. When using the cosine similarity between samples to measure the distance, if two samples belong to the same category, their cosine values ​​are increased to enhance the similarity; otherwise, the cosine values ​​are reduced to enhance the distinction.

[0029] Furthermore, the classifier loss includes knowledge graph loss and entropy separation loss;

[0030] The knowledge graph loss is used to increase the distance between different categories and reduce the distance within the same category;

[0031] The entropy separation loss uses the entropy of the classifier output to expand the boundary between known and unknown categories.

[0032] A second aspect of the present invention provides an open set domain adaptive behavior recognition system based on CSI.

[0033] An open set domain adaptive behavior recognition system based on CSI includes an acquisition module and a recognition module:

[0034] The acquisition module is configured to: acquire the WiFi signal to be identified and extract the channel state information CSI therein;

[0035] The recognition module is configured to: input the channel state information CSI into the trained open set domain adaptive behavior recognition model to recognize and classify different behavior actions;

[0036] Among them, the open set domain adaptation behavior recognition model adopts the structure of a generative adversarial network, with the goal of minimizing the domain difference between the source domain and the target domain, constructs a loss function to constrain the training process, continuously aligns the centroids of the same category between the source domain and the target domain, adjusts the distance between the samples in the target domain and the centroid of the source domain category, and aligns the samples in the target domain with the samples of known categories in the source domain until the conditions for stopping training are met.

[0037] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in a CSI-based open set domain adaptive behavior recognition method as described in the first aspect of the present invention.

[0038] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the CSI-based open set domain adaptive behavior recognition method as described in the first aspect of the present invention are implemented.

[0039] One or more of the above technical solutions have the following beneficial effects:

[0040] The CSI open set domain adaptation behavior recognition model based on semantic information proposed in the present invention first uses adversarial domain adaptation to achieve effective alignment between the target domain and the source domain, and then divides the category boundaries according to the semantic information in the CSI data through the semantic category comparison module and the semantic comparison mapping module to distinguish known classes from unknown classes.

[0041] In addition, SeOSDA also introduces entropy separation loss and knowledge graph module to further enhance the discrimination effect.

[0042] Finally, the spherical logistic regression layer is used instead of the fully connected layer for classification, which improves the classification accuracy of the model.

[0043] The SeOSDA model shows excellent classification effect in open set domain adaptive behavior recognition based on CSI, and can well solve the negative transfer problem caused by the source domain label space being a subset of the target domain label space.

[0044] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0046] Figure 1 It is a plan view of the experimental environment of the first embodiment;

[0047] Figure 2 It is a schematic diagram of a data acquisition device of the first embodiment;

[0048] Figure 3 This is the overall structure diagram of the SeOSDA model of the first embodiment;

[0049] Figure 4 This is a schematic diagram of the SCA module effect of the first embodiment;

[0050] Figure 5 The schematic diagram of the SCM module of the first embodiment;

[0051] Figure 6 This is a schematic diagram of the behavior sample knowledge graph of the first embodiment. DETAILED DESCRIPTION

[0052] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0053] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0054] Embodiment 1

[0055] In one embodiment of the present disclosure, a method for identifying an open set domain adaptive behavior based on CSI is provided, comprising the following steps:

[0056] Obtain the WiFi signal to be identified and extract the channel state information CSI therein;

[0057] Input the channel state information CSI into the trained open set domain adaptive behavior recognition model to identify and classify different behavior actions;

[0058] Among them, the open set domain adaptation behavior recognition model adopts the structure of a generative adversarial network, with the goal of minimizing the domain difference between the source domain and the target domain, constructs a loss function to constrain the training process, continuously aligns the centroids of the same category between the source domain and the target domain, adjusts the distance between the samples in the target domain and the centroid of the source domain category, and aligns the samples in the target domain with the samples of known categories in the source domain until the conditions for stopping training are met.

[0059] Furthermore, the open set domain adaptation behavior recognition model includes a feature extractor, a semantic category alignment module, a semantic contrast mapping module, and an adversarial domain adaptation module;

[0060] The feature extractor adopts a CNN network to extract features from the channel state information CSI;

[0061] The semantic category alignment module is used to align the centroids of the same category between the source domain and the target domain;

[0062] The semantic contrast mapping module is used to adjust the distance between the samples in the target domain and the centroid of the source domain categories;

[0063] The adversarial domain adaptation module is used to align samples in the target domain with samples of known categories in the source domain.

[0064] Furthermore, the loss function includes a semantic category alignment loss, a semantic contrast mapping loss, and an adversarial domain adaptation loss, which correspond to a semantic category alignment module, a semantic contrast mapping module, and an adversarial domain adaptation module, respectively;

[0065] The semantic category alignment loss includes contrastive center loss and cross-domain based center loss, which are used to compress source domain samples belonging to the same category and minimize the distance between the centroids of the same category between the source domain and the target domain respectively;

[0066] The semantic contrast mapping loss is used to minimize the distance between the known category samples in the target domain and the corresponding source domain category centroids, and maximize the distance between the unknown category samples in the target domain and the centroids of all known categories in the source domain;

[0067] The adversarial domain adaptation loss includes a binary cross entropy loss and a classifier loss, which are respectively used to strengthen the similarity between samples of the same category and to enhance the distinction between samples of different categories.

[0068] Furthermore, the adversarial domain adaptation module includes a domain discriminator, a classifier, and a gradient reversal layer;

[0069] The domain discriminator is used to distinguish whether the sample is from the source domain or the target domain;

[0070] The classifier is used to classify the behavior according to the features extracted by the feature extractor;

[0071] The gradient reversal layer is used to flip the sign of the gradient in the reverse process.

[0072] Furthermore, the classifier uses a spherical logistic regression layer to replace the fully connected layer in the traditional classifier, and regards the feature as a sphere, where a circle on the sphere corresponds to a hyperplane in the feature space.

[0073] Furthermore, it also includes introducing the idea of ​​knowledge graph in distance calculation. When using the cosine similarity between samples to measure the distance, if two samples belong to the same category, their cosine values ​​are increased to enhance the similarity; otherwise, the cosine values ​​are reduced to enhance the distinction.

[0074] Furthermore, the classifier loss includes knowledge graph loss and entropy separation loss;

[0075] The knowledge graph loss is used to increase the distance between different categories and reduce the distance within the same category;

[0076] The entropy separation loss uses the entropy of the classifier output to expand the boundary between known and unknown categories.

[0077] The following describes in detail the implementation process of an open set domain adaptive behavior recognition method based on CSI in this embodiment.

[0078] This embodiment selects three indoor environments with large differences to collect behavioral action data and obtains a data set; constructs an open set domain adaptive behavior recognition model SeOSDA based on semantic information; uses the data set to train and test the model, solves the cross-domain recognition domain difference problem when the source domain label category is a subset of the target domain label category, effectively utilizes data resources in different environments, and realizes accurate and efficient cross-domain human behavior recognition. The specific steps are:

[0079] S1. Experimental environment deployment

[0080] Three indoor environments with large differences were selected for the experiment, namely an activity room (environment A), a conference room (environment B) and a large factory building (environment C); a data acquisition device was deployed in each of the above three environments, and the device included a wireless signal transmitter and a wireless signal receiver. Both the transmitter and the receiver were computers equipped with an Intel 5300 network card.

[0081] Specifically, three experimental environments with great differences are as follows Figure 1 As shown, the area of ​​the A environment activity room is 76.5m 2 In this room, there is a 3D printer and several small machines, as well as some sundries. During the data collection process, staff will enter and exit this room from time to time.

[0082] The area of ​​the B environment meeting room is 86.4m 2 ,There are some obstacles such as wooden tables and chairs placed in the room. ,Compared with environment A, people came and left more frequently in environment B during the ,data collection period.

[0083] C environment large workshop area is 512m 2 In this spacious factory building, there are three glass rooms and many large iron mechanical instruments. Compared with the previous two environments, there are fewer people coming in and out of environment C during data collection.

[0084] Data acquisition devices such as Figure 2 As shown, in this embodiment, a human behavior recognition platform based on the widely commercialized Intel5300 wireless device is designed and implemented, aiming to collect the data required for the experiment and verify the performance of the designed model. The platform consists of two parts: software and hardware.

[0085] In terms of hardware, this embodiment uses two computers as core devices, and these computers are installed with the Linux system of Ubuntu 12.04 LTS version. In order to improve the quality of signal transmission, each computer is connected to three gain antennas, which can effectively enhance the strength and stability of wireless signals. Among them, one computer is set as a transmitter, responsible for sending wireless signals, and the other computer is used as a receiver, responsible for receiving and processing these signals. Both computers are installed with Intel 5300 wireless network cards to ensure smooth wireless communication.

[0086] In terms of software, this embodiment constructs two main interfaces: a data acquisition interface and an online real-time recognition interface; the data acquisition interface is not only responsible for data collection, but also can display the channel waveform transformation diagram in real time, so that the fluctuation of the CSI signal during the acquisition process can be visualized; and the online real-time recognition interface carries another important function, which uses the trained model to recognize human behavior in real time and display the recognition results on the host computer immediately.

[0087] S2. Experimental data collection

[0088] Organize m experimenters to stand between the transmitter and the receiver respectively, and perform the behavioral actions required by the experiment. Each action is collected n times continuously to collect experimental data.

[0089] Specifically, in this embodiment, a WiFi signal transmitter (Tx) and a WiFi signal receiver (Rx) are set in each environment, and the transmitter and receiver are each connected to three gain antennas. The height of the external gain antenna is 1.2m, and the distance between the signal transmitter (Tx) and the signal receiver (Rx) is 3.5m. The CSI Tool toolkit is used to collect data. The number of subcarriers of the CSI signal is 30. The acquired CSI signal contains 30 channel features, so a CSI data of size 3×3×30 can be obtained in each sampling process.

[0090] Table 1 Data volume in different environments

[0091]

[0092] As shown in Table 1, this embodiment sets the sampling frequency of data collection to 500 Hz to ensure that each sample contains 2000 sampling points; data of seven representative actions are collected in three different environments, including "empty" (no action state), "jump", "bend", "run", "sit", "walk" and "wave". At the same time, this embodiment selects 7 people with different heights, weights, genders and body shapes in each of the three environments. Each participant completes the collection of the above 7 behaviors in the three environments, and each behavior is repeated 101 times; therefore, the total number of samples collected in each environment is 4949.

[0093] S3, Dataset Division

[0094] The data set in S2 is divided into two parts according to a certain ratio, one part is used as a training set to train the model, and the other part is used as a test set to test the optimization degree of the model.

[0095] Specifically, the data collected by each person in different environments are divided into training set and test set according to 7:3.

[0096] S4. Construct an open set domain adaptive behavior recognition model SeOSDA based on semantic information and perform training and testing.

[0097] This embodiment designs a SeOSDA model based on a generative adversarial network. Its overall structure is as follows: Figure 3 As shown in the figure, the main modules include feature extractor G, semantic category alignment module (Semantic Categorical Alignment, SCA), semantic contrast mapping module (Semantic Contrastive Mapping, SCM) and adversarial domain adaptation module (Adversarial Domain Adaptation, ADA); all modules are trained simultaneously and work together, combining the semantic information of CSI data and complex feature representation to better classify each known behavior category and unknown behavior category. The following is an explanation of each model:

[0098] 1. Build a feature extractor;

[0099] The feature extractor G, which is equivalent to the generator in the generative adversarial network, adopts the CNN network. By inputting the CSI signal into the feature extractor, key feature information can be extracted to provide a basis for subsequent processing.

[0100] 2. Construct the semantic category alignment module (SCA);

[0101] The purpose of the semantic category alignment module is to make each known class more centralized and the alignment between the source and target domains more accurate, aiming to compress the representation of known classes and distinguish each class from other classes in the known categories.

[0102] The SCA module consists of two parts:

[0103] First, the contrastive center loss is used to enhance the distinguishability of the features generated by source domain samples, so that the same categories in the source domain are as close as possible and different categories are more separable.

[0104] Secondly, based on the cross-domain center loss, each centroid of the known category in the target domain is aligned with the corresponding centroid of the same category in the source domain, trying to narrow the domain difference of the same category in the source domain and the target domain and match their distributions.

[0105] In this way, different categories in the source domain samples will be easier to distinguish, and the centroids of known categories in the target domain will be more accurately aligned with the centroids of corresponding source domain categories.

[0106] SCA module effects Figure 4 As shown, and are two known category samples in the source domain, and are two known category samples in the target domain that are the same as the source domain categories, and are two unknown category samples in the target domain. After the SCA module, the two known category samples in the source domain are as close as possible to the same category, and the different categories are more separable, and the known category samples in the target domain are close to the same category samples in the source domain.

[0107] Specifically, in order to compress source domain samples belonging to the same class in the feature space, this module applies the following contrast center loss to the source domain samples, as shown in formula (1):

[0108]

[0109] Where m represents the number of samples in a small batch during training, represents the i-th training sample from the source domain, Representation Class is the centroid in the source domain, and δ is a constant used to prevent the denominator from being zero.

[0110] To align two centroids of known classes between the source and target domains, this module attempts to minimize the distance between the centroid pairs. in and denote the centroid of category k from the source and target domains, respectively.

[0111] Due to the randomness and bias in each mini-batch, this module aligns the global centroid instead of the local centroid; however, it is not easy to obtain the global centroid directly; this embodiment partially updates them with the local centroid at each iteration. Specifically, the initial global centroid of the source domain is first calculated based on the prediction results of the pre-trained model, as shown below:

[0112]

[0113] Among them, n k represents the number of samples with label k, Indicates the calculation of the i-th sample with label k The local centroid of . The pre-trained model is trained on the supervised source domain.

[0114] For the target sample, this embodiment uses the prediction result as the pseudo label, and in each iteration, a set of local centroids is calculated using a small batch of samples Where I represents the number of iterations, and the local centroid is calculated as the average of all samples in each iteration; then the source domain centroid is updated by reweighting through equations (3) and (4) and the target domain centroid

[0115]

[0116] Here, ρ(·,·) is defined as

[0117] Finally, the classification center alignment loss is formulated as shown in formula (5):

[0118]

[0119] 3. Construct a semantic contrastive mapping module (SCM);

[0120] The function of the previous SCA module is to align the centroids of the same category between the source domain and the target domain. For non-centroid samples in the target domain, i.e., samples of unknown categories, this embodiment uses a contrast loss function to encourage known samples to move toward their centroids and force unknown samples to stay away from all centroids of known classes; in this way, non-centroid samples in the target domain can be aligned, and this embodiment calls this process semantic contrast mapping (SCM).

[0121] The SCM module effect is as follows Figure 5 As shown, and are two known category samples in the source domain, and are two known category samples in the target domain that are the same as the source domain categories, and are two unknown category samples in the target domain, is the centroid of the known category. After passing through the SCM module, the two known category samples move toward their centroids respectively, and the unknown category samples in the target domain move away from the centroids of the two known category samples.

[0122] Since the pseudo labels of the target domain samples are not completely correct, this embodiment selects reliable samples whose classification probability exceeds the threshold. In the method of this embodiment, this module sets the threshold to 1 / N+1. The purpose of the SCM module is to reduce the distance between the known category samples and their centroids, and at the same time expand the distance between the unknown category samples and the known category samples. The loss function is:

[0123]

[0124] Among them, if x t is predicted to be a known N category, then z = 0, if x t If it is predicted as an unknown category, z = 1. kIt is represented as the distance between the known category samples in the target domain and the corresponding source domain category centroid; D u Represents the distance between samples of unknown categories in the target domain and all known categories in the source domain.

[0125] D k , D u Its function settings are as follows:

[0126]

[0127] Wherein, ρ represents the cosine similarity; in order to ensure effective and accurate distance measurement, this embodiment also uses a hyperparameter ω to reweight the distance calculated in the loss; M k is the classification adaptive range of the neighborhood radius of measurement category k, M k The definition is shown in formula (9):

[0128]

[0129] 4. Construct the Adversarial Domain Adaptation (ADA) module;

[0130] The main goal of the adversarial domain adaptation module is to preliminarily align samples in the target domain with known category samples in the source domain, or to classify these samples into unknown categories; the adversarial domain adaptation module includes a domain discriminator, a classifier based on spherical logistic regression, and a gradient reversal layer (GRL); the domain discriminator is equivalent to the discriminator in the generative adversarial network, which aims to effectively distinguish whether the sample is from the source domain or the target domain; the classifier based on spherical logistic regression is used to classify behavioral actions according to the features extracted by the feature extractor; the gradient reversal layer is used to flip the sign of the gradient in the reverse process.

[0131] Specifically, the domain discriminator D in the adversarial domain adaptation module is trained to effectively distinguish whether the sample is from the source domain or the target domain; the feature extractor G attempts to minimize the domain difference between the source domain and the target domain samples, and the classifier C is used to classify the extracted behavioral features. When trying to establish boundaries for unknown samples, this embodiment uses binary cross entropy loss. The adversarial loss is shown in formula (10):

[0132]

[0133] Among them, x t represents the input sample, y=N+1 represents the model prediction sample x t is correct.

[0134] Therefore, the loss function L of the adversarial domain adaptation module ADA It can be expressed as:

[0135]

[0136] Among them, L cls is the classifier loss. The following describes how to construct the classifier loss.

[0137] In the adversarial domain adaptation module, this embodiment uses a spherical logistic regression layer (SLR) to replace the fully connected layer in the traditional classifier for classification, which can improve the feature representation ability, generalization ability and robustness of the model, making it more suitable for processing and classification tasks of complex data sets; this layer regards the feature as a sphere, and the spherical surface A circle on the feature space R n A hyperplane in . The circle can be represented by w T z+b=0, where w is the unit normal vector and b is the bias in [-r, r]. Similar to Euclidean logistic regression, the SLR layer is defined as:

[0138]

[0139] Among them, w k ∈R n ,||w k ||=1,b k ∈[-r,r]; w T z+b=0 The classification circle boundary on b k The constraint ∈[-r,r] can be modeled by k = rtanh(b' k ) to enforce, where b' k ∈R are the parameters to be learned.

[0140] This embodiment incorporates the idea of ​​knowledge graph into it. By calculating the cosine similarity between samples, the similarity between samples of the same category is further strengthened, and the distinction between samples of different categories is enhanced, thereby effectively further expanding the boundaries between samples of each category. When calculating the cosine similarity, this embodiment adopts a strategy: if two samples belong to the same category, their cosine values ​​are increased to enhance the similarity; otherwise, the cosine value is reduced to enhance the distinction.

[0141] Knowledge graphs such as Figure 6 As shown, samples 1 and 2 are from category 1, and samples 3 and 4 are from category 2. Before optimization, all objects are connected to each other. After optimization, the relations from different categories, such as d 13 ,d 14 ,d 23 ,d 24are removed, and relations from the same category (e.g. d 12 d 34 ) is retained, and its cosine distance is close to 1.

[0142] This process can force the feature extractor G to extract the distinguishing features of each category. The features of the same category are more similar, and the features of different categories are more different, which can help the classifier C to identify each sample more easily; therefore, the knowledge graph can assist the training of CNN and improve the feature extraction capability.

[0143] In this embodiment, the intra-class distance is defined as d mn , m,n∈N(I), where N(I) is the sample set of class I, m,n are samples from the same class I, and the inter-class distance is defined as d ij , i∈N(I), j∈N(J), where i and j are samples from class I and class J respectively; in order to maximize the distance between different categories and minimize the distance within the same category, the loss guided by the knowledge graph can be calculated, that is, the knowledge graph loss:

[0144]

[0145] Where K is the total number of categories, m∈N(I),n∈N(J) means that m and n belong to different categories, and i,j∈N(I) means that i and j are from the same category. According to the definition of inter-class distance, formula (13) can be written as formula (14):

[0146]

[0147] Among them, x' m and x' n are samples from I and J respectively, x i 'and x' j are samples from the same category I.

[0148] In addition, in the classifier, this embodiment adds an entropy separation loss (ES) to use the entropy of the classifier output to expand the boundary between known categories and unknown categories.

[0149] The distance ρ between the entropy and the threshold boundary is defined as |H(p)-ρ|, where p is the output of the target domain sample classification; this embodiment defines ρ=log(K s ) / 2,K s Represents the total number of categories in the source domain; at the same time, the confidence threshold parameter m is introduced, so that the final form of the entropy separation loss is shown in formula (15):

[0150]

[0151] Therefore, L in formula (11)cls Should be:

[0152] L cls =γL K +ζL es (16)

[0153] Among them, L K is the knowledge graph loss, L es is the entropy separation loss. γ and ζ are the weights of these constraint losses respectively.

[0154] 5. SeOSDA model training and experimental results analysis;

[0155] The overall constraint loss formula of the SeOSDA model is as follows:

[0156]

[0157] Among them, L ADA , L SCA , L SCM They are adversarial domain adaptation loss, semantic category alignment loss, and semantic contrast mapping loss, respectively. α, β, φ, λ, and ψ are the weights of each loss, respectively. The SeOSDA model is trained together under different constraints to enable it to achieve the effect of open set domain adaptation model recognition.

[0158] The training process is:

[0159] First, the model is pre-trained to update the model parameters. Second, in the adversarial learning stage, the source domain and target domain samples are preliminarily aligned through the ADA module to calculate the global centroid of the samples. After that, the global centroid of the samples is updated through the SCA module, and the distance between the source domain centroid and the target domain centroid is calculated to select target domain samples of known categories. At the same time, the distance between the known category samples and the target domain sample centroid is calculated through the SCM module. Finally, the model is trained by optimizing the loss function of formula (17) to update the network parameters and achieve good recognition of the target domain samples by the SeOSDA model under open set conditions.

[0160] In the experiment of this embodiment, a total of 6 cross-domain recognitions were performed in the three environments A, B, and C to verify the effectiveness of the model; and compared with other algorithms in the same environment, namely: CNN, DANN, CDAN, OSBP, and STA.

[0161] Table 2 Recognition accuracy of the model for all categories in the target domain

[0162]

[0163]

[0164] As can be seen from Table 2, the SeOSDA model significantly outperforms other algorithms in all six cross-domain experiments, and the average recognition accuracy of all categories in the target domain reaches 76.0%.

[0165] Table 3 Recognition accuracy of the model for only known categories in the target domain

[0166]

[0167] As can be seen from Table 3, except for the case where domain B is the source domain and domain A is the target domain, the other five experiments are also better than other algorithms, and the average recognition accuracy of only known categories in the target domain reaches 75.9%. Among them, due to its poor cross-domain recognition ability, the CNN algorithm has an average recognition accuracy of only 30.7% for all categories and only 41.0% for known categories. As traditional DA algorithms, the DANN algorithm and the CDAN algorithm have improved their recognition accuracy for all categories and only known categories, but they have not reached the recognition standard of the open set domain adaptation algorithm. As open set domain adaptation algorithms, the OSDP algorithm and the STA algorithm have an average recognition accuracy of 69.7% and 56.4% for all categories, and an average recognition accuracy of 61.3% and 64.5% for only known categories. They are better than traditional DA algorithms and each has its own advantages and disadvantages, but they are all lower than the SeOSDA model proposed in this project.

[0168] In summary, the SeOSDA algorithm proposed in this embodiment can effectively perform open set domain adaptive behavior recognition and is advanced.

[0169] Embodiment 2

[0170] In one embodiment of the present disclosure, a CSI-based open set domain adaptive behavior recognition system is provided, including an acquisition module and a recognition module:

[0171] The acquisition module is configured to: acquire the WiFi signal to be identified and extract the channel state information CSI therein;

[0172] The recognition module is configured to: input the channel state information CSI into the trained open set domain adaptive behavior recognition model to recognize and classify different behavior actions;

[0173] Among them, the open set domain adaptation behavior recognition model adopts the structure of a generative adversarial network, with the goal of minimizing the domain difference between the source domain and the target domain, constructs a loss function to constrain the training process, continuously aligns the centroids of the same category between the source domain and the target domain, adjusts the distance between the samples in the target domain and the centroid of the source domain category, and aligns the samples in the target domain with the samples of known categories in the source domain until the conditions for stopping training are met.

[0174] Embodiment 3

[0175] The purpose of this embodiment is to provide a computer-readable storage medium.

[0176] A computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in a CSI-based open set domain adaptive behavior recognition method as described in Embodiment 1 of the present disclosure.

[0177] Embodiment 4

[0178] The purpose of this embodiment is to provide an electronic device.

[0179] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the open set domain adaptive behavior recognition method based on CSI as described in the first embodiment of the present disclosure are implemented.

[0180] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A CSI-based open set domain adaptive behavior recognition method, characterized in that: include: Obtain the WiFi signal to be identified and extract the channel state information CSI therein; The channel state information CS I is input into the trained open set domain adaptive behavior recognition model to identify and classify different behavior actions; Among them, the open set domain adaptation behavior recognition model adopts the structure of a generative adversarial network, with the goal of minimizing the domain difference between the source domain and the target domain, constructs a loss function to constrain the training process, continuously aligns the centroids of the same category between the source domain and the target domain, adjusts the distance between the samples in the target domain and the centroid of the source domain category, and aligns the samples in the target domain with the samples of known categories in the source domain until the conditions for stopping training are met.

2. The open set domain adaptive behavior recognition method based on CS I as claimed in claim 1, characterized in that: The open set domain adaptation behavior recognition model includes a feature extractor, a semantic category alignment module, a semantic contrast mapping module and an adversarial domain adaptation module; The feature extractor adopts a CNN network to extract features from the channel state information CS I; The semantic category alignment module is used to align the centroids of the same category between the source domain and the target domain; The semantic contrast mapping module is used to adjust the distance between the samples in the target domain and the centroid of the source domain categories; The adversarial domain adaptation module is used to align samples in the target domain with samples of known categories in the source domain.

3. The open set domain adaptive behavior recognition method based on CS I as claimed in claim 2, characterized in that: The loss function includes a semantic category alignment loss, a semantic contrast mapping loss, and an adversarial domain adaptation loss, which correspond to a semantic category alignment module, a semantic contrast mapping module, and an adversarial domain adaptation module, respectively; The semantic category alignment loss includes contrastive center loss and cross-domain based center loss, which are used to compress source domain samples belonging to the same category and minimize the distance between the centroids of the same category between the source domain and the target domain respectively; The semantic contrast mapping loss is used to minimize the distance between samples of known categories in the target domain and the corresponding centroids of source domain categories, while maximizing the distance between samples of unknown categories in the target domain and the centroids of all known categories in the source domain; The adversarial domain adaptation loss includes a binary cross entropy loss and a classifier loss, which are respectively used to strengthen the similarity between samples of the same category and to enhance the distinction between samples of different categories.

4. The open set domain adaptive behavior recognition method based on CS I as claimed in claim 3, characterized in that: The adversarial domain adaptation module includes a domain discriminator, a classifier, and a gradient reversal layer; The domain discriminator is used to distinguish whether the sample is from the source domain or the target domain; The classifier is used to classify the behavior according to the features extracted by the feature extractor; The gradient reversal layer is used to flip the sign of the gradient in the reverse process.

5. The open set domain adaptive behavior recognition method based on CS I as claimed in claim 4, characterized in that: The classifier uses a spherical logistic regression layer to replace the fully connected layer in the traditional classifier, and regards the feature as a sphere. A circle on the sphere corresponds to a hyperplane in the feature space.

6. The open set domain adaptive behavior recognition method based on CS I as claimed in claim 3, characterized in that: It also includes the idea of ​​introducing knowledge graphs in distance calculation. When using cosine similarity between samples to measure distance, if two samples belong to the same category, their cosine values ​​are increased to enhance similarity. On the contrary, the cosine value is reduced to enhance the discrimination.

7. The open set domain adaptive behavior recognition method based on CS I as claimed in claim 3, characterized in that: The classifier loss includes knowledge graph loss and entropy separation loss; The knowledge graph loss is used to increase the distance between different categories and reduce the distance within the same category; The entropy separation loss uses the entropy of the classifier output to expand the boundary between known and unknown categories.

8. An open domain adaptive behavior recognition system based on CS I, characterized in that: Including acquisition module and identification module: The acquisition module is configured to: acquire the WiFi signal to be identified and extract the channel state information CS I therein; The recognition module is configured to: input the channel state information CS I into the trained open set domain adaptive behavior recognition model to recognize and classify different behavior actions; Among them, the open set domain adaptation behavior recognition model adopts the structure of a generative adversarial network, with the goal of minimizing the domain difference between the source domain and the target domain, constructs a loss function to constrain the training process, continuously aligns the centroids of the same category between the source domain and the target domain, adjusts the distance between the samples in the target domain and the centroid of the source domain category, and aligns the samples in the target domain with the samples of known categories in the source domain until the conditions for stopping training are met.

9. An electronic device, comprising: a memory for non-transitory storage of computer readable instructions; a processor for executing the computer-readable instructions; Wherein, when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is executed.

10. A storage medium, characterized in that: The computer-readable instructions are non-transitorily stored, wherein when the computer-readable instructions are executed by a computer, the method of any one of claims 1 to 7 is performed.

Images