Action perception model construction system and method based on physical layer semantics, server

Abstract

The application discloses a kind of action perception model construction system and method based on physical layer semantics, server belongs to pose perception model construction technical field, the construction method includes: using the method of aggregation training realizes the model collaborative training in large-scale distributed wireless perception system.In aggregation, each local node is customized personalized aggregation model, different local nodes downloaded aggregation model is not necessarily same.The greater the model difference, the worse the effect of collaboration between models (i.e.model aggregation), the slower the model convergence speed.Therefore, based on the model difference generates personalized aggregation model, on the one hand, it can increase the similarity between models, help local models to obtain more help from other nodes, speed up the convergence speed of model training, on the other hand, by setting personalized aggregation model, it can better adapt to the environment, and has higher prediction accuracy in the environment.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of gesture perception model construction, and more particularly relates to a motion perception model construction system and method based on physical layer semantics and a server. BACKGROUND

[0002] In recent years, wireless perception technology and systems based on various wireless radio frequency signals (such as Wi-Fi, Bluetooth, RFID) have been widely studied and used to support a variety of emerging applications including smart medical care, smart home, tactile Internet, and augmented reality / virtual reality (AR / VR). A large number of research results show that by monitoring the changes in wireless signal propagation and reflection patterns caused by human motion in the surrounding environment, we can detect and identify a variety of human actions and human poses, such as walking, falling, sitting, etc. under non-contact, non-line-of-sight, and weak / no light conditions.

[0003] Existing related research work mostly uses wireless samples collected from a limited number of receivers to train a global model, and then directly deploys the trained global model in a wider environment to deal with wireless perception tasks in large-scale scenarios. However, wireless signals and their statistical characteristics exhibit high spatiotemporal heterogeneity, i.e. factors such as the deployment location of the receiver / transmitter, the surrounding environment, the user's orientation, and the user's motion habits will all cause the sample features and data distribution of the wireless datasets collected by different receivers to be not completely consistent. This will result in slower convergence speed when training a global model based on a limited dataset, and the prediction accuracy of the model is not high. SUMMARY

[0004] In view of the above defects or improvement needs of the prior art, the present application provides a motion perception model construction system and method based on physical layer semantics and a server, which aims to improve the convergence speed of the perception model training and the prediction accuracy of the model.

[0005] To achieve the above-mentioned purpose, according to one aspect of the present application, a motion perception model construction method based on physical layer semantics is provided, comprising:

[0006] Step S1: the local node k' acquires labeled local data, and the labeled local data is the physical layer semantic feature of the radio frequency signal with action label; k' = 1, 2, 3, …, K', K' is the number of local nodes participating in aggregated training;

[0007] Step S2: the local node k' downloads the aggregated model corresponding to the current t-th iteration from the central server and performs local training with the labeled local data to obtain a local model and then uploads the local model Uploaded to the central server; wherein, local training uses physical layer semantic features as input and corresponding actions as output for model training;

[0008] Step S3: After receiving K′ local models, the central server determines whether the iteration termination condition has been met. If not, it calculates the aggregate model of each local node k′ in the (t+1)th iteration. Let t = t + 1, and jump to step S2; if so, end the iteration, and each local node uses its current local model as its final local model; wherein, the iteration ends when the maximum number of iterations is exceeded or the global loss is lower than a preset value; aggregate model For local models Subtract the coordination factor, which is positively correlated with the local model. The degree of difference between the model and other local models.

[0009] In one embodiment, step S1 includes:

[0010] Step S11: Acquire the radio frequency signal under a certain action. The radio frequency signal is the channel state information (CSI) received by the receiver after the transmitter's transmitted signal is reflected by an obstacle. The obstacle includes static environmental obstacles and the target of the operation performing the certain action.

[0011] Step S12: Extract the radio frequency signal features of the Channel State Information (CSI) to obtain the corresponding physical layer semantic features φ k′ The physical layer semantic feature φ k′ Including environment-related semantic features u k′ and action-related semantic features υ k′ .

[0012] In one embodiment, the environment-related semantic feature u k′ Including the environment-related channel attenuation factor A n Propagation delay τ n The signal arrives at the incident angle ψ n Semantic features related to actions υ k′ Including the action-related channel attenuation factor A m Propagation delay τ m The signal arrives at the incident angle ψ m Doppler frequency shift ρ m .

[0013] In one embodiment, the aggregation model The calculation formula is:

[0014]

[0015] In the formula, denotes the other local nodes in K' except for the local node k', ||·|| is the Euclidean norm, R(x) is a nonlinear attention inducing function with x as the argument, is the gradient of R(x), is an increasing function, and β is an adjustment coefficient obtained by experimental adjustment.

[0016] In one embodiment, the form of the R(x) function is:

[0017]

[0018] where σ is an experimentally determined parameter of the same order of magnitude as x.

[0019] In one embodiment, the method further comprises: determining the local model of the second type of local node set K" according to the data of the first type of local node set K', the first type of local node set K' being the local nodes participating in the aggregated training, and the second type of local node set K" being the local nodes whose local data sets are not labeled and do not participate in the aggregated training; specifically comprising:

[0020] converting any physical layer semantic feature φ k of the local node k ∈ K into an embedding vector K = K' ∪ K";

[0021] calculating the semantic distance between the embedding vector of the local node i' ∈ K' and the embedding vector of the local node j' ∈ K'; and calculating the semantic distance between the embedding vector of the local node k" ∈ K" and the embedding vector of each local node k' ∈ K';

[0022] calculating the model similarity M(ω i′ ,ω j′ ) between the local model ω i′ of the local node i' and the local model ω j′ of the local node j';

[0023] training the deep feedforward network G to fit the corresponding mapping function with the semantic distance as input and the model similarity M(ω i′ ,ω j′ ) as output;

[0024] inputting the semantic distance into the trained deep feedforward network G to obtain the model similarity M(ω i′ ,ω j′ ) between the local node k" and each local node k'.k′ ,ω k″ ),ω k′ ,ω k″ is a local model of the local node k' and the local node k'', respectively;

[0025] The local models ω k′ are weightedly summed to obtain a local model ω k″ of the local node k'', and the weighted weight of the local model ω k′ is proportional to the model similarity M(ω k′ ,ω k″ ).

[0026] In one of the embodiments, the semantic distance and the model similarity are respectively calculated by the cosine distance.

[0027] According to another aspect of the present application, a central server participating in the action perception model construction method is provided, comprising:

[0028] A receiving module is configured to receive the local model ω k' = 1, 2, 3, …, K', K' being the number of local nodes participating in the aggregated training;

[0029] A determining module is configured to determine whether an iteration end condition is reached after receiving the K' local models, if not, triggering the iteration module to work, and if yes, ending the iteration;

[0030] An iteration module is configured to calculate the aggregated model ω of the local node k' in the t+1 iteration round after being triggered by the determining module, and deliver it to the corresponding local node k'; wherein the iteration end condition is that the maximum iteration round is exceeded or the global loss is lower than a preset value; the aggregated model ω is the local model ω minus a coordination amount, and the coordination amount is positively related to the difference degree between the local model ω and other local models.

[0031] In one of the embodiments, comprising:

[0032] A conversion module is configured to convert the arbitrary physical layer semantic feature φ k of the local node k ∈ K into an embedding vector K = K' ∪ K'';

[0033] A semantic distance calculation module is configured to calculate the semantic distance d between the embedding vector φ of the local node i' ∈ K' and the embedding vector φ of the local node j' ∈ K'.and the embedding vector of the local node k'' in K'' and the embedding vector of the local node k'' in K'' semantic distance

[0034] a model similarity calculation module, configured to calculate a model similarity M(ω i′ ,ω j′ ) of the local model ω i′ of the local node i' and the local model ω j′ of the local node j';

[0035] a network training module, configured to train the deep feedforward network G with the semantic distance as input and the model similarity M(ω i′ ,ω j′ ) as output;

[0036] a migration module, configured to input the semantic distance into the trained deep feedforward network G to obtain the model similarity M(ω k′ ,ω k″ ) of the local node k'' and each local node k', ω k′ and ω k″ being the local model of the local node k' and the local node k'' respectively, and to obtain the local model ω k′ of the local node k'' by weighted summation of all local models ω k′ , the weighted weight of the local model ω k′ being proportional to the model similarity M(ω k′ ,ω k″ );

[0037] wherein K' is a first set of local nodes, K'' is a second set of local nodes, the first set of local nodes K' is a local node participating in aggregated training, and the second set of local nodes K'' is a local node without label and not participating in aggregated training in a local data set.

[0038] According to another aspect of the present application, there is provided an action perception model construction system, comprising a central server and a plurality of local nodes, the central server performing aggregated training with K' local nodes,

[0039] the central server being the central server described above;

[0040] each local node k' participating in aggregated training comprising:

[0041] a downloading module, configured to download an aggregated model corresponding to a current t-th round of iteration from the central server

[0042] a local training module configured to train the aggregated model with a local dataset to obtain a local model The local dataset is a physical layer semantic feature of a radio frequency signal with an action label, and the local training takes the physical layer semantic feature in the local dataset as input and an action as output to train the model.

[0043] an uploading module configured to upload the local model to the central server

[0044] k' = 1, 2, 3,..., K'.

[0045] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0046] 1. The action perception model construction method disclosed by the present application adopts an aggregated training method to realize model collaborative training in a large-scale distributed wireless perception system. When performing collaborative training, the local model is uploaded instead of the local dataset, which not only greatly reduces the communication overhead of the system but also effectively reduces the risk of user privacy data leakage. In addition, when performing aggregation, instead of aggregating into a common global model, a personalized aggregated model is customized for each local node, and the aggregated models downloaded by different local nodes are not necessarily the same. For any local model, based on the difference between the local model and other local models, the corresponding aggregated model parameters are adjusted in iterations to obtain different aggregated models. The greater the model difference, the worse the effect of collaboration between models (i.e., model aggregation), and the slower the model convergence speed. Therefore, generating personalized aggregated models based on model differences can increase the similarity between models, which is beneficial for local models to obtain more help from other nodes and speed up the convergence speed of model training. On the other hand, by customizing the personalized aggregated model, the model obtained by each local training is not necessarily the same, which can better adapt to the environment and has a higher prediction accuracy in the environment.

[0047] 2. Further, only part of the local nodes need to be labeled, and after determining the local model of the labeled local nodes through aggregated training, the mapping relationship between the model similarity and the local data semantic distance is learned. For other local nodes, the local data can not be labeled, and only the semantic distance between the local data and the labeled local data needs to be calculated. Then, according to the mapping relationship, the model parameters of the existing local model are migrated to determine the local model of the local nodes without labeled samples. In this way, the labeling cost can be saved, and multiple local models can be quickly determined.

[0048] 3. Further, the semantic feature u k′ ​including a channel attenuation factor A related to the environment n , a propagation time delay τ n , a signal arrival incidence angle ψ n , a semantic feature υ related to the action k′ including a channel attenuation factor A related to the action m , a propagation time delay τ m , a signal arrival incidence angle ψ m , a Doppler shift ρ m By selecting the above semantic features, the relative position, relative distance and motion path between the moving target and the wireless transceiver device and the surrounding physical environment can be comprehensively described, which helps the trained model to make more accurate judgments on unknown target actions.

[0049] 4. Further, the R(x) function is selected in the form of This is conducive to learning more useful knowledge from other local models with higher similarity to the local model, so as to better train the local model.

[0050] 5. Further, when performing model migration, the cosine distance is used to calculate the semantic distance and the model similarity, so as to obtain faster convergence speed and higher model accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 a scene diagram of a wireless sensing node of an embodiment;

[0052] Figure 2 a step flowchart of an action perception model construction method of an embodiment;

[0053] Figure 3 a system framework diagram of an action perception model construction system of an embodiment;

[0054] Figure 4 a step flowchart of model migration of an embodiment;

[0055] Figure 5 a performance comparison diagram of three learning methods of an embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0057] As Figure 1As shown in the scene diagram of the wireless sensing node in an embodiment, a wireless signal transceiver device, that is, a transmitter and a receiver, is deployed, the transmitter transmits wireless radio frequency signals under a given frequency band, the receiver collects wireless radio frequency signals under the given frequency band after the wireless radio frequency signals are reflected by an action target and a static environment, the wireless radio frequency signals are analyzed, semantic features related to the static environment and semantic features related to the action can be extracted, and the semantic features can be used for training of a perception model.

[0058] Due to the wide application of human action perception, wireless sensing as one of the directions of action perception has also developed rapidly. In a complex wireless sensing environment, spatio-temporal heterogeneity is a major challenge for the development of wireless sensing, that is, different users record different distribution of sensing data from different positions in different environments, thereby affecting the transferability of the model trained in different scenes, resulting in low prediction accuracy of the model. Based on this, the present application provides an action perception model construction system and method based on physical layer semantics and a server, which adopts an aggregated training manner to customize personalized aggregated models for different local nodes, improves the convergence speed of the global model, and the personalized model has high prediction accuracy in the environment.

[0059] Embodiment 1

[0060] As Figure 2 As shown is a step flowchart of the action perception model construction method in an embodiment.

[0061] The construction method comprises the following steps:

[0062] Step S1: The local node k' acquires labeled local data, and the labeled local data is a physical layer semantic feature of a radio frequency signal with an action label.

[0063] Wherein, k' = 1, 2, 3, …, K', and K' is the number of local nodes participating in the aggregated training.

[0064] It can be understood that different local nodes correspond to different perception environments, each local node is deployed with a wireless signal transceiver device, and wireless signal samples under a given frequency band are collected as a local data set.

[0065] For example, local node 1 is an empty classroom with tables and chairs, local node 2 is a spacious hall, and local node 3 is an office with sofas and tables and other furniture. The wireless signal transmitter is a router that transmits WiFi signals, and the wireless signal receiver is a desktop computer equipped with an Intel 5300 network card, which uses Linux CSI Tool software to measure and record the CSI (Channel State Information) in the channel. The above transceiver works in the 5.825 GHz frequency band (i.e., channel 165) of the wireless channel, wherein the router broadcasts WiFi packets at a rate of 1000 packets per second, and after reflection by the static environment and the moving target, the receiver activates three antennas to receive the CSI information in the channel.

[0066] Step S1 specifically includes:

[0067] Step S11: Obtain the radio frequency signal under a certain action, the radio frequency signal being the channel state information CSI received by the receiver after the transmission signal of the transmitter is reflected by the obstacle, the obstacle including the static environment obstacle and the moving target performing a certain action.

[0068] Specifically, the user performs a certain action in the environment, the transmitter transmits a radio frequency signal in a selected channel, and after the transmission signal is reflected by the static environment obstacle and the moving target performing a certain action, the receiver receives the channel state information CSI in the selected channel. The label of the channel state information CSI is the action currently performed by the target.

[0069] For example, in the environment, 16 moving targets perform 6 gestures (push-pull, wave, clap, slide, circle, and zigzag) in 5 positions, 5 directions for each position, and 5 instances for each gesture, respectively collect the corresponding channel state information CSI and mark the corresponding action, form 12,000 gesture samples (16 users x 5 positions x 5 directions x 6 gestures x 5 instances), and construct the local labeled data set.

[0070] Step S12: Extract the radio frequency signal features of the channel state information CSI to obtain the corresponding physical layer semantic features k′ The physical layer semantic features k′ include environment-related semantic features u k′ and action-related semantic features υ k′ .

[0071] The main feature of the wireless signal recorded by the receiver is the CSI of the wireless link connecting the transmitter and the receiver, which is reflected and scattered by the moving user and located in the static environment. Specifically, the CSI recorded at time t, subcarrier frequency θ, and antenna a can be written as follows in the classical form:

[0072]

[0073] wherein, and represent static channel and dynamic channel respectively, the dynamic channel refers to the signal reflected from the moving target, and the static channel refers to the direct path signal and the reflected signal from the static object such as wall, for each channel A l and τ l represent channel attenuation factor and propagation delay respectively. Since the Wi-Fi network interface controller measures the CSI in a discrete manner in time (packet interval), frequency (subcarrier bandwidth) and space (sensor distance), the measured CSI data is discrete in time, frequency and space, so the propagation delay τ n (θ, a) in the above formula can be expressed as: m (t, θ, a).

[0074]

[0075]

[0076] wherein, τ0, ψ0, ρ0 are the propagation delay, signal arrival incidence angle and Doppler frequency shift of the reference signal H(0, 0, 0) respectively. And Δa n , Δa m are the spatial position difference of the corresponding receiving antenna between H(t, θ, a) and the reference signal H(0, 0, 0) for channel n and channel m respectively; Δt m is the packet sending time difference between H(t, θ, a) and the reference signal H(0, 0, 0) for channel m; Δθ m is the subcarrier frequency difference between H(t, θ, a) and the reference signal H(0, 0, 0) for channel m.

[0077] As can be seen from the above formula, the channel state information CSI is closely related to two kinds of physical layer semantic features, namely the semantic features related to the physical layer environment (denoted as P semantic) and the semantic features related to the action (denoted as G semantic). The P semantic includes the semantic features related to the physical environment, such as the environmental layout, and the relative position and direction between the transmitter, the receiver and the human user, which are embodied in the environment-related channel attenuation factor A n , the propagation delay τ n , the signal arrival incidence angle ψ n = Δa n · ψ0, which can be expressed as, The G semantic includes the semantic features related to the gesture, such as the body coordinates of the user and the motion pattern of the gesture, which are embodied in the channel attenuation factor A m , the propagation delay τ m, signal arrival angle of incidence ψ m = Δa m · ψ0, Doppler shift may be expressed as

[0078] In other embodiments, other parameters can also be parsed as semantic features, such as signal angle of departure AOD.

[0079] The physical layer semantic features of the wireless signal are expressed as a combination of P semantic and G semantic φ k′ = <u k′ ,υ k′ >. The semantic features of the physical layer are specific to the environment and the user, so each local node has unique semantic features φ k′ .

[0080] Step S2: The local node k' downloads the aggregated model corresponding to the current tthiteration from the central server and performs local training with the labeled local data set to obtain the local model Then upload the local model to the central server.

[0081] where the local training takes the physical layer semantic features as input and the corresponding action as output to train the model.

[0082] In the considered distributed wireless sensing scenario, the data recorded by each receiver cannot be exposed to other receivers, but its local model can be shared with the central server and other receivers, so a personalized federated learning method is used for collaborative training of the model. In the collaborative training process, each target is locally trained at a receiver k' using its local data set , and the training target is to make the loss function F k′ (ω k′ ) converge to the expected extent.

[0083] It can be understood that in the initial iteration, the central server can set an initial aggregated model.

[0084] In each iteration, the local node k' downloads the aggregated model corresponding to the current tthiteration from the central server Let be the initial model for local training of the local node k' in the current iteration, and after determining the initial model for local training in each iteration, local training is performed using the local data set. Specifically, E rounds of local iterations are performed locally by the random gradient descent method, i.e.:

[0085]

[0086] wherein η is a learning rate, is an unbiased random gradient on the local training set.

[0087] After the local node k' completes the local iteration, the local node k' sends the updated local model to the central server, and the central server performs model coordination.

[0088] Step S3: After the central server receives the K' local models, it determines whether the iteration end condition is reached. If not, it calculates the aggregated model for the t+1 iteration of each local node k'. Let t = t + 1, and jump to step S2; if yes, end the iteration, and each local node takes its current local model as its final local model; wherein the iteration end condition is that the maximum iteration round is exceeded or the global loss is lower than the preset value; the aggregated model is the local model minus the coordination amount, and the coordination amount is positively related to the difference between the local model and other local models.

[0089] Specifically, if the iteration end condition is not reached, the iteration continues until the iteration end condition is reached. When the iteration continues, a personalized aggregated model needs to be generated for each local node according to the customization requirements. The customization principle is to adjust the local model according to the model gap between the local model and other local models. The larger the gap, the larger the adjustment amount. Through adjustment, the gap between models can be gradually reduced, and the similarity between models can be increased, which is beneficial to the local model to obtain more help from other nodes and accelerate the convergence speed of model training. Moreover, by generating a personalized aggregated model for each local node, it does not require a common aggregated model, which can make each local model better adapt to the environment and have higher prediction accuracy in the environment.

[0090] In an embodiment, the formula for updating the aggregated model is:

[0091]

[0092] wherein, denotes other local nodes in the K' local nodes except the local node k', ||·|| is the Euclidean norm, reflects the difference between the local model of the local node k' and , R(x) is a nonlinear attention inducing function with x as the independent variable, is an increasing function with the difference between the local models as the independent variable. The larger the difference, the larger the function value, is the gradient calculation of R(x), is also an increasing function, β is an adjustment coefficient obtained by experimental adjustment, used to ensure that the adjustment quantity is not too large or too small, and can realize the adjustment effect on the model parameters, which is usually determined by multiple experimental comparisons and adjustments.

[0093] In an embodiment, the form of the R(x) function can be expressed as follows:

[0094]

[0095] In the formula, σ is a parameter determined according to experiments, and its order of magnitude is the same as x. Multiple values can be selected, and the optimal value can be determined by experimental comparison. It should be noted that the form of the R(x) function is not limited to the above form, as long as it meets the above individual customization rules. For example, a smooth edge absolute deviation function SCAD can also be used.

[0096] By executing steps S2-S3 multiple times, the iteration is stopped when the maximum iteration round is exceeded or the global loss is lower than the preset value. Through the above iteration, the overall goal of federated learning can be achieved, that is,

[0097]

[0098]

[0099]

[0100] γ is a regularization parameter, and β = αγ.

[0101] The local node uses the latest local model as the trained local model, which is used to perceive the user's action in the environment based on the physical layer semantic features in the environment.

[0102] Embodiment 2

[0103] Considering that another challenge faced by wireless perception is that it is very difficult to collect high-quality labeled samples, the user needs to receive the radio frequency signal after making the corresponding action in the corresponding environment, which is complex and time-consuming. To solve this problem, the embodiment determines the local model of the local node of the unlabeled sample based on the training results of embodiment 1.

[0104] As Figure 3Fig. 1 shows a system framework diagram of an action perception model construction system in an embodiment, including a central server and K local nodes, which form a local node set K, and are divided into a first type of local node set K' with labeled wireless signal samples and a second type of local node set K" without wireless signal samples. For simplicity of description, k e K represents a local node in the local node set K, k' e K' represents a local node in the local node set K', and k" e K" represents a local node in the local node set K". For the first type of local node set K', each local node k' e K' participates in the aggregated training and determines a local model in the manner of Embodiment 1. Then, the local model of the local node k" e K" is obtained based on the local model of the local node k' e K'.

[0105] Therefore, in an embodiment, after step S3, it further includes:

[0106] Step S4: determining the local model of the second type of local node set K" according to the data of the first type of local node set K', the first type of local node set K' being the local nodes participating in the aggregated training, and the second type of local node set K" being the local nodes without labeled data and not participating in the aggregated training.

[0107] As shown in Fig. 2, the model migration in an embodiment includes the following steps: Figure 4 Fig. 2 shows a step flow diagram of model migration in an embodiment, and the specific process is as follows:

[0108] Step S41: converting any physical layer semantic feature φ k of the local node k e K into an embedding vector K = K' U K".

[0109] Specifically, the central server obtains at least one physical layer semantic feature of each local node to form a global semantic knowledge base. To measure the distance of the physical layer semantic information, it is necessary to map the high-dimensional physical layer semantic information into a low-dimensional vector space. For this purpose, the physical layer semantic information can be embedded into an embedding vector space using a knowledge graph embedding method such as transE, etc. For the physical layer semantic feature φ k′ of the local node k' e K', it can be converted into an embedding vector For the physical layer semantic feature φ k″ of the local node k" e K", it can be converted into an embedding vector

[0110] Step S42: calculating the semantic distance d between the embedding vector of the local node i' e K' and the embedding vector of the local node j' e K', and calculating the semantic distance d Embedding vector of each local node k′∈K′ semantic distance

[0111] Specifically, calculate the semantic distance between pairwise local models in the first-class local node set K′. And the semantic distance between a local model k″ in the second type of local node set K″ and every local model in the first type of local node set K′.

[0112] In one embodiment, semantic distance can be calculated using cosine distance. For any local node i,j∈K, the semantic distance is calculated. The specific calculation formula is as follows:

[0113]

[0114] Step S43: Calculate the local model ω of local node i′ i′ The local model ω of local node j′ j′ Model similarity M(ω) i′ ,ω j′ ).

[0115] Specifically, model similarity can be calculated using cosine distance. For any local node i,j∈K, the model similarity M(ω) is calculated. i ,ω j The specific calculation formula is as follows:

[0116]

[0117] It should be noted that the calculation of semantic distance and model similarity is not limited to using cosine distance. In other embodiments, Euclidean distance or other methods can also be used. In this embodiment, compared with Euclidean distance, using cosine distance can achieve faster convergence speed and higher model accuracy.

[0118] Step S44: Using semantic distance As input, with model similarity M(ω) i′ ,ω j′ The deep feedforward network G is trained using ) as the output.

[0119] Semantic distance calculated using data from the first type of local node set Model similarity M(ω) i′ ,ω j′ Using samples as pairs, a deep feedforward network G is trained to form a mapping relationship between semantic distance and model similarity, M = G(S).

[0120] Step S45: Calculate the semantic distance Input the trained deep feedforward network G to obtain the model similarity M(ω k′ ,ω k″ ) of the local node k'' with each local node k'.

[0121] wherein ω k′ and ω k″ are local models of the local node k' and the local node k'' respectively.

[0122] Step S46: Weighted sum of all local models ω k′ to obtain the local model ω k′ of the local node k'', and the weighted weight of the local model ω k′ is proportional to the model similarity M(ω k′ ,ω k″ ).

[0123] The higher the model similarity is, the greater the weight is.

[0124] In an embodiment, the calculation formula of the local model ω k″ is as follows:

[0125]

[0126] m * is a parameter of the deep feedforward network G, R(x) is a nonlinear attention inducing function with x as an independent variable, is an increasing function, and α is an adjustable coefficient obtained according to multiple experiments.

[0127] In the embodiment, only the data of part of the local nodes need to be labeled, the local model thereof is determined through aggregated training, and then the mapping relationship between the model similarity and the semantic distance of the local data is learned. For other local nodes, the local data thereof can not be labeled, only the semantic distance between the local data and the labeled local data needs to be calculated, and then the model parameters of the existing local model are migrated to determine the local model of the local node without labeled samples, so that the labeling cost can be saved, and multiple local models can be quickly determined.

[0128] Embodiment 3

[0129] The application also relates to a central server participating in the action perception model construction method, which comprises:

[0130] a receiving module configured to receive the local model ω k' obtained by each local node k' participating in the aggregated training in the current tth iteration. k' = 1, 2, 3, …, K', and K' is the number of the local nodes participating in the aggregated training.

[0131] a determination module configured to determine whether an iteration end condition is reached after receiving the K' local models, if not, trigger the iteration module to work, if yes, end the iteration;

[0132] an iteration module configured to calculate an aggregated model of the local node k' in the t+1th iteration round after being triggered by the determination module and send it to the corresponding local node k'; wherein the iteration end condition is that the maximum iteration round is exceeded or the global loss is lower than a preset value; the aggregated model is the local model minus the coordination amount, the coordination amount is positively correlated with the difference between the local model and other local models.

[0133] Further, the central server further comprises:

[0134] a conversion module configured to convert any physical layer semantic feature φ k of the local node k∈K into an embedding vector K=K'∪K";

[0135] a semantic distance calculation module configured to calculate a semantic distance between the embedding vector of the local node i'∈K' and the embedding vector of the local node j'∈K'; and a semantic distance between the embedding vector of the local node k"∈K" and the embedding vector of each local node k'∈K';

[0136] a model similarity calculation module configured to calculate a model similarity M(ω i′ ,ω j′ ) between the local model ω i′ of the local node i' and the local model ω j′ of the local node j';

[0137] a network training module configured to train a deep feedforward network G with the semantic distance as input and the model similarity M(ω i′ ,ω j′ ) as output;

[0138] a migration module configured to input the semantic distance into the trained deep feedforward network G to obtain the model similarity M(ω k′ ,ω k″ ) between the local node k" and each local node k', wherein ω k′ and ω k″ are the local models of the local node k' and the local node k", respectively, and all the local models ωk′ We obtain the local model ω of the local node k″ by weighted summation. k′ Local model ω k′ The weighted average is proportional to the model similarity M(ω). k′ ,ω k″ );

[0139] Where K′ is the first type of local node set, K″ is the second type of local node set, the first type of local node set K′ consists of local nodes that participate in the aggregation training, and the second type of local node set K″ consists of local nodes in the local dataset that are unlabeled and do not participate in the aggregation training.

[0140] Example 4

[0141] This invention also relates to a motion-aware model construction system, comprising a central server and multiple local nodes. The central server and K′ local nodes perform aggregate training. The central server is the one described in Example 3. Each local node k′ participating in the aggregate training includes:

[0142] The download module is used to download the aggregation model corresponding to the current iteration t from the central server.

[0143] The local training module is used to train the aggregation model on a local dataset. Perform local training to obtain a local model The local dataset consists of physical layer semantic features with action labels. Local training uses the physical layer semantic features in the local dataset as input and the corresponding actions as output to train the model.

[0144] The upload module is used to upload local models. Uploaded to the central server.

[0145] For details on the specific functions of each module, please refer to the previous introduction, which will not be repeated here.

[0146] Example 5

[0147] This embodiment verifies the proposed solution using specific experimental data.

[0148] Dataset selection: Eighteen local nodes were set up, each collecting local data using a publicly available gesture dataset consisting of 18 gesture classes performed by 17 human users and recorded by Wi-Fi receivers. Each receiver was equipped with three antennas, enabling the recording of CSI measurements at a sampling rate of 1000Hz.

[0149] Model setup: A two-layer convolutional neural network (CNN) with two ReLU activations, a pooling layer in between, and a softmax layer at the end of the network is used. In addition, the model is trained using an SGD optimizer with a learning rate of 0.005.

[0150] Experimental results: The proposed method is compared with two typical machine learning methods (using only local learning and using federated averaging algorithm). Among them, federated averaging means that the aggregation model downloaded by all local nodes participating in the aggregation training is the same, while the proposed method is to customize a personalized aggregation model for each local node. Among the 18 local nodes, 12 local nodes with labeled samples are selected as source nodes, and the remaining 6 local nodes without labeling and training are selected as target nodes. As shown in FIG. 3, the performance comparison of the three learning methods is shown. As can be seen from the figure, for source nodes 1-12, the proposed method is better than using only local training and federated averaging in most source local nodes, which means that the proposed method based on the collaborative personalized model training strategy can effectively alleviate the problems of label scarcity and data heterogeneity, and can effectively improve the model prediction accuracy. For target nodes 13-18, the global model obtained by using the federated averaging algorithm on the source nodes 1-12 is directly used for prediction of the actions of the target nodes 13-18, and the prediction accuracy is obviously lower than that of the local model obtained by the model migration of the proposed method. Figure 5

[0151] In summary, the proposed method customizes a personalized model for each receiver to effectively alleviate the problems of data heterogeneity and label scarcity in wireless systems and improve the model prediction accuracy.

[0152] Those skilled in the art will readily understand that the above is only a preferred embodiment of the present application, and is not intended to limit the present application, and any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.​

Claims

1. A method for constructing an action perception model based on physical layer semantics, characterized in that, Comprising: Step S1: local node acquire tagged local data, the tagged local data being a physical layer semantic feature of the radio frequency signal with the action tag; , the number of local nodes participating in the aggregation training; Step S2: Local Node Download the current number from the central server. Aggregation model corresponding to round iteration The local model is obtained by training on labeled local data. Then the local model Uploaded to the central server; wherein, local training uses physical layer semantic features as input and corresponding actions as output for model training; Step S3: The central server receives After determining whether the iteration termination condition has been met for each local model, if not, then calculate the iteration termination condition for each local node. The Round-Iterative Aggregation Model ,make If so, proceed to step S2; if yes, end the iteration, and each local node uses its current local model as its final local model; wherein, the iteration termination condition is exceeding the maximum iteration round or the global loss is lower than a preset value; aggregate model For local models Subtract the coordination factor, which is positively correlated with the local model. Differences between the model and other local models; The step S1 comprises: Step S11: acquiring a radio frequency signal under certain action, the radio frequency signal being channel state information (CSI) received by a receiver after a transmitter's transmitted signal is reflected by an obstacle, the obstacle including a static environmental obstacle and an application target performing certain action; Step S12: extract the radio frequency signal features of channel state information CSI to obtain corresponding physical layer semantic features The physical layer semantic features include environment-related semantic features and action-related semantic features ; Aggregation model The calculation formula is; In the formula, represents other local nodes in the K local nodes except the local node , is the Euclidean norm, is a nonlinear attention induction function with as the independent variable, is a gradient calculation on , is an increasing function, is an adjustment coefficient obtained by experimental adjustment; The function has the form: wherein According to the parameters of the experiment, the order of magnitude is the same as the same.

2. The action-aware model construction method of claim 1, wherein, Environment-related semantic features Including channel attenuation factors related to the environment , propagation time delays , signal arrival angles of incidence Action-related semantic features Including channel attenuation factors related to the action , propagation time delays , signal arrival angles of incidence , Doppler shifts . 3.The action-aware model construction method of claim 1, wherein, The method further includes: based on the first type of local node set Data determines the second type of local node set The local model, the first type of local node set The second type of local node set refers to the local nodes participating in the aggregation training. These are local nodes in the local dataset that are unlabeled and did not participate in the aggregation training; specifically, they include: local node arbitrary physical layer semantic features Convert to embedding vector , ; computing the embedding vector of the local node computing the semantic distance between the embedding vector of the local node and the embedding vector of the local node computing the semantic distance between the embedding vector of the local node and the embedding vector of each local node computing the semantic distance between the embedding vector of the local node and the embedding vector of each local node computing the semantic distance between the embedding vector of the local node and the embedding vector of each local node computing the semantic distance between the embedding vector of the local node and the embedding vector of each local node computing local node local model of the local node and the local node the local model of the local node model similarity ; semantic distance As input, based on model similarity For the output, a deep feedforward network Train the program to fit the corresponding mapping function; semantically distance inputting the trained deep feedforward network obtaining local nodes a model similarity with each local node , , a local model of local node and local node , respectively for all local models weighted sum, resulting in local model of local node , local model weighted weight of local model is proportional to model similarity 4. The action-aware model construction method of claim 3, wherein, Respectively calculate semantic distance and model similarity through cosine distance.

5. A central server participating in the action-aware model construction method of claim 1, characterized in that, Comprising: a receiving module configured to receive local models from each of the local nodes participating in the federated training in the current round of iterations the local model resulting from the current round of iterations , , is the number of local nodes participating in the federated training The determination module is used to determine the condition of the received data. After determining whether the local model has reached the iteration termination condition, if not, the iteration module is triggered; if so, the iteration ends. An iteration module is configured to calculate the local model of each local node after being triggered by the determination module The first iteration of the aggregation model And send to the corresponding local node ; wherein the iteration end condition is to exceed the maximum iteration round or the global loss is lower than the preset value; the aggregation model is the local model Subtract the coordination amount, which is positively related to the difference between the local model And other local models.

6. The central server of claim 5, wherein, Comprising: conversion module, local node any physical layer semantic feature converted into an embedding vector , ; a semantic distance calculation module, configured to calculate a semantic distance between the embedding vector of the local node and the embedding vector of the local node and a semantic distance between the embedding vector of the local node and the embedding vector of each local node and the embedding vector of each local node and the embedding vector of each local node and the embedding vector of each local node and the embedding vector of each local node and the embedding vector of each local node and the embedding vector of each local node ; The model similarity calculation module is used to calculate the similarity of local nodes. local model and local nodes local model Model similarity ; The network training module is used to measure semantic distance. As input, based on model similarity For the output, a deep feedforward network Conduct training; The migration module is used to transfer semantic distance. Input the trained deep feedforward network Get the local node With each local node Model similarity , , Local nodes and local nodes The local model, for all local models Weighted summation yields the local node local model Local model The weighted average is proportional to the model similarity. ; wherein, is a first set of local nodes, is a second set of local nodes, the first set of local nodes is a local node participating in the aggregated training, the second set of local nodes is a local node not labeled and not participating in the aggregated training.

7. A motion-aware model building system, comprising a central server and multiple local nodes, wherein the central server and... The feature is that local nodes perform aggregated training. The central server is the central server of claim 5 or 6; Each local node participating in the aggregated training comprising: a download module, configured to download the current first aggregated model corresponding to the wheel iteration ; a local training module configured to train the aggregated model with a local dataset to obtain a local model wherein the local dataset is a physical layer semantic feature of the radio frequency signal with action label, and the local training is performed with the physical layer semantic feature in the local dataset as input and the corresponding action as output for model training. The upload module is used to upload the local model. Uploaded to the central server; 。

Images