Training method of wireless sensing pre-training model

Abstract

The invention provides a wireless sensing pre-training model training method based on asymmetric local consistency rearrangement, and the method comprises the steps: carrying out the phase alignment of original signal information, and obtaining a sample original feature map of the original signal information; inputting the original sample feature map into a first neural network of a wireless sensing pre-training model to obtain original sample features; performing first down-sampling on the sample original feature map to obtain a plurality of initial local region features of the sample original feature map; based on a first rearrangement rule, performing feature rearrangement on the plurality of initial local region features to obtain a plurality of enhanced local region features; inputting a sample enhanced feature graph generated by the plurality of enhanced local region features into a second neural network of the wireless sensing pre-training model to obtain sample enhanced features; and constructing similarity loss according to the original sample features and the enhanced sample features, and training a wireless sensing pre-training model to obtain a trained wireless sensing pre-training model.

Description

Technical Field

[0001] The present disclosure relates to the fields of signal processing and artificial intelligence technology, and more specifically, to a training method for a wireless perception pre-training model based on asymmetric local consistency rearrangement. Background Art

[0002] Currently, the biggest obstacle to developing learning-based wireless perception models is obtaining large-scale, high-quality manually annotated datasets. However, wireless signals lack intuitiveness and interpretability, making them difficult to annotate manually like visual image data. In addition, the high correlation between wireless signals and the data collection environment forces the dataset to be collected on a large scale in various environments, further increasing the difficulty of dataset acquisition.

[0003] In order to solve the above problems, a common approach is to use sensors of other modalities (such as visual cameras) for synchronous acquisition, and indirectly realize the labeling of wireless signals by labeling the data of other modalities. However, the additional overhead caused by problems such as synchronization and calibration between different modalities still limits the deployment and application of wireless systems in reality.

[0004] Although it is difficult to obtain large-scale, high-quality manually annotated wireless signal data, large-scale unlabeled wireless data is relatively easy to obtain. In recent years, self-supervised contrastive learning has made it easier to develop downstream tasks by learning general semantic representations from a large amount of unlabeled data and fine-tuning the pre-trained model with a small amount of labeled data. This approach has shown great potential in improving the performance of downstream tasks and reducing the dependence on large-scale labeled data. However, existing self-supervised contrastive learning techniques are mainly designed for conventional data such as images and languages. When directly applied to wireless signal data, they are prone to learning meaningless shortcut information, which is not helpful for downstream task development.

[0005] Therefore, based on the characteristics of wireless signals, designing a general and efficient wireless signal self-supervised contrastive learning pre-training method to obtain an efficient wireless signal pre-training model is of great significance for large-scale wireless sensing applications. Summary of the invention

[0006] In view of this, the present disclosure provides a training method for a wireless sensing pre-training model.

[0007] One aspect of the present disclosure provides a training method for a wireless sensing pre-training model, including: performing phase alignment on original signal information to obtain a sample original feature map of the original signal information; inputting the sample original feature map into a first neural network of the wireless sensing pre-training model to obtain sample original features; performing a first down-sampling on the sample original feature map to obtain multiple initial local area features of the sample original feature map; based on a first rearrangement rule, performing feature rearrangement on the multiple initial local area features respectively to obtain multiple enhanced local area features; and inputting the sample enhanced feature map generated by the multiple enhanced local area features into a second neural network of the wireless sensing pre-training model to obtain sample enhanced features; constructing a similarity loss based on the sample original features and the sample enhanced features, training the wireless sensing pre-training model, and obtaining a trained wireless sensing pre-training model.

[0008] Another aspect of the present disclosure provides a training method for a wireless perception model, wherein the wireless perception model includes a functional module and a first neural network module in the wireless perception pre-training model trained based on the training method of the wireless perception pre-training model of the present disclosure, and the functional module is arranged after the first neural network module. The training method of the wireless perception model includes: using labeled sample wireless signals to fine-tune the first neural network module and the functional module.

[0009] Another aspect of the present disclosure provides a training device for a wireless sensing pre-training model, including: a phase alignment module, used to perform phase alignment on original signal information to obtain a sample original feature map of the original signal information; a first neural network module, used to input the sample original feature map into the first neural network of the wireless sensing pre-training model to obtain the sample original features; a first downsampling module, used to perform a first downsampling on the sample original feature map to obtain multiple initial local area features of the sample original feature map; a first rearrangement module, used to perform feature rearrangement on the multiple initial local area features based on a first rearrangement rule to obtain multiple enhanced local area features; and a second neural network module, used to input the sample enhanced feature map generated by the multiple enhanced local area features into the second neural network of the wireless sensing pre-training model to obtain the sample enhanced features; a training module, used to construct a similarity loss according to the sample original features and the sample enhanced features, train the wireless sensing pre-training model, and obtain a trained wireless sensing pre-training model.

[0010] According to the above-mentioned embodiments of the present disclosure, a local consistency rearrangement data enhancement method is proposed. By introducing smaller interference only in a limited area, it can prevent it from destroying the global semantics in the signal and maximize the shared information between the original signal and its enhanced version. It is suitable for a self-supervised comparative learning method for wireless signals and can be used for model pre-training on large-scale unlabeled wireless datasets, significantly improving the performance of downstream tasks and reducing the demand for large-scale labeled data. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0012] Figure 1 An exemplary system architecture to which at least one of a training method for a wireless sensing pre-training model and a training method for a wireless sensing model can be applied according to an embodiment of the present disclosure is schematically shown;

[0013] Figure 2 A flowchart of a training method for a wireless sensing pre-training model according to an embodiment of the present disclosure is schematically shown;

[0014] Figure 3A A schematic diagram schematically shows signal local consistency rearrangement data enhancement according to an embodiment of the present disclosure;

[0015] Figure 3B Schematically showing the effect of signal local consistency rearrangement data enhancement according to an embodiment of the present disclosure;

[0016] Figure 4 Schematically showing the effect of local consistency rearrangement data enhancement of the original signal according to an embodiment of the present disclosure;

[0017] Figure 5 A schematic diagram schematically shows the overall training process of the wireless sensing pre-training model according to an embodiment of the present disclosure;

[0018] Figure 6 A flowchart of a method for training a wireless sensing model according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 7 A schematic diagram of developing downstream tasks based on a wireless perception model according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 8 A block diagram schematically shows a training device for a wireless sensing pre-training model according to an embodiment of the present disclosure;

[0021] Fig. 9 A block diagram schematically shows a training device for a wireless sensing model according to an embodiment of the present disclosure; and

[0022] Fig.10 A block diagram of an electronic device suitable for implementing at least one of the training method of a wireless perception pre-training model and the training method of a wireless perception model of the present disclosure is schematically shown according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0024] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0025] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0026] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0027] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security, and national security.

[0028] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0029] The present disclosure proposes a training method for a wireless sensing pre-training model based on asymmetric local consistency rearrangement, the core contents of which include the following parts: (1) using local consistency rearrangement data enhancement to construct positive and negative sample pairs; (2) asymmetric data enhancement to avoid shortcut information; (3) bringing positive sample pairs closer and negative sample pairs farther apart during contrastive learning.

[0030] Figure 1 The exemplary system architecture 100 is schematically shown to which at least one of the training method of the wireless sensing pre-training model and the training method of the wireless sensing model can be applied according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0031] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0032] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0033] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0034] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0035] It should be noted that the training method of the wireless perception pre-training model provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the training device of the wireless perception pre-training model provided in the embodiment of the present disclosure can generally be set in the server 105. The training method of the wireless perception pre-training model provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the training device of the wireless perception pre-training model provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the training device method of the wireless perception pre-training model provided in the embodiment of the present disclosure can also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or can also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Correspondingly, the training device of the wireless perception pre-training model provided in the embodiment of the present disclosure can also be set in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0036] For example, the original signal information may be originally stored in any one of the first terminal device 101, the second terminal device 102, or the third terminal device 103 (for example, the first terminal device 101, but not limited thereto), or stored on an external storage device and may be imported into the first terminal device 101. Then, the first terminal device 101 may locally execute the training method of the wireless perception pre-training model provided in the embodiment of the present disclosure, or send the original signal information to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the original signal information may execute the training method of the wireless perception pre-training model provided in the embodiment of the present disclosure.

[0037] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0038] Figure 2 The flowchart of the training method of the wireless sensing pre-training model according to the embodiment of the present disclosure is schematically shown.

[0039] like Figure 2As shown, the method includes operations S201 to S206.

[0040] In operation S201, phase alignment is performed on the original signal information to obtain a sample original feature map of the original signal information.

[0041] According to an embodiment of the present disclosure, the original signal information may represent information of a wireless signal received by a receiving antenna. Phase alignment may include at least one of alignment of frequency point phases on a single signal and alignment of adjacent antenna signals between multiple signals, and may not be limited thereto. The sample original feature graph may represent a two-dimensional feature graph after phase alignment of a wireless signal recorded in a coordinate system with frequency as the horizontal coordinate and antenna as the vertical coordinate.

[0042] In operation S202, the sample original feature map is input into the first neural network of the wireless sensing pre-training model to obtain the sample original feature.

[0043] According to an embodiment of the present disclosure, the wireless sensing pre-training model may adopt a twin network model or other neural network models with two branches, which are not limited here. The first neural network may extract features from the original feature map of the sample to obtain the original features of the sample.

[0044] In operation S203, a first downsampling is performed on the original feature map of the sample to obtain a plurality of initial local area features of the original feature map of the sample.

[0045] According to an embodiment of the present disclosure, the first downsampling may be achieved by downsampling the matrix or by convolution, but is not limited thereto.

[0046] For example, for a frame of radar signal , where W and H represent the dimensions of the matrix, and S×S represents a smaller local area, then the radar signal can be first downsampled to ,in is a global description of X, S 2 Contains a local detail of X.

[0047] In operation S204, based on the first rearrangement rule, the plurality of initial local region features are rearranged to obtain a plurality of enhanced local region features.

[0048] According to an embodiment of the present disclosure, the first rearrangement rule may represent a preset rearrangement rule, for example, it may include: rearranging a feature at a certain position to another preset position, etc., and may not be limited thereto. The aforementioned set rule is followed when performing feature rearrangement on each initial local area feature. Through feature rearrangement, data enhancement of the initial local area feature can be achieved to obtain an enhanced local area feature.

[0049] For example, the same rearrangement is performed on each initial local region feature , the enhanced local region features after rearrangement can be , can be obtained by executing formula (1):

[0050] Formula (1)

[0051] In formula (1), Represents the index of the local zone in the global zone, The initial local area characteristics can be characterized. It can represent the enhancement of local area features.

[0052] In operation S205, a sample enhanced feature map generated by a plurality of enhanced local area features is input into a second neural network of a wireless sensing pre-training model to obtain a sample enhanced feature.

[0053] According to an embodiment of the present disclosure, when the wireless sensing pre-training model adopts a twin network model, the first neural network and the second neural network can represent two branches of model parameter sharing. When the wireless sensing pre-training model adopts a neural network model with two branches, the initial model parameters of the first neural network and the second neural network can be set to be the same.

[0054] According to an embodiment of the present disclosure, a plurality of enhanced local area features are reorganized to obtain a sample enhanced feature map. The second neural network can extract features from the sample enhanced feature map to obtain sample enhanced features.

[0055] Figure 3A The schematic diagram of signal local consistency rearrangement data enhancement according to an embodiment of the present disclosure is schematically shown.

[0056] like Figure 3A As shown, the radar signal The sample original feature map 310 is subjected to a first downsampling and then locally expanded to obtain a set 320 of multiple initial local area features, including an initial local area feature 321. Based on the first rearrangement rule, the initial local area feature 321 is locally rearranged to obtain an enhanced local area feature 331. Each initial local area feature in the set 320 of multiple initial local area features is subjected to a local rearrangement operation according to the aforementioned first rearrangement rule to obtain a set 330 of multiple enhanced local area features, including the aforementioned enhanced local area feature 331. By combining the sets 330 of multiple enhanced local area features, a sample enhanced feature map 340 can be obtained after the sample original feature map 310 is subjected to local consistency rearrangement data enhancement.

[0057] Figure 3BThe effect diagram of signal local consistency rearrangement data enhancement according to an embodiment of the present disclosure is schematically shown.

[0058] like Figure 3A and Figure 3B As shown, the above-mentioned local consistency rearrangement method, by dividing the signal into multiple local areas of the same size and non-overlapping, and adopting the same rearrangement method in all local areas, can make the local consistency rearrangement data enhancement satisfy the singular value consistency, thereby maintaining the semantic consistency of the wireless signal before and after enhancement.

[0059] In operation S206, a similarity loss is constructed according to the original features of the samples and the enhanced features of the samples, and the wireless sensing pre-training model is trained to obtain a trained wireless sensing pre-training model.

[0060] Through the above-mentioned embodiments of the present disclosure, a local consistency rearrangement data enhancement method is proposed, which can prevent the global semantics in the signal from being destroyed by introducing smaller interference only in a limited area, and can maximize the shared information between the original signal and its enhanced version. It is a self-supervised comparative learning method suitable for wireless signals, which can be used for model pre-training on large-scale unlabeled wireless datasets, significantly improving the performance of downstream tasks and reducing the demand for large-scale labeled data.

[0061] In conjunction with specific embodiments, Figure 2 The method shown is further explained.

[0062] According to an embodiment of the present disclosure, in addition to performing local consistency rearrangement data enhancement in the spectral space of the signal, local consistency rearrangement data enhancement can also be performed in the original space of the signal. That is, before operation S201, the original signal local consistency rearrangement data enhancement is first performed. The method may include: performing a second downsampling on the original signal information to obtain multiple initial local area information of the original signal information. Based on the second rearrangement rule, the multiple initial local area information are respectively rearranged to obtain multiple enhanced local area information. According to the multiple enhanced local area information, the data-enhanced original signal information is generated.

[0063] It should be noted that the second downsampling, second rearrangement rule, and information rearrangement in this method can have the same or similar implementation methods as the aforementioned first downsampling, first rearrangement rule, and feature rearrangement, and will not be repeated here.

[0064] Figure 4 The effect diagram of data enhancement of local consistency rearrangement of the original signal according to an embodiment of the present disclosure is schematically shown.

[0065] like Figure 4As shown, the phase alignment operation is performed on the original signal information 410 to obtain a first sample feature map 411. The original signal information 410 is first subjected to local consistency rearrangement data enhancement in the original signal space, and then the phase alignment operation is performed to obtain a second sample feature map 412.

[0066] Through the above-mentioned embodiments of the present disclosure, local consistency rearrangement is performed in the original space of the signal, which is equivalent to dispersing the energy of the target to multiple positions in the original space, which can enhance the spatial diversity of the wireless signal and improve the feature learning performance of the pre-trained model.

[0067] According to an embodiment of the present disclosure, the above operation S201 may include: obtaining a frequency phase difference of a signal frequency representing the same received signal in the original signal information and an antenna phase difference between different received signals represented in the original signal information. According to the frequency phase difference and the antenna phase difference, the original signal information is phase aligned to obtain a sample original feature map of the original signal information.

[0068] For example, consider a wireless signal that starts from the transmitting antenna, is reflected by the target at (x, y), and is finally received by the receiving antenna. The phase difference between adjacent frequency points in the received signal can be calculated using formula (2):

[0069] Formula (2)

[0070] In formula (2), represents the frequency difference between adjacent frequency points, c represents the speed of light, Indicates the frequency phase difference.

[0071] The phase difference between adjacent antennas can be calculated using formula (3):

[0072] Formula (3)

[0073] In formula (3), d represents the distance between adjacent receiving antennas. represents the wavelength of the adjacent receiving antenna, Indicates the antenna phase difference.

[0074] By compensating for phase shift and superimposing wireless signals from different frequencies and antennas, the signal from the target (x, y) will be enhanced, and the signals from other locations will be suppressed. To this end, the signal from the target (x, y) will be separated, and the extracted signal can be expressed as formula (4):

[0075] Formula (4)

[0076] In formula (4), Indicates the wireless signal received by the device, a and e represent the receiving antenna index and signal frequency index respectively, A and E represent the total number of receiving antennas and signal frequency respectively, Represents the original feature map of the sample, Is an imaginary number.

[0077] Through the above-mentioned embodiments of the present disclosure, signal information can be preprocessed into original feature maps of samples, which is beneficial to subsequent local consistency rearrangement data enhancement.

[0078] According to an embodiment of the present disclosure, in the process of performing the above operation S204, the possible number of rearrangements obtained by using the random feature rearrangement method includes S 2 !Since not all rearrangements have the same importance, considering that the proximity of signals is important, the rearrangement of signals can be limited to column rearrangement and row rearrangement.

[0079] On this basis, the above operation S204 may include: for each initial local area feature, performing a matrix multiplication calculation on at least one of a row permutation matrix and a column permutation matrix and the initial local area feature to obtain an enhanced local area feature.

[0080] For example, ,in represents the row permutation matrix, represents the row-column permutation matrix, and the rearranged signal can be expressed as formula (5):

[0081] Formula (5)

[0082] In formula (5), The initial overall matrix representing multiple initial local region features, represents the overall representation of the row permutation matrix corresponding to the initial overall matrix, represents the overall representation of the column permutation matrix corresponding to the overall matrix, An enhanced overall matrix representing multiple enhanced local region features.

[0083] In addition, the same rearrangement method is used in all non-overlapping local areas, which is equivalent to rearranging the convolution kernel of the first convolution layer of the model. Therefore, the above formula (5) can also be replaced by formula (6) to achieve:

[0084] Formula (6)

[0085] In formula (6), W is the convolution kernel parameter, * refers to the convolution operation, yes The rearrangement result of a local area is the enhanced local area feature mentioned above.

[0086] Through the above-mentioned embodiments of the present disclosure, the same rearrangement method is adopted in all local areas, so that performing a local consistency rearrangement operation on the signal is equivalent to the convolution kernel of the first layer of the rearrangement model, providing an efficient local consistency rearrangement execution method.

[0087] Furthermore, in the process of performing the aforementioned original signal local consistency rearrangement data enhancement, the rearrangement operation shown in formula (5) or formula (6) can be applied to the original wireless signal R, and the rearranged original signal can be expressed as formula (7), for example:

[0088] Formula (7)

[0089] Then, the phase alignment operation of operation S201 is performed. The essence of the local consistency rearrangement enhancement of the original signal is to spread the target energy to other locations, which can further enhance the spatial richness of the wireless signal.

[0090] According to the embodiments of the present disclosure, based on the above method, the original features of the signal and the enhanced features after local consistency rearrangement data enhancement can be obtained. Further, positive and negative sample pairs can be constructed based on these features to support the completion of the training of the above operation S206.

[0091] In the process of realizing the concept of the present disclosure, the inventor discovered that the core of self-supervised contrastive learning is to construct positive and negative sample pairs, shorten the distance between the positive sample pairs output by the network, and increase the distance between the negative sample pairs, so that the feature space meets a certain consistency, thereby learning common semantic features. Existing positive and negative sample pairs are usually constructed based on data enhancement methods, and these methods are mostly designed for image vision and other data. When directly applied to wireless signal data, only some meaningless shortcut information can be learned. Therefore, how to design efficient data enhancement for wireless signals and avoid shortcuts is the core challenge of applying self-supervised contrastive learning to wireless signals.

[0092] According to an embodiment of the present disclosure, the number of original signal information is multiple. Corresponding to the process of constructing positive and negative sample pairs, the above operation S206 may include: determining a positive sample pair based on the sample original features of the target original signal information and the sample enhanced features of the target original signal information, wherein the target original signal information represents any one of the multiple original signal information. Determine a negative sample pair based on the sample original features of the target original signal information and the sample enhanced features of the other original signal information, wherein the other original signal information represents any one of the multiple original signal information except the target original signal information. Construct a similarity loss based on the first similarity metric value of the positive sample pair and the second similarity metric value of the negative sample pair, and train the wireless sensing pre-training model.

[0093] For example, an unlabeled data set containing N original signal information can be expressed as First, the data enhancement operation of the local consistency rearrangement of the original signal can obtain a data set of the original signal information with data enhancement. Secondly, the phase alignment operation in operation S201 can be performed to obtain a data set of the original feature map of the sample that has been data enhanced. Then, combined with the local consistency rearrangement data enhancement operation of operations S203~S204, a data set of sample enhanced feature maps can be obtained. ,in It means that the original signal has undergone local consistency rearrangement data enhancement and the sample original feature map has undergone two local consistency rearrangement data enhancement operations. In addition, for the unlabeled dataset Only performing the phase alignment operation in operation S201 can obtain a data set of the original feature map of the sample without data enhancement. In this embodiment, is called a positive sample pair, are called negative sample pairs.

[0094] It is worth noting that the conventional method of constructing positive and negative sample pairs adopts a symmetric data enhancement method, that is, the sample pairs are composed of enhanced signals, which will cause the model to learn shortcut features. To solve this problem, the present invention adopts an asymmetric data enhancement method, which uses the wireless signal and its enhanced version to construct positive and negative sample pairs instead of using both enhanced signals.

[0095] According to the embodiments of the present disclosure, the goal of the pre-training of the self-supervised contrastive learning model is to shorten the distance between the features of the positive sample pairs and exclude the distance between the features of the negative sample pairs, so that the feature space satisfies a certain consistency. To this end, the wireless sensing pre-training model can adopt two branches f that are shared by model parameters or have consistent initial parameter information. q (.) and f k (.)constitute.

[0096] For example, the first neural network is f q (.), the second neural network is f k (.). Corresponding to the above similarity loss, the goal of model training can be to minimize the InfoNCE loss function as shown in formula (8):

[0097] Formula (8)

[0098] In formula (8), , , , Respectively represent the positive and negative sample pair features extracted by the model, is the feature similarity measurement function, is the temperature coefficient, and K is the number of negative sample pairs.

[0099] Through the above-mentioned embodiments of the present disclosure, an asymmetric data enhancement method is adopted to maximize the shared information between the original wireless signal and its enhanced version, rather than maximizing the shared information between the two enhanced versions, which can significantly reduce the shortcut features in the pre-trained model and is conducive to training an efficient wireless perception pre-trained model.

[0100] According to an embodiment of the present disclosure, corresponding to the process of adjusting the model parameters, the above operation S206 may also include: in the tth round of training, according to the tth similarity loss value of the similarity loss, adjusting the first model parameter in the first neural network to obtain the first adjusted model parameter. According to the momentum coefficient, the first adjusted parameter and the second model parameter in the second neural network, a momentum summation calculation is performed to obtain the second adjusted model parameter in the second neural network, and the initial parameter information of the first model parameter is the same as the initial parameter information of the second model parameter.

[0101] According to the embodiments of the present disclosure, the twin network is usually a weight-sharing model to process multiple inputs, which includes a gradient-computable branch for updating network parameters and a gradient-untrainable branch. The data domain of the entire pre-trained model depends on the input data of the gradient-trainable branch. Therefore, the original signal information can be input into the gradient-trainable branch, i.e., the first neural network f q (.), input the enhanced version into the gradient untrainable branch, i.e., the second neural network f k (.), this operation prevents the model from fitting to potential shortcut features introduced by data augmentation.

[0102] For f q The model parameters in (.) can be updated by the back propagation algorithm based on the InfoNCE loss function shown in formula (8).

[0103] For f k The model parameters in (.) can be updated by the momentum summation method in formula (9):

[0104] Formula (9)

[0105] In formula (9), Respectively represent f q (.) and f k (.) model parameters, =0.999 represents the momentum coefficient, and its value can be adjusted according to actual business needs, but is not limited to this.

[0106] Figure 5 The schematic diagram schematically shows the overall training process of the wireless sensing pre-training model according to an embodiment of the present disclosure.

[0107] like Figure 5 As shown, the sample original feature map obtained after performing phase alignment on the original signal information 510 can be input into the model f q (.) 520, extract the original features of the sample 521. The sample enhanced feature map obtained by sequentially performing operations S501 to S503 on the original signal information 510 can be input into the model f k (.) 530, extract sample enhancement features (or )531.

[0108] In operation S501 , local consistency rearrangement data enhancement is performed in an original space of a signal.

[0109] In operation S502, phase alignment is performed.

[0110] In operation S503, local consistency rearrangement data enhancement is performed in the spectral space of the signal.

[0111] According to the embodiments of the present disclosure, the number of negative sample pairs K is usually a relatively large value, such as K=16385. This results in the need for a large batch of negative sample pair data, resulting in large memory requirements and high hardware costs. Figure 5 As shown, a queue 540 can be used to store the f in the training process. k (.) output, each time only needs to sample from queue 540 samples, and then with Construct negative sample pairs, combine with InfoNCE loss, and train the model.

[0112] It should be noted that, corresponding to the above process of performing local consistency rearrangement data enhancement on the original signal information 510 to obtain a sample enhanced feature map, only operations S501~S502 may be performed, or only operations S502~S503 may be performed, which is not limited here.

[0113] According to an embodiment of the present disclosure, corresponding to the model determination process, the above operation S206 may also include: determining a trained wireless sensing pre-training model according to the trained first neural network.

[0114] For example, after large-scale unlabeled data pre-training, the final f q (.) is the wireless perception model officially used in actual business.

[0115] Through the above embodiments of the present disclosure, a training method for a wireless perception pre-training model based on asymmetric local consistency rearrangement is realized. This method maximizes the shared information between the original radio frequency signal and its enhanced version by utilizing local consistency rearrangement data enhancement, and introduces only small interference in a limited area, which can prevent it from destroying the global semantics in the wireless signal. By utilizing asymmetric enhancement to eliminate shortcuts, an efficient wireless signal pre-training model is obtained.

[0116] Figure 6 The flowchart of the training method of the wireless sensing model according to the embodiment of the present disclosure is schematically shown.

[0117] According to an embodiment of the present disclosure, the wireless perception model includes a functional module and a first neural network module in the wireless perception pre-training model obtained by training based on the above method. The functional module is arranged after the first neural network module.

[0118] like Figure 6 As shown, the method includes operation S601.

[0119] In operation S601, the first neural network module and the function module are fine-tuned and trained using labeled sample wireless signals.

[0120] According to the embodiments of the present disclosure, for different downstream tasks, only a small amount of labeled data needs to be collected. q (.)Minor adjustments can complete the development of downstream tasks.

[0121] Figure 7 A schematic diagram of developing downstream tasks based on a wireless perception model according to an embodiment of the present disclosure is schematically shown.

[0122] like Figure 7 As shown, the input may be a wireless signal 701. The pre-trained model 710 may adopt the first neural network module f in the wireless sensing pre-trained model trained based on the above method. q (.) Corresponding to different downstream tasks, the functional module 720 may adopt any one of the following: a classifier module 721 , a regression module 722 , and a decoder module 723 .

[0123] For example, the first embodiment of the present disclosure verifies the performance improvement of the wireless sensing pre-training model obtained by the present disclosure in the behavior recognition task and the reduction of the demand for labeled samples. Behavior recognition belongs to a classification task. It only needs to add a classifier at the end of the pre-training model and use a small amount of labeled data for fine-tuning. The performance measurement indicator is the classification accuracy. The data set contains four types of behaviors: single-person walking, standing, squatting, and multi-person walking. The overall data set is divided into three parts: pre-training data set, training set, and test set. First, the model is pre-trained on an unlabeled pre-training data set, then the model is fine-tuned on a labeled training set, and finally the model performance is evaluated on the test set. The pre-training data set contains 226,008 samples, the training set contains 11,760 samples, and the test set contains 4,557 samples.

[0124] The experimental results show that without using the pre-trained model disclosed in this invention, the behavior recognition accuracy on the test set is 85.211% when training is performed directly on the training set. However, by using the method disclosed in this invention, the model is first pre-trained on the pre-trained data set, then fine-tuned on the training set, and finally tested on the test set, the behavior recognition accuracy is improved to 92.298%, a significant improvement of 7.087%. Further reducing the size of the training set to 50% (5880 samples) for fine-tuning the pre-trained model, the behavior recognition accuracy is 91.703%; reducing the size of the training set to 10% (1176 samples), the accuracy is 89.217%, which is still 4.006% higher than the model trained from scratch. Therefore, the pre-trained model disclosed in this invention significantly improves the accuracy of behavior recognition while reducing the need for labeled data.

[0125] For example, the second embodiment of the present disclosure verifies the performance improvement of the wireless sensing pre-training model obtained in the present disclosure in the 3D posture estimation task and the reduction of the demand for labeled samples. 3D posture estimation is a structural prediction task. It only needs to add a local proposal module at the end of the pre-training model to extract the features of each individual, and then input the individual features into the 3D posture estimation network to complete the posture estimation. The performance measurement indicator is the average joint point error (unit: mm). The data set contains 226,008 samples in the pre-training data set, 47,304 samples in the training set, and 22,192 samples in the test set. The model is first pre-trained on an unlabeled pre-training data set, then fine-tuned on a labeled training set, and finally the model performance is evaluated on the test set.

[0126] The experimental results show that without using the pre-trained model of the present invention, the posture estimation error directly trained on the training set is 134.90 mm. However, using the method of the present invention, pre-training is first performed on the pre-training data set, and then fine-tuning is performed on the training set. The posture estimation error on the final test set is 121.13 mm, and the accuracy is significantly improved by 13.77 mm. If the size of the training set is reduced to 50% (23,652 samples), the posture estimation error after fine-tuning is 126.76 mm; if the size of the training set is reduced to 10% (4,730 samples), the error is 147.28 mm, and the performance of the model after reducing the training set by 50% is still 8.14 mm higher than the model trained from scratch. Therefore, the pre-trained model of the present invention significantly improves the accuracy of 3D posture estimation and reduces the need for labeled data.

[0127] For example, the third embodiment of the present disclosure verifies the performance improvement of the wireless sensing pre-training model obtained in the present disclosure in the task of generating human contours and the reduction of the demand for labeled samples. Human contour generation is a dense prediction task. It only requires adding a decoder module at the end of the pre-training model and fine-tuning it with a small amount of labeled data. The performance measurement indicator is the average intersection over union (IoU). The data set contains 226,008 samples in the pre-training data set, 17,520 samples in the training set, and 4,675 samples in the test set. First, the model is pre-trained on an unlabeled pre-training data set, then the model is fine-tuned on a labeled training set, and finally the model performance is evaluated on the test set.

[0128] The experimental results show that without using the pre-trained model of the present invention, the IoU generated by directly training the human body contour on the training set is 0.699. However, using the method of the present invention, pre-training is first performed on the pre-training data set, and then fine-tuning is performed on the training set. The IoU on the final test set is increased to 0.727, a significant improvement of 0.028. If the size of the training set is reduced to 50% (8760 samples), the IoU after fine-tuning is 0.712; if the size of the training set is reduced to 10% (1752 samples), the IoU is 0.686, which is only slightly lower than the model trained directly from scratch by 0.013. Therefore, the pre-trained model of the present invention significantly improves the accuracy of human body contour generation while reducing the need for labeled data.

[0129] According to the above embodiments of the present disclosure, the effectiveness of the proposed method can be verified by three RF sensing tasks. Experimental results show that the present disclosure significantly improves the performance of various downstream tasks. Even if only 10% of the training data set is used, the performance of the wireless sensing pre-training model based on the present disclosure is still comparable to the model trained from scratch using 100% of the training data set, demonstrating that the present disclosure can significantly reduce the demand for large-scale annotated data sets for downstream task models.

[0130] Figure 8 A block diagram of a training device for a wireless sensing pre-training model according to an embodiment of the present disclosure is schematically shown.

[0131] like Figure 8 As shown, the training device 800 of the wireless sensing pre-training model includes a phase alignment module 810, a first neural network module 820, a first downsampling module 830, a first rearrangement module 840, a second neural network module 850 and a training module 860.

[0132] The phase alignment module 810 is used to perform phase alignment on the original signal information to obtain a sample original feature map of the original signal information.

[0133] The first neural network module 820 is used to input the sample original feature map into the first neural network of the wireless sensing pre-training model to obtain the sample original feature.

[0134] The first down-sampling module 830 is used to perform a first down-sampling on the original feature map of the sample to obtain a plurality of initial local area features of the original feature map of the sample.

[0135] The first rearrangement module 840 is used to rearrange the multiple initial local area features based on the first rearrangement rule to obtain multiple enhanced local area features.

[0136] The second neural network module 850 is used to input the sample enhanced feature map generated by multiple enhanced local area features into the second neural network of the wireless sensing pre-training model to obtain the sample enhanced features.

[0137] The training module 860 is used to construct a similarity loss according to the original features of the samples and the enhanced features of the samples, train the wireless sensing pre-training model, and obtain a trained wireless sensing pre-training model.

[0138] According to an embodiment of the present disclosure, the training device of the wireless sensing pre-training model also includes a second down-sampling module, a second rearrangement module and an original signal data enhancement module.

[0139] The second down-sampling module is used to perform a second down-sampling on the original signal information to obtain a plurality of initial local area information of the original signal information.

[0140] The second rearrangement module is used to rearrange the multiple initial local area information based on the second rearrangement rule to obtain multiple enhanced local area information.

[0141] The original signal data enhancement module is used to generate data-enhanced original signal information according to multiple enhanced local area information.

[0142] According to an embodiment of the present disclosure, the phase alignment module includes a phase difference acquisition unit and a phase alignment unit.

[0143] The phase difference acquisition unit is used to acquire the frequency phase difference of the signal frequency representing the same received signal in the original signal information and the antenna phase difference between different received signals represented in the original signal information.

[0144] The phase alignment unit is used to perform phase alignment on the original signal information according to the frequency phase difference and the antenna phase difference to obtain a sample original feature map of the original signal information.

[0145] According to an embodiment of the present disclosure, the first rearrangement module includes a local consistency rearrangement unit.

[0146] The local consistency rearrangement unit is used to perform matrix multiplication calculation on at least one of the row permutation matrix and the column permutation matrix with the initial local area feature for each initial local area feature, so as to obtain an enhanced local area feature.

[0147] According to an embodiment of the present disclosure, the number of original signal information is multiple. The training module includes a positive sample pair determination unit, a negative sample pair determination unit and a similarity loss construction unit.

[0148] The positive sample pair determination unit is used to determine the positive sample pair according to the sample original feature of the target original signal information and the sample enhanced feature of the target original signal information, wherein the target original signal information represents any one of the multiple original signal information.

[0149] A negative sample pair determination unit is used to determine a negative sample pair according to the sample original features of the target original signal information and the sample enhanced features of other original signal information, wherein the other original signal information represents any original signal information other than the target original signal information among the multiple original signal information. And

[0150] The similarity loss construction unit is used to construct a similarity loss according to a first similarity measurement value of the positive sample pair and a second similarity measurement value of the negative sample pair, so as to train the wireless sensing pre-training model.

[0151] According to an embodiment of the present disclosure, the training module includes a first model parameter adjustment unit and a second model parameter calculation unit.

[0152] The first model parameter adjustment unit is used to adjust the first model parameter in the first neural network according to the tth similarity loss value of the similarity loss during the tth round of training to obtain the first adjusted model parameter.

[0153] The second model parameter calculation unit is used for the parameter calculation unit, which is used to perform momentum summation calculation according to the momentum coefficient, the first adjusted parameter and the second model parameter in the second neural network to obtain the second adjusted model parameter in the second neural network, and the initial parameter information of the first model parameter is the same as the initial parameter information of the second model parameter.

[0154] According to an embodiment of the present disclosure, the training module includes a pre-training model determination unit.

[0155] The pre-trained model determination unit is used to determine a trained wireless sensing pre-trained model according to the trained first neural network.

[0156] Fig. 9 The block diagram of the training device of the wireless perception model according to the embodiment of the present disclosure is schematically shown. The wireless perception model 900 includes a functional module and a first neural network module in the wireless perception pre-training model obtained by training based on the training method of the wireless perception pre-training model. The functional module is arranged after the first neural network module.

[0157] like Fig. 9 As shown, the training device 900 for the wireless sensing model includes a fine-tuning module 910 .

[0158] The fine-tuning module 910 is used to perform fine-tuning training on the first neural network module and the functional module using the labeled sample wireless signals.

[0159] According to an embodiment of the present disclosure, the functional module includes any one of the following: a classifier module, a regression module, and a decoder module.

[0160] According to the embodiments of the present invention, any multiple of the modules and units, or at least part of the functions of any multiple of them, can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules and units can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules and units can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules and units can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0161] For example, any of the phase alignment module 810, the first neural network module 820, the first downsampling module 830, the first rearrangement module 840, the second neural network module 850 and the training module 860, or the fine-tuning module 910 can be combined into one module / unit for implementation, or any one of the modules / units can be split into multiple modules / units. Alternatively, at least part of the functions of one or more of these modules / units can be combined with at least part of the functions of other modules / units and implemented in one module / unit. According to an embodiment of the present disclosure, at least one of the phase alignment module 810, the first neural network module 820, the first down-sampling module 830, the first rearrangement module 840, the second neural network module 850 and the training module 860, or the fine-tuning module 910 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the phase alignment module 810, the first neural network module 820, the first down-sampling module 830, the first rearrangement module 840, the second neural network module 850 and the training module 860, or the fine-tuning module 910 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0162] It should be noted that the training device part of the wireless sensing pre-training model in the embodiment of the present disclosure corresponds to the training method part of the wireless sensing pre-training model in the embodiment of the present disclosure. The description of the training device part of the wireless sensing pre-training model specifically refers to the training method part of the wireless sensing pre-training model, which will not be repeated here.

[0163] The training device part of the wireless perception model in the embodiment of the present disclosure corresponds to the training method part of the wireless perception model in the embodiment of the present disclosure. The description of the training device part of the wireless perception model specifically refers to the training method part of the wireless perception model, which will not be repeated here.

[0164] Fig.10 A block diagram of an electronic device suitable for implementing at least one of the training method of a wireless perception pre-training model and the training method of a wireless perception model of the present disclosure is schematically shown according to an embodiment of the present disclosure. Fig.10 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0165] like Fig.10 As shown, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 to a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0166] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0167] According to an embodiment of the present disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The system 1000 may further include one or more of the following components connected to the input / output (I / O) interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output (I / O) interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0168] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0169] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, at least one of the training methods of the wireless perception pre-training model and the training method of the wireless perception model according to the embodiments of the present disclosure is implemented.

[0170] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0171] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 1002 and / or the RAM 1003 described above and / or one or more memories other than the ROM 1002 and the RAM 1003 .

[0172] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement at least one of the training methods of the wireless perception pre-training model and the training method of the wireless perception model provided by the embodiment of the present disclosure.

[0173] When the computer program is executed by the processor 1001, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0174] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0175] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0176] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0177] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A training method for a wireless sensing pre-training model, comprising: Performing phase alignment on the original signal information to obtain a sample original feature map of the original signal information; Inputting the sample original feature map into the first neural network of the wireless sensing pre-training model to obtain the sample original feature; Performing a first downsampling on the original feature map of the sample to obtain a plurality of initial local area features of the original feature map of the sample; Based on the first rearrangement rule, rearrange the plurality of initial local area features respectively to obtain a plurality of enhanced local area features; and Inputting a sample enhanced feature map generated by a plurality of the enhanced local area features into a second neural network of the wireless sensing pre-training model to obtain a sample enhanced feature; A similarity loss is constructed according to the original features of the samples and the enhanced features of the samples, and the wireless sensing pre-training model is trained to obtain a trained wireless sensing pre-training model.

2. The method according to claim 1, further comprising: Before the phase alignment is performed on the original signal information, Performing a second down-sampling on the original signal information to obtain a plurality of initial local area information of the original signal information; Based on the second rearrangement rule, the plurality of initial local area information are rearranged to obtain a plurality of enhanced local area information; as well as According to the plurality of enhanced local area information, data-enhanced original signal information is generated.

3. The method according to claim 1, wherein: The performing phase alignment on the original signal information to obtain the sample original feature map of the original signal information comprises: Acquire a frequency point phase difference of a signal frequency point representing a same received signal in the original signal information and an antenna phase difference between different received signals represented in the original signal information; The original signal information is phase aligned according to the frequency point phase difference and the antenna phase difference to obtain a sample original feature map of the original signal information.

4. The method according to claim 1, wherein: The step of rearranging the plurality of initial local area features to obtain a plurality of enhanced local area features comprises: For each of the initial local area features, matrix multiplication calculation is performed on at least one of a row permutation matrix and a column permutation matrix and the initial local area feature to obtain the enhanced local area feature.

5. The method according to claim 1, wherein: The number of the original signal information is multiple; the similarity loss is constructed according to the original features of the samples and the enhanced features of the samples, and the wireless sensing pre-training model is trained, including: Determine a positive sample pair according to a sample original feature of target original signal information and a sample enhanced feature of the target original signal information, wherein the target original signal information represents any one of the plurality of original signal information; Determining a negative sample pair according to the sample original features of the target original signal information and the sample enhanced features of other original signal information, wherein the other original signal information represents any one of the multiple original signal information except the target original signal information; and The similarity loss is constructed according to the first similarity measurement value of the positive sample pair and the second similarity measurement value of the negative sample pair, and the wireless sensing pre-training model is trained.

6. The method according to claim 1 or 5, wherein: The constructing a similarity loss according to the original features of the sample and the enhanced features of the sample, and training the wireless sensing pre-training model comprises: During the t-th round of training, adjusting the first model parameters in the first neural network according to the t-th similarity loss value of the similarity loss to obtain first adjusted model parameters; and A momentum summation calculation is performed based on the momentum coefficient, the first adjusted parameters and the second model parameters in the second neural network to obtain the second adjusted model parameters in the second neural network, and the initial parameter information of the first model parameters is the same as the initial parameter information of the second model parameters.

7. The method according to claim 1, wherein: The step of training the wireless sensing pre-training model to obtain a trained wireless sensing pre-training model includes: According to the trained first neural network, a trained wireless sensing pre-training model is determined.

8. A method for training a wireless sensing model, wherein: The wireless sensing model includes a functional module and a first neural network module in a wireless sensing pre-training model obtained by training based on the method according to any one of claims 1 to 7, wherein the functional module is arranged after the first neural network module, and the method includes: The first neural network module and the functional module are fine-tuned and trained using labeled sample wireless signals.

9. The method according to claim 8, wherein: The functional module includes any one of the following: a classifier module, a regression module and a decoder module.

10. A training device for a wireless sensing pre-training model, comprising: A phase alignment module, used to perform phase alignment on the original signal information to obtain a sample original feature map of the original signal information; A first neural network module, used for inputting the sample original feature map into a first neural network of a wireless sensing pre-training model to obtain the sample original feature; A first downsampling module, used for performing a first downsampling on the original feature map of the sample to obtain a plurality of initial local area features of the original feature map of the sample; A first rearrangement module, configured to rearrange the plurality of initial local region features based on a first rearrangement rule to obtain a plurality of enhanced local region features; and A second neural network module is used to input a sample enhancement feature map generated by a plurality of the enhanced local area features into a second neural network of the wireless sensing pre-training model to obtain a sample enhancement feature; The training module is used to construct a similarity loss according to the original features of the samples and the enhanced features of the samples, train the wireless sensing pre-training model, and obtain a trained wireless sensing pre-training model.

Images