FTTR-based environmental sensing methods, devices, media, and software products

Abstract

This application discloses an environment sensing method, device, medium, and program product based on FTTR, belonging to the field of communication technology. The method includes: acquiring a multi-link channel state information (CSI) tensor, which includes CSI data from at least two FTTR sources corresponding to the gateways within a target area during a target time window; performing singular value decomposition on the multi-link CSI tensor to determine the noise subspace basis of the link dimensions; separating crystal oscillator noise from the multi-link CSI tensor based on the noise subspace basis of the link dimensions to obtain a motion tensor of the sensed object; and determining the environmental state information of the target area based on the motion tensor of the sensed object. This approach reduces the dependence of the sensing process on the motion of the sensed object, enabling environmental structure sensing in static scenes, thereby effectively improving the stability and robustness of environmental sensing.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to an environment sensing method, device, medium and program product based on FTTR. Background Technology

[0002] For asynchronous WiFi sensing systems in a distributed Fiber to the Room (FTTR) architecture, related technologies mainly rely on Channel State Information (CSI) collected by a single Access Point (AP), using amplitude changes, phase transitions, or machine learning methods to identify human activity. However, these methods overly depend on the dynamic changes in CSI caused by human movement, making it difficult to effectively detect static environmental structural changes such as furniture movement and the opening and closing of doors and windows. Summary of the Invention

[0003] This application provides an environmental sensing method, device, medium, and program product based on FTTR, to at least solve the problem that related environmental sensing methods are difficult to effectively sense changes in static environmental structures.

[0004] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide an environment perception method based on FTTR, comprising: acquiring a multi-link channel state information (CSI) tensor, wherein the multi-link CSI tensor includes CSI data of at least two FTTR gateway corresponding links within a target area in a target time window; performing singular value decomposition on the multi-link CSI tensor to determine the noise subspace basis of the link dimension; separating crystal oscillator noise from the multi-link CSI tensor based on the noise subspace basis of the link dimension to obtain a motion tensor of the sensing object; and determining the environmental state information of the target area based on the motion tensor of the sensing object.

[0005] In a second aspect, embodiments of this application provide an electronic device, which includes a processor and a memory. The memory stores programs or instructions that can run on the processor, and when the programs or instructions are executed by the processor, they implement the steps of the method described in the first aspect above.

[0006] Thirdly, embodiments of this application provide a computer-readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect above.

[0007] Fourthly, embodiments of this application provide a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the steps of the method described in the first aspect above.

[0008] In this embodiment, a multi-link channel state information (CSI) tensor is obtained. This multi-link CSI tensor includes CSI data of at least two FTTR slave gateways within the target area during the target time window. Singular value decomposition is performed on the multi-link CSI tensor to determine the noise subspace basis of the link dimension. Based on the noise subspace basis of the link dimension, crystal oscillator noise is separated from the multi-link CSI tensor to obtain the motion tensor of the sensed object. Based on the motion tensor of the sensed object, the environmental state information of the target area is determined. This embodiment utilizes the physical characteristic that at least two FTTR slave gateways share the clock of the FTTR master gateway. The noise subspace basis of the link dimension achieves adversarial decoupling between crystal oscillator noise and the motion tensor of the sensed object. Crystal oscillator noise is used as a carrier of environmental structure, reducing the dependence of the sensing process on the motion of the sensed object, enabling environmental structure perception in static scenes, thereby effectively improving the stability and robustness of environmental perception.

[0009] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0011] Figure 1 This application illustrates a network architecture diagram of a distributed FTTR whole-house WiFi system provided in some embodiments. Figure 2 The flowchart of an environment perception method based on FTTR provided in some embodiments of this application is shown; Figure 3 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0012] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0013] Currently, environmental sensing technology mainly relies on single AP devices to collect CSI (Continuous Sensor Indicator) data, identifying human activity through amplitude changes, phase transitions, or machine learning models. In distributed FTTR (Firmware Transmission Recognition) systems, multiple slave gateways use independent temperature-compensated crystal oscillators, resulting in frequency drift of ±20ppm. This causes asynchronous and non-common-mode phase shifts in the CSI data of each transmission link, leading to a significant decrease in the sensing performance of traditional differential phase methods.

[0014] Related solutions typically treat crystal oscillator noise as interference and employ Kalman filtering and wavelet transform for denoising. However, these methods struggle to distinguish crystal oscillator noise from the low-rank structural features of human perception signals, leading to a high false alarm rate. Furthermore, traditional deep learning models such as CNN-LSTM often directly input the original CSI sequence without considering the spatial topology information of multiple links, easily causing feature aliasing and limiting the model's generalization ability. In summary, related perception methods still heavily rely on dynamic changes in CSI caused by human movement, making it difficult to effectively identify static environmental structural changes such as furniture movement and the opening and closing of doors and windows.

[0015] To address the aforementioned problems in the environmental perception process, this application provides an environmental perception method based on FTTR. This method utilizes the physical characteristic that at least two FTTR slave gateways share the clock of the FTTR master gateway. It achieves adversarial decoupling between crystal oscillator noise and the motion tensor of the perceived object through the noise subspace basis of the link dimension, and uses crystal oscillator noise as the carrier of environmental structure to reduce the dependence of the perception process on the motion of the perceived object, thereby realizing environmental structure perception in static scenes.

[0016] Please see Figure 1 , Figure 1 The diagram illustrates the network architecture of a distributed FTTR whole-house WiFi system provided in some embodiments of this application. As shown in the figure, the system 100 includes an application layer 110, an edge server 120, an FTTR master gateway 130, and N FTTR slave gateways, where N ≥ 2.

[0017] Application layer 110 serves as the central hub for smart home operations. It receives environmental perception results from the edge server, including environmental status information such as "someone is walking in the living room" and "the bedroom door is open," and triggers device linkage operations such as turning on lights and alarms according to preset linkage strategies.

[0018] Edge server 120 or FTTR master gateway 130 serves as a data aggregation node, used to receive all CSI data collected by FTTR slave gateways, and determine environmental status information based on the CSI data from FTTR slave gateways.

[0019] FTTR main gateway 130, including embedded processor, Wi The Fi 6 baseband chip, fiber optic access interface, and Precision Time Protocol (PTP) clock synchronization module are used for fiber-to-the-home access and serve as a time reference source, providing PTP clock synchronization signals.

[0020] N FTTRs from the gateway, including a low-power control chip and Wi-Fi. The system includes a Fi 6 chip, high-precision crystal oscillators, and fiber optic transceivers. N FTTR slave gateways are distributed across various rooms and interconnected with the FTTR master gateway via fiber optic links. Each slave gateway is equipped with an independent Wi-Fi RF module, supporting the 802.11ax protocol and 2×2 Multiple Input Multiple Output (MIMO) mode, and has a built-in independent high-precision crystal oscillator, such as a TCXO temperature-compensated crystal oscillator.

[0021] All of the aforementioned gateways support CSI acquisition functionality. Through Linux drivers or custom firmware, they output complex CSI in-phase and quadrature (I / Q) raw data on N transceiver links and 30 subcarriers on each link, at a sampling frequency of 100Hz.

[0022] Please see Figure 2 , Figure 2 The diagram illustrates a flowchart of an FTTR-based environment awareness method according to some embodiments of this application. This method can be executed by a gateway device or a server, specifically applied to the aforementioned edge server 120 or FTTR main gateway 130. As shown in the figure, method 200 may include the following steps: Step 210: Obtain the multi-link channel state information (CSI) tensor, which includes CSI data of at least two FTTRs from the corresponding links of the gateway within the target area in the target time window.

[0023] In one exemplary embodiment, a multi-link CSI tensor is acquired within a preset target time window at a sampling rate of 100Hz. The multi-link CSI tensor This includes CSI data from at least two FTTR slave gateway links within the target area. Where N is the total number of links corresponding to at least two FTTR slave gateways; K is the number of subcarriers; and W is the number of frames in the time window. The target time window can be set according to actual needs, for example, W=100 frames.

[0024] It should be noted that if a certain FTTR slave gateway supports 2×2 MIMO (i.e., 2 transmit antennas and 2 receive antennas), then the gateway can simultaneously establish 2 × 2 = 4 independent links. If there are 3 FTTR slave gateways in the target area, and each FTTR slave gateway is 2×2 MIMO, then the total number of links is N = 3 × 4 = 12 links. A three-dimensional complex tensor is constructed from the CSI complex data of N links, K subcarriers, and W time frames. .

[0025] Step 220: Perform singular value decomposition on the multi-link CSI tensor to determine the noise subspace basis of the link dimension.

[0026] Continuing with the above embodiments, the above multi-link CSI tensor... The factors are expanded and singular value decomposed along the spatial link dimension, subcarrier dimension, and time dimension, respectively, to obtain factor matrices corresponding to these dimensions. The link dimension represents different FTTR communication links within the target area, the subcarrier dimension represents different subcarriers in the frequency domain, and the time dimension represents different time frames for acquiring CSI data. Since all slave gateways in the FTTR system share the master gateway clock, the frequency and phase fluctuations (i.e., crystal noise) caused by the crystal oscillators within the slave gateways are strongly correlated across links, thus exhibiting a low-rank structure along the link dimension. The noise subspace basis of the link dimension can be determined based on the factor matrix of the spatial link dimension. For example, singular value decomposition is performed on the factor matrix of the spatial link dimension, and the noise subspace basis of the link dimension is the singular vector corresponding to the target singular value that is greater than a preset threshold among multiple singular values.

[0027] Step 230: Based on the noise subspace basis of the link dimension, the crystal oscillator noise is separated from the multi-link CSI tensor to obtain the motion tensor of the sensed object.

[0028] Among them, the motion tensor of the perceived object is a multidimensional tensor used to characterize the motion features of the perceived object, which includes the human body and various moving objects.

[0029] Continuing with the above embodiments, based on the aforementioned link-dimensional noise subspace basis, crystal oscillator noise can be reconstructed from the multi-link CSI tensor. The crystal oscillator noise is separated from the crystal to obtain the motion tensor of the sensed object.

[0030] In this way, by utilizing the common-mode clock characteristics of at least two FTTRs from the gateway and using crystal oscillator noise as an environmental structure carrier, the motion tensor of the sensed object can be decoupled from the multi-link CSI tensor in a static scene, thereby reducing the dependence on human motion during the sensing process.

[0031] Step 240: Determine the environmental state information of the target area based on the motion tensor of the perceived object.

[0032] Continuing with the above embodiments, the motion tensor of the perceived object can be input into a preset deep learning model or neural network model to obtain environmental state information of the target area, such as "the door is detected to be open" but no one moves, "there is someone in the kitchen" but no one moves, etc.

[0033] In this embodiment, the physical characteristic of at least two FTTR slave gateways sharing the clock of the FTTR master gateway is utilized. The adversarial decoupling of crystal oscillator noise and motion tensor of the sensed object is achieved through the noise subspace basis of the link dimension. The crystal oscillator noise is used as the carrier of the environmental structure, reducing the dependence of the sensing process on the motion of the sensed object. This enables the perception of the environmental structure in a static scene, thereby effectively improving the stability and robustness of environmental perception and providing true "silent perception" capability for whole-house intelligence.

[0034] In some embodiments, obtaining the multi-link channel state information (CSI) tensor in step 210 above includes: Step 211: Obtain CSI data from at least two FTTRs corresponding to the gateway links within the target area. By parsing the CSI data, obtain the I-channel data and Q-channel data of the subcarriers corresponding to each link.

[0035] In an exemplary embodiment, at a sampling rate of 100Hz, CSI data of at least two FTTRs from the corresponding links of the gateway within a preset target time window are acquired. By parsing the CSI data, I-channel data and Q-channel data of the corresponding subcarriers of each link are obtained.

[0036] Step 212: Based on the I-channel data and Q-channel data, determine the absolute amplitude of each frame subcarrier signal within the target time window, as well as the inter-frame phase difference between each frame subcarrier signal.

[0037] Continuing with the above embodiments, based on the I-channel data and Q-channel data of the subcarriers corresponding to each link, the absolute amplitude of the subcarrier signal in each frame within the target time window is determined. For example, for each frame subcarrier corresponding to each link, the sum of the squares of the I-channel data and the Q-channel data is determined as the absolute amplitude, i.e. To eliminate the carrier frequency offset (CFO) caused by the independent crystal oscillators of the FTTR master and slave gateways, the complex conjugate multiplication method is used to extract the inter-frame phase difference between subcarrier signals in each frame. For example, for each link n ∈ [1,N] and each subcarrier k ∈ [1,K], the complex conjugate multiplication between adjacent frames is calculated to obtain the inter-frame differential phase ΔΦ[n,k,t] ∈ [ [π, π], forming a three-dimensional difference phase tensor .

[0038] Step 213: Determine the multi-link CSI tensor based on the absolute amplitude and inter-frame phase difference of each link subcarrier.

[0039] Continuing with the above embodiments, the multi-link CSI tensor is determined based on the absolute amplitude of the link subcarriers and the inter-frame phase difference.

[0040] In this embodiment, by calculating the absolute amplitude and inter-frame phase difference of each frame subcarrier signal and completing the standardized construction of the multi-link CSI tensor, a data foundation is provided for subsequent noise subspace decomposition, adversarial decoupling, and environmental state information perception.

[0041] In some embodiments, step 220 above, performing singular value decomposition on the multi-link CSI tensor to determine the noise subspace basis of the link dimension, includes: Step 221: Decompose the multi-link CSI tensor into multiple factor matrices, including factor matrices representing the link dimension.

[0042] In one exemplary embodiment, a higher-order singular value decomposition (HOSVD) is performed on the multi-link CSI tensor to determine the noise subspace basis of the link dimension. For example, a Tucker decomposition is performed on the multi-link CSI tensor T as follows:

[0043] Where G is the kernel tensor, , , R1 represents the rank-retained matrix for the spatial link dimension, R2 represents the rank-retained matrix for the subcarrier dimension, and R3 represents the rank-retained matrix for the time dimension.

[0044] Step 222: Perform singular value decomposition on the factor matrix of the link dimension, and determine the singular vector corresponding to the target singular value as the basis of the noise subspace; wherein, the target singular value is the singular value among the multiple singular values ​​of the factor matrix of the link dimension that is greater than a preset threshold.

[0045] Continuing with the above embodiments, utilizing the physical characteristic that all slave gateways in an FTTR system share the master gateway's clock, crystal oscillator noise exhibits strong correlation across all links, thus displaying a low-rank structure in the link dimension. This application embodiment selects a factor matrix for the link dimension. Perform singular value decomposition to obtain a left singular vector, a diagonal matrix, and a right singular vector. The elements on the diagonal of the diagonal matrix are singular values. Multiple singular values ​​are arranged in descending order, and a predetermined number of the top-ranked singular values ​​are selected. The singular values ​​are determined as the target singular values. The singular vectors corresponding to the singular values ​​of each target are determined as the basis of the noise subspace. This can be understood as... The singular vectors corresponding to the r1 target singular values ​​are a subset of the singular vectors corresponding to the r1 singular values, representing the link dimension. The number of core factors identified as "common-mode crystal noise" in the data.

[0046] In some embodiments, step 230 above, based on the noise subspace basis of the link dimension, separates the crystal oscillator noise from the multi-link CSI tensor to obtain the motion tensor of the sensed object, including: Step 231: Determine the factor matrix of crystal oscillator noise in the link dimension based on the noise subspace basis of the link dimension.

[0047] In one exemplary embodiment, based on the aforementioned noise subspace basis in the link dimension, a factor matrix of crystal oscillator noise in the link dimension is constructed. .

[0048] Step 232: Generate the crystal oscillator noise tensor based on the factor matrix of the crystal oscillator noise in the link dimension and the factor matrices of other dimensions besides the factor matrix of the link dimension in the multiple factor matrices.

[0049] Continuing with the above embodiments, based on the factor matrix of crystal oscillator noise in the link dimension And factor matrices in multiple factor matrices other than the link dimension factor matrix, such as the subcarrier dimension factor matrix. Factor matrix with time dimension Generate crystal oscillator noise tensor ,in, This is the part of the kernel tensor corresponding to the noise component.

[0050] Step 233: Determine the motion tensor of the sensing object based on the multi-link CSI tensor and the crystal oscillator noise tensor.

[0051] Continuing with the above embodiments, the crystal oscillator noise tensor is subtracted from the multi-link CSI tensor to obtain the motion tensor of the sensed object. The formula is as follows:

[0052] The motion tensor of the sensed object eliminates the nonlinear interference of time-varying crystal oscillator noise and retains only the channel changes caused by the motion of the sensed object.

[0053] In some embodiments, step 240 above, determining the environmental state information of the target area based on the motion tensor of the perceived object, includes: Step 241: Generate spatial dimension features based on the physical quantities in the motion tensor of the perceived object.

[0054] In one exemplary embodiment, the motion tensor of the perceived object is... Expanding along the spatial link dimension yields an N×K×W complex tensor. Multiple physical quantities are extracted from this complex tensor and stacked into channels to generate a spatial dimension feature. This spatial dimension feature can characterize the spatial distribution relationship between at least two FTTRs corresponding to multiple links from the gateway.

[0055] Step 242: Perform feature extraction processing on the spatial dimension features to obtain the frequency domain dimension features.

[0056] Continuing with the above embodiments, preset convolution kernels and pooling kernels are used to perform feature extraction processing on the above spatial dimension features, and the frequency domain characteristics of the motion of the perceived object and changes in the environment are further explored to obtain frequency domain dimension features.

[0057] Step 243: Deep spatial-frequency features are obtained by fusing the frequency domain features and spatial dimension features.

[0058] Continuing with the above embodiments, a cross-domain convolutional kernel (e.g., 3×5) is used to fuse the aforementioned frequency domain features and spatial domain features, capturing the deep correlation of spatial diversity features of multiple gateway nodes, and then extracting deep space-frequency features that characterize the network topology and spatial semantic information within the target area. These deep space-frequency features are hierarchically cascaded spatial and frequency domain features, overcoming the limitations of single-domain feature representation and improving the accuracy of environmental state perception.

[0059] Step 244: Input the deep spatial frequency features into the deep learning model to obtain the environmental state information of the target area.

[0060] Continuing with the above embodiments, unidirectional pooling is used to further compress the frequency domain dimension, converting the feature map of deep spatial-frequency features into a one-dimensional feature vector. This converted one-dimensional feature vector is then input into a pre-trained deep learning model to mine the temporal evolution patterns of perceived object activities and output environmental state information for target areas, such as human activity, furniture movement, and door / window opening / closing. This environmental state information includes, but is not limited to, the activity state of the perceived object (e.g., stationary or moving) and the probability distribution results for room differentiation. The deep learning model described above can employ a Long Short-Term Memory (LSTM) network with Dropout to prevent overfitting, gated recurrent units, residual networks, etc.

[0061] In some embodiments, in step 241 above, the physical quantity includes at least one of a first physical quantity, a second physical quantity, and a third physical quantity; (1) The first physical quantity is the de-environmentalized relative amplitude. The difference between the instantaneous amplitude and the mean amplitude of each frame subcarrier in the motion tensor of the sensed object is determined as the first physical quantity. The mean amplitude is the average of the instantaneous amplitudes of each frame subcarrier. In this embodiment, by subtracting the mean amplitude from the instantaneous amplitude of each frame subcarrier, the fixed channel amplitude caused by static environments such as walls and fixed objects can be eliminated, and multipath interference from the static background environment can be stripped away. Only the dynamic amplitude fluctuation component caused by human activity and object movement is retained, thereby achieving static environment background stripping.

[0062] (2) The second physical quantity is the first-order amplitude difference, which is the difference between the amplitudes of the subcarrier signals of two adjacent frames in the motion tensor of the sensing object. The embodiments of this application can suppress low-frequency slowly varying noise such as crystal oscillator drift and environmental temperature drift by performing differential operations on the amplitudes of the subcarrier signals of adjacent frames, and capture dynamic energy changes caused by micro-movements or macro-movements of personnel.

[0063] (3) The third physical quantity is the anti-disturbance differential phase, which is the inter-frame phase difference between subcarriers in the motion tensor of the sensed object. In this embodiment, the inter-frame phase difference is obtained to cancel at least two constant frequency offset differential phases such as the crystal oscillator frequency offset from the gateway and the common-mode phase offset introduced by link asynchrony.

[0064] In some embodiments, step 242 above involves feature extraction processing of the spatial dimension features to obtain frequency domain dimension features, including: The spatial dimension features of each link are convolved by the target convolution kernel to obtain the convolutional features corresponding to each link; the convolutional features corresponding to each link are pooled by the target pooling kernel to obtain the frequency domain dimension features.

[0065] In an exemplary embodiment, the process of feature extraction of spatial dimension features may include the following steps: (1) First-layer unidirectional receptive field convolution. That is, the target convolution kernel is used to convolve the spatial dimension features of each link. For example, the target convolution kernel is a 1×7 asymmetric convolution kernel. In this way, the signals of different spatial links can be avoided from aliasing too early, so that the network learns only the frequency selective fading features of a single link in the first stage.

[0066] (2) Unidirectional spatial fidelity pooling. This involves using a target pooling kernel to pool the convolutional features corresponding to each link, resulting in frequency domain features. For example, if the target pooling kernel is a 1×4 pooling kernel, dimensionality reduction is only applied to the subcarrier dimension, keeping the number of spatial links unchanged. This method can compress the feature dimension while preserving the physical spatial topology between at least two FTTR gateways.

[0067] In this embodiment, a network structure of "unidirectional receptive field convolution + spatial fidelity pooling + cross-domain fusion" is used to retain the spatial topology of at least two FTTRs from the gateway, avoiding feature aliasing and reducing the number of model parameters.

[0068] In some embodiments, step 240 described above further includes: Step 245: Based on the noise subspace basis, determine the crystal oscillator noise signals of at least two FTTRs from each link corresponding to the gateway.

[0069] In one exemplary embodiment, the crystal oscillator noise tensor can be generated based on the noise subspace basis. ,Although It is a three-dimensional tensor, but since the crystal oscillator offset is a global common mode, the noise is highly correlated in the subcarrier dimension (K). Therefore, in practical engineering implementation, for each slave gateway link i, the noise signal of K subcarriers can be averaged or the master subcarrier can be selected to obtain a single-channel timing noise signal. . The dimension is 1×W, which is a one-dimensional time-series signal with a length of W frames. Each element is a complex number (I / Q value), representing the crystal oscillator noise signal of the i-th link at time t. This crystal oscillator noise signal has been freed from the influence of human motion and only reflects hardware noise and environmental multipath structure.

[0070] Step 246: Determine the noise cross-correlation matrix based on the cross-correlation function values ​​between the crystal oscillator noise signals of each link.

[0071] The noise cross-correlation matrix is ​​composed of the cross-correlation function values ​​between the crystal oscillator noise signals of each link. The diagonal elements in the matrix are used to indicate the noise power of each link, and the off-diagonal elements are used to indicate the degree of noise correlation between links.

[0072] Continuing with the above embodiment, for each pair of links (i,j), calculate the normalized cross-correlation function value:

[0073] in, This represents the pure crystal oscillator noise (complex timing signal) of the i-th link; E[·] represents the conjugate of the noise signal of the j-th link and is shifted to the right by τ frames; E[·] represents the expectation operation, which is used to calculate the average within a finite window in specific applications; τ represents the time offset, which is in time frames and is used to compensate for the small time offset that may exist between the two link noise signals.

[0074] Among all possible time delays τ, find the one that makes The largest τ values ​​form the cross-correlation matrix. : , This represents the maximum similarity between the noise signals of links i and j under optimal time alignment. Its value ranges from [0, 1]. The closer the value is to 1, the more "common source" the noise of the two links is, that is, the more stable the multipath structure of the environment and the more similar the paths are.

[0075] It reflects the correlation strength of noise propagation between links i and j, and its distribution is determined by the set of wireless signal propagation paths determined by the static / semi-static physical environment such as room layout, furniture placement, building materials, and the opening and closing of doors and windows.

[0076] Step 247: Based on the singular value distribution characteristics of the noise cross-correlation matrix, determine the noise cross-correlation entropy, which is used to indicate the environmental structural complexity of the target area.

[0077] Continuing with the above embodiment, singular value decomposition is performed on the noise cross-correlation matrix C to obtain singular values ​​σ1≥σ2≥ ≥σN≥0. Define normalized singular values:

[0078] The noise cross-correlation entropy is then:

[0079] in, The noise cross-correlation entropy is a scalar measure of the environmental structural complexity of the target region. A small noise cross-correlation entropy indicates that the noise correlation among links is consistent and the environmental structure is simple; a large noise cross-correlation entropy indicates that the noise correlation is dispersed and the environmental structure is complex.

[0080] Step 248, in response to determining the environmental change type of the target area as the target change type based on the noise cross-correlation entropy, perform at least one of the following: (1) Regenerate spatial dimension features based on the physical quantities in the motion tensor of the perceived object, and obtain environmental state information of the target area based on the regenerated spatial dimension features. That is, repeat the steps from "generating spatial dimension features based on the physical quantities in the motion tensor of the perceived object" in step 241 to "obtaining environmental state information of the target area" in step 244 above.

[0081] (2) Adjust the model parameters of the deep learning model.

[0082] Continuing with the above embodiments, the structural changes in the target area's environment are determined based on the noise cross-correlation entropy, and the type of environmental change is identified. For example, if the environmental change is a temporary disturbance caused by the rapid passage of a sensing object, the model parameters of the deep learning model remain unchanged; if the environmental change is a permanent change in the environmental structure, such as furniture movement or the opening and closing of doors and windows, incremental model updates or recalibration are triggered; if the environmental change is a slow change, such as seasonal changes or channel temperature drift, a time sliding window mechanism can be introduced to dynamically retain recent steady-state data and iteratively update the environmental reference feature distribution; relying on unsupervised adaptive strategies, the deep learning model spontaneously adapts to gradual environmental changes, maintaining stable sensing performance over the long term.

[0083] When the environmental change type of the target area is determined to be a permanent change in the environmental structure based on the noise cross-correlation entropy, the spatial dimension features can be regenerated based on the physical quantities in the motion tensor of the perceived object. The environmental state information of the target area can be determined based on the regenerated spatial dimension features to update the feature statistics. Alternatively, the model parameters of the deep learning model can be adjusted to adapt the deep learning model to the current environmental structure.

[0084] In this embodiment, the traditional understanding of "noise needs to be eliminated" is overturned. Crystal oscillator noise is used as a carrier of environmental multipath structure. The environmental complexity is quantified by noise cross-correlation entropy, enabling the detection of environmental changes in a static state. Furthermore, the update strategy of the deep learning model is adaptively adjusted according to different types of environmental changes, enabling long-term unsupervised stable operation of the deep learning model.

[0085] In some embodiments, the method for determining the type of environmental change in the target area in step 248 above includes: If the change in the noise cross-correlation entropy of the target time window compared to the noise cross-correlation entropy of the adjacent previous time window is greater than a preset change threshold, it is determined that the environmental structure of the target area has undergone structural changes; based on the temporal characteristics of the change, the type of environmental change in the target area is determined.

[0086] In an exemplary embodiment, the change in entropy between adjacent time points is calculated, that is, the change in the noise cross-correlation entropy of the current target time window compared to the noise cross-correlation entropy of the previous adjacent time window:

[0087] Set a threshold for the amount of change ,like If so, it is determined that the environmental structure of the target area has undergone structural changes.

[0088] Based on the temporal characteristics of the changes, the type of environmental change in the target area is determined. For example, when the change in entropy at adjacent moments experiences an instantaneous spike followed by a recovery, the type of environmental change is a temporary disturbance caused by the rapid passage of a sensed object; when the change in entropy at adjacent moments is a permanent jump, the type of environmental change is a permanent alteration of the environmental structure; and when the change in entropy at adjacent moments is a slow drift, the type of environmental change is a slow change in the environment.

[0089] In this embodiment, utilizing the physical characteristic that at least two FTTR slave gateways share the clock of the FTTR master gateway, adversarial decoupling between crystal oscillator noise and the motion tensor of the sensed object is achieved through the noise subspace basis of the link dimension. Crystal oscillator noise is used as a carrier of environmental structure, reducing the dependence of the sensing process on the motion of the sensed object, thus enabling environmental structure perception in static scenes and effectively improving the stability and robustness of environmental perception. Furthermore, through... By leveraging the temporal characteristics, the deep learning model can be automatically fine-tuned or its parameter distribution updated, thus addressing the issue of "expiration and failure" of related models and enabling long-term unsupervised and stable operation of deep learning models.

[0090] The FTTR-based environment perception method provided in this application can be applied to at least one of the following scenarios: (1) Smart elderly care: fall detection, bed exit alarm, sedentary reminder, etc.; (2) Smart home: automatic light switch, air conditioning linkage, security arming, etc.; (3) Smart Hotel: Unmanned Room Detection, Guest Room Occupancy Statistics, Energy Saving Control, etc.; (4) Smart office: meeting room occupancy identification, workstation usage analysis, etc.; (5) Medical rehabilitation: monitoring of patient microtremors, assessment of rehabilitation movements, etc.; (6) Industrial Internet of Things: equipment vibration monitoring, personnel intrusion detection, etc.; (7) Integration with AR / VR: Dynamically adjust the position of virtual objects based on environmental changes; (8) Integration with 5G+WiFi 7: as the basic sensing module of the 6G "integrated sensing" system.

[0091] Figure 3This diagram illustrates the hardware structure of an electronic device implementing the embodiments of this application. Referring to the diagram, at the hardware level, the electronic device 300 includes a processor 310, and optionally includes an internal bus 320, a network interface 330, and a memory. The memory may include main memory 341, such as high-speed random-access memory (RAM), and may also include non-volatile memory 342, such as at least one disk storage device. Of course, the electronic device 300 may also include other hardware required for other services.

[0092] The processor 310, network interface 330, and memory can be interconnected via an internal bus 320, which can be an Advanced Microcontroller Bus Architecture (AMIC) bus, Wishbone bus, Open Core Protocol (OCP) bus, Avalon bus, etc. The bus can be categorized as an address bus, data bus, control bus, etc. For ease of illustration, only a single bidirectional arrow is used in this diagram, but this does not imply that there is only one bus or one type of bus.

[0093] The memory stores programs. Specifically, the program may include program code, which includes computer operation instructions. The memory may include main memory 341 and non-volatile memory 342, and provides instructions and data to the processor 310.

[0094] Processor 310 reads the corresponding computer program from non-volatile memory 342 into memory and then runs it, forming a device for locating the target user at the logical level. Processor 310 executes the program stored in memory and specifically performs the following: Figure 2 The methods disclosed in the embodiments shown achieve the functions and beneficial effects of the methods described in the preceding method embodiments, and will not be repeated here.

[0095] The above is as stated in this application. Figure 2The methods disclosed in the illustrated embodiments can be applied to or implemented by processor 310. Processor 310 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the hardware or by instructions in software form within processor 310. Processor 310 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0096] The electronic device can also execute the methods described in the preceding method embodiments and achieve the functions and beneficial effects of the methods described in the preceding method embodiments, which will not be repeated here.

[0097] Of course, in addition to software implementation, the electronic device 300 of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0098] This application also proposes a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform... Figure 2 The methods disclosed in the embodiments shown achieve the functions and beneficial effects of the methods described in the preceding method embodiments, and will not be repeated here.

[0099] The computer-readable storage medium mentioned above includes read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc.

[0100] Furthermore, embodiments of this application also provide a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, implement the following process: Figure 2 The methods disclosed in the embodiments shown achieve the functions and beneficial effects of the methods described in the preceding method embodiments, and will not be repeated here.

[0101] The embodiments of this application can be applied to various scenarios of electronic device collaboration or interconnection, including: collaboration and interconnection between mobile phones and laptops / tablets; collaboration and interconnection between mobile terminals and smart TVs / monitors; collaboration and interconnection between mobile phones or tablets and in-vehicle entertainment systems; collaboration and interconnection between mobile terminals and smart conferencing systems, etc. This satisfies users' diverse needs in smart home, smart office, and smart travel scenarios.

[0102] In summary, the above description is merely a preferred embodiment of this application and does not limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0103] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0104] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0105] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0106] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A method for environment perception based on FTTR, characterized in that, include: Obtain a multi-link channel state information (CSI) tensor, wherein the multi-link CSI tensor includes CSI data of at least two FTTRs from the corresponding links of the gateway within the target area in the target time window; Singular value decomposition is performed on the multi-link CSI tensor to determine the noise subspace basis of the link dimension; Based on the noise subspace basis of the link dimension, crystal oscillator noise is separated from the multi-link CSI tensor to obtain the motion tensor of the sensed object; Based on the motion tensor of the perceived object, the environmental state information of the target area is determined.

2. The method according to claim 1, characterized in that, The acquisition of the multi-link channel state information (CSI) tensor includes: Acquire CSI data from at least two FTTRs within the target area from the corresponding links of the gateway, and obtain the I-channel data and Q-channel data of each link's corresponding subcarrier by parsing the CSI data; Based on the I-channel data and the Q-channel data, determine the absolute amplitude of each frame subcarrier signal within the target time window, as well as the inter-frame phase difference between each frame subcarrier signal; The multi-link CSI tensor is determined based on the absolute amplitude of each link subcarrier and the inter-frame phase difference.

3. The method according to claim 1, characterized in that, The step of performing singular value decomposition on the multi-link CSI tensor to determine the noise subspace basis of the link dimension includes: The multi-link CSI tensor is decomposed into multiple factor matrices, including factor matrices of the link dimension. Singular value decomposition is performed on the factor matrix of the link dimension, and the singular vector corresponding to the target singular value is determined as the basis of the noise subspace; wherein, the target singular value is the singular value greater than a preset threshold among the multiple singular values ​​corresponding to the factor matrix of the link dimension.

4. The method according to claim 3, characterized in that, The noise subspace basis based on the link dimension separates crystal oscillator noise from the multi-link CSI tensor to obtain the motion tensor of the sensed object, including: Based on the noise subspace basis of the link dimension, the factor matrix of crystal oscillator noise in the link dimension is determined; Based on the factor matrix of the crystal oscillator noise in the link dimension, and the factor matrices of other dimensions besides the factor matrix of the link dimension in the plurality of factor matrices, a crystal oscillator noise tensor is generated. The motion tensor of the sensed object is determined based on the multi-link CSI tensor and the crystal oscillator noise tensor.

5. The method according to any one of claims 1 to 4, characterized in that, The step of determining the environmental state information of the target region based on the motion tensor of the perceived object includes: Based on the physical quantities in the motion tensor of the perceived object, spatial dimension features are generated; The spatial dimension features are subjected to feature extraction processing to obtain the frequency domain dimension features; By fusing the frequency domain features and the spatial domain features, deep spatial-frequency features are obtained. The deep spatial frequency features are input into a deep learning model to obtain environmental state information of the target region.

6. The method according to claim 5, characterized in that, The physical quantity includes at least one of the first physical quantity, the second physical quantity, and the third physical quantity; The difference between the instantaneous amplitude and the mean amplitude of each frame subcarrier in the motion tensor of the sensed object is determined as the first physical quantity, wherein the mean amplitude is the average value of the instantaneous amplitude of each frame subcarrier. The difference in amplitude between two adjacent subcarrier signals in the motion tensor of the sensed object is determined as the second physical quantity; The inter-frame phase difference between subcarriers in each frame of the motion tensor of the sensed object is determined as the third physical quantity.

7. The method according to claim 5, characterized in that, The step of performing feature extraction processing on the spatial dimension features to obtain frequency domain dimension features includes: The spatial dimension features of each link are convolved by the target convolution kernel to obtain the convolutional features corresponding to each link; The convolutional features corresponding to each link are pooled using a target pooling kernel to obtain frequency domain features.

8. The method according to claim 5, characterized in that, Also includes: Based on the noise subspace basis, the crystal oscillator noise signal of each link corresponding to the at least two FTTRs is determined; The noise cross-correlation matrix is ​​determined based on the cross-correlation function values ​​between the crystal oscillator noise signals of each link; Based on the singular value distribution characteristics of the noise cross-correlation matrix, the noise cross-correlation entropy is determined, which is used to indicate the environmental structural complexity of the target area. In response to determining the environmental change type of the target area as the target change type based on the noise cross-correlation entropy, perform at least one of the following: Based on the physical quantities in the motion tensor of the perceived object, spatial dimension features are regenerated, and environmental state information of the target area is obtained based on the regenerated spatial dimension features. Adjust the model parameters of the deep learning model.

9. The method according to claim 8, characterized in that, The method for determining the type of environmental change in the target area includes: If the change in the noise cross-correlation entropy of the target time window compared to the noise cross-correlation entropy of the adjacent previous time window is greater than a preset change threshold, it is determined that the environmental structure of the target area has undergone a change. Based on the temporal characteristics of the changes, the type of environmental change in the target area is determined.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing programs or instructions that can run on the processor, the programs or instructions being executed by the processor to implement the steps of the method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 9.

12. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the steps of the method as described in any one of claims 1 to 9.

Images