Spatial region layout reconstruction based on radio frequency measurement

Abstract

Certain aspects of the present disclosure provide techniques and apparatus for training and using a machine learning model to predict a layout of a spatial region based on an input data set from samples of a spatial environment. An example method generally includes receiving an input data set including a plurality of samples from a spatial region. Each sample of the plurality of samples typically includes at least channel state information data. A machine learning model is trained to predict a layout of the spatial region based on the input data set. The predicted layout of the spatial region generally includes a plurality of bounding boxes defining different regions of the spatial region.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Greek patent application serial number 20220100405, entitled “Spatial Area Layout Reconstruction Based on Radio Frequency Measurements,” filed on May 17, 2022 and assigned to the assignee of this application, the entire contents of which are hereby incorporated herein by reference. Background Art

[0003] Aspects of the present disclosure relate to using machine learning to estimate the layout of a spatial region based on radio frequency measurements.

[0004] In a wireless communication system, information about the layout of a spatial region in which operations are performed and a position estimate within the spatial environment (e.g., relative to one or more network entities) can be used for various purposes. For example, the layout information and the position estimate can be used to help identify various parameters for subsequent transmissions in the wireless communication system (such as identifying one or more directional beams to be used in communications between a network entity (e.g., a base station) and a user equipment), identifying a beamforming pattern to be applied to allow directionality in signal processing, and the like. In another example, the position estimate can be used to detect devices entering and leaving different areas (e.g., defined based on a radius from a given device). The layout information and the position estimate can also be used for many other purposes, such as emergency management within a spatial region, spatial management, and the like.

[0005] Generally speaking, RF measurements within a spatial region may differ due to various factors within the spatial region. For example, RF interference sources such as interfering network entities may affect RF measurements in some portions of the spatial region. In another example, hard surfaces such as walls, support columns, etc. may introduce variance or noise in RF measurements obtained within the spatial region. Because RF measurements are typically noisy, it may be difficult to accurately estimate the layout of the spatial region based solely on these RF measurements. Summary of the invention

[0006] Certain aspects provide a method for training a machine learning model to predict a layout of a spatial region based on radio frequency measurements. An example method generally includes receiving an input data set including a plurality of samples from a spatial region. Each sample in the plurality of samples generally includes at least channel state information data. The machine learning model is trained to predict a layout of the spatial region based on the input data set. The predicted layout of the spatial region generally includes a plurality of bounding boxes defining different regions of the spatial region.

[0007] Certain aspects provide a method for predicting a layout of a spatial region based on radio frequency measurements. An example method generally includes receiving an input data set including a plurality of samples from the spatial region. Generally, each sample in the plurality of samples includes at least channel state information data. A layout of the spatial region is predicted based on a machine learning model and the received input data set. The layout of the spatial region generally includes a plurality of bounding boxes defining different zones of the spatial region. The predicted layout of the spatial region is output.

[0008] Other aspects provide: a processing system configured to perform the aforementioned methods and those described herein; a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of the processing system, cause the processing system to perform the aforementioned methods and those described herein; a computer program product embodied on a computer-readable storage medium, the computer program product comprising code for performing the aforementioned methods and those further described herein; and a processing system comprising components for performing the aforementioned methods and those further described herein.

[0009] The following description and associated drawings set forth in detail certain illustrative features of the one or more aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings depict example features of certain aspects of the disclosure and therefore are not to be considered limiting of the scope of the disclosure.

[0011] Figure 1 Radio frequency measurements in a spatial region and radio frequency measurements captured during traversal of a path in the spatial region are illustrated.

[0012] Figure 2 Depicted are example representations of points along a path in a region of space in accordance with aspects of the present disclosure.

[0013] Figure 3 Depicted is an example pipeline for predicting a layout of a spatial region based on a multi-dimensional set of samples representing data captured while traversing a path through the spatial region and a set predictor model in accordance with aspects of the present disclosure.

[0014] Figure 4 Depicted are examples of generating a machine learning model for a representation of each sample in a multidimensional set of samples representing data captured while traversing a path through a spatial region in accordance with aspects of the present disclosure.

[0015] Figure 5 Depicted are examples of a machine learning model that generates a bounding box from a representation of a multi-dimensional sample representing data captured while traversing a path through a region of space in accordance with aspects of the present disclosure.

[0016] Figure 6 Depicted is the transformation of bounding boxes representing different portions of a spatial region into a layout of the spatial region in accordance with aspects of the present disclosure.

[0017] Figure 7 Depicted are example visual representations of temporal data used in predicting a layout of a spatial region in accordance with aspects of the present disclosure.

[0018] Figure 8 Depicted are examples of machine learning models that predict layouts of spatial regions based on channel state information and visual representations of temporal data in accordance with aspects of the present disclosure.

[0019] Fig. 9 Depicted are example operations for training a machine learning model to predict a layout of a spatial region based on an input data set of multi-dimensional samples from the spatial region in accordance with aspects of the present disclosure.

[0020] Fig.10 Depicted are example operations for predicting a layout of a spatial region based on a machine learning model and an input data set of multi-dimensional samples from the spatial region in accordance with aspects of the present disclosure.

[0021] Fig.11 Depicted are example implementations of a processing system on which a machine learning model may be trained to predict the layout of a spatial region in accordance with aspects of the present disclosure.

[0022] Fig.12 Depicted are example implementations of a processing system on which a machine learning model can be used to predict a layout of a spatial region in accordance with aspects of the present disclosure.

[0023] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements and features of one aspect may be beneficially incorporated in other aspects without further recitation. DETAILED DESCRIPTION

[0024] Aspects of the present disclosure provide techniques and apparatus for predicting the layout of a spatial region based on measurements obtained while traversing the spatial region.

[0025] Information about the layout of the spatial region can be used for various tasks. For example, the information can be used to determine how the spatial region will be used, to generate virtual reality or extended reality scenes in the spatial region, for traffic management in the spatial region, for location prediction, etc. Location prediction can be a powerful tool to help change the task. For example, a wireless device can use active positioning to predict the location of the wireless device in the spatial environment based on signals received from one or more transmitters (e.g., base stations, gNodeBs, etc.) in the spatial environment. Based on the predicted position, the wireless device can then identify the parameters to be used in wireless communication. For example, position estimation can allow beamforming or beam selection to be performed in a manner that maximizes (or at least increases) the strength of the signaling received by a device (e.g., UE) in a wireless communication system. In another example, position estimation can be used in passive positioning. In passive positioning, the user equipment can use radio frequency measurements to predict the positioning of other devices in the spatial environment. Generally speaking, the positioning of other devices in the spatial environment can be based on disturbances to the wireless signal caused by objects that hinder the direct line of sight path between the receiving device and the sending device.

[0026] Within a spatial environment, a wireless receiver may receive signaling from one or more transmitting devices (such as base stations, access points, repeaters, etc.). Due to various objects in the spatial environment and changes to the signals caused by these objects (e.g., reflections, attenuation, interference, etc.), measurements (such as channel state information (CSI) measurements) may change as the receiving device moves within the spatial environment (e.g., as a user of a mobile device moves within a spatial region). Due to the variance (or noise) present in the signal measurements within the spatial environment, it may be challenging to reconstruct the layout of the spatial region using only the signal measurements. In addition, because a path through the spatial region may allow for many different layouts of the spatial region, information about the path alone may be insufficient for reconstruction of the layout of the spatial region.

[0027] Various aspects of the present disclosure provide techniques that allow signal measurements and other information to be used to predict the layout of a spatial environment. Generally speaking, the layout of a spatial region may include information that identifies discrete portions of the spatial region (e.g., rooms in a building), openings between different portions of the spatial region (e.g., doorways or other open passageways), and the arrangement of these discrete portions of the spatial region. Because signal measurements typically provide some information about the spatial region, and because other information (such as timing information derived from signal measurements and / or the velocity and acceleration of a device that captures radio frequency measurements while traversing a path through the spatial region) may also provide information about the spatial region, various aspects of the present disclosure allow data captured along a path through the spatial region to be used in predicting the layout of the spatial environment. Therefore, in order to predict the layout of a spatial region, a subset of possible points in the spatial region may be sampled, which may reduce the time and amount of data used to predict the layout of the spatial region.

[0028] Example RF measurements in the space region

[0029] Figure 1 Examples of radio frequency measurements in a region of space and captured during traversal of a path 112 in the region of space are illustrated.

[0030] As illustrated, the measurement diagram 100 illustrates the radio frequency measurement at each point in the spatial region. To simplify the illustration, the measurement diagram 100 assumes a spatial region with three rooms separated by walls 102 and 104 and a transmitter 106 located in the upper left corner of the spatial region. However, it should be recognized that the spatial region for which the measurement diagram 100 can be generated may include any number of continuous or non-continuous boundaries (e.g., walls) and any number of transmitters. Within the measurement diagram 100, the signal measurement is strongest in the region close to the transmitter 106, as illustrated by the area of ​​the layout with lower brightness values ​​(or darker colors, if it is colored), and the signal measurement generally becomes weaker with the distance from the transmitter 106, as illustrated by the area with higher brightness values ​​(or brighter colors, if it is colored). This observation is generally consistent with the nature of radio communications, in which the received power of the signal decreases with increasing distance from the sender device (assuming that there is a clear line of sight between the receiving device and the sender device).

[0031] Furthermore, it can be seen that walls 102 and 104 may affect signal measurements captured in different portions of the spatial region represented by measurement graph 100. For example, due to signal attenuation caused by wall 102, signal measurements in first room 107 may be weaker than measurements at the far end (relative to transmitter 106) of second room 109, even though the distance between any point in first room 107 and transmitter 106 may be shorter than the distance between a point at the right side of second room 109 and transmitter 106. Similarly, due to signal attenuation caused by walls 102 and 104 and the distance from transmitter 106, signal measurements in third room 108 may be weaker than measurements in first room 107 or second room 109. Thus, based on the assumption that a single transmitter 106 exists in the spatial region, the layout of the spatial region may be inferred by examining signal measurements captured throughout the spatial region represented by measurement graph 100 and identifying points in the spatial region where signal measurements change significantly relative to the origin associated with the single transmitter 106. Points where the signal measurement changes dramatically may indicate a wall or other boundary separating different portions of the spatial region, while a gradual change in the signal measurement may indicate an increase in distance from the transmitter 106 within the same unobstructed portion of the spatial region.

[0032] Although the radio frequency measurements illustrated in the measurement graph 100 generally provide a lot of information about the layout of the spatial region from which the measurement graph 100 is generated, it may be impractical to obtain measurements at every location within the spatial region. Instead, the device may generate a measurement graph 110 with measurements obtained along a path 112 that traverses the spatial region. As illustrated, the measurements obtained along the path 112 in the measurement graph 110 may be a sparse subset of possible measurements that may be obtained within the spatial region, as illustrated by the measurement graph 100. However, the measurements obtained along the path 112 may include additional information that defines how the path 112 was generated. For example, because a device cannot be in multiple places at the same time, each measurement may be associated with a timestamp indicating when the measurement was obtained. Because each measurement may be associated with a timestamp, the measurements generated along the path 112 may be organized into an ordered set in which the first element in the ordered set corresponds to the earliest measurement (e.g., the measurement at time 0), and the last measurement in the ordered set corresponds to the latest measurement obtained while traversing the path 112. Additionally, information regarding direction of travel, speed, acceleration, and other movement information may be derived from timing information associated with each measurement captured while traversing path 112 and / or from one or more sensors at the device generating these measurements.

[0033] It should be noted that the measurements obtained while traversing path 112 may not include actual positioning information associated with each measurement. For example, in a building, it may be difficult to obtain accurate data from a satellite positioning system (e.g., GPS, GLONASS, GALILEO, etc.) due to the inability to obtain signaling from a sufficient number of satellites. In another example, the mobile device that is collecting these measurements may not have access to external visual data that can help locate the user within the spatial region. Therefore, path 112 may represent a traversal through an undefined spatial region defined according to timestamps and directional acceleration and velocity information, from which the layout of the spatial region can be inferred.

[0034] To generate an input data set that can be used to train a machine learning model to predict the layout of a spatial region, as discussed in further detail below, the measurements obtained while traversing path 112 may be transformed into a set of multidimensional samples, each multidimensional sample in the set representing a discrete measurement obtained while traversing path 112. To allow the set of multidimensional samples to represent measurements obtained while traversing path 112 and preserve information about the order in which these measurements were obtained, the multidimensional samples may therefore include information about the measurements as well as other contextual information associated with the measurements.

[0035] Figure 2 An example multidimensional sample 200 representing measurements obtained at points along a path in a spatial region is illustrated in accordance with aspects of the present disclosure. As illustrated, the multidimensional sample 200 may be a four-dimensional multidimensional sample (e.g., a 4×1 vector) having elements 202, 204, 206, and 208. Element 202 may include a signal measurement obtained at a particular point in the spatial region. The signal measurement may include, for example, a measurement of signal strength (e.g., in decibels or decibel-milliwatts), a signal-to-noise ratio (SNR), a signal-to-interference-plus-noise ratio (SINR), or other measurements indicating the strength or quality of a signal obtained at a point along a path in the spatial region. Element 208 may be associated with a timestamp associated with the measurement in element 202.

[0036] At the same time, elements 204 and 206 can provide additional contextual information about the motion of the device. For example, elements 204 and 206 may include two-dimensional speed and acceleration information. The directional speed and acceleration information may be represented, for example, by the speed and acceleration information on the first axis (dimension) in element 204 and the speed and acceleration information on the second axis (dimension) in element 206. In another example, although not illustrated, element 204 may include two-dimensional speed information, and element 206 may include two-dimensional acceleration information. By including two-dimensional speed and acceleration information in multidimensional sample 200, multidimensional sample set can be used to predict the two-dimensional layout of spatial region, as discussed in further detail below. Generally speaking, speed and acceleration information can be obtained from various sensors on the device, such as accelerometers, gyroscopes, or other motion sensing devices integrated with or connected to the device when traversing the path through the spatial region to obtain measurements.

[0037] Example ensemble predictor model for predicting the layout of a spatial region based on multidimensional samples from the spatial region type

[0038] To predict the layout of a spatial region, aspects of the present disclosure use a set of multidimensional data representations (e.g., vectors) representing measurements obtained along a path through the spatial region as input to a set predictor model that predicts the membership of a set of bounding boxes representing the spatial region based on the set of multidimensional data representations. The set predictor model can be trained (as described herein with respect to Figures 3 to 5 Detailed discussion further) can predict the shapes of bounding boxes (or other bounding polygons or ellipses) that define different portions of the spatial region, and a predicted layout of the spatial region can be generated by matching points that define these bounding boxes.

[0039] Figure 3 An example pipeline 300 for predicting a layout of a spatial region based on a multi-dimensional set of samples representing data captured while traversing a path through the spatial region and a set predictor model in accordance with aspects of the present disclosure is illustrated.

[0040] Generally, a path through a region of space can be traversed (e.g., Figure 1 112) to obtain N measurements. Each measurement can be associated with a specific location in the spatial region and a specific timestamp at which the measurement was taken. Based on the N measurements, the input sequence 310 can be generated as a set of N multidimensional samples {x1, x2, ..., x N}, in some aspects, each sample x is as described above with respect to Figure 2The discussion is formatted. Input sequence 310 may be provided as input to an ensemble predictor model 320, and the ensemble predictor model may generate a series of bounding box coordinates 330 (or coordinates of another bounding shape). For example, bounding box coordinates 330 may include coordinates for each of a plurality of rooms in a spatial region.

[0041] As illustrated, the ensemble predictor model 320 includes a first machine learning model 322 and a second machine learning model 324. The first machine learning model 322 is typically trained to map the input sequence 310 to the input sequence {z1, z2, ..., z N The second machine learning model 324 is trained to use the input sequence {z1, z2, …, z N} as input for generating coordinates of bounding boxes representing different definable spaces (e.g., rooms) in the spatial region. In some aspects, the first machine learning model 322 can be a transformer encoder machine learning model that encodes the input sequence 310 into a representation of the input sequence, and the second machine learning model 324 can be a multi-layer perceptron (MLP) or other neural network that generates coordinates of the bounding box based on the representation of the input sequence generated by the transformer encoder machine learning model.

[0042] Figure 4 An example 400 of using a first machine learning model 322 to generate a representation for each sample in a set of multidimensional samples representing data captured while traversing a path through a spatial region is illustrated in accordance with aspects of the present disclosure. As discussed, the first machine learning model 322 can be a transformer encoder model that encodes an input sequence 310 of multidimensional samples representing measurements in a spatial region into an output sequence 410 of the input sequence 310. In general, the input sequence 310 and the output sequence 410 can include N samples, with each sample in the input sequence 310 of multidimensional samples being mapped to a corresponding representation in the output sequence 410. That is, the dimensions and length of the input sequence 310 can be the same as the dimensions and length of the output sequence 410.

[0043] The first machine learning model 322 may be a transformer encoder constructed as a neural network that is trained to encode samples in the input sequence 310 into representations such as latent space representations based on correlations between sequences of samples in the input sequence 310. For example, the transformer encoder may include one or more self-attention layers that process different sequences from a set of input sequences and a feed-forward network that applies a linear transformation to the input sequence using different parameters to encode each multidimensional sample in the input sequence 310 into a corresponding representation in the output sequence 410, which may be fed as input to the second machine learning model 324 to generate bounding boxes (e.g., bounding box coordinates 330) that define discrete portions of a spatial region and layouts of the spatial region from these bounding boxes.

[0044] To train the first machine learning model 322, supervised learning techniques may be used. The training data set used to train the first machine learning model 322 may include a path (e.g., Figure 1 The multidimensional sample set of measurements obtained when the path 112 illustrated in FIG. 1 is taken is labeled with the coordinates of each of the bounding boxes in the plurality of bounding boxes defined for the spatial region. For example, Figure 6 As illustrated in and described in further detail below, a bounding box may be defined according to coordinates of two opposite corners of the box (e.g., the upper left corner and the lower right corner). In some aspects, where a room may have a non-rectangular shape, a bounding box may be defined according to the coordinates of each vertex in the room, such that an n-sided polygon is defined according to n coordinates.

[0045] Figure 5 An example 500 of generating a bounding box from a representation of a multi-dimensional sample representing data captured while traversing a path through a spatial region using a second machine learning model 324 according to aspects of the present disclosure is illustrated. As illustrated, an input data set (e.g., Figure 4 The representation of each sample in the plurality of multidimensional samples in the output sequence 410 illustrated in can be flattened into a vector 510. Assuming that each sample in the plurality of multidimensional samples has four dimensions (however, it should be recognized that the multidimensional samples may include fewer dimensions or larger dimensions, and the use of four-dimensional samples is merely illustrative), the vector 510 may have a dimension of 1 times (n*4), and each discrete group of four elements represents a specific sample in the plurality of multidimensional samples. That is, elements (i-1)*4 to (i-1)*4+3, i∈{1,…,n} may be associated with the i-th sample in the plurality of multidimensional samples in the input sequence 310. The vector 510 may be input into a second machine learning model 324, which generates a set 520 of bounding boxes representing different discrete portions (e.g., rooms) in a spatial region.

[0046] As discussed, in some aspects, the second machine learning model 324 can be a multilayer perceptron, which is typically a neural network with several layers that globally aggregates information and maps representations of multidimensional samples to coordinates of bounding boxes in a metric space. The second machine learning model 324 can be trained to generate bounding boxes and layouts of these bounding boxes by matching sets of points corresponding to coordinates of bounding boxes representing different discrete regions of the spatial region.

[0047] In some aspects, the second machine learning model can be trained to match the set of points corresponding to the coordinates of the bounding boxes based on minimization of the slope distance between the coordinates of the bounding boxes (serving as the loss function to be minimized) to generate the layout of the spatial region. In general, the slope distance can be an evaluation metric that evaluates the distance between points in different point clouds (in this example, the distance between the coordinates of the bounding boxes in the bounding box set 520 and the coordinates of the bounding boxes in the candidate layout of the spatial region). Given the distance represented as In the case of a bounding box set 520 and a desired layout of the spatial region represented by Y, the slope distance can be expressed by the equation:

[0048]

[0049] in Represents a bounding box collection The coordinates of the i-th sample in , and y j Represents the coordinates of the jth sample in the layout Y generated by matching the set of points of the bounding boxes in the set of bounding boxes 520. In some aspects, the second model may be further trained to minimize the average intersection-over-union (IoU) measure between the bounding boxes. In general, the average IoU measure may represent the ratio of the area of ​​the box where two bounding boxes intersect (e.g., the area where the two bounding boxes overlap) to the area of ​​the union of the two bounding boxes (e.g., the total area enclosed by the two bounding boxes). In general, an IoU measure close to 1 may indicate a large amount of overlap between the two bounding boxes, while an IoU measure close to 0 may indicate a small amount of overlap between the two bounding boxes. Because two discrete areas of a spatial region (e.g., two rooms in a building) cannot overlap each other, minimizing the average IoU measure may allow the second machine learning model to predict layouts of spatial regions that constitute valid layouts of the spatial region.

[0050] In some aspects, the second machine learning model can be trained to match a set of points corresponding to the coordinates of each of the plurality of bounding boxes based on minimization of a Hungarian loss metric corresponding to the coordinates of each of the plurality of bounding boxes. In general, the Hungarian loss metric defines a loss between different point pairs (in this example, the bounding box set The magnitude of the match between the coordinates of the bounding boxes in the image) is such that the second machine learning model 324 optimizes the assignment of the inferred bounding box coordinates to the real data bounding box coordinates. The second machine learning model 324 can be further trained to minimize the average IoU measurement between the bounding boxes.

[0051] Figure 6 The transformation of bounding boxes representing different portions of a spatial region into a layout of the spatial region is illustrated in accordance with aspects of the present disclosure. As discussed, Figure 3 The second machine learning model 324 illustrated in generates a bounding box set 610 including bounding boxes 620, 630, and 640. Bounding box 620 is defined relative to points 622 and 624, which represent two opposite corners of bounding box 620. Similarly, bounding box 630 is defined relative to points 632 and 634, and bounding box 640 is defined relative to points 642 and 644.

[0052] Based on the techniques discussed above (e.g., minimization of slope distance, minimization of Hungarian loss, minimization of average IoU, etc.), Figure 3 The ensemble predictor model 320 illustrated in FIG. 6 generates a layout 650 by matching sets of points for bounding boxes 620, 630, and 640. In this example, the points defining bounding box 620 may be translated to points 622' and 624' in two-dimensional space. At the same time, bounding box 630 may be defined according to translated points 632' and 634', with point 632' being positioned so that the upper left corner of bounding box 630 matches the lower left corner of bounding box 620. Finally, bounding box 640 may be defined according to translated points 642' and 644'. When bounding box 640 is moved to its position in layout 650, the upper right corner of bounding box 640 matches with point 624' defining the lower right corner of bounding box 620, the upper left corner of bounding box 640 defined by point 642' matches with the upper right corner of bounding box 630, and the lower left corner of bounding box 640 matches with the lower right corner of bounding box 630 defined by point 634'.

[0053] In some aspects, a multidimensional sample set captured by traversing a path in a spatial region may be associated with any one of a plurality of candidate layouts having bounding boxes of different sizes and different layouts of the bounding boxes. Because the multidimensional sample set may be associated with any one of the plurality of candidate layouts, the ensemble predictor model may be trained as a probabilistic model in which a layout is predicted based on a predicted layout distribution in the spatial region for any given input data set. In this case, the ensemble predictor model may predict a layout of the spatial region in which the multidimensional sample set is captured based on a joint distribution over parameters of the ensemble predictor model and the layout distribution.

[0054] The joint distribution can be defined according to the equation:

[0055] p(w,D)=p(D|w)p(w)

[0056] Where w represents the model parameters, D represents the distribution over the layout range, p represents the probability, and p(D|w) represents the probability distribution conditional on a specific set of model parameters. The posterior distribution over the weights of the ensemble predictor model can be expressed by the equation:

[0057]

[0058] The posterior term ∫f(w)p(w|D)dw can be used to predict which of multiple layouts is a possible layout for a given input of multi-dimensional samples captured by traversing a path through an unknown spatial region. However, because finding the true posterior p(w|D) is an intractable problem, an approximate probability distribution q can be defined θ (w), and can be based on the approximate probability distribution q θ (w) to optimize the posterior distribution. When predicting the posterior distribution, the KL divergence measure can be defined according to the equation:

[0059]

[0060] as well as

[0061]

[0062] Therefore, the KL divergence measure can be expressed according to the equation:

[0063]

[0064] The resulting probability distribution may be made "peaky" to reduce the level of uncertainty in the predictions of the layout of the spatial region generated by the ensemble predictor model. In some cases, where there is some uncertainty about the appropriate layout for a given path through the spatial region as input, the resulting probability distribution may be flat or have multiple peaks.

[0065] It should be noted that many spatial regions may be laid out with openings in walls that may affect the signal measurements obtained within these spatial regions. For example, while walls may attenuate signals, openings such as doors may provide a clear line of sight for a transmitter to send signaling to a receiving device. The ensemble predictor model discussed herein may be trained with data from these environments to predict the layout of such spatial regions, including openings between different bounding boxes representing doors or other interruptions in barriers between discrete portions of the spatial region. In order to identify openings in these spatial regions, a machine learning model may be trained using a training data set that includes points representing the vertices of the bounding boxes defining the room and the locations of the openings between different discrete portions of the spatial region. Because these openings may not attenuate signals in the way that walls or other barriers would attenuate signals, the model may identify these openings based at least in part on identifying local spikes in signal measurements (e.g., local spikes in received power, SNR, SINR, etc.), which are defined by signal measurements indicating a degraded signal quality (e.g., a decrease in received power, SNR, SINR, etc.) relative to the measurement associated with the local spike.

[0066] Example transformer model predicting the layout of a spatial region based on signal measurement samples from the spatial region

[0067] In general, a spatial region may include multiple discrete portions (e.g., rooms) and passages (e.g., doors) between different portions of the spatial region. To predict the layout of a spatial region (including these passages between different portions of the spatial region), aspects of the present disclosure may use signal measurements (such as channel state information (CSI) measurements) and temporal data derived therefrom as input to a machine learning model trained to generate a predicted layout of a spatial region.

[0068] The channel state information data may be a complex input including multiple components, such as phase sine, phase cosine, and amplitude components. For a wireless device having four antennas measuring 64 complex subtones from a received signal, each channel state information sample (which includes the aforementioned phase and amplitude components for each antenna and each subtone) may therefore produce a 4×64×3 tensor. Although the channel state information may typically have a timestamp assigned to the channel state information based on the time at which the signal on which the channel state information was generated was received, the mapping from the data in the frequency domain to the time domain may include multiple peaks corresponding to different arrival times of different multipath components caused by reflections, diffraction, etc. However, due to bandwidth limitations and the computational power available on the hardware, these peaks may be blurred and shifted, so that some peaks (corresponding to the arrival times of some multipath components) merge with other more dominant peaks. For example, the signal based on which the channel state information sample is generated may include a line-of-sight component that arrives from the transmitter at a time defined by the line-of-sight distance between the transmitter and the wireless device and the speed of light. However, non-line-of-sight components may randomly shift the arrival times associated with the signals based on which the channel state information samples are generated; therefore, the arrival time data may not provide reliable, accurate data that can be used by a machine learning model to predict the layout of the spatial region in which the wireless device operates.

[0069] However, time difference of arrival (TDoA) data may provide useful information about signals that have arrived at a wireless device or about the same signal arriving at different antennas at the wireless device. However, because the number of arriving signals may differ between different antennas, and because it may be impractical to identify whether a signal is a line-of-sight signal or a multipath component, aspects of the present disclosure derive TDoA data from channel state information samples by calculating a set of possible time differences between different multipath components or by using pseudo TDoA data derived from channel state information samples by calculating a set of possible products between different multipath components.

[0070] Figure 7 Illustrated is the generation of an example visual representation of temporal data for use in predicting a layout of a spatial region in accordance with aspects of the present disclosure.

[0071] In order to generate a visual representation 740 of TDoA data for a received signal (the visual representation may be a visual graph or other graphical data representing correlation between received signals at different antennas of a wireless device), the signal 710 may be transformed into time delay information 720 and 730 based on an inverse Fourier transform of the signal 710. In general, the signal 710 may be represented by a signal strength measured over a time window, and the time delay information 720 and 730 may be generated as a matrix including a plurality of time intervals (bins) (e.g., a number of intervals represented by num_bins). In one example, the time delay information 720 may be the result of applying an inverse Fourier transform to the signal 710, and the time delay information 730 may be the result of transposing the time delay information 720 (e.g., rotating the time delay information 720 by 90 degrees so that a matrix having a dimension of 1×num_bins is transformed into a matrix having a dimension of num_bins×1).

[0072] A visual representation 740 of TDoA data for received signals may be generated based on an interval-by-interval combination of the time delay information 720 and the time delay information 730. For example, the visual representation 740 of TDoA data may be generated by subtracting the time delay information 730 from the time delay information 720 for each interval in a matrix representing the time delay information 730 and the time delay information 720. As illustrated, the resulting visual representation 740 of the TDoA information may be an image constructed as a two-dimensional matrix having dimensions num_bins×num_bins and includes information identifying timing correlations between signals received by different antennas at the wireless device. In some aspects, the visual representation 740 may include a set of values ​​represented by high brightness values ​​or bright colors indicating possible pseudo TDoA values ​​for signals received by different antennas at the wireless device.

[0073] To generate TDoA data that can be used by a machine learning model to predict the layout of a spatial region, as discussed in further detail herein, the TDoA data can be generated as a discrete visual representation for each antenna in the wireless device. Thus, for a device having n antennas, the visual representation of the TDoA information can be a stack of n visual representations of the TDoA information, each individual visual representation being associated with a specific one of the n antennas in the wireless device.

[0074] Figure 8 Depicted is an example of a machine learning model 800 for predicting a layout of a spatial region based on channel state information and a visual representation of temporal data in accordance with aspects of the present disclosure.

[0075] As illustrated, the machine learning model 800 includes a channel state information branch 810 and a time information branch 820. As illustrated, the channel state information branch 810 of the machine learning model 800 includes a multilayer perceptron 812, a transformer neural network 814, and an activation function 816 that generates an intermediate output from the channel state information received as an input to the machine learning model 800. The multilayer perceptron 812 generally transforms the channel state information input into the machine learning model 800 into a representation of a different dimension. For example, channel state information samples on 64 sub-tones using a number of antennas a and a number of components c (e.g., 4 antennas and 3 components, as discussed above) can be transformed into 64-dimensional samples, which can be provided as input to the transformer neural network 814.

[0076] The transformer neural network 814 may include multiple layers and include an activation function, such as a rectified linear unit (ReLU), a Gaussian error linear unit (GELU), a continuous differential exponential unit (CELU), etc., to generate a representation of the input channel state information. The generated representation may be fed as an input to the activation function 816, which generates an activated representation for each sample of the channel state information input into the machine learning model 800.

[0077] At the same time, the time information branch 820 uses the received channel state information to generate an attention output from the time data derived from the channel state information. As illustrated, the time information branch 820 includes a time data exporter 822, a visual transformer neural network 824, and an activation function 826. The generation of the attention output by the visual transformer neural network 824 and the activation function 826 can be performed using one or more instances of the visual transformer neural network 824 and the activation function 826. To generate the attention output, the time data exporter 822 can first generate multiple visual representations of the time information (e.g., images representing the TDoA relationship between different signals) from the channel state information received as input to the machine learning model 800. As discussed, to generate these visual representations of the time information from the channel state information, the channel state information samples represented as received signal strength over a period of time can be transformed into a one-dimensional timing information matrix using an inverse Fourier transform. The generated one-dimensional timing information matrix may be transposed to generate a second timing information matrix, and the generated one-dimensional timing information matrix and the second timing information matrix may be combined to generate a two-dimensional visual representation of time data (e.g., TDoA data) for the antennas of the wireless device. In general, the time data exporter 822 may generate a visual representation of the time data for each antenna, and thus generate a stack of a visual representations of the time data for a wireless device having a number of antennas.

[0078] The sequence of channel state information data provided as input to the machine learning model 800 can therefore be represented as a sequence of visual representations of temporal data for the wireless device. The sequence can be input into a visual transformer neural network 824 to generate an output, which can include a temporal attention block trained to identify temporal relationships between different components. The attention output generated by the visual transformer neural network 824 can then be fed as an input to an activation function 826, which generates an attention output from the temporal data derived from the channel state information data. Similar to the activation function 816, the activation function 826 can be a Gaussian error linear unit that implements a function that multiplies the input by the cumulative density function of the normal distribution at the input.

[0079] The activated representation of the channel state information generated by the channel state information branch 810 and input into the machine learning model 800 may be averaged at the channel state information averaging block 818. The attention output generated by the time information branch 820 may be averaged at the time information averaging block 828, or may be generated based on an input sequence including a special marker. The averaged activated representation of the channel state information or the output of the channel state information branch 818 based on a combination of the channel state information sequence and the special vector and the averaged attention output for the channel state information may be combined at the combiner block 830 and further processed using the global activation layer 832 to generate an activated combined representation of the channel state information and the time data derived from the channel state information.

[0080] The post-activation combined representation may be provided as input to a first regression head 834 that predicts the number of rooms and openings between the rooms in the spatial region and a second regression head 836 that predicts the coordinates of the rooms and openings between the rooms in the spatial region. The first regression head 834 may be a linear unit trained to generate one-dimensional scalar data from a higher-dimensional post-activation combined representation. The second regression head 836 may also be a linear unit; however, to identify the coordinates of the rooms and the openings between the rooms, the second regression head 836 may generate a plurality of coordinate outputs defining the locations of the rooms and the openings between the rooms. For example, as discussed above, a room may be defined as a set of points, such as diagonal points (e.g., an upper left point and a lower right point) on a two-dimensional plane for a regular four-sided room (e.g., a room shaped as a rectangle). Similarly, the location of an opening from one room to another room may be defined as a pair of points in a two-dimensional plane. The pair of points may generally be located on a polygon or other shape representing the boundary of the room, or at least close to the polygon or other shape.

[0081] In some aspects, the predicted layout of the spatial region defined by the outputs of the first regression head 834 and the second regression head 836 can be further refined in post-processing operations on the outputs. For example, these post-processed outputs can refine the predicted layout of the spatial region by minimizing (or at least reducing) free space in the predicted layout, minimizing (or at least reducing) overlap between rooms, moving openings to the boundaries of rooms, etc.

[0082] Example operations for predicting a layout of a spatial region based on samples from the spatial region

[0083] Fig. 9 The example of a method that can be used by a computing device (e.g., Fig.11 The system 1100 illustrated in FIG. 11A performs example operations 900 to train a machine learning model to predict a layout of a spatial region based on an input data set of samples from the spatial region. The computing device may be, for example, a server, a server cluster, or other computing device that can use a set of samples and a layout of a spatial region in which these samples are captured to train a machine learning model to predict the layout of a spatial region.

[0084] As illustrated, operation 900 begins at box 910, receiving an input data set. Generally speaking, the input data set includes multiple samples from a spatial region. Each sample may include at least channel state information data. In some aspects, the sample may also include positioning data and time data. In some aspects, the channel state information data may include power measurements, such as signal strength measured at a given position and time in three-dimensional space (e.g., in decibels or decibel-milliwatts), calculated SNR, calculated SINR, etc. In some aspects, the positioning data may include acceleration data and velocity data generated by one or more motion sensors on a device that captures power measurements (such as accelerometers, gyroscopes, compasses, or other motion sensors or positioning sensors integrated or coupled with the device). The acceleration data and velocity data may include directional acceleration and velocity for each dimension in multiple dimensions. For example, in a two-dimensional mapping scenario, the acceleration and velocity data may include acceleration and velocity data for a forward-backward axis and for a lateral (left-right) axis. However, in a three-dimensional mapping scenario, the acceleration and velocity data may include acceleration and velocity data for a forward-backward axis, a lateral axis, and a vertical axis.

[0085] At block 920, operations 900 continue with training a machine learning model to predict a layout of a spatial region based on an input data set. Generally, the predicted layout of the spatial region includes a plurality of bounding boxes defining different regions of the spatial region.

[0086] In some aspects, training the machine learning model includes training a first machine learning model (e.g., Figure 8 ), and a second machine learning model (e.g., a channel state information branch 810 illustrated in FIG. 8 ) trained to generate attention outputs from temporal data derived from the channel state information data. Figure 8 , or the time information branch 820 illustrated in FIG.

[0087] In some aspects, training the machine learning model may further include training the machine learning model to generate an activated combined average based on applying an activation function to a combination of the time-averaged output of the first model and the time-averaged output of the second model.

[0088] In some aspects, training the machine learning model may further include training the machine learning model to output information defining a predicted layout of the spatial region based on the plurality of regression heads and the post-activation combined average.

[0089] In some aspects, the temporal data derived from the channel state information can be a visual representation of TDoA information for each of a plurality of antennas of the wireless device associated with the channel state information data for each channel state information sample in the input data set. The first machine learning model can be a transformer neural network, and the second machine learning model can include any number of instances of a visual transformer and a multi-headed attention block that generates an attention output from the temporal data based on the visual representation of the TDoA information for each of the plurality of antennas. In some aspects, a visual representation of the TDoA information for each of the plurality of antennas can be generated based on an inverse Fourier transform of the channel state information data and a transposed version of the inverse Fourier transform of the channel state information data.

[0090] In some aspects, the predicted layout of the spatial region also includes one or more of: a predicted number of zones in the spatial region, a predicted number of openings between zones in the spatial region, predicted coordinates of each zone in the spatial region, and predicted coordinates of each opening between zones in the spatial region.

[0091] In some aspects, the plurality of samples may include multi-dimensional samples including channel state information data, positioning data, and time data for each sample in the input data set. Training the machine learning model may generally include training an ensemble predictor model including a first machine learning model and a second machine learning model. The first machine learning model (e.g., Figure 3 The first machine learning model 322 illustrated in FIG. 3 may be trained to generate a plurality of multidimensional samples (e.g., based on Figure 3 The multi-dimensional sample input sequence 310 is generated by Figure 4 The second machine learning model (e.g., Figure 3The second machine learning model 324 illustrated in FIG. 3 can be trained to generate bounding boxes for discrete portions of the spatial region and a layout of the spatial region based on a representation of each sample in the plurality of multidimensional samples.

[0092] In some aspects, the first model may be a transformer encoder.Training the transformer encoder may include training the transformer encoder to generate an output sequence from an input data set that identifies local correlations within a sequence of samples in the data set.

[0093] In some aspects, training the second machine learning model may include training the model to generate bounding boxes and layouts of spatial regions from these bounding boxes. Each bounding box can generally be defined as a set of coordinates in a multidimensional space.

[0094] In some aspects, to generate a layout of the spatial area, training the second machine learning model may also include training the second machine learning model to match a set of points corresponding to the coordinates of the bounding boxes based on minimization of a slope distance between the coordinates of the bounding boxes. The second model may be further trained to minimize an average intersection-over-union (IoU) measure between the bounding boxes. As discussed, by minimizing the average IoU measure between the bounding boxes, a layout may be generated that identifies the physical impossibility of two discrete rooms overlapping each other in the spatial environment.

[0095] In some aspects, training the second machine learning model may further include training the second machine learning model to match a set of points corresponding to the coordinates of the bounding box based on minimization of a Hungarian loss metric. The second machine learning model may further be trained to minimize an average IoU measure between the bounding boxes.

[0096] In some aspects, training the machine learning model includes training the machine learning model to predict a layout of a spatial region based on a predicted layout distribution in the spatial region for an input data set. The ensemble predictor model may be trained to predict a distribution of layouts based on a joint distribution over parameters of the ensemble predictor model and the predicted layout distribution. In some aspects, the ensemble predictor model may be trained to predict a posterior distribution over weights of the ensemble predictor model. Because computing a posterior distribution may be a thorny problem, the posterior distribution may be approximated based on a Kullback-Leibler (KL) divergence measure of an approximate probability distribution over weights of the ensemble predictor model.

[0097] In some aspects, the machine learning model can be trained to generate bounding boxes with non-continuous coordinates. Generally speaking, these non-continuous coordinates can represent openings between different discrete parts of a spatial region, such as doors between rooms, open passages between rooms, etc.

[0098] Fig.10 The example of a method that can be used by a computing device (e.g., Fig.12The system 1200 illustrated in FIG. 1 performs example operations 1000 to predict a layout of a spatial region based on a machine learning model and an input data set of samples from the spatial region. The computing device may include, for example, a mobile phone, a tablet computer, a laptop computer, or other device that can capture signal measurements from multiple locations within a spatial environment and use the signal measurements and motion information to generate multi-dimensional samples that can be used by the machine learning model to predict the layout of the spatial region.

[0099] As illustrated, operation 1000 begins at block 1010, receiving an input data set including multiple samples from a spatial region. As discussed, each of the multiple samples from the spatial region typically includes at least channel state information data. In some aspects, each sample may be a multidimensional sample including channel state information, positioning data, and time data. In some aspects, the channel state information data may include power measurements, such as signal strength (e.g., in decibels) measured at a given position and time in three-dimensional space, calculated SNR, calculated SINR, etc. In some aspects, the positioning data may include directional acceleration data and velocity data. The acceleration data and velocity data may include acceleration and velocity for each dimension in multiple dimensions. For example, in a two-dimensional mapping scenario, the acceleration and velocity data may include acceleration and velocity data for the forward-backward axis and for the lateral (left-right) axis. However, in a three-dimensional mapping scenario, the acceleration and velocity data may include acceleration and velocity data for the forward-backward axis, the lateral axis, and the vertical axis.

[0100] Operations 1000 proceed to block 1020 where a layout of the spatial region is predicted based on the machine learning model and the received input data set. Generally, the predicted layout of the spatial region includes a plurality of bounding boxes defining different regions of the spatial region.

[0101] In some aspects, predicting a layout of a spatial region includes predicting a layout of a spatial region based on a first machine learning model (e.g., Figure 8 The channel state information branch 810 illustrated in FIG. 8 is an intermediate output generated from the channel state information data and by a second machine learning model (e.g., Figure 8 The temporal information branch 820 illustrated in FIG. 8 predicts the layout from the attention output generated from the temporal data derived from the channel state information data.

[0102] In some aspects, predicting the layout of the spatial region further comprises generating an activated combined average based on an activation function applied to a combination of the temporal average output of the first model and the temporal average output of the second model.

[0103] In some aspects, predicting a layout of the spatial region further comprises generating information defining the predicted layout of the spatial region based on the plurality of regression heads and the post-activation combining average.

[0104] In some aspects, the temporal data includes a visual representation of time difference of arrival (TDoA) information for each of a plurality of antennas of the wireless device associated with the channel state information data for each channel state information sample in the input data set. The first machine learning model can be a transformer neural network, and the second machine learning model can be a visual transformer that generates an attention output from the temporal data based on the visual representation of the TDoA information for each of the plurality of antennas. In some aspects, the visual representation of the TDoA information for each of the plurality of antennas can be generated based on an inverse Fourier transform of the channel state information data and a transposed version of the inverse Fourier transform of the channel state information data.

[0105] In some aspects, the predicted layout of the spatial region also includes at least one of: a predicted number of zones in the spatial region, a predicted number of openings between zones in the spatial region, predicted coordinates of each zone in the spatial region, and predicted coordinates of each opening between zones in the spatial region.

[0106] In some aspects, predicting the layout of the spatial region includes using a first machine learning model (e.g., Figure 3 exemplified in the first machine learning model 322) to generate multiple samples (e.g., based on Figure 3 The multi-dimensional sample input sequence 310 is generated by Figure 4 A second machine learning model (e.g., Figure 3 The second machine learning model 324 illustrated in FIG. 3 is used to generate bounding boxes for discrete portions of the spatial region and a layout of the spatial region.

[0107] In some aspects, the first machine learning model can be a transformer encoder. The transformer encoder can be configured to generate an output sequence having the same dimension as the input dataset from the input dataset. The output sequence typically identifies local correlations within a sequence of samples in the input dataset.

[0108] In some aspects, the second machine learning model can be a model configured to generate bounding boxes and layouts of spatial regions from these bounding boxes.Each bounding box can be defined according to a set of coordinates in a multidimensional space.

[0109] In some aspects, to generate a layout of the spatial region, the second machine learning model may match a set of points corresponding to the coordinates of the bounding boxes based on minimizing the slope distance between the coordinates of the bounding boxes. The second machine learning model may further minimize the average IoU measure between the bounding boxes. As discussed, by minimizing the average IoU measure between the bounding boxes, a layout can be generated that identifies the physical impossibility of two discrete rooms overlapping each other in the spatial environment.

[0110] In some aspects, to generate a layout of the spatial region, the second machine learning model may match a set of points corresponding to the coordinates of the bounding boxes based on minimization of the Hungarian loss metric. The second model may be further trained to minimize the average IoU measurement between the bounding boxes.

[0111] In some aspects, the second machine learning model is configured to generate a bounding box by mapping a representation of each sample in the plurality of multi-dimensional samples generated by the first machine learning model to coordinates of the bounding box.

[0112] In some aspects, predicting a layout of the spatial region includes predicting a layout of the spatial region over a plurality of possible layouts for an input data set (e.g., based on a probability distribution over the plurality of possible layouts). As discussed, because a set of multidimensional samples captured by traversing a path through the spatial region may reasonably be associated with several different layouts of the spatial region, the probability distribution over the plurality of possible layouts may be used to identify a layout having a highest probability of being associated with an input data set comprising the plurality of multidimensional samples.

[0113] In some aspects, the predicted layout may include multiple bounding boxes. At least one of the multiple bounding boxes may include non-contiguous coordinates. These non-contiguous coordinates may represent, for example, doors, windows, or other openings between different discrete portions of the spatial area.

[0114] Operation 1000 proceeds to block 1030 where the predicted layout of the spatial region is output, such as Figure 6 Layout 650 depicted in .

[0115] Example processing system for predicting a layout of a spatial region based on samples from the spatial region

[0116] Fig.11 An example processing system 1100 is depicted for training a machine learning model to predict a layout of a spatial region based on samples from the spatial region, such as described herein, for example, with respect to Fig. 9 described.

[0117] The processing system 1100 includes a central processing unit (CPU) 1102, which may be a multi-core CPU in some examples. Instructions executed at the CPU 1102 may be loaded, for example, from a program memory associated with the CPU 1102, or may be loaded from the memory 1124.

[0118] The processing system 1100 also includes additional processing components customized for specific functions, such as a graphics processing unit (GPU) 1104 , a digital signal processor (DSP) 1106 , a neural processing unit (NPU) 1108 , a multimedia processing unit 1110 , and a wireless connectivity component 1112 .

[0119] An NPU, such as NPU 1108, is typically a dedicated circuit configured to implement control and arithmetic logic for executing machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. An NPU is sometimes alternatively referred to as a neural signal processor (NSP), a tensor processing unit (TPU), a neural network processor (NNP), an intelligence processing unit (IPU), a vision processing unit (VPU), or a graphics processing unit.

[0120] An NPU, such as NPU 1108, is configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a system on a chip (SoC), while in other examples, the NPU may be part of a dedicated neural network accelerator.

[0121] The NPU can be optimized for training or inference, or in some cases configured to balance performance between the two. For NPUs that can perform both training and inference, the two tasks can generally still be performed independently.

[0122] NPUs designed to accelerate training are generally configured to accelerate the optimization of new models, which is a highly computationally intensive operation that involves inputting an existing data set (usually labeled or tagged), iterating on the data set, and then adjusting model parameters (such as weights and biases) to improve model performance. Generally speaking, optimization based on error predictions involves passing back through the layers of the model and determining the gradient to reduce the prediction error.

[0123] NPUs designed to accelerate inference are generally configured to operate on complete models. Thus, such NPUs can be configured to input new data segments and quickly process the data segments through the trained model to generate model outputs (e.g., inferences).

[0124] In one specific implementation, the NPU 1108 is part of one or more of the CPU 1102 , the GPU 1104 , and / or the DSP 1106 .

[0125] In some examples, one or more of the processors of processing system 1100 may be based on the ARM or RISC-V instruction set.

[0126] The processing system 1100 also includes a memory 1124, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 1124 includes computer-executable components that are executable by one or more of the aforementioned processors of the processing system 1100.

[0127] Specifically, in this example, the memory 1124 includes a data set receiving component 1124A and a model training component 1124B. The depicted components and other components not depicted may be configured to perform various aspects of the methods described herein.

[0128] Generally speaking, the processing system 1100 and / or its components may be configured to perform the methods described herein.

[0129] Fig.12 An example processing system 1200 is depicted for predicting the layout of a spatial region using a machine learning model and an input data set of multi-dimensional samples, such as described herein, for example, with respect to Fig.10 described.

[0130] Processing system 1200 includes a central processing unit (CPU) 1202, which in some examples may be a multi-core CPU. Processing system 1200 also includes additional processing components customized for specific functions, such as a graphics processing unit (GPU) 1204, a digital signal processor (DSP) 1206, and a neural processing unit (NPU) 1208. CPU 1202, GPU 1204, DSP 1206, and NPU 1208 may be similar to those described above with respect to Fig.11 The CPU 1102, GPU 1104, DSP 1106 and NPU 1108 are discussed.

[0131] In some examples, connectivity component 1212 may include, for example, subcomponents for third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., 4G LTE), fifth generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. Connectivity component 1212 may be further connected to one or more antennas (not shown).

[0132] In some examples, one or more of the processors of processing system 1200 may be based on the ARM or RISC-V instruction set.

[0133] The processing system 1200 also includes a memory 1224, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 1224 includes computer-executable components that are executable by one or more of the aforementioned processors of the processing system 1200.

[0134] Specifically, in this example, the memory 1224 includes a data set receiving component 1224A, a layout prediction component 1224B, a layout output component 1224C, and a machine learning model component 1224D (such as a Fig.11 The depicted components, as well as other components not depicted, may be configured to perform various aspects of the methods described herein.

[0135] Generally speaking, the processing system 1200 and / or its components may be configured to perform the methods described herein.

[0136] It is noted that in other aspects, such as where the processing system 1200 is a server computer, various elements of the processing system 1200 may be omitted. For example, in other aspects, the multimedia component 1210, the connectivity component 1212, the sensor 1216, the ISP 1218, and / or the navigation component 1220 may be omitted.

[0137] Sample Clauses

[0138] Specific implementation details for each aspect are described in the following numbered clauses.

[0139] Item 1: A computer-implemented method comprising: receiving an input data set comprising a plurality of samples from a spatial region, each of the plurality of samples comprising at least channel state information data; and training a machine learning model to predict a layout of the spatial region based on the input data set, wherein the predicted layout of the spatial region comprises a plurality of bounding boxes defining different regions of the spatial region.

[0140] Clause 2: A method as described in clause 1, wherein the channel state information data comprises power measurements at a given location and time in three-dimensional space.

[0141] Clause 3: A method according to any one of clauses 1 or 2, wherein training the machine learning model comprises: training a first machine learning model that generates intermediate outputs from the channel state information data; and training a second machine learning model that generates attention outputs from temporal data derived from the channel state information data.

[0142] Clause 4: A method according to Clause 3, wherein training the machine learning model also includes training the machine learning model to generate a post-activation combined average based on applying an activation function to a combination of the time-averaged output of the first machine learning model and the time-averaged output of the second machine learning model.

[0143] Clause 5: The method of clause 4, wherein training the machine learning model further comprises training the machine learning model to output information defining the predicted layout of the spatial region based on multiple regression heads and the post-activation combined average.

[0144] Clause 6: A method according to any one of clauses 3 to 5, wherein: the temporal data includes a visual representation of time difference of arrival (TDoA) information for each of a plurality of antennas of a wireless device associated with the channel state information data for each sample in the input data set, the first machine learning model includes a transformer neural network, and the second machine learning model includes a visual transformer neural network, which generates the attention output from the temporal data based on the visual representation of the TDoA information for each of the plurality of antennas.

[0145] Clause 7: The method of clause 6, further comprising generating the visual representation of the TDoA information for each of the plurality of antennas based on an inverse Fourier transform of the channel state information data and a transposed version of the inverse Fourier transform of the channel state information data.

[0146] Clause 8: A method according to any one of clauses 1 to 7, wherein the predicted layout of the spatial region also includes one or more of the following: a predicted number of zones in the spatial region, a predicted number of openings between zones in the spatial region, predicted coordinates of each zone in the spatial region, and predicted coordinates of each opening between zones in the spatial region.

[0147] Clause 9: A method according to any one of clauses 1 to 8, wherein: the multiple samples include multidimensional samples, the multidimensional samples include the channel state information data, positioning data and time data for each sample in the input data set, and training the machine learning model includes: training a first machine learning model to generate a representation of each sample in the multiple samples, and training a second machine learning model to generate the multiple bounding boxes for discrete parts of the spatial region based on the representation of each sample in the multiple samples.

[0148] Clause 10: The method of clause 9, wherein the positioning data comprises acceleration data and velocity data for a wireless device associated with the channel state information data.

[0149] Clause 11: A method according to any one of clauses 9 or 10, wherein: the first machine learning model includes a transformer encoder, and training the first machine learning model includes training the transformer encoder to generate an output sequence from the input data set, wherein the output sequence identifies local correlations within a sequence of samples in the input data set.

[0150] Clause 12: The method of clause 11, wherein training the second machine learning model comprises training the second machine learning model to generate: the bounding boxes, wherein each bounding box comprises a set of coordinates in a multidimensional space; and a layout of the spatial region from the bounding boxes.

[0151] Clause 13: A method according to clause 12, wherein training the second machine learning model also includes training the second machine learning model to match a set of points corresponding to the coordinates of the bounding boxes based on minimization of one or more of a slope distance between the coordinates of the bounding boxes or a Hungarian loss metric corresponding to the coordinates of each of the multiple bounding boxes to generate the layout of the spatial region.

[0152] Clause 14: The method of clause 13, wherein training the second machine learning model further comprises training the second machine learning model to minimize an average intersection over union (IoU) measurement between the bounding boxes.

[0153] Clause 15: A method according to any one of clauses 9 to 14, wherein training the machine learning model also includes training the machine learning model to predict the layout of the spatial region based on the predicted layout distribution in the spatial region for the input data set.

[0154] Clause 16: A method according to any one of clauses 9 to 15, wherein training the machine learning model comprises training the machine learning model to predict a layout distribution in the spatial region for the input data set based on a joint distribution over parameters of the machine learning model and the predicted layout distribution.

[0155] Clause 17: A method according to clause 16, wherein training the machine learning model comprises training the machine learning model to predict a posterior distribution over the weights of the machine learning model, wherein the posterior distribution is approximated based on a Kullback-Leibler (KL) divergence measure of an approximate probability distribution over the weights.

[0156] Clause 18: A method according to any one of clauses 9 to 17, wherein training the machine learning model also includes training the machine learning model to generate the bounding box with non-contiguous coordinates.

[0157] Clause 19: A computer-implemented method comprising: receiving an input data set comprising a plurality of samples from a spatial region, each sample comprising at least channel state information data; predicting a layout of the spatial region based on a machine learning model and the received input data set, wherein the predicted layout of the spatial region comprises a plurality of bounding boxes defining different regions of the spatial region; and outputting the predicted layout of the spatial region.

[0158] Clause 20: The method of clause 19, wherein the channel state information data comprises power measurements at a given location and time in three-dimensional space.

[0159] Clause 21: A method according to any one of clauses 19 or 20, wherein predicting the layout of the spatial region comprises predicting the layout based on: intermediate outputs generated by a first machine learning model from the channel state information data, and attention outputs generated by a second machine learning model from temporal data derived from the channel state information data.

[0160] Clause 22: A method according to clause 21, wherein predicting the layout of the spatial region also includes generating a post-activation combined average based on an activation function applied to a combination of the time-averaged output of the first machine learning model and the time-averaged output of the second machine learning model.

[0161] Clause 23: The method of clause 22, wherein predicting the layout of the spatial region further comprises generating information defining the predicted layout of the spatial region based on a plurality of regression heads and the post-activation combined average.

[0162] Clause 24: A method according to any one of clauses 21 to 23, wherein: the temporal data includes a visual representation of time difference of arrival (TDoA) information for each of a plurality of antennas of a wireless device associated with the channel state information data for each sample in the input data set, the first machine learning model includes a transformer neural network, and the second machine learning model includes a visual transformer neural network, which generates the attention output from the temporal data based on the visual representation of the TDoA information for each of the plurality of antennas.

[0163] Clause 25: The method of clause 24, further comprising generating the visual representation of the TDoA information for each of the plurality of antennas based on an inverse Fourier transform of the channel state information data and a transposed version of the inverse Fourier transform of the channel state information data.

[0164] Clause 26: A method according to any one of clauses 19 to 25, wherein the predicted layout of the spatial region also includes at least one of the following: a predicted number of zones in the spatial region, a predicted number of openings between zones in the spatial region, predicted coordinates of each zone in the spatial region, and predicted coordinates of each opening between zones in the spatial region.

[0165] Clause 27: A method according to any one of clauses 19 to 26, wherein: the multiple samples include multidimensional samples, the multidimensional samples include the channel state information data, positioning data and time data for each sample in the input data set; and predicting the layout of the spatial region includes: using a first machine learning model to generate a representation of each sample in the multiple samples; and using a second machine learning model and the representation of each sample in the multiple samples to generate the bounding box for discrete parts of the spatial region and the layout of the spatial region.

[0166] Clause 28: A method according to clause 27, wherein: the first machine learning model includes a transformer encoder, which is configured to generate an output sequence having the same dimension as the input dataset from the input dataset, and the output sequence identifies local correlations within a sequence of samples in the input dataset; and the second machine learning model is configured to generate: the bounding boxes, each of which includes a set of coordinates in a multidimensional space; and a layout of the spatial region from the bounding boxes.

[0167] Clause 29: A processing system comprising: a memory including computer-executable instructions; and one or more processors configured to execute the computer-executable instructions and cause the processing system to perform a method according to any one of clauses 1 to 28.

[0168] Clause 30: A processing system comprising means for performing the method according to any one of clauses 1 to 28.

[0169] Clause 31: A non-transitory computer readable medium comprising computer executable instructions which, when executed by one or more processors of a processing system, cause the processing system to perform the method of any of clauses 1 to 28.

[0170] Clause 32: A computer program product embodied on a computer readable storage medium, the computer program product comprising code for performing the method according to any one of clauses 1 to 28.

[0171] Additional considerations

[0172] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. The examples discussed herein are not limited to the scope, applicability or aspects set forth in the claims. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. For example, without departing from the scope of the present disclosure, the functions and arrangements of the elements discussed may be changed. Various examples may omit, replace or add various processes or components as appropriate. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted or combined. In addition, the features described with respect to some examples may be combined in some other examples. For example, any number of aspects set forth herein may be used to implement a device or practice method. In addition, the scope of the present disclosure is intended to cover such devices or methods practiced using other structures, functions, or structures and functions other than the various aspects of the present disclosure set forth herein or different from the various aspects of the present disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of the claims.

[0173] As used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0174] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items (including single members). For example, "at least one of a, b, or c" is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination with multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0175] As used herein, the term "determining" encompasses a wide variety of actions. For example, "determining" may include calculating, computing, processing, deriving, investigating, searching (e.g., searching in a table, database, or other data structure), ascertaining, and the like. Furthermore, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Furthermore, "determining" may include resolving, selecting, choosing, establishing, and the like.

[0176] The method disclosed herein includes one or more steps or actions for implementing the method. The steps and / or actions of the method can be interchangeable with each other without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions can be modified without departing from the scope of the claims. In addition, the various operations of the method described above can be performed by any suitable component capable of performing the corresponding function. The component may include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs) or processors. Generally speaking, in the case of the operations illustrated in the accompanying drawings, those operations may have corresponding corresponding components with similar numbers plus functional components.

[0177] The following claims are not intended to be limited to the various aspects shown herein, but should be given the full scope consistent with the language of the claims. Within the claims, unless specifically stated otherwise, reference to an element in the singular form is not intended to mean "one and only one", but "one or more". Unless otherwise specifically stated, the term "some" refers to one or more. No claim element should be interpreted according to the provisions of 35 U.S.C. § 112 (f), unless the element is explicitly recorded using the phrase "a component for...", or in the case of a method claim, the element is recorded using the phrase "a step for...". All structural and functional equivalents of the elements of the various aspects described throughout this disclosure that are known or will be known later to a person of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is explicitly recorded in the claims.

Claims

1. A computer-implemented method comprising: receiving an input data set comprising a plurality of samples from a spatial region, each sample of the plurality of samples comprising at least channel state information data; as well as A machine learning model is trained to predict a layout of the spatial region based on the input data set, wherein the predicted layout of the spatial region includes a plurality of bounding boxes defining different regions of the spatial region.

2. The method of claim 1, wherein the channel state information data comprises power measurements at a given location and time in three-dimensional space.

3. The method of claim 1, wherein training the machine learning model comprises: training a first machine learning model that generates an intermediate output from the channel state information data; as well as A second machine learning model is trained to generate attention outputs from temporal data derived from the channel state information data.

4. The method of claim 3, wherein training the machine learning model further comprises training the machine learning model to generate an activated combined average based on applying an activation function to a combination of a time-averaged output of the first machine learning model and a time-averaged output of the second machine learning model.

5. The method of claim 4, wherein training the machine learning model further comprises training the machine learning model to output information defining the predicted layout of the spatial region based on a plurality of regression heads and the post-activation combined average.

6. The method according to claim 3, wherein: The temporal data comprises a visual representation of time difference of arrival (TDoA) information for each of a plurality of antennas of a wireless device associated with the channel state information data for each sample in the input data set, The first machine learning model comprises a transformer neural network, and The second machine learning model includes a visual transformer neural network that generates the attention output from temporal data based on the visual representation of the TDoA information for each of the plurality of antennas.

7. The method of claim 6, further comprising generating the visual representation of the TDoA information for each of the plurality of antennas based on an inverse Fourier transform of the channel state information data and a transposed version of the inverse Fourier transform of the channel state information data.

8. A method according to claim 1, wherein the predicted layout of the spatial region also includes one or more of the following: a predicted number of zones in the spatial region, a predicted number of openings between zones in the spatial region, predicted coordinates of each zone in the spatial region, and predicted coordinates of each opening between zones in the spatial region.

9. The method according to claim 1, wherein: The plurality of samples comprises multi-dimensional samples, the multi-dimensional samples comprising the channel state information data, positioning data and time data for each sample in the input data set, and Training the machine learning model includes: training a first machine learning model to generate a representation of each sample in the plurality of samples, and A second machine learning model is trained to generate the plurality of bounding boxes for discrete portions of the spatial region based on the representation of each sample in the plurality of samples.

10. The method of claim 9, wherein the positioning data comprises acceleration data and velocity data for a wireless device associated with the channel state information data.

11. The method according to claim 9, wherein: The first machine learning model includes a transformer encoder, and Training the first machine learning model includes training the transformer encoder to generate an output sequence from the input data set, the output sequence identifying local correlations within a sequence of samples in the input data set.

12. The method of claim 11, wherein training the second machine learning model comprises training the second machine learning model to generate: the bounding boxes, wherein each bounding box comprises a set of coordinates in a multi-dimensional space; and A layout of the spatial region from the bounding box.

13. The method of claim 12, wherein training the second machine learning model further comprises training the second machine learning model to match a set of points corresponding to the coordinates of the bounding boxes based on minimization of one or more of a slope distance between the coordinates of the bounding boxes or a Hungarian loss metric corresponding to the coordinates of each of the plurality of bounding boxes to generate the layout of the spatial region.

14. The method of claim 13, wherein training the second machine learning model further comprises training the second machine learning model to minimize an average intersection over union (IoU) measurement between the bounding boxes.

15. The method of claim 9, wherein training the machine learning model further comprises training the machine learning model to predict the layout of the spatial region based on a predicted layout distribution in the spatial region for the input data set.

16. A method according to claim 9, wherein training the machine learning model includes training the machine learning model to predict the layout distribution in the spatial region for the input data set based on a joint distribution over parameters of the machine learning model and the predicted layout distribution.

17. A method according to claim 16, wherein training the machine learning model includes training the machine learning model to predict a posterior distribution over the weights of the machine learning model, and the posterior distribution is approximated based on a Kullback-Leibler (KL) divergence measure of an approximate probability distribution over the weights.

18. The method of claim 9, wherein training the machine learning model further comprises training the machine learning model to generate the bounding box with non-contiguous coordinates.

19. A computer-implemented method comprising: receiving an input data set comprising a plurality of samples from a spatial region, each sample comprising at least channel state information data; predicting a layout of the spatial region based on the machine learning model and the received input data set, wherein the predicted layout of the spatial region includes a plurality of bounding boxes defining different regions of the spatial region; and The predicted layout of the spatial region is output.

20. The method of claim 19, wherein the channel state information data comprises power measurements at a given location and time in three-dimensional space.

21. The method of claim 19, wherein predicting the layout of the spatial region comprises predicting the layout based on: an intermediate output generated by a first machine learning model from the channel state information data, and An attention output generated by a second machine learning model from temporal data derived from the channel state information data.

22. The method of claim 21, wherein predicting the layout of the spatial region further comprises generating an activated combined average based on an activation function applied to a combination of a time-averaged output of the first machine learning model and a time-averaged output of the second machine learning model.

23. The method of claim 22, wherein predicting the layout of the spatial region further comprises generating information defining the predicted layout of the spatial region based on a plurality of regression heads and the post-activation combined average.

24. The method of claim 21, wherein: The temporal data comprises a visual representation of time difference of arrival (TDoA) information for each of a plurality of antennas of a wireless device associated with the channel state information data for each sample in the input data set, The first machine learning model comprises a transformer neural network, and The second machine learning model includes a visual transformer neural network that generates the attention output from the temporal data based on the visual representation of the TDoA information for each of the plurality of antennas.

25. The method of claim 24, further comprising generating the visual representation of the TDoA information for each of the plurality of antennas based on an inverse Fourier transform of the channel state information data and a transposed version of the inverse Fourier transform of the channel state information data.

26. A method according to claim 19, wherein the predicted layout of the spatial region also includes at least one of the following: a predicted number of zones in the spatial region, a predicted number of openings between zones in the spatial region, predicted coordinates of each zone in the spatial region, and predicted coordinates of each opening between zones in the spatial region.

27. The method of claim 19, wherein: The plurality of samples comprises multi-dimensional samples, the multi-dimensional samples comprising the channel state information data, positioning data and time data for each sample in the input data set; and Predicting the layout of the spatial region comprises: generating, using a first machine learning model, a representation of each sample in the plurality of samples; as well as The bounding boxes for discrete portions of the spatial region and the layout of the spatial region are generated using a second machine learning model and the representation of each sample in the plurality of samples.

28. The method of claim 27, wherein: The first machine learning model comprises a transformer encoder configured to generate, from the input data set, an output sequence having the same dimension as the input data set, the output sequence identifying local correlations within a sequence of samples in the input data set; and The second machine learning model is configured to generate: the bounding boxes, wherein each bounding box comprises a set of coordinates in a multidimensional space, and A layout of the spatial region from the bounding box.

29. A system comprising: A memory, wherein executable instructions are stored in the memory; as well as a processor configured to execute the executable instructions so as to cause the system to: receiving an input data set comprising a plurality of samples from a spatial region, each sample of the plurality of samples comprising at least channel state information data; and A machine learning model is trained to predict a layout of the spatial region based on the input data set, wherein the predicted layout of the spatial region includes a plurality of bounding boxes defining different regions of the spatial region.

30. A system comprising: A memory, wherein executable instructions are stored in the memory; as well as a processor configured to execute the executable instructions so as to cause the system to: receiving an input data set comprising a plurality of samples from a spatial region, each sample comprising at least channel state information data; predicting a layout of the spatial region based on the machine learning model and the received input data set, wherein the predicted layout of the spatial region includes a plurality of bounding boxes defining different regions of the spatial region; and The predicted layout of the spatial region is output.

Images