SYSTEM AND METHOD FOR TRAINING A RECURRING NEURAL NETWORK FOR LOCALIZING A DEVICE IN AN ENVIRONMENT - Patent application

Abstract

The present disclosure provides a method and a system for training a neural network suitable for identifying the position of a device in an environment based on signals received by the device. The method includes training a double-regressor neural network to identify a position from labeled data, the double-regressor neural network including a feature extractor and a double-regressor including two regressors, the method further including training the parameters of the double-regressor using labeled data and unlabeled data such that each of the two regressors identifies the same labeled position while processing the labeled data and different positions while processing the unlabeled data, and training the parameters of the feature extractor using an adversarial discriminator to extract domain-invariant features having statistical characteristics of the labeled data from the unlabeled data according to the adversarial discriminator such that each of the two regressors identifies the same position while processing the domain-invariant features.

Description

[Technical Field]

[0001] The present disclosure relates generally to device localization, and more particularly to systems and methods for training recurrent neural networks for device localization in an environment. [Background technology]

[0002] Domain adaptation (DA) aims to transfer knowledge from a well-labeled source domain to facilitate unlabeled target learning. Turning to specific tasks such as indoor (Wi-Fi) localization, learning a cross-domain regressor to mitigate domain shift is essential. However, in contrast to classification tasks, such tasks rely on regression because the trained neural network needs to identify locations in a continuous space, while the training data can, in practice, only come from samples in discrete locations. Summary of the Invention [Problem to be solved by the invention]

[0003] Regression, as opposed to classification, is a major paradigm with a wide range of applications. Domain adaptation regression (DAR) extends it by generalizing a regressor from a labeled source domain to an unlabeled target domain. Deep learning has brought about remarkable changes in diverse regression applications across many domains. Nevertheless, training high-quality deep models relies on large labeled datasets. Furthermore, in many real-world regression applications, accurately annotating abundant training instances is time-consuming and labor-intensive. A solution to such problems is to leverage ready-made labeled data from related domains and apply domain adaptation techniques to overcome domain shifts or dataset biases.

[0004] Deep domain adaptation methods have achieved significant progress in domain adaptive classification (DAC) problems. However, despite these advances in DAC, learning invariant representations in the deep regime of DAR remains underdeveloped. One reason for this deficiency lies in the difference in loss functions during training of classifiers and regressors. The commonly used loss function in regression is the squared loss (L2), while in classification it is the cross-entropy loss (CE) with a softmax activation function. Softmax allows activation values ​​of different categories to compete with each other. Introducing this competition mechanism allows classifiers to quickly adapt to changes in feature scale. However, in regression tasks, regressors may not have such adaptability.

[0005] Therefore, there is a need for a domain adaptation technique for regressors. [Means for solving the problem]

[0006] Some embodiments recognize that the challenges of domain adaptation for recurrent neural networks can be addressed using adversarial classifiers applied at the feature extraction level. The goal of such classifiers is to train a feature extractor to extract features from target-domain measurements that have a similar statistical distribution to the features extracted from source-domain measurements. However, arriving at such a classifier presents a problem in that the feature extractor is neither a classifier nor a regressor, requiring an additional decision unit to judge the quality of the extracted features. Such a decision unit is absent in typical regression architectures used for device localization.

[0007] To that end, some embodiments disclose a bi-regressor neural network and a three-stage training procedure for domain-adaptive regression. The bi-regressor neural network includes a feature extractor and a bi-regressor including two regressors. The two regressors have the same architecture with potentially different weights and biases. The three-stage training procedure is used to train the bi-regressor neural network for localization of a device within an environment based on signals received by the device located within the environment. The signals may include different types of radio frequency (RF) signals, such as received signal strength indicator (RSSI) signals and channel state information (CSI) signals, at various signal levels. The different types of RF signals include ultra-wideband (UWB) signals, Wi-Fi signals, and inertial measurement unit (IMU) signals. The three-stage training procedure includes training the bi-regressor neural network with labeled data in the environment and / or with unlabeled data in a modified or new environment. In one embodiment, the labeled data includes signal measurements, labeled with coordinates of the measurement locations. In one embodiment, the unlabeled data includes measurements of signals at unidentified locations in the environment, and the labeled data measurements and the unlabeled data measurements are sampled at the same or different discrete locations in the environment.

[0008] Each training stage of the three-stage training procedure is described below. During the first training stage, a bi-regressor neural network is trained using labeled data to identify locations in the continuous space of the environment from the labeled data. In particular, a feature extractor is trained to extract features from the labeled data, and each regressor is trained to determine locations in the continuous space of the environment from the features extracted by the feature extractor. In other words, the goal of each regressor is to learn a regression that can map the extracted features to locations in the continuous space of the environment. The need for two regressors stems from the need to have a decision unit for the features extracted by the feature extractor and the ability to determine whether the input sensor signals are collected from an environment similar to the training data or from an unidentified environment.

[0009] During the first stage of training, the feature extractor and the two regressors are trained using the loss function L r The regressors are trained on labeled data based on the loss function L. As a result, the parameters of the two regressors are most likely to be the same or very similar. Therefore, when both regressors process the features of the labeled data, both regressors will produce similar outputs that approximate the ground truth data. In one embodiment, the loss function L r is based on the mean squared error (MSE) criterion between the labeled data and the coordinate estimates.

[0010] During the second stage of training, two regressors are trained using labeled data and unlabeled data, producing the same correct output for the labeled data but different outputs for unlabeled data from different domains. In this way, the two regressors are trained to produce different outputs for the unlabeled data if the unlabeled data has a statistical distribution different from that of the labeled data. As a result, the two regressors are trained to identify whether the statistical distribution of the unlabeled data has a similar or different distribution to that of the labeled data. As a result, these two regressors can act as judges of the similarity between the input statistical distribution and the statistical distribution of the labeled data.

[0011] Furthermore, the difference between the outputs of both regressors on unlabeled data can be maximized by minimizing the overlap / similarity between the outputs of both regressors. To minimize the overlap between the outputs of both regressors, the overlap between the bi-regressor outputs needs to be quantified. To quantify such overlap, some embodiments use the Jaccard similarity coefficient (IoU score) from the field of object detection using visual sensors. However, the non-differentiable nature of the IoU score poses a problem in optimizing the bi-regressor neural network weights. To mitigate such problems, a soft similarity function L s The soft similarity function L s Minimizing σ reduces the overlap between the outputs of the two regressors and further increases the difference between the outputs of the two regressors on the unlabeled data.

[0012] However, we still require that both regressors should produce the same output for labeled data. To that end, in the second stage of training, we use a soft similarity function L s In addition to minimizing the loss function L r (used in the first stage of training) and the soft similarity function L s An objective function based on is minimized.

[0013] Therefore, the combination of the two regressors can be envisioned as an implicit discriminator to directly distinguish between labeled and unlabeled data (i.e., detect target samples outside the source samples / supports).

[0014] Given these new capabilities of the two regressors, during the third training stage, a feature extractor is trained with an adversarial classifier to use the two regressors as judges to extract domain-invariant features for the unlabeled data using the statistical distribution of the labeled data. The extracted domain-invariant features are such that when processed by the two regressors, each of the two regressors identifies the same location while processing the domain-invariant features.

[0015] Some embodiments recognize that domain adaptation becomes difficult when the distribution difference is large. As used herein, distribution difference refers to the difference between the outputs of two regressors. For example, in the case of multimodal indoor localization, the distribution difference becomes relatively significant due to the fact that a change in one multipath component can contribute to the entire sensor signal in either a constructive or destructive manner. Some embodiments recognize that such a problem can be solved by constructing two intermediate domains between the source domain and the target domain and gradually eliminating the mismatch between them to achieve statistical distribution matching. The intermediate domains are constructed based on data augmentation of labeled data and unlabeled data.

[0016]

number

[0017] After performing the above three-stage training procedure, a trained bi-regressor neural network is obtained. The trained bi-regressor neural network includes a trained feature extractor and two trained regressors. The trained bi-regressor neural network can be used to determine a location for input unlabeled data. Here, the two trained regressors of the trained bi-regressor neural network generate similar locations for the input unlabeled data. To this end, some embodiments are based on the recognition that the architecture of the trained bi-regressor neural network can include only a single trained regressor, rather than two trained regressors, for determining a location. Alternatively, in some embodiments, the trained bi-regressor neural network may include a trained feature extractor and the original regressor (the regressor obtained from the first stage of training), which may provide additional robustness of the regression output.

[0018] Accordingly, one embodiment discloses a computer-implemented method for training a neural network suitable for locating a device within an environment based on signals received by the device located within the environment.The method employs a processor with stored instructions implementing the method, which instructions, when executed by the processor, perform steps of the method, the steps including collecting labeled data including measurements of the signal, the labeled data labeled with coordinates of locations of the measurements, and further including collecting unlabeled data including measurements of the signal at unidentified locations in the environment, the measurements of the labeled data and the measurements of the unlabeled data being sampled at the same or different discrete locations in the environment, and further including training a bi-regressor neural network to identify locations in a continuous space of the environment from the labeled data during a first training stage using the labeled data, the bi-regressor neural network including a feature extractor configured to extract features from the labeled data and a bi-regressor including two regressors having identical architecture, each regressor receiving features from the feature extractor. and training parameters of the feature extractor using the labeled data and the unlabeled data during a second training stage with fixed parameters of the feature extractor trained during the first training stage using the labeled data and the unlabeled data, such that each of the two regressors identifies the same labeled location while processing the labeled data and identifies different locations while processing the unlabeled data; and training parameters of the feature extractor using the adversarial classifier during a third training stage with fixed parameters of the bi-regressor trained during the second training stage using the labeled data and the unlabeled data to extract domain-invariant features from the unlabeled data using statistical properties of the labeled data according to an adversarial classifier, such that each of the two regressors identifies the same location while processing domain-invariant features.

[0019] Accordingly, another embodiment discloses a system for training a neural network suitable for localizing a device within an environment based on signals received by the device located within the environment.The system comprises a processor and a memory having stored thereon instructions that, when executed by the processor, cause the system to collect labeled data including measurements of the signal, the labeled data labeled with coordinates of locations of the measurements; the instructions, when executed by the processor, further cause the system to collect unlabeled data including measurements of the signal at unidentified locations in the environment, the measurements of the labeled data and the measurements of the unlabeled data being sampled at the same or different discrete locations in the environment; and the instructions, when executed by the processor, further cause the system to train a bi-regressor neural network to identify locations in a continuous space of the environment from the labeled data during a first training phase using the labeled data, the bi-regressor neural network including a feature extractor configured to extract features from the labeled data and a bi-regressor including two regressors having identical architecture, each regressor extracting features from the labeled data received from the feature extractor. and trained to determine locations in the continuous space of the environment from the features, the instructions, when executed by the processor, further cause the system to train parameters of the bi-regressor during a second training stage with fixed parameters of the feature extractor trained during the first training stage using the labeled data and the unlabeled data, such that each of the two regressors identifies the same labeled location while processing the labeled data and identifies different locations while processing the unlabeled data, and train parameters of the feature extractor using the adversarial classifier during a third training stage with fixed parameters of the bi-regressor trained during the second training stage using the labeled data and the unlabeled data, to extract domain-invariant features from the unlabeled data using statistical properties of the labeled data according to an adversarial classifier, such that each of the two regressors identifies the same location while processing domain-invariant features.

[0020] Accordingly, yet another embodiment discloses a non-transitory computer-readable storage medium having embodied thereon a program executable by a processor for performing a method for training a neural network suitable for locating a device within an environment based on signals received by the device located within the environment.the method includes collecting labeled data including measurements of the signal, the labeled data labeled with coordinates of locations of the measurements; the method further includes collecting unlabeled data including measurements of the signal at unidentified locations in the environment, the measurements of the labeled data and the measurements of the unlabeled data being sampled at the same or different discrete locations in the environment; the method further includes, during a first training stage using the labeled data, training a bi-regressor neural network to identify locations in a continuous space of the environment from the labeled data, the bi-regressor neural network including a feature extractor configured to extract features from the labeled data and a bi-regressor including two regressors having an identical architecture, each regressor trained to determine a location in the continuous space of the environment from the features received from the feature extractor; The method further includes training parameters of the bi-regressor during a second training stage using the labeled data and the unlabeled data with fixed parameters of the feature extractor trained during the first training stage, such that each of the two regressors identifies the same labeled location while processing the labeled data and identifies different locations while processing the unlabeled data; and training parameters of the feature extractor during a third training stage using the adversarial classifier with fixed parameters of the bi-regressor trained during the second training stage using the labeled data and the unlabeled data, to extract domain-invariant features from the unlabeled data using statistical properties of the labeled data according to an adversarial classifier, such that each of the two regressors identifies the same location while processing domain-invariant features.

[0021] The present invention will now be described in detail with reference to the accompanying drawings, in which the drawings are not necessarily to scale, with emphasis instead generally being placed upon illustrating the principles of embodiments of the present disclosure. [Brief explanation of the drawings]

[0022] [Figure 1A] FIG. 1 shows a schematic diagram of a bi-regressor neural network architecture according to some embodiments of the present disclosure. [Figure 1B] 10 shows a schematic diagram of a second training stage according to some embodiments of the present disclosure. [Figure 1C] 10 shows a schematic diagram of a third training stage according to some embodiments of the present disclosure. [Figure 2A] FIG. 1 illustrates a schematic diagram of the architecture of a trained bi-regressor neural network for determining the location of a device for input unlabeled data, according to some embodiments of the present disclosure. [Figure 2B] 1 illustrates the architecture of a bi-regressor neural network for determining a location of a device, according to some embodiments of the present disclosure, where the bi-regressor neural network includes a single trained regressor. [Figure 2C] 10 illustrates a bi-regressor neural network architecture for determining a device's location, according to some other embodiments of the present disclosure. [Figure 3] 1 illustrates control of a robot in an autonomous factory according to some embodiments of the present disclosure. [Figure 4] FIG. 1 is a schematic diagram illustrating a system for training a neural network suitable for localizing a device in an environment, according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0023] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details. In other instances, devices and methods are shown only in block diagram form in order to avoid obscuring the present disclosure.

[0024] As used in this specification and claims, the words "for example," "for example," "e.g.," "e.g.," and the verbs "comprise," "have," "include," and other verb forms thereof, when used in conjunction with a list of one or more components or other items, should each be construed as open-ended, meaning that the list should not be viewed as excluding other additional components or items. The phrase "based on" means based at least in part on. Furthermore, it should be understood that the phraseology and terminology used herein are for purposes of description and should not be considered limiting. Any headings used within this description are for convenience only and have no legal or restrictive effect.

[0025] Domain adaptation (DA) aims to transfer knowledge from a well-labeled source domain to facilitate unlabeled target learning. Turning to specific tasks such as indoor (Wi-Fi) localization, learning a cross-domain regressor to mitigate domain shift is essential. However, in contrast to classification tasks, such tasks rely on regression because the trained neural network needs to identify locations in a continuous space, while the training data can, in practice, only come from samples in discrete locations.

[0026] Regression is a technique for determining the relationship between independent variables or features and a dependent variable or outcome. Regression may be used as a method for predictive modeling in machine learning, where an algorithm is used to predict a continuous outcome. Domain adaptation regression extends regression by generalizing the regressor from a labeled source domain to an unlabeled target domain.

[0027] For example, a regressor may be trained with labeled data in one environment. However, during the online testing phase, the regressor may receive unlabeled data in a new environment. As a result, a regressor trained with labeled data may suffer from performance degradation when the regressor processes unlabeled data. Therefore, a domain adaptation method for regressors is needed.

[0028] Some embodiments recognize that the challenges of domain adaptation for recurrent neural networks can be addressed using adversarial classifiers applied at the feature extraction level. The goal of such classifiers is to train a feature extractor to extract features from target-domain measurements that have a similar statistical distribution to the features extracted from source-domain measurements. However, arriving at such a classifier presents a problem in that the feature extractor is neither a classifier nor a regressor, requiring an additional decision unit to judge the quality of the extracted features. Such a decision unit is absent in typical regression architectures used for device localization.

[0029] To that end, some embodiments disclose a bi-regressor neural network and a three-stage training procedure for domain-adaptive regression. Figure 1A shows a schematic diagram of the architecture of a bi-regressor neural network 100 according to some embodiments of the present disclosure. The bi-regressor neural network 100 includes a feature extractor F(·) 103 and a bi-regressor including two regressors 105 and 107.

number

[0030] Each training stage of the three-stage training procedure is described below.

[0031] During the first training stage, the bi-regressor neural network 100 is trained using labeled data 101 to identify locations in the continuous space of the environment from the labeled data 101. In particular, the feature extractor 103 is trained to extract features from the labeled data 101, and each of the regressors, namely 105 and 107, is trained to determine locations in the continuous space of the environment from the features extracted by the feature extractor 103. In other words, the goal of each regressor is to learn a regression that can map the extracted features to locations in the continuous space of the environment. The need for two regressors 105 and 107 stems from the need to have a decision unit for the features extracted by the feature extractor 103.

[0032] In some embodiments, during the first training stage, the feature extractor 101 and the two regressors 105 and 107 use a loss function L r The two regressors 105 and 107 are trained using the labeled data 101 based on the loss function L. As a result, the parameters of the two regressors 105 and 107 are mostly similar. Therefore, when the two regressors 105 and 107 process the extracted features, they generate similar labeled positions that approximate the ground truth data. In one embodiment, the loss function L r is based on the mean square error (MSE) criterion. For example, the first training stage is performed in a supervised manner as follows:

number

[0033] 1B shows a schematic diagram of a second training stage according to some embodiments of the present disclosure. During the second training stage, the parameters of the feature extractor 101 trained during the first training stage are fixed, and the two regressors 105 and 107 are trained using the labeled data 101 and the unlabeled data 109 to generate the same correctly labeled location for the labeled data 101 but different locations for the unlabeled data 109. In this manner, the two regressors 105 and 107 are trained to generate different outputs (i.e., locations) for the unlabeled data 109 if the unlabeled data 109 has a statistical distribution different from that of the labeled data 101. As a result, the two regressors 105 and 107 are trained to identify whether the statistical distribution of the unlabeled data 109 has a similar or different distribution to that of the labeled data 101. As a result, the two regressors 105 and 107 can be judgers of statistical distribution.

[0034] Furthermore, the difference between the outputs of the two regressors 105 and 107 on unlabeled data can be maximized by minimizing the overlap / similarity between the two regressors 105 and 107. To minimize the overlap between the two regressors 105 and 107, the overlap between the outputs of the two regressors needs to be quantified. To quantify the overlap between the two regressors 105 and 107, some embodiments use the Jaccard similarity coefficient (IoU score) from the field of object detection using visual sensors. However, the non-differentiable nature of the IoU score poses a challenge to optimizing the weights of the bi-regressor neural network 100. To alleviate this challenge, a soft similarity function Ls111 between the concatenated outputs of the two components G and R of the two regressors is implemented. The output of the first regression component is denoted as G(f), and the output of the second regression component is denoted as R(G(f)), which is also the output of the regression. The concatenated output h = [G, R(G(f))].

number

[0035] However, it is still necessary that the two regressors 105 and 107 should produce the same output for the labeled data 101. To that end, the second stage of training not only minimizes the soft similarity function Ls 111 but also the loss function L r An objective function based on the (used in the first stage of training) and the soft similarity function Ls111 is minimized. The objective function minimization is given as:

number

[0036] Therefore, the combination of the two regressors 105 and 107 can be envisioned as an implicit classifier to directly distinguish between labeled data 101 and unlabeled data 109.

[0037] 1C shows a schematic diagram of the third training stage according to some embodiments of the present disclosure. In the third training stage, the parameters of the feature extractor 103 are trained by the adversarial classifier 113 using fixed parameters of the two regressors 105 and 107 trained during the second training stage using labeled data and unlabeled data. During the third training stage, the parameters of the feature extractor 103 are trained by the adversarial classifier 113 to extract domain-invariant features of the unlabeled data 109 using the statistical distribution of the labeled data 101, using the two regressors 105 and 107 as judges. The extracted domain-invariant features, when processed by the two regressors 105 and 107, are such that each of the two regressors 105 and 107 identifies the same position while processing the domain-invariant features.

[0038] In addition, some embodiments recognize that domain adaptation becomes difficult when the distribution difference is large. As used herein, distribution difference refers to the difference between the outputs of the two regressors 105 and 107. For example, in the case of multimodal indoor localization, the distribution difference becomes relatively significant. Some embodiments recognize that such a problem can be solved by constructing two intermediate domains between the source domain and the target domain and gradually eliminating the mismatch between them to achieve statistical distribution matching. The intermediate domains are constructed based on data augmentation of the labeled data 101 and the unlabeled data 109. Initially, a fixed ratio λ is selected, for example, λ is set to 0.7.

number

[0039]

number

[0040] After performing the above three-stage training procedure, a trained bi-regressor neural network is obtained. During the online, i.e., real-time, stage, the trained bi-regressor neural network can be used to determine the location of the device in the environment for input unlabeled data.

[0041] 2A shows a schematic diagram of the architecture of a trained bi-regressor neural network 200 for determining the location of a device for input unlabeled data 201, according to some embodiments of the present disclosure. The trained bi-regressor neural network 200 includes a trained feature extractor 203 obtained from the third training stage and two trained regressors 205 and 207 obtained from the second training stage. The unlabeled data 201 is input to the trained feature extractor 203. The trained feature extractor 203 extracts domain-invariant features from the unlabeled data 201. The extracted domain-invariant features are further processed by the two trained regressors 205 and 207. Each of the two trained regressors 205 and 207 outputs the same location of the device in the environment.

[0042] Because the two trained regressors 205 and 207 of the trained bi-regressor neural network 200 output the same location for the input unlabeled data 201, some embodiments are based on the recognition that the architecture of the trained bi-regressor neural network 200 can include only one trained regressor, rather than two, to determine location. Such an architecture of a bi-regressor neural network is shown in FIG. 2B.

[0043] 2B shows the architecture of a bi-regressor neural network 209 for determining location according to some other embodiments of the present disclosure. The bi-regressor neural network 209 includes a trained feature extractor 203 and a trained regressor 207. Unlabeled data 201 is input to the trained feature extractor 203. The trained feature extractor 203 extracts domain-invariant features from the unlabeled data 201. The extracted domain-invariant features are further processed by the trained regressor 207 to output the location of the device within the environment.

[0044] Alternatively, in some embodiments, the bi-regressor neural network 209 may include trained regressor 205 instead of trained regressor 207 .

[0045] Some embodiments are based on the recognition that a trained bi-regressor neural network for determining location may include a trained feature extractor 203 and an original regressor (the regressor obtained from the first stage of training). Such a bi-regressor neural network is shown in FIG. 2C. FIG. 2C shows the architecture of a bi-regressor neural network 211 for determining location according to some other alternative embodiments of the present disclosure. The bi-regressor neural network 211 includes the trained feature extractor 203 obtained from the third training stage and two regressors 105 and 107 obtained from the first training stage. Alternatively, in some embodiments, the bi-regressor neural network 211 may include the trained feature extractor 203 obtained from the third training stage and one of the two regressors 105 and 107 obtained from the first training stage.

[0046] According to some embodiments, the position of a device in an environment output by a trained biregressor neural network (e.g., trained biregressor neural network 200 or 209) may be used to control the device in the environment to perform a task. For example, the device may be a robot and the environment may be an autonomous factory. Based on the position of the robot in the autonomous factory, the robot may be controlled to perform a task, such as reaching a target position, as described below in FIG. 3.

[0047] 3 illustrates control of a robot 301 in an autonomous factory 300 in accordance with some embodiments of the present disclosure. The robot 301 has a trained bi-regressor neural network 200 embedded therein. The robot 301 also includes a controller 303. The robot 301 receives unlabeled data including signal measurements at unidentified locations within the autonomous factory 300. The unlabeled data is input to the trained bi-regressor neural network 200 to determine a position 305 of the robot 301 within the autonomous factory 300. Based on the position 305 of the robot 301, the controller 303 determines a motion path 307 connecting the position 303 and a target position 309 within the autonomous factory 300. The controller 303 then controls the robot 301 according to the motion path 307. In some alternative embodiments, the controller 303 may make decisions regarding obstacle avoidance based on the position 305 of the robot 301.

[0048] Additionally, in some embodiments, the robot 301 may also be tracked within the autonomous factory 300 based on the location of the robot 301. Additionally or alternatively, the trained bi-regressor neural network 200 can be used in any environment where asset (e.g., robot) tracking is required or important, such as an autonomous factory 300, warehouse, or smart home.

[0049] 4 is a schematic diagram illustrating a system 400 for implementing the methods of the present disclosure. The computing device 400 includes a power supply 401, a processor 403, a memory 405, and a storage device 407, all connected to a bus 409. A high-speed interface 411, a low-speed interface 413, a high-speed expansion port 415, and a low-speed connection port 417 may be connected to the bus 409. A low-speed expansion port 419 may also be connected to the bus 409. An input interface 421 may be connected to an external receiver 423 and an output interface 425 via the bus 409. The receiver 427 may be connected to an external transmitter 429 and a transmitter 431 via the bus 409. An external memory 433, an external sensor 435, a machine 437, and an environment 439 may also be connected to the bus 409. One or more external input / output devices 441 may also be connected to the bus 409. A network interface controller (NIC) 443 may be adapted to connect to a network 445 through bus 409 so that data or other data may be rendered on, among other things, a third-party display device, a third-party imaging device, and / or a third-party printing device external to computing device 400.

[0050] The memory 405 may store instructions executable by the computing device 400, as well as any data that may be utilized by the methods and systems of the present disclosure. The memory 405 may include random access memory (RAM), read-only memory (ROM), flash memory, or any other suitable memory system. The memory 405 may be a volatile memory unit and / or a non-volatile memory unit. The memory 405 may also be another form of computer-readable medium, such as a magnetic disk or an optical disk.

[0051] The storage device 407 may be adapted to store supplemental data and / or software modules used by the computing device 400. The storage device 407 may include a hard drive, an optical drive, a thumb drive, an array of drives, or any combination thereof. Additionally, the storage device 407 may include an array of devices, including computer-readable media such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid-state memory device, or a device in a storage area network or other configuration. The instructions may be stored on an information carrier. When executed by one or more processing units (e.g., the processor 403), the instructions perform one or more methods, such as those described above.

[0052] Computing device 400 may be linked via bus 409 to an optional display interface or user interface (HMI) 447 adapted to connect computing device 400 to a display device 449 and keyboard 451, which may include, among other things, a computer monitor, a camera, a television, a projector, or a mobile device. In some implementations, computing device 400 may include a printer interface for connecting to a printing device, which may include, among other things, a liquid inkjet printer, a solid ink printer, a large-scale commercial printer, a thermal printer, a UV printer, or a dye sublimation printer.

[0053] The high-speed interface 411 manages bandwidth-intensive operations for the computing device 400, and the low-speed interface 413 manages less bandwidth-intensive operations. This allocation of functionality is by way of example only. In some implementations, the high-speed interface 411 may be coupled to memory 405, a user interface (HMI) 444, a keyboard 451 and a display 449 (e.g., via a graphics processor or accelerator), and a high-speed expansion port 415 that may accept various expansion cards via the bus 409.

[0054] In one implementation, low-speed interface 413 is coupled to storage device 407 and low-speed expansion port 417 via bus 409. Low-speed expansion port 417, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled to one or more input / output devices 441. Computing device 400 may be connected to a server 453 and a rack server 455. Computing device 400 may be implemented in several different forms. For example, computing device 400 may be implemented as part of a rack server 455.

[0055] This description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with an enabling description for implementing one or more exemplary embodiments. Contemplated are various changes that may be made in the function and arrangement of elements without departing from the spirit and scope of the disclosed subject matter as set forth in the claims.

[0056] In the following description, specific details are given for a thorough understanding of the embodiments. However, it will be understood by those skilled in the art that the embodiments may be practiced without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in block diagram form so as not to obscure the embodiments in unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail to avoid obscuring the embodiments. Furthermore, like reference numbers and names in the various drawings indicate like elements.

[0057] Also, particular embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. While a flowchart may describe operations as a sequential process, many of the operations can be performed in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process may terminate when its operations are completed, but may have additional steps not discussed or included in the diagram. Moreover, not all operations in any specifically described process may occur in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, the termination of the function may correspond to a return of the function to the calling function or the main function.

[0058] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. Manual or automatic implementations may be performed or at least assisted by machine, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored on a machine-readable medium. The necessary tasks may be performed by a processor.

[0059] The various methods or processes outlined herein may be coded as software executable on one or more processors using any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine language code or intermediate code that runs on a framework or virtual machine. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0060] The embodiments of the present disclosure may be embodied as a method, of which an example is provided. The acts performed as part of the method may be ordered in any suitable manner. Thus, embodiments may be constructed in which acts are performed in a different order than illustrated, including simultaneously performing some acts shown as sequential acts in the exemplary embodiment.

[0061] Furthermore, embodiments of the present disclosure and the functional operations described herein may be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware including the structures disclosed herein and their structural equivalents, or one or more combinations thereof. Furthermore, some embodiments of the present disclosure may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by or to control the operation of a data processing apparatus. Furthermore, the program instructions may be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal generated to encode information for transmission to a suitable receiving device for execution by a data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or one or more combinations thereof.

[0062] According to embodiments of the present disclosure, the term "data processing apparatus" can encompass all types of apparatus, devices, and machines for processing data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. The apparatus can include special-purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the apparatus can also include code that creates an execution environment for the computer program, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0063] Computer programs (which may also be referred to as or described as programs, software, software applications, modules, software modules, scripts, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored as part of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, a single file dedicated to the program in question, or multiple cooperating files, e.g., a file that stores one or more modules, subprograms, or portions of code.

[0064] A computer program can be deployed to be executed on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network. Computers suitable for executing computer programs can include, for example, and may be based on, by way of example, general-purpose or special-purpose microprocessors or both, or any other type of central processing unit. Typically, a central processing unit receives instructions and data from a read-only memory or a random-access memory or both. The essential elements of a computer are a central processing unit for executing instructions and one or more memory devices for storing instructions and data.

[0065] Generally, a computer will also include one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, or be operably coupled to receive data from or transfer data to them, or both. However, a computer need not have such devices. Furthermore, a computer can be incorporated into another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0066] To provide for user interaction, embodiments of the subject matter described herein may be implemented on a computer having a display device, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, that displays information to the user, as well as a keyboard and pointing device, such as a mouse or trackball, that allows the user to provide input to the computer. Other types of devices may also be used to provide user interaction; for example, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; input from the user may be received in any form, including acoustic input, voice (speech) input, or tactile input. Furthermore, a computer may interact with a user by sending documents to and receiving documents from a device used by the user, for example, by sending a web page to a web browser on the user's client device in response to a request received from the web browser.

[0067] Embodiments of the subject matter described herein may be implemented in a computing system that includes a back-end component, e.g., a data server, or includes a middleware component, e.g., an application server, or includes a front-end component, e.g., a client computer having a graphical user interface or web browser through which a user can interact with an implementation of the subject matter described herein, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communications network. Examples of communications networks include local area networks ("LANs") and wide area networks ("WANs"), e.g., the Internet.

[0068] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0069] Although the present disclosure has been described with reference to certain preferred embodiments, it is to be understood that various other adaptations and modifications can be made within the spirit and scope of the disclosure. It is therefore the object of the appended claims to cover all such variations and modifications that come within the true spirit and scope of the disclosure.

Claims

1. 1. A computer-implemented method for training a neural network suitable for locating a device within an environment based on signals received by the device located within the environment, the method using a processor with stored instructions implementing the method, the instructions, when executed by the processor, performing steps of the method, the steps comprising: collecting labeled data including measurements of the signal, the labeled data being labeled with coordinates of locations of the measurements, the step further comprising: collecting unlabeled data comprising measurements of the signal at unidentified locations in the environment, the measurements of the labeled data and the measurements of the unlabeled data being sampled at the same or different discrete locations in the environment, the step further comprising: training a bi-regressor neural network to identify locations in a continuous space of the environment from the labeled data during a first training stage using the labeled data, the bi-regressor neural network including a feature extractor configured to extract features from the labeled data and a bi-regressor including two regressors having an identical architecture, each regressor trained to determine locations in the continuous space of the environment from the features received from the feature extractor; training parameters of the bi-regressors during a second training stage using the labeled data and the unlabeled data with fixed parameters of the feature extractor trained during the first training stage, such that each of the two regressors identifies the same labeled location while processing the labeled data and identifies different locations while processing the unlabeled data; and training parameters of the feature extractor using an adversarial classifier during a third training stage with fixed parameters of the bi-regressor trained during the second training stage using the labeled data and the unlabeled data to extract domain-invariant features from the unlabeled data according to an adversarial classifier using statistical properties of the labeled data, such that each of the two regressors identifies the same location while processing the domain-invariant features.

2. 2. The computer-implemented method of claim 1, wherein during the first training stage, the bi-regressor neural network is trained using a loss function such that each of the two regressors identifies the same location while processing the labeled data, the loss function being based on a mean squared error (MSE) criterion.

3. 2. The computer-implemented method of claim 1, wherein during the second training stage with fixed parameters of the feature extractor trained during the first training stage, the parameters of the bi-regressors are trained based on a soft similarity function such that each of the two regressors identifies different locations while processing the unlabeled data.

4. 4. The computer-implemented method of claim 3, wherein during the second training stage with fixed parameters of the feature extractor trained during the first training stage, the parameters of the bi-regressors are trained based on an objective function, the objective function being based on a loss function and the soft similarity function, such that each of the two regressors identifies identically labeled locations while processing the labeled data and identifies different locations while processing the unlabeled data.

5. 4. The computer-implemented method of claim 3, wherein during the third training stage with fixed parameters of the bi-regressor trained during the second training stage, parameters of the feature extractor and the adversarial classifier are trained based on an adversarial loss function and the soft similarity function, wherein the adversarial loss function is configured to mitigate domain distribution shifts.

6. 2. The computer-implemented method of claim 1, wherein during the third training stage with fixed parameters of the bi-regressor trained during the second training stage, parameters of the feature extractor and the adversarial classifier are trained based on maximization of an adversarial loss function.

7. The computer-implemented method of claim 1 , wherein the method further comprises determining the augmented labeled data and the augmented unlabeled data that form a plurality of intermediate domains.

8. 8. The computer-implemented method of claim 7, further comprising mitigating a domain shift between the augmented labeled data and the augmented unlabeled data based on an adversarial relationship between the adversarial classifier and the feature extractor to align statistical properties of the labeled data and the unlabeled data.

9. 8. The computer-implemented method of claim 7, wherein the augmented labeled data is determined by linearly combining the labeled data and the unlabeled data in a fixed ratio such that the augmented labeled data resembles the labeled data, and the augmented unlabeled data is determined by linearly combining the unlabeled data and the labeled data in the fixed ratio such that the augmented unlabeled data resembles the unlabeled data.

10. The method further comprises: inputting unlabeled data collected by the device into a trained bi-regressor neural network, the trained bi-regressor neural network including the feature extractor trained during the third training stage and at least one of the two regressors trained during the second training stage, the method further comprising:

2. The computer-implemented method of claim 1, comprising processing the unlabeled data with the trained bi-regressor neural network to output a location of the device within the environment.

11. The method further comprises: inputting unlabeled data collected by the device into a trained bi-regressor neural network, the trained bi-regressor neural network including the feature extractor trained during the third training stage and at least one of the two regressors trained during the first training stage using the labeled data, the method further comprising:

2. The computer-implemented method of claim 1, comprising processing the unlabeled data with the trained bi-regressor neural network to output a location of the device within the environment.

12. The method further comprises: determining a motion path for the device to perform a task within the environment based on the location of the device within the environment; and controlling the device based on the motion path.

13. The computer-implemented method of claim 11 , further comprising tracking the device within the environment based on the outputted location of the device.

14. 12. The computer-implemented method of claim 11, wherein the device is a robot and the environment is one of an autonomous factory, a warehouse, or a smart home.

15. 1. A system for training a neural network suitable for locating a device within an environment based on signals received by the device located within the environment, the system comprising: a processor; and a memory having stored thereon instructions that, when executed by the processor, cause the system to: and causing the system to collect labeled data including measurements of the signal, the labeled data being labeled with coordinates of locations of the measurements. The instructions, when executed by the processor, further cause the system to: and collecting unlabeled data including measurements of the signal at unidentified locations in the environment, the measurements of the labeled data and the measurements of the unlabeled data being sampled at the same or different discrete locations in the environment, the instructions, when executed by the processor, further cause the system to: and during a first training stage using the labeled data, training a bi-regressor neural network to identify locations in a continuous space of the environment from the labeled data, the bi-regressor neural network including a feature extractor configured to extract features from the labeled data and a bi-regressor including two regressors having identical architecture, each regressor trained to determine a location in the continuous space of the environment from the features received from the feature extractor. The instructions, when executed by the processor, further cause the system to: during a second training stage, using the labeled data and the unlabeled data to train parameters of the bi-regressors with fixed parameters of the feature extractor trained during the first training stage, such that each of the two regressors identifies the same labeled location while processing the labeled data and identifies different locations while processing the unlabeled data; and during a third training stage, using fixed parameters of the bi-regressor trained during the second training stage using the labeled data and the unlabeled data, training parameters of the feature extractor using the adversarial classifier to extract domain-invariant features from the unlabeled data according to an adversarial classifier using statistical properties of the labeled data, such that each of the two regressors identifies the same location while processing the domain-invariant features.

16. 16. The system of claim 15, wherein during the first training stage, the bi-regressor neural network is trained using a loss function such that each of the two regressors identifies the same location while processing the labeled data, the loss function being based on a mean squared error (MSE) criterion.

17. 16. The system of claim 15, wherein during the second training stage with fixed parameters of the feature extractor trained during the first training stage, the parameters of the bi-regressors are trained based on a soft similarity function such that each of the two regressors identifies different locations while processing the unlabeled data.

18. 18. The system of claim 17, wherein the parameters of the bi-regressors are trained based on an objective function, the objective function being based on a loss function and the soft similarity function, such that during the second training stage with fixed parameters of the feature extractor trained during the first training stage, each of the two regressors identifies identically labeled locations while processing the labeled data and different locations while processing the unlabeled data.

19. The processor further comprises: and configured to input unlabeled data collected by the device into a trained bi-regressor neural network, the trained bi-regressor neural network including the feature extractor trained during the third training stage and at least one of the two regressors trained during the second training stage, the processor further comprising: The system of claim 15 , configured to process the unlabeled data using the trained bi-regressor neural network to output a location of the device within the environment.

20. 1. A non-transitory computer-readable storage medium having embodied thereon a program executable by a processor for performing a method for training a neural network suitable for localizing a device in an environment based on signals received by the device located in the environment, the method comprising: collecting labeled data including measurements of the signal, the labeled data being labeled with coordinates of locations of the measurements, the method further comprising: collecting unlabeled data comprising measurements of the signal at unidentified locations in the environment, the measurements of the labeled data and the measurements of the unlabeled data being sampled at the same or different discrete locations in the environment, the method further comprising: during a first training stage using the labeled data, training a bi-regressor neural network to identify locations in a continuous space of the environment from the labeled data, the bi-regressor neural network including a feature extractor configured to extract features from the labeled data and a bi-regressor including two regressors having an identical architecture, each regressor trained to determine a location in the continuous space of the environment from the features received from the feature extractor, the method further comprising: training parameters of the bi-regressors during a second training stage using the labeled data and the unlabeled data with fixed parameters of the feature extractor trained during the first training stage, such that each of the two regressors identifies the same labeled location while processing the labeled data and identifies different locations while processing the unlabeled data; and training parameters of the feature extractor using an adversarial classifier during a third training stage with fixed parameters of the bi-regressor trained during the second training stage using the labeled data and the unlabeled data to extract domain-invariant features from the unlabeled data according to an adversarial classifier using statistical properties of the labeled data, such that each of the two regressors identifies the same location while processing the domain-invariant features.

Images