Method and apparatus for locating RFID tags

Abstract

A radio frequency identification (RFID) system includes an array of antennas that distinguishes line-of-sight (LOS) paths from non-line-of-sight (NLOS) paths. The distance between adjacent antennas in the antenna array is less than half the wavelength of the system's radio frequency (RF) signal. Each antenna in the antenna array is also digitally controlled to vary the relative phase difference between the antennas, thereby enabling digital steering of the antenna array over angles of arrival (AOA) between 0 and π. The digital steering generates a plot of signal amplitude as a function of AOA. LOS paths are distinguished from NLOS paths based on the shape (e.g., depth, slope, etc.) of the extrema (e.g., maxima or minima) of the plot. [Selection diagram] Figure 3-2

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority under 35 U.S.C. §119(e) to U.S. application Ser. No. 62 / 577,530, filed Oct. 26, 2017, and U.S. application Ser. No. 62 / 477,796, filed Mar. 28, 2017, each of which is incorporated by reference in its entirety. [Background technology]

[0002] Radio Frequency Identification (RFID) technology has applications in many commercial areas, such as access control, animal tracking, security, and toll collection. A typical RFID system includes a tag (also called a transponder) and a reader (also called an interrogator). The reader includes an antenna for transmitting radio frequency (RF) signals and receiving RF signals reflected or radiated by the tag. The tag may also include an antenna and an application specific integrated circuit (ASIC) or microchip. A unique electronic product code can be assigned to the tag to distinguish it from other tags.

[0003] RFID systems can use either active or passive tags. An active tag contains a transmitter to radiate an RF signal to a reader and a power source (e.g., a battery) to power the transmitter. In contrast, a passive tag does not contain a power source and derives power generated by the reader through induced current in the tag's antenna. In a passive RFID system, the reader transmits a signal with the reader antenna to excite the tag antenna. Once the tag is powered on (excited), it transmits stored data back to the reader.

[0004] A signal emitted or reflected by a tag can reach a reader via one or more paths. For example, the signal can travel along a straight line from the tag to the reader (called a line-of-sight or LOS path). The signal can also be reflected or scattered from obstacles (e.g., walls and other objects distributed throughout the environment) before reaching the reader. These paths are called non-line-of-sight (NLOS) paths. In some cases, a given signal can take multiple paths to a receiver, and several copies of the signal may reach the receiver. The reader sees each copy of the signal as originating from a different direction or angle of arrival. This phenomenon is called "multipath" in the field of RFID technology.

[0005] Multipath can cause unwanted interference and ghosts. If different copies of a signal overlap in time, they can interfere with each other. Destructive interference causes fading. If different copies of a signal do not overlap with each other, subsequent copies can appear as "ghosts." These ghosts can fool a receiver into thinking that extra RFID tags are present. Summary of the Invention

[0006] Embodiments of the present technology include methods and systems for locating radio frequency identification (RFID) tags. One example includes receiving a plurality of first RFID signals from a first RFID tag at a first unknown location using a system having one or more antennas or RFID tag readers. A processor coupled to the antenna designates the first RFID tag as a first virtual reference tag based on the plurality of first RFID signals. The antenna receives at least one second RFID signal from a second RFID tag at a second unknown location. The processor then determines a location of the first RFID tag relative to the first virtual reference tag based on the at least one second RFID signal.

[0007] Another example of the present technology uses a first antenna to receive a first line-of-sight (LOS) signal from an RFID tag. A processor coupled to the first antenna estimates a first angle of arrival, a first phase difference, and a first frequency difference of the first LOS signal to determine a change in the first phase difference with respect to the first frequency difference. A second antenna receives a second line-of-sight (LOS) signal from the RFID tag. The processor estimates a second angle of arrival, a second phase difference, and a second frequency difference of the second LOS signal to determine a change in the second phase difference with respect to the second frequency difference. The processor then estimates a location of the RFID tag based on the first angle of arrival, the change in the first phase difference with respect to the first frequency difference, the second angle of arrival, and the change in the second phase difference with respect to the second frequency difference.

[0008] Yet another example includes receiving, by a plurality of antennas, at least one RFID signal from at least one reference RFID tag. A processor operatively coupled to the antenna determines an estimated location of the reference RFID tag based on the RFID signal. It performs a comparison between the estimated location of the reference RFID tag and an actual location of the reference RFID tag. The processor is calibrated based on the comparison between the estimated location of the reference RFID tag and the actual location of the reference RFID tag. The antenna receives at least one RFID signal from an RFID tag at an unknown location. And the processor determines an estimated location of the RFID tag based on the RFID signal.

[0009] Yet another example includes receiving, by a plurality of antennas, reference RFID signals from respective reference RFID tags at respective known locations, the antennas also receiving at least one RFID signal from an RFID tag at an unknown location, and a processor coupled to the antennas determining the location of the RFID tag based on the RFID signals and the reference RFID signals.

[0010] Yet another example includes receiving a reference RFID signal from at least one reference RFID tag using an antenna array, a processor determining a receive sensitivity pattern of the antenna array based on the reference RFID signal, the antenna array receiving an RFID signal from an RFID tag at an unknown location, and the processor determining a location of the RFID tag based on the RFID signal and the receive sensitivity pattern of the antenna array.

[0011] Another example of the techniques of the present invention includes monitoring an RFID tag by receiving, with at least one antenna, a plurality of RFID signals from the RFID tag over a period of time. A processor coupled to the antenna estimates a plurality of possible trajectories of the RFID tag over a period of time based on the plurality of RFID signals. The processor then identifies a first trajectory of the plurality of possible trajectories as corresponding to a line-of-sight (LOS) path between the antenna and the RFID tag.

[0012] Another example method for locating an RFID tag includes receiving a signal from the RFID tag with a plurality of antennas. A processor generates a first digital representation of a response detected by a first antenna of the plurality of antennas and a second digital representation of a response detected by a second antenna of the plurality of antennas. The processor generates a plurality of sums of the first digital representation and the second digital representation, each of which is a relative phase difference representing a different angle of arrival for the signal from the RFID tag. The processor uses the sums to estimate the location of the RFID tag.

[0013] Embodiments of the present invention include devices, systems and methods for locating a radio frequency identification (RFID) tag. In one example, a method for locating an RFID tag includes detecting a signal from the RFID tag to a transmitter by a plurality of antennas. One or more analog-to-digital converters (ADCs) generate a first digital representation of a response detected by a first antenna of the plurality of antennas and a second digital representation of a response detected by a second antenna of the plurality of antennas. A processor coupled to the ADC generates a plurality of sums of the first digital representation and the second digital representation. Each sum of the plurality of sums is a relative phase difference representing a different angle of arrival for the signal from the RFID tag. The method also includes estimating a location of the RFID tag based on the plurality of sums.

[0014] All combinations of the foregoing concepts and additional concepts discussed in more detail below (provided such concepts are not mutually inconsistent) are considered to be part of the inventive subject matter disclosed herein. In particular, all combinations of subject matter recited in the claims appearing at the end of this disclosure are considered to be part of the inventive subject matter disclosed herein. Terms explicitly used in any disclosure incorporated herein by reference should be given the meaning most consistent with the specific concepts disclosed herein. [Brief description of the drawings]

[0015] Those skilled in the art will appreciate that the drawings are primarily for illustrative purposes and are not intended to limit the scope of the inventive subject matter described herein. The drawings are not necessarily to scale, and in some instances, various aspects of the inventive subject matter disclosed herein may be exaggerated or enlarged in the drawings to facilitate understanding of different features. In the drawings, like reference characters generally refer to like features (e.g., functionally similar and / or structurally similar elements). [Figure 1-1]FIG. 1A illustrates an example system for locating radio frequency identification (RFID) tags in an environment having line-of-sight (LOS) and non-line-of-sight (NLOS) paths between the RFID tag and a receiver. [Figure 1-2] FIG. 1B illustrates an example system for estimating the angle of arrival (AOA) of an incident signal. [Figure 1-3] FIG. 1C is a plot showing an example of composite signal amplitude versus angle of arrival / phase difference before (solid trace) and after (multiple lines) deconvolution or other correction by an antenna receive sensitivity pattern. [Figure 1-4] FIG. 1D shows a plot of RFID tag signal amplitude versus angle and elevation angle for four different RFID tags, each at a different AOA relative to the antenna. [Diagram 2] FIG. 2 is a block diagram of a transmitter and receiver suitable for use in the system of FIG. 1A. [Figure 3-1] FIG. 3A is a flow chart illustrating a method for locating an RFID tag using a system such as the one shown in FIG. 1A. [Figure 3-2] FIG. 3B illustrates locating an RFID tag using a virtual reference RFID tag, a reference RFID tag, and measurements by multiple readers from different AOAs. [Figure 3-3] 3C-3F are frames of a video showing the measured location of an RFID tag, indicated by a circle, and a bounding box drawn around an object tagged with the RFID tag using a neural network or other computer vision technique. [Diagram 3-4] FIG. 3G is a flow chart illustrating a method for associating RFID tag measurements with image data. [Figure 4] FIG. 4 illustrates the LOS and NLOS signal paths from a moving RFID tag to a pair of antennas, as well as the corresponding real and "ghost" velocity vectors and trajectories acquired by the LOS and NLOS signals. [Figure 5-1]FIG. 5A illustrates a retail and stockroom RFID tag location system. [Figure 5-2] FIG. 5B shows RFID tag transmitters and receivers on items for sale in the retail store of FIG. 5A. [Figure 6] FIG. 6 shows a graphical user interface (GUI) on a smartphone or tablet displaying employee and product locations derived from RFID tag location data. [Figure 7-1] 7A-7D show how the GUI can be used to select products or other items by RFID tag for a particular action. [Figure 7-2] Same as above. [Figure 8] FIG. 8 illustrates how a GUI can show real-time and / or historical activity of RFID tags in a store, stockroom, warehouse, or other environment monitored by an RFID tag location system. [Figure 9-1] 9A-9D show how the GUI can be used to pull items from a picklist based on RFID tag location data, for example, to plan and track pick paths in a stockroom or warehouse. [Figure 9-2] Same as above. [Figure 10] FIG. 10 shows how the GUI can be used to identify and locate flown items by their RFID tags. [Figure 11] FIG. 11 shows how the GUI can be used to fulfill a stock request for an item by RFID tag. [Figure 12] FIG. 12 illustrates how the GUI can be used to indicate the location of selected products tagged with RFID tags on the sales floor and / or in the stockroom. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] To date, RFID location technology has not lived up to its hype. Combined with computer vision technology, the RFID location technology of the present invention offers unprecedented speed and accuracy. In fact, it can be more than 300 times more accurate than conventional RFID location technology. For example, using the systems and methods disclosed below, RFID tags can be located to within 50 cm, 40 cm, 30 cm, 25 cm, 20 cm, 15 cm, 10 cm, 5 cm, or 2.5 cm of their actual location. This speed and accuracy allows it to be used to track RFID tagged items in real time, down to the slightest movement. This level of speed and accuracy makes it possible to find and restock items almost instantly, and track interactions between RFID tags. For RFID tags on products in a store, this can yield data about customer interactions with products at an item-by-item level, enabling autonomous checkout.

[0017] All of the techniques disclosed herein can be used with each other, except where they are not physically compatible. For example, an RFID tag location system can use multiple RFID tag readers to interrogate reference tags, virtual reference tags and RFID tags from many angles of arrival to create (multipath) signatures based on the received signals. Such a system can locate tags in two or three dimensions relative to each other and / or to absolute (known) locations. The resulting locations can be correlated with video data to train neural networks or manage store or warehouse operations. Location information can also be displayed on smartphones, tablets or other devices for inventory and supply chain management, etc., as described in more detail below.

[0018] 1. Multipath and RFID Signals To address the multipath problem in known radio frequency identification (RFID) technologies to accurately locate RFID tags, the systems, methods and apparatus described herein use an array of antennas to distinguish RF signals traveling along a line-of-sight (LOS) path from RF signals traveling along a non-line-of-sight (NLOS) path. The distance between adjacent antennas in the array of antennas can be less than half the wavelength of the system's radio frequency (RF) signal. Each antenna in the antenna array is also digitally controlled to vary its relative phase difference with respect to other antennas in the antenna array. Each distinct phase setting of the antenna array corresponds to a distinct angle of arrival (AOA) measured by the antenna array. This allows the antenna array to be digitally steered across an elevation angle AOA between 0 and π (i.e., between 0 and 180 degrees) and an azimuth angle AOA between 0 and 2π (i.e., between 0 and 360 degrees), as long as the array includes three or more antennas.

[0019] Digital steering then allows for the generation of a plot or other representation of signal amplitude as a function of AOA. LOS paths are differentiated from NLOS paths based on the extremes (e.g., maxima or minima) of the plot. For example, the highest (lower), steepest maximum (minimum) can be the AOA corresponding to the LOS path. Triangulation with the AOA against two or more distinct LOS paths yields a three-dimensional (3D) RFID tag location. Theoretically, this approach can locate items to perfect accuracy under perfect environmental conditions. Under realistic indoor conditions, location accuracy can be better than 50 cm using this technique.

[0020] The above estimated LOS paths can be used to determine the location of the RF tag via a triangulation scheme. A first antenna or group of antennas is used to estimate a first LOS path to the RF tag, and a second antenna or group of antennas is used to estimate a second LOS path to the same RF tag. Then, by triangulating between the two LOS paths, an estimate of the RF tag's location in 3D is provided.

[0021] The above approach utilizes digital steering of the antenna array and can be cost-effective in practice. In addition, this approach can be conveniently scaled up to multiple antenna arrays. These antenna arrays can be distributed in a given space (e.g., in the ceiling of a store or warehouse) to ensure that at least two antenna arrays have a LOS path with the RFID tag in the space. This can be particularly advantageous in indoor environments where there may be multiple obstacles. Examples of indoor applications of this RFID approach include retail stores, libraries, and warehouses, among others (see more detail below).

[0022] Digital steering can also be used to locate other RF transceivers, including those found in smartphones, wearables, tablets, laptops and other portable electronic devices equipped with WiFi, Bluetooth or similar antennas. Similar to RFID tag location, described briefly above and in more detail below, a transmitter emits a trigger signal to a device having a WiFi, Bluetooth or other RF transceiver. In response to this trigger signal, the device radiates a response that is detected by two or more receivers via LOS and / or NLOS paths. A processor coupled to the receiver steers the receiver sensitivity pattern by digitally steering the AOA for different combinations of receivers (e.g., paired combinations of receivers) and looking for the strongest signal as a function of AOA.

[0023] 2. A system for distinguishing between LOS and NLOS paths FIG. 1A shows a system 100 for distinguishing a LOS path 11 to a device or item having an RF transceiver, such as an RFID tag 10, a smartphone, a wearable computing device, a tablet or a laptop, from an NLOS path 13. The system 100 includes an RFID reader (transmitter) 110 and two receivers 120a and 120b (collectively referred to as receivers 120, also referred to as receiver antennas 120). The reader 110 and receiver 120 are coupled to a processor 130. Two receivers 120 are shown in FIG. 1A for illustrative purposes. In practice, the system 100 can include three or more receivers 120. These receivers 120 can be arranged in a one-dimensional (1D) or two-dimensional (2D) array. In another example, the receivers 120 are randomly or irregularly distributed in a given space.

[0024] The receiver 120 may form (part of) a phased antenna array, in which case the distance d between the two receivers 120a and 120b is substantially less than or equal to half the carrier wavelength λ of the radio frequency (RF) signal used to interrogate the RFID tag 10, i.e. d≦λ / 2. The system 100 may be configured to operate at any one of a variety of carrier wavelengths (and therefore a variety of carrier frequencies).

[0025] For example, the system 100 can use RF signals in the extremely high frequency (UHF) region of the electromagnetic spectrum (e.g., about 850 MHz to about 960 MHz) or microwave signals (e.g., 2.45 GHz). The corresponding carrier wavelength is about 31 cm to about 35 cm for UHF signals and about 12.2 cm for microwave signals. In this case, the distance d between the two receivers 120a and 120b can be substantially less than 17.5 cm for UHF frequencies or less than 6.1 cm for microwave frequencies. In other applications, such as outdoor applications, the system can operate at lower frequencies (e.g., 13.56 MHz, 125 kHz, etc.) with corresponding longer wavelengths (e.g., 22 meters, 2400 meters, etc.). To locate WiFi or Bluetooth devices, the system can operate in the unlicensed Industrial, Scientific, and Medical (ISM) bands from 2.0 GHz to 2.4 GHz, or any other suitable band (e.g., 5 GHz). Higher frequencies (shorter wavelengths) generally provide more accurate position estimates than lower frequencies (longer wavelengths).

[0026] The two receivers 120a and 120b each include an antenna 122a and 122b (collectively referred to as receiver antennas 122) to receive RF signals. The receiver antennas 122 can be digitally controlled to vary the phase difference of the signals they receive from the RFID tag 10. This digital control allows convenient steering of the two antennas 122 to different angles of arrival (AOAs).

[0027] In one embodiment, the reader 110 and the receiver 120 can be located in a single enclosure to form an integrated device. The processor 130 can also be integrated into the device. In another embodiment, the reader 110, the receiver 120 and the processor 130 can be distributed in different locations. For example, the receiver 120 can be located in a location that has a clear view of the space being monitored for tags 10 (e.g., on the ceiling of a room), while the processor 130 is located in a location that is better accessible to personnel (e.g., in a control room). The reader 120 can be connected to the processor 130 via one or more wired connections or through a wireless link (e.g., a WiFi link).

[0028] In operation, the reader 110 radiates an RF signal towards the RFID tag 10. In one embodiment, the reader 110 transmits the RF signal throughout a given space (e.g., a room). In another embodiment, the reader 110 radiates the RF signal with a smaller divergence to steer or sweep the RF signal throughout the space. In either case, when the RFID tag 10 is within the given space, the RFID tag 10 can radiate a response signal as understood in the art of RFID tags.

[0029] Depending on the locations of the RFID tag 10 and the receiver 120, the response signal may propagate directly from the RFID tag 10 to the receiver 120 along the LOS path 11 without being reflected or scattered. The response signal may propagate in other directions as well. For example, the response signal may be reflected or scattered off of a wall 12 (or any other obstacles distributed throughout a given space). In this case, the response signal reaches the receiver 120 along one or more NLOS paths 13. As mentioned above, this can create multipath problems and impair the accuracy and reliability of the system 100.

[0030] The system 100 shown in FIG. 1 can distinguish signals along LOS path 11 from signals along NLOS path 13 based on their respective angles of arrival (AOA). The distinction can be made by determining the angles of arrival that correspond to extrema (e.g., maxima and minima) of the antenna's receive sensitivity pattern. For example, the system processor 130 can coherently sum signals received by adjacent antennas 122 at each of several phase differences, each of which corresponds to a different AOA. The coherent sum that produces a maximum corresponds to the AOA at which the LOS and NLOS signals arrive. In the absence of attenuation, the highest and steepest maximum generally corresponds to the LOS path 11, while the other maximum corresponds to the AOA for the NLOS path 13.

[0031] 1B shows a receiver 120 used to estimate the AOAθ based on the phase difference of the signals received by the two antennas 122a and 122b. The RF signals 125a and 125b arriving at the two antennas 122a and 122b, respectively, can be considered to be substantially parallel to each other, provided that the distance d is sufficiently small (d<λ / 2) compared to the distance between the receiver 120 and the RFID tag 10. In this case, the signals 125a and 125b have the same AOAθ with respect to the antenna plane 15 defined by the two antennas 122a and 122b. Without being bound to any particular theory or mode of operation, the phase difference Δφ between the two signals 125a and 125b detected by the two antennas 122a and 122b, respectively, can be written as:

number

[0032] Equation (1) also represents the digital steering of the antenna 122 towards a different AOAθ. In this case, the phase difference Δφ between the two antennas 122a and 122b can be adjusted, for example, by applying a digital delay to one or both antennas 122a and 122b. This digital delay cancels the propagation delay Δ shown in FIG. 1B. Once the phase difference Δφ changes, the AOAθ changes accordingly, which means that the antennas 122a and 122b are steered towards different AOAθ to receive the signals 125a and 125b.

[0033] The phase difference Δφ may vary over a range such that the corresponding AOAθ varies from 0 to π. At each AOAθ, the corresponding signal amplitude may be recorded. The signal amplitude may be a coherent sum of the signals detected by the two antennas 122a and 122b. Upon completing a scan of the AOAθ, a plot may be generated showing the signal amplitude as a function of the AOAθ to find the LOS path 11 (see, e.g., FIG. 1C and the discussion below).

[0034] In the system 100, the processor 130 can be used to control the scanning of the AOA θ by controlling the amount of delay applied to the antenna 122. The step size of the scan Δθ can be about π / 1000 to π / 10 (e.g., about π / 1000, about π / 500, about π / 200, about π / 100, about π / 50, about π / 20, or about π / 10, including any value and subrange therebetween).

[0035] Processor 130 can also take advantage of estimated, known, or measured symmetries to reduce scanning and / or processing time. For example, processor 130 can select and digitally calculate phase differences Δφ to steer antenna 122 at symmetric angles (e.g., ±45°) instead of asymmetric angles (e.g., −45° and +44°). Because the angles are symmetric, they yield antisymmetric results (e.g., results that have only a sign difference) and can therefore be calculated in approximately half the time as asymmetric angles.

[0036] In addition, knowledge of the antenna pattern can be used to reduce the number of angles that need to be calculated for a given measurement accuracy. For example, the sensitivity of the antenna 122 may change rapidly around certain angles. At or near these angles, the step size of the scan Δθ can be reduced to sample more AOA and obtain more accurate results. In contrast, at angles where the sensitivity of the antenna 122 is relatively constant, the step size of the scan Δθ can be increased to sample fewer, thereby reducing scan and processing times.

[0037] FIG. 1C shows a plot 150 of conceptual signal amplitude A versus AOAθ, i.e., A(θ), for an RFID signal received by a pair of antennas 122 as shown in FIG. 1A. The top trace 151 represents a composite signal formed by digitally incrementing the phase difference between the signals received by the antennas. In this case, the composite signal includes a first maximum 155a near −3π / 8 and a second maximum 155b near +π / 4. The second maximum 155b is relatively high and narrow (sharp), while the first maximum 155a is relatively short and wide. In this case, the high and narrow second maximum 155b corresponds to a signal arriving at the receiver along a LOS path (e.g., path 11), while the wide and short first maximum 155a corresponds to a signal arriving at the receiver along a NLOS path (e.g., path 13).

[0038] The processor 130 can further be used to modify the antenna receive sensitivity pattern S(θ) from the signal amplitude A, resulting in a line 152 shown along the horizontal axis. This modification can simplify the identification of angles of arrival corresponding to LOS and NLOS paths between the antenna and the RFID tag. Without being bound to any particular theory of operation mode, this modification can be done by calibrating the antenna design to exclude the calibrated gain pattern or by modifying the measured signal amplitude A measured This can be done by effectively deconvolving the calibration pattern from (curve 151), which shows that the measured signal is essentially the true signal A convolved with the antenna receive sensitivity pattern S(θ). true This is because of the convolution of amplitudes.

number

[0039] The antenna receiving sensitivity pattern S(θ) is determined by a known radiation pattern (e.g., A true ) can be measured using a reference antenna with the radiation source A measured Then, the receiver sensitivity pattern S(θ) can be calculated according to equation (2).

[0040] After deconvolution or other modification, the amplitude curve 151 is transformed into two peaks 156a and 156b. The higher peak 156b corresponds to the LOS path and the lower peak 156a corresponds to the NLOS path between the antenna and the RFID tag. If desired, the processor can fit a curve (e.g., a Lorentzian or Gaussian) to the peak 156 to generate a more accurate estimate of the AOA for the LOS and NLOS paths.

[0041] 3. Estimating the location of an RFID tag Based on the AOA of the LOS paths, the processor 130 can estimate the location of the RFID tag 10 using a triangulation scheme. More than one group of antenna arrays can be used. For example, a first antenna array, such as the two antennas 122, is used to identify a first LOS path between the RFID tag 10 and the first antenna array. A second antenna array (not shown) is used to identify a second LOS path between the RFID tag 10 and the second antenna array. The location where the two LOS paths intersect each other (or where the error between them is minimized) is likely to be the location of the RFID tag 10 in the plane of the LOS paths to the first and second antenna arrays.

[0042] Optionally, the processor can estimate the distance between each antenna and the RFID tag based on the amplitude or received signal strength indication (RSSI) of each LOS signal, or based on the slope of the phase difference over the frequency difference. With two or more distance estimates, the processor can trilaterate the location of the RFID tag in addition to or instead of triangulating based on the AOA. These distance estimates can be used to more accurately or uniquely estimate the location of the RFID tag without the AOA.

[0043] The slope of the phase difference over the frequency difference is related to a technique regularly used in radar and radar-like systems, where the phase of the received signal is directly compared to the phase of the transmitted signal. For an item (a tag in this example) at a given distance from the reader, this phase offset should vary in a predictable way with its carrier frequency. Thus, capturing this relative phase offset φ at multiple carrier frequencies, f, allows an estimate of the distance from the reader as follows:

number

[0044] 4 Training and operation of the RFID tag location system Before becoming operational, the RFID tag location system may undergo a training phase. In this training phase, the RFID tag location system estimates the location of a reference RFID tag or other transceiver of known location. The system calibrates itself by comparing the estimated locations of the reference RFID tags with their actual locations. Once training is complete, the system is capable of locating unknown RFID tags, smartphones and / or other devices. The system may repeat training periodically (e.g., at night, on weekends, etc.) or when desired (e.g., in response to a user command).

[0045] To see how an exemplary system (e.g., the system of FIG. 1A) determines and estimates the LOS and NLOS paths, consider a reader that emits a continuous wave (cw) RF interrogation signal at a wavelength of λ. In a first (training) phase, the reader interrogates a set of tags whose locations are known. These tags are called reference tags. Each reference tag receives this interrogation signal and emits a response signal that is received in turn by each of k=1 to K antennas, each of which is located along a line segment of length D, where x k = kD / (K-1) is the lateral position of the kth antenna. (Other antenna arrangements are possible.) Each antenna in the array detects the output of the tag and produces a complex baseband signal s k Emits.

[0046] In the absence of multipath, the expected spatial response of each antenna to a tag with an arrival angle θ is:

number

[0047] (Since the system is not a beamforming system, it is not necessary to account for the gain.) The power received across the antenna array in the θ direction can be calculated as follows:

number

[0048] B(θ) is also referred to as the multipath profile of the antenna array because it (i.e., B(θ)) takes into account the radiated power from signals along the LOS and NLOS paths. The system measures the multipath profile of the antenna array for all of the reference tags and at each of several AOAs for one or more readers. Once the processor determines the AOA for the LOS path between the antenna and the reference tag, the processor can calculate the position of the reference tag using triangulation and / or trilateration as described above.

[0049] The above techniques can be extended to 2D (or even 3D) antenna array topologies. For a simple 2D array, for example a 2×2 uniform rectangular array, assuming isotropic elements in the xy plane, the steering vector is given by:

number

number

[0050] The power experienced at each 3D angle (θ,φ) and therefore the 3D multipath profile is calculated by:

number

[0051] FIG. 1D shows 3D multipath signatures measured for RFID tags at different positions and angles of arrival relative to a common receiver (antenna). Each plot shows RFID tag signal amplitude versus azimuth and elevation angles. The peaks represent the LOS and NLOS paths between the tag and the antenna, with the highest and steepest peak in each plot representing the LOS path. These multipath signatures can be compared to each other to determine the relative AOA and RFID tag location, as described in more detail below.

[0052] After the system has completed training (measured all multipath profiles for desired angles of arrival), it enters a second (operational) phase in which it runs in that environment (or a replicated environment). In the operational phase, the system queries non-reference tags (i.e., tags whose locations are unknown) and calculates a multipath profile for each unknown tag / reader combination. The system compares the multipath profile for each unknown tag with the multipath profile for the reference tag to determine the location of the unknown tag.

[0053] The system can estimate the location of an unknown tag by taking a weighted sum of three or more reference tag locations, where the weights depend on the distance between the corresponding multipath profiles. For example, a reference tag location whose multipath profile more closely matches that of an unknown tag may be weighted more heavily than another reference tag location. The exact weights can be determined using an appropriate distance metric, e.g., Euclidean distance or "metric learning," which affects the locations of both the reference tag and the estimated locations of the other unknown tags. Alternatively, or in addition, the system can cluster the reference and unknown tags according to a property (e.g., multipath profile) and determine a representative example of that property for weighting.

[0054] The system can repeat the training phase, e.g., periodically, to account for changes in the environment, such as changes in the number and location of reference tags, and changes in the number, type and location of obstacles that cause multipath effects. The system can also be tested in a third (post-training) phase, in which an unknown tag is moved through a series of known locations within the environment, e.g., using a drone or robot. As in the operation phase, the system measures the location of the unknown tag and compares the measured location to the coordinates of the robot or drone to determine an optimal distance metric (metric learning) for weighting the reference tag locations.

[0055] In some cases, instead of computing a solution (e.g., AOA and LOS paths) from one antenna array and then overlaying another solution from another antenna array, raw data from both arrays can be taken and a single composite solution generated. This allows multiple solutions for arrays spaced farther than λ / 2 to be obtained. Solutions with spurious signals can be ruled out by checking the plausibility of the resulting position estimate.

[0056] 5 Transmitters and receivers for LOS and NLOS determination FIG. 2 illustrates an RFID system 200 that includes multiple readers 210a-210n (collectively, RFID readers 210) and multiple receivers 220a-220n (collectively, receivers 220). The RFID system 200 also includes a processor 230, a common local oscillator (LO) 240, and analog front ends 250a-250n (collectively, front ends 250). Each reader 210 is grouped with a corresponding receiver 220 and a corresponding front end 250 as shown in FIG. 2. Other arrangements of the readers 210, receivers 220, and front ends 250 are possible. For example, one or more receivers 220 and / or one or more front ends 250 can share a common reader 210.

[0057] Each reader 210 includes a corresponding digital-to-analog converter (DAC) 218. The input of the DAC 218 is coupled to the processor 230, and the output of the DAC 218 is coupled to a low pass filter 216. In operation, the DAC 218 generates an analog representation of the digital RFID tag interrogation signal generated by the processor 230. The filter 216 removes high frequency spurs and noise from the analog RFID tag interrogation signal. The output of the filter 216 is coupled to an intermediate frequency (IF) input of the mixer 214. The LO input of the mixer 214 is coupled to the LO 240. The mixer 214 mixes the analog RFID tag interrogation signal with a high frequency (e.g., 902-928 MHz) carrier from the LO 240 to generate an RF output, which is coupled to the power amplifier 212. The power amplifier 212 amplifies the RF output and couples it to the circulator 256, which transmits the amplified RF output to the antenna 252a via the band pass filter 254a. Antenna 225a may be any suitable single antenna element. Circulator 256 substantially prevents the amplified RF power from propagating to or through receiver 220. Antenna 252 transmits the amplified RF power to an RFID tag, which responds with its own analog response signal.

[0058] The antenna 252 receives the response signals from the RFID tags and couples them to a bandpass filter 254, which filters the response signal and couples it to a circulator 256. The circulator 256 then couples all or substantially all of the response signal to a low noise amplifier (LNA) 222. The LNA 222 boosts the amplitude of the response signal and couples it to a mixer 224, which mixes the response signal with an LO to create a downconverted RFID signal. The lowpass filter 226 removes high frequency noise and spurs from the downconverted RFID signal, which is digitized by an analog-to-digital converter (ADC) 228 and provided to the processor 230.

[0059] The antennas 252 shown in FIG. 2 form an antenna array with a fixed or known phase difference between pairs of adjacent antennas 252. Components and connections between components of the receiver 220 and the front end 250 can be calibrated, adjusted, lengthened, or shortened to provide a known and stable phase relationship between signals received by the nearest adjacent antennas 252. For example, at least one antenna of each pair of adjacent antennas may be coupled to a phase tuner (not shown) to set or adjust the phase relationship between the adjacent antennas 252. The relative phase relationship between adjacent antennas 252 can also be digitally measured and calibrated using a processor (e.g., the processor 230 or a different processor not shown in FIG. 2). Maintaining a fixed phase relationship between adjacent antennas 252 allows digital steering of the antenna's receive sensitivity pattern by digitally adjusting the phase difference between the signals.

[0060] The system architecture shown in Figure 2 can be used to locate any wireless system, including Bluetooth and WiFi, which just operate at different frequencies. A system for locating RFID, Bluetooth and / or WiFi devices can include multiple copies of the components shown in Figure 2, one copy for each type of device, operating at a different frequency band (e.g., 865-868 MHz or 902-928 MHz for RFID, 2400-2835.2 MHz for Bluetooth and 2.4 GHz or 5 GHz for WiFi).

[0061] 6. Method for estimating RFID tag location FIG. 3A illustrates a method 300 of estimating the location of an RFID tag, smartphone, or other device having an RF transceiver using a system such as that shown in FIGS. 1A and 2. In step 302, a transmitter radiates or transmits an RFID tag interrogation signal to one or more RFID tags within a volume of interest, e.g., a store, stock room, warehouse, or other environment in which the RFID tags are used. (Step 302 can be omitted when locating devices using active transmitters, e.g., cellular, WiFi, or Bluetooth transmitters.) The RFID tags respond to the RFID tag interrogation signal by radiating an analog RFID signal, called a response signal. Two or more antennas receive the response signals in step 304. One or more ADCs digitize the analog response signals in step 306. In addition, electronic components coupled to the antennas can also downconvert and filter the analog response signals to facilitate subsequent processing. The resulting digital RFID signals can be stored and processed in real time, post-processed, or both.

[0062] As described above, a processor coupled to the electronic components uses the digital RFID signal to identify the AOA of the signal relative to the antenna. For example, the processor can electronically steer the antenna's receive sensitivity pattern across one or more AOAs in step 308. In one embodiment, the AOA can be selected a priori. For example, angles between 0 and π can be scanned using a uniform step size (e.g., about π / 1000 to about π / 10). Alternatively, or in addition, the AOA can be selected based on previous measurements to reduce processing time. For example, at angles where the antenna sensitivity changes rapidly, the processor can use a smaller step size and take more samples. Additionally, the processor can use information about the RFID tag and the environment (including symmetry considerations) to select an AOA that is more likely to produce results to reduce processing time.

[0063] The processor can select a possible AOA based on a principal component analysis (PCA) of previously received signals. For example, the antenna can monitor the movement of a designated RFID tag. Between successive acquisitions of response signals by the antenna, the RFID tag may move by a small amount ΔL, which may be much smaller than the distance between the RFID tag and the antenna. In this case, the AOA corresponding to the stronger signal of these adjacent measurements may be substantially the same, so the AOA estimated in the previous measurement can be used in the subsequent measurement.

[0064] Each candidate AOA corresponds to a particular phase offset (also called a phase setting) measured by the antenna. Thus, the processor can determine the signal strength for each AOA by digitally adjusting the phase difference between the digitized RIFD signals from two or more antennas (e.g., nearest adjacent antennas) that have a known phase relationship, and then coherently summing the digitized RFID signals in step 310. This steers the receiver sensitivity pattern of the antenna through each corresponding AOA. It also results in the signal amplitude and phase detected by the antenna as a function of AOA (phase difference between the antennas). The steering produces a plot of the signal amplitude as a function of AOA (see, e.g., FIG. 1C).

[0065] In optional step 312, the processor may deconvolute or correct the antenna pattern from the plot of signal amplitude (see peak 152 in FIG. 1C). This facilitates determining the LOS path by examining the height of the peaks. Generally, the highest peak corresponds to the signal traveling along the LOS path. For a more accurate estimation, the processor may fit a curve to the peaks, for example using a polynomial or nonlinear regression, to estimate the AOA based on coefficients used to reduce or minimize errors associated with the curve fit.

[0066] After the antenna pattern has been deconvolved or corrected from the signal amplitude and phase, the processor may look for minima (valleys) instead of maxima (peaks). In this case, the processor may identify the LOS and NLOS paths based on valley depth, valley width, valley slope, or some combination thereof. For example, the processor may identify the deepest, steepest valley in the representation of the signal amplitude relative to the AOA that corresponds to the AOA for the null along the LOS path to the RFID tag. Other valleys may correspond to angles relative to the null of other NLOS paths to the RFID tag.

[0067] In step 314, the processor compares the amplitude and phase with the AOA offset to determine the angle of arrival that most likely represents the LOS channel for the RFID tag. The processor may identify the LOS and NLOS paths based on the height of the maximum, the width of the maximum, the slope (rate of change) of the signal amplitude versus AOA, the curve fit coefficients, or a combination thereof. For example, the processor may look for the highest, steepest maximum in the representation of the signal amplitude versus AOA. This maximum represents the angle at which a peak in the antenna's receiver sensitivity pattern is exhibited along the RFID tag's LOS path. Other maxima may represent angles at which peaks are exhibited along the RFID tag's NLOS path.

[0068] In some cases, the processor associates the response signal with an expected response from the RFID tag. This can be done, for example, in step 308 of method 300. In this case, the system (e.g., of FIG. 1A) builds or uses a library of expected responses from the RFID tag. Each expected response corresponds to a distinct AOA. The processor compares the detected response signal to the expected responses to find the closest expected response. The AOA of the closest expected response is deemed to be the AOA of the response signal. This technique is similar to matched filtering and can increase the signal-to-noise ratio (SNR) by up to 20 dB or more.

[0069] In optional step 316, the processor estimates the location of the RFID tag in the environment using the different angles of arrival from different pairs of antennas. For example, the processor may triangulate the location of the RFID tag in at least two dimensions (e.g., in a plane parallel to the floor) using two or more estimated angles of arrival in that same plane. If the antennas are in different planes, the processor may estimate the location of the RFID tag in 3D space based on three or more estimated angles of arrival in different planes.

[0070] If more than one antenna is used in step 304, and the antennas are not all collinear, then each RFID receiver can find the angle to the tag in 3D space, so that in optional step 316 the location of the tag can be determined without the constraints of antenna arrays in different planes.

[0071] In another optional step 318, the processor can track the change in the tag's location over time. More specifically, it can smoothly map the various changes in the tag's location to a path in 2D or 3D space. To do this, the system measures the tag's location at many points in time, for example, once every second or every few seconds. It calculates the tag's location at each point in time and performs a vector distance determination between successive positions to determine the tag's speed. The processor can classify the tag's speed by speed and direction and, based on the speed and direction, determine the likely trajectories of the tag and which are (likely) carrying or moving the RFID tag. For example, if the tag is moving at a walking pace toward the exit, the system can determine that a customer is checking out or taking the tagged item to the store exit. Alternatively, if the tag is moving to and from a stockroom in a short time, the system can determine that an employee is restocking or shelving the tagged item.

[0072] The system can also use measurements at many points in time to distinguish LOS signals from NLOS signals. If the system detects an LOS signal and one or more NLOS signals, each appearing as a separate "ghost" tag, it can build a trajectory for each signal. The trajectory for an LOS signal should vary smoothly, whereas the trajectory for an NLOS signal can change direction sharply as the tag moves relative to the antenna and obstacles that scatter or reflect the NLOS signal. More specifically, the processor can use the time-varying measurements of the LOS and NLOS signals to generate a principal vector of the tag's trajectory. Vectors that resolve to a smooth trajectory are likely to be LOS, while vectors that resolve to a rough trajectory or impossible trajectory (e.g., due to some given or predetermined maximum speed of a human) are dismissed as multipath (NLOS) rays.

[0073] In systems with multiple pairs of antennas (i.e., three or more antennas), the processor can perform steps 302, 304, 306, 308, 310, 312, and 314 for different combinations of readers and antenna pairs to derive additional LOS and NLOS path information for one or more of the RFID tags in the environment. With a single reader and three or more antennas, for example, the processor can calculate the angle of arrival for the LOS path to each pair of adjacent antennas. If the midpoints of the line segments connecting different pairs of antennas are in different locations, then each pair of antennas can have a LOS path to an RFID tag with a different angle of arrival.

[0074] The processor may also perform steps 302, 304, 306, 308, 310, 312, and 314 for a combination of single pairs of antennas and multiple readers. For example, the readers may emit signals synchronized in time and phase such that the readers interrogate RFID tags in a staggered or round-robin fashion. The processor uses information about the time and phase of the interrogation signal and the position of each reader relative to the antenna pair to determine angles of arrival for the LOS and NLOS paths.

[0075] For systems with multiple readers, the processor can resolve the angle of arrival of each major component of the detected signal based on the location of the reader that triggered the signal. The processor can determine that those readers with matching locations / intersecting angles of arrival share the same LOS path to the RFID tag. The processor can use this information to determine that other lines are the result of multipath (i.e., NLOS) rather than LOS paths.

[0076] Another method is to map the tag trajectory to the trajectory of a human being within the field of view of one or more cameras. The cameras can be positioned to monitor the same volume that the antennas use to monitor RFID tags. This method can be used in conjunction with the above method to provide a single trajectory as opposed to multiple trajectories with some vertical or horizontal shift. More specifically, the cameras can be used to detect groups of moving pixels (e.g., representing people or objects tagged with RFID tags). A processor coupled to the cameras determines the trajectory of the groups and assigns the RFID tag location to the group with the matching trajectory. The processor can also segment and out the human body or perform pose estimation. For example, the processor can evaluate the difference in the trajectory of a bag being swung in the person's hand relative to the trajectory of the person.

[0077] The reader, antenna and processor may repeatedly perform steps 302, 304, 306, 308, 310, 312 and 314. In one embodiment, steps 302-314 are performed at regular intervals. For example, the steps may have a repetition rate of about 0.1 Hz to about 100 Hz (e.g., about 0.1 Hz, about 0.2 Hz, about 0.5 Hz, about 1 Hz, about 2 Hz, about 5 Hz, about 10 Hz, about 20 Hz, about 50 Hz or about 100 Hz, including any value and subrange therebetween).

[0078] In another embodiment, steps 302-314 can be executed at predetermined times in response to a command or a trigger event, or both. For example, steps 302-314 can be executed periodically (e.g., every hour, every day, during an evening inventory, etc.). They can be executed in response to the arrival of a new shipment, a stock or restock event, a user command, or the detection of possible theft. For example, a store manager can initiate process 300 when the store opens in the morning and closes in the evening. Or, a processor can initiate process 300 automatically, for example, at predetermined times or in response to data from other sensors, including video cameras monitoring the same space as the RFID tag location system.

[0079] In yet another embodiment, steps 302-314 can be repeated more or less frequently in response to measured changes in the number or location of RFID tags. For example, the processor can determine that a first RFID tag is moving if its location or corresponding LOS path angle of arrival changes smoothly as a function of time. The processor can associate the movement of the first RFID tag with the movement of the person from video or image data of the person, or information about a second RFID tag, smartphone, or other RF transceiver carried by or attached to the person. If the processor associates the movement of the first RFID tag with the movement of the person, the processor can determine that the person is also carrying the first RFID tag.

[0080] The processor can use this information regarding the movement of the first RFID tag along with knowledge of the location of the first RFID tag to trigger other actions. For example, when the first RFID tag reaches a particular area or volume or crosses a boundary surrounding an area or volume, the processor can debit a person's account for the purchase of an item associated with the first RFID tag. The processor can also update product inventory to reflect the movement or purchase of products or sound an alarm if the movement of the first RFID tag is unauthorized.

[0081] 7 Virtual Reference Tags The system shown in Figures 1 and 2 and the process shown in Figure 3A can be used to identify "virtual reference RFID tags" or "virtual reference tags," which are RFID tags that can be used to generate accurate location estimates for other RFID tags. Using virtual reference tags provides greater location accuracy for scenarios where there are multiple tags between the references, and the denser the environment, the more accurate the location. Virtual reference tags also allow for the measurement of relative distances between items, even in the absence of non-virtual reference tags. For example, knowing that tag B is between tags A and C can be very helpful, even if the exact locations of tags A and C are not known.

[0082] A simple way of looking at reference tags is shown in FIG. 3B, which shows a system with several RFID readers 320a-320c (collectively, RFID readers 320) that interrogate RFID tags in a store or other environment. The RFID tag readers measure the LOS and NLOS signatures of the RFID tags at different angles of arrival (AOAs). A plot from each reader shows the LOS signatures for a subset of the tags. A processor 328 wirelessly coupled to the RFID readers 320 compares these signatures with each other to obtain information regarding the relative locations of the RFID tags. This processor 328 may also be coupled via a smartphone, tablet or computer to a remote server or a computer network such as the Internet to share and use information regarding the tag locations, as described in more detail below.

[0083] The processor 328 can determine the RFID tag location by fitting the signature to a curve representing the receiver sensitivity pattern of the RFID reader, determining the peaks (maximums) of each curve, and interpolating between adjacent peaks to determine the Euclidean distance between the peaks. This Euclidean distance represents the error or deviation in the AOA for the corresponding RFID tag and RFID reader. For a pair of AOAs, if neither is known, the error represents the difference in the AOA (i.e., the relative AOA), and if one AOA is known, the other can be estimated. Multiple relative (or absolute) AOAs for a single RFID tag can be used to estimate the relative (or absolute) location of the RFID tag. For example, collecting more data about the RFID tag by taking more measurements with more RFID readers over more AOAs improves the location estimation accuracy beyond 50 cm, 40 cm, 30 cm, 25 cm, 20 cm, 15 cm, 10 cm, or 5 cm.

[0084] 3B illustrates how this can be used to locate an RFID tag 322 in an unknown position relative to other RFID tags and relative to one or more known positions. This shows a 1D diagram with reference tags 324a and 324b (collectively, reference tags 324) in known positions on either end of a linear rack. Virtual reference tags 326a-326c and an unknown RFID tag 322 are in the rack between the reference tags 324.

[0085] The plots below for the first and second RFID tag readers 320a, 320b show RFID tag signal amplitude versus angle of arrival for different RFID tags. These profiles show multipath signatures similar to those described above, with the highest peaks representing the LOS path 323 between the tag and the RFID tag reader 320. (The symbol on each peak matches the symbol for the corresponding tag on the rack.)

[0086] There are two plots for the first RFID tag reader 320a, the upper plot shows the multipath signature without any obstruction between the RFID tag reader 320a and the tags, and the lower plot shows the multipath signature with the presence of a person 321 between the RFID tag reader 320a and the tag on the left. Note that the person 321 attenuates / changes the multipath signature for some tags but not others, and does not affect the multipath signature received by the second and third RFID tag readers 320b, 320c.

[0087] The processor 328 can determine the relative location of the tags by comparing their multipath signatures to one another. In this example, the tag signature for the RFID tag closest to any of the reference tags 324 (e.g., RFID tags 326a and 326b) has the lowest error when compared to the tag signature of the corresponding reference tag 324. The error metric used to compare the multipath signatures may be mean square error (MSE), dynamic temporal distortion (DTW), or any other metric that can be used to compare the similarity of signatures. Using the example of MSE, the lower the metric, the more similar the multipath signatures are. In a scenario where a multipath signature for RFID tag 322 is compared to multipath signatures for reference tags 324a and 324b, if the error between the multipath signature for RFID tag 322 and the reference tag 324a is less than the error between the multipath signature for RFID tag 322 and the reference tag 324b, then the processor 328 determines that RFID tag 322 is closer to the reference tag 324a than to the reference tag 324b. If the error between the multipath signature for RFID tag 322 and the reference tag 324b is twice the error between the multipath signature for RFID tag 322 and the reference tag 324a, then the RFID tag 322 may be twice as far from the reference tag 324a as from the reference tag 324b. Other nonlinear weightings may also be valid.

[0088] If repeated RFID signal measurements indicate that RFID tags 326a and 326b are not moving, they can be added as "virtual reference tags" even though their absolute positions are not known (at least to the same level of accuracy of the position of the reference tag 324). This process can continue for other stationary RFID tags. For example, RFID tag 326c is closest to RFID tag 326a, so its RFID signature should be most similar to that of RFID tag 326a. If repeated measurements indicate that RFID tag 326c is also stationary, it may be designated as a virtual reference tag as well. Continuing this process, the system can establish an order of RFID tags (and therefore items tagged by the RFID tags). An error metric can be used as a surrogate for relative distance, and an estimate of absolute position can be established based on the relative distance and the known position of the reference tag 324.

[0089] Since the above approach relies on the relative error of the tag signature, it can be further improved by using signatures at multiple readers, calculating the error by reader, and summing (or combining) the errors at the different readers. This is where virtual reference tags can be used to reduce location measurement errors. Once the system has identified all of the stationary RFID tags, designated them as virtual reference tags, and located them at least relative to their nearest neighbors, the processor 328 can locate the desired RFID tag 322 based on their multipath signatures relative to one or more nearby virtual reference tags 326. Depending on the measurement error between different combinations of multipath signatures for the RFID tag 322 and the virtual reference tags 326, the processor 328 can improve the accuracy of its estimate of the RFID tag's actual location to within, for example, 50 cm, 40 cm, 30 cm, 25 cm, 20 cm, 15 cm, 10 cm, or 5 cm of its actual location. The accuracy becomes better as the number of virtual reference tags 326 increases and the accuracy of the location of each virtual reference tag improves.

[0090] The above 1D example can also be extended to 2D by placing a reference tag in a 2D space (e.g., a wall or floor) and comparing the errors of the tag, the reference tag and the virtual reference tag in this space. This example can be further extended to 3D by placing a reference tag in a 3D space and comparing nearest neighbors.

[0091] This approach can be further improved by anything that changes the RF communication channel between the tag and the reader. This can include moving the reader, changing the frequency at which the reader is operating, or even when a person (or object) moves into the space occupied by the reader and / or tag. For example, consider a person 321 walking between the virtual reference tag 326 and the first and second RFID readers 320a, 320b as shown in FIG. 3B. The person attenuates or scatters the RFID signals from the virtual reference tag 326 (as well as the unknown RFID tag 322 and the reference tag 324) that propagate towards the first and second RFID readers 320a, 320b. This creates a new set of signatures for the first and second RFID readers 320a, 320b that are completely uncorrelated with the set of signatures prior to the channel change and can therefore be used to reduce errors in the estimated location of the virtual reference tag 326 and the unknown RFID tag 322.

[0092] The problem with using virtual reference tags tends to be around the amount of processing power used to compare every tag signature to every other tag signature. The amount of processing power can be reduced by first comparing the RFID tag's signature to its last signature. If the signature has not changed, no comparison needs to be performed. Other ways to reduce processing power include using sources of side information (e.g., video of the area around the RFID tag and previous location information) to limit comparisons to signatures of RFID tags known to be close enough to be of concern.

[0093] To identify or designate an RFID tag as a virtual reference tag, the system measures the location of the RFID tag (e.g., using the methods described above) several times over a period of time as the environment around the RFID tag changes. These location estimates may be distributed over a region or volume whose size depends on the noise and measurement uncertainty. As the number of measurements increases, the average location estimate may converge to a smaller region or volume whose size is limited by the underlying measurement uncertainty. Once the size of the region or volume reaches a predefined threshold, the processor sets an appropriate tag location (e.g., the centroid of the region or volume) and uses this tag location as a reference for similar tags. The system may repeat this process until a desired number or set of RFID tags have been added to the pool of virtual reference tags. Once a reference tag is set (location is known), the method for calculating this reference tag location has been proven reliable and can be used to estimate the locations of other tags.

[0094] If the location of an RFID tag changes, the system can remove it from the virtual reference tag pool unless similar RFID tags show similar changes (e.g., due to changes in the environment, such as human disturbance as shown in FIG. 3B). The system can identify changes in a group of RFID tags by looking at the signature of each RFID tag in the group relative to the signatures of each part of the group. The system can also look for changes (or lack of change) in the signatures received at other AOAs from the group. In FIG. 3B, for example, a person 321 can change the LOS signature received in a correlated manner by the first and second RFID readers 320a, 320b, but should not affect the LOS signature received by the third RFID reader 320c. The combination of correlated changes from a particular AOA and no changes in other AOAs between the group of RFID tags can indicate that the group is not moving.

[0095] The change in RFID position may be discounted in one or more of the following circumstances: For example, if there is no relative change between different RFID tags, it may be likely that all of the tags are blocked or moving together; in another embodiment, the relative change is below a predefined threshold (e.g., measurement uncertainty); and in yet another embodiment, the relative change is short enough (e.g., within one, two, or three measurement cycles).

[0096] Information about the virtual reference tag can be combined with other information to improve RFID tag location accuracy. For example, an RFID tag location system can use the (estimated) positions of the virtual reference tag, product count data, and visual data from one or more cameras to determine the average density of products between virtual and / or real reference tags. In addition, visual data can be used to determine if a person is or was close enough to pick up or set down a product or RFID tag (pose / range estimation can be done similarly). If the visual data indicates that a virtual reference tag or a product with a virtual reference is or has been moved, the system can remove that virtual reference tag from the pool of virtual reference tags. Conversely, if the visual data indicates that a particular RFID tag has not moved for an extended period of time (e.g., hours or days), the system can designate that RFID tag as a virtual reference tag.

[0097] 8. Computer Vision Systems and Computer Vision Systems Training The RFID technology described above can be used to train a computer vision system to locate and / or identify different objects captured by a camera in or coupled to the computer vision system. For example, a computer vision system may include or be coupled to multiple cameras, which may be positioned to monitor a wide-angle area. Additionally, light sources emitting light of different wavelengths and / or intensities can be used to generate different environments to enhance the training of the computer vision system. Training images are acquired by the camera under different environments.

[0098] A combined computer vision / RFID tag location system can cross-reference the timestamp data of the scanned barcode / transaction, the item of the barcode / transaction, and the camera data corresponding to the location of the register or checkout kiosk to pull frames containing those items. A processor performing object detection on those frames can draw a bounding box around the images to generate additional tagged images.

[0099] During training of a computer vision system, e.g., a processor running an artificial neural network, the RFID technology described above is used to locate and identify objects in training images. These objects are separated into separate images, which are fed into a training set for the computer vision system (e.g., an artificial neural network). If the computer vision system does not distinguish between objects that are too close together, reinforcement learning can be used to filter out the objects. In addition, training can train the computer vision system to recognize objects such as light fixtures, doors, and shopping carts that may be irrelevant during the use of the computer vision system, e.g., in a retail store. These objects can then be removed from the training data set, and frames containing images of these objects can be marked as occluded frames. Because the RFID technology described above can automatically identify objects with RFID tags in an efficient manner, a large database of product images can be built without the need for a human to verify or inspect the contents of the database.

[0100] In certain cases, there may be errors in the location of the tag relative to the actual object. In these situations, one can plot the location of the tag from frame to frame against the location of the human or object from frame to frame (e.g., achieving a group of consecutive pixels moving via optical flow or re-identification), and average the distance from each set of consecutive pixel groups to each RFID tag across n frames, and group them based on which pairing resulted in the lowest error / average distance. Kalman filters work well to filter / group objects and / or objects. First and second derivatives of the RFID tag and pixel blob motion functions can also be incorporated to weight the matches. If the goal is to attribute the product to a person, one can perform person detection in each frame and simply select pixels corresponding to the person. If the goal is to collect annotated images, one can use person detection to simply ignore pixels corresponding to the person in the bounding box generation / pixel segmentation.

[0101] Figures 3C-3F show a sequence of video frames showing the movement of a shirt tagged with an RFID tag. The small circle represents the estimated location of the RFID tag attached to the shirt. The box is around a blob of pixels that the system has correlated to the RFID tag motion obtained from the RFID signal received by the RFID tag reader. As shown in Figures 3C-3F, there is significant error in where the object actually is from frame to frame versus where the system estimates the RFID tag is located, but the system is still able to give the pixel blob an RFID location estimate.

[0102] FIG. 3G illustrates a method 340 for correlating RFID tag location estimates with video data. This process 340 can be used to train a neural network to recognize items tagged with RFID tags or to correlate the motion of tagged and untagged objects (e.g., clothing and people with RFID tags). The process 340 begins by recognizing, segmenting out, and ignoring pixel blobs representing humans in an image with a trained neural network (342). Next, the remaining pixel blobs in the image are assigned a tracker that watches their motion from frame to frame (344). Each pixel blob is then matched with the RFID tag that most closely follows its trajectory given a certain threshold (346). Once pixel groups have been paired to their respective RFID tags, object detection can be performed on each pixel group in each frame to discard images that do not clearly match the RFID tag description due to environmental occlusions such as bags, carts, jackets, humans, and other such occlusion sources (348). Thinking of the tag data as being in pixel blobs also helps reduce or eliminate errors between tag location and item location.

[0103] Another note is that the RFID location provides constant feedback to the artificial neural network so that it is constantly learning what was right and wrong in each frame. This extends to autonomous checkout and human / product interaction, where the process 340 can be used to teach the vision system what was right and wrong on a frame by frame basis.

[0104] 9. Tracking moving RFID tags FIG. 4 illustrates how the RFID system and process described above may be used to track a moving RFID tag 402 in an obstacle-filled environment, such as a store, stockroom, or warehouse. In this example, a pair of RFID tag readers 410a and 410b (collectively, RFID tag readers 410) interrogate the RFID tag 402 by transmitting RFID interrogation signals at regular intervals, e.g., at a rate of about 0.01 Hz to about 1.0 Hz. The RFID tag readers 410 may change their interrogation rate based on the signals received from the RFID tag 410. If the RFID tag's response signal indicates that the RFID tag is moving fast, changing speed, or changing direction, the RFID tag readers 410 may increase their interrogation rate to provide finer spatial and temporal resolution of the RFID tag's motion. Conversely, if the RFID tag's response signal indicates that the RFID tag is stationary or moving slowly, the RFID tag readers 410 may decrease their interrogation rate to conserve energy. The RFID tag readers 410 can jointly or independently increase or decrease their interrogation rates in response to the relative motion of the RFID tags 402 .

[0105] The RFID tag reader 410 can broadcast interrogation signals over a wide range of angles, for example via an isotropically radiating antenna, or can scan them over different angles with an antenna array as described above. A processor (not shown) wirelessly coupled to the RFID tag reader 410 uses the RFID signals from the RFID tag 402 to calculate a velocity vector 481 and trajectory 491 for the RFID tag 402. To calculate a given velocity vector 481, the processor can determine the position 471 of the RFID tag at different moments in time and then determine a vector connecting those positions in 2D or 3D space. Scaling that vector by the time difference gives the velocity vector.

[0106] The processor can use the position measurements and / or the speed vector 481 to determine the trajectory 491 of the RFID tag. This can be a historical trajectory (i.e., where the RFID tag 402 has been) or a predicted trajectory (i.e., where the RFID tag 402 is going based on its estimated speed). If desired, the current speed, recent trajectory and / or predicted trajectory of the RFID tag can be displayed on a smart phone, tablet or other electronic device and used to initiate a transaction (e.g., a sale of an item associated with the RFID tag 402), prevent misplacement or theft, or track the item as it passes through the warehouse, as described in more detail below. If the speed vector 481 and the trajectory 491 indicate that the RFID tag 402 is not moving, the processor can select the RFID tag 402 as a virtual reference RFID tag, as described above.

[0107] The processor can also use the position measurements, velocity vector 481 and trajectory 491 to distinguish a "real" RFID tag, such as RFID tag 410 of FIG. 4, from a false signal or "ghost" RFID tag 482. In this case, ghost RFID tag 482 results from multipath effects. More specifically, FIG. 4 shows that RFID signals propagating between real RFID tag 402 and RFID tag readers 410a, 410b can take LOS paths 411a, 411b, resulting in accurate measurements of the RFID tag's position, velocity and trajectory. These RFID signals can also take NLOS paths 413 between real RFID tag 402 and RFID tag reader 410. In this example, some portion of the RFID energy emitted by RFID tag 402 reflects or scatters off walls 412 to the first receiver 410a. And when RFID tag 402 is in a particular position, this wall 412 prevents it from sending an RFID signal to second receiver 410b. In essence, wall 412 causes first RFID tag reader 410a to receive spurious RFID signals and stops second RFID tag reader 410b from receiving any RFID signals when RFID tag 402 is between wall 412 and first RFID tag reader 412a.

[0108] In this case, processing the pseudo RFID signal simply results in the appearance of ghost RFID tag 482 as shown in FIG. 4, complete with ghost location 473, ghost velocity vector 483 and ghost trajectory 493. The processor can distinguish ghost tag 482 from the corresponding real tag 402 based on the discontinuity between ghost velocity vector 483 and ghost trajectory 493 and / or based on the similarity between ghost velocity vector 483 and real velocity vector 481, and between ghost trajectory 493 and real trajectory 491. In particular, the real and ghost velocities and trajectories appear mirror symmetric about a line or plane defined by wall 412. The processor can use this mirror symmetry and the abrupt discontinuity at the beginning and end of ghost trajectory 493 (where wall 412 begins and ends) to distinguish between real RFID tag 402 and ghost RFID tag 482.

[0109] A camera 420 can be used to track the RFID tag 402 as well. In FIG. 4, the camera 420 takes a picture (e.g., at video rate or in the vicinity) of a person 401 who is carrying an RFID tag 402. A processor can use an artificial neural network to recognize the person 401 appearing in the image (e.g., as a person in general, as an employee, or as a specific person) and correlate the person's movements with the movements of the RFID tag 402. If the RFID tag 402 is on a name tag, wristband, or ID card, this can be done as part of the process of training the neural network to recognize the person associated with the RFID tag 402. If the neural network is already trained, the processor can use the overlapping or matching behavior of the person and the RFID tag 402 to perform tracking or to trigger another action, such as selling an item carried by the person or to which the RFID tag 402 is attached.

[0110] 10 RFID Tag Location System for Retail Space and Stockrooms 5A and 5B show different views of an RFID tag location system for a store 500. The RFID tag location system includes several RFID tag readers 510 distributed throughout the store's sales floor 590, stock rooms 592, and dressing rooms 580. The RFID tag readers 510 can be located, for example, at or near the ceiling, as shown in FIG. 5B, to provide a clearer line of sight to the RFID tags 502 on the merchandise on the sales floor 590 and in the stock rooms 592. Placing the RFID tag readers 510 above the RFID tags 502 also allows the 3D position of each tag to be determined using azimuth and elevation information obtained from the RFID signals received by the RFID tag readers.

[0111] RFID tags 502 can be distributed throughout the store 500, including items for sale such as clothing and other merchandise. The RFID tags 502 can be embedded in the items or attached to the items by tags, clips or stickers. There may be other types of RFID tags in the store 500, including reference RFID tags 504 at known locations such as fixed or movable clothing racks, walls or tables. In addition, some of the movable RFID tags 502 can be designated as virtual reference RFID tags 506 if they remain stationary for a long enough period of time. Also, some RFID tags can be attached to ID cards 508a and 508b (collectively, ID cards 508), key fobs, bracelets or other items worn or carried by employees 503a-503c (collectively, employees 503) or customers 501. These ID cards 508 can identify specific employees and their locations. Similarly, there may be an RFID tag 502 embedded or attached to a shopping bag, basket or cart at an entrance 596 to the store.

[0112] RFID tag reader 510 communicates wirelessly with processor / controller 530 via wireless router 540 or other suitable device. A camera, shown disposed with RFID tag reader 510, similarly communicates with processor 530. Processor 530 can in turn communicate with tablets 530 and smartphones 540 carried by customers 501 and employees 503. It can also communicate, via a suitable communications network, e.g., the Internet, with one or more servers, databases, or other remote devices that track the store's inventory and operations.

[0113] In operation, the RFID tag reader 510 measures the location, speed and trajectory of the RFID tag as described above. This information is used to monitor inventory and trigger actions associated with items tagged with the RFID tag 502. For example, if the RFID tag reader 510 detects the RFID tag 502 moving toward the store exit 598 and the camera 520 detects the customer 501 moving along the same trajectory, the processor 530 can trigger an automatic purchase of the associated item by the customer 501. This allows the customer 501 to skip the checkout 582 and save time. The processor 530 can also direct the customer 501 and employees 503 to specific items based on the precise RFID location estimate by the RFID tag reader 520. This functionality can be used to direct the customer 501 to a desired item, such as a shirt of a particular size or color, to control and re-shelve inventory such as items left in a locker room, or to determine how inventory placement impacts sales. These and other uses are described in more detail below.

[0114] 11 Application of High Precision Object Location Using RFID Technology The RFID tag localization techniques described above provide fine spatial resolution and high accuracy, making them suitable for a wide number of applications, many of which are not possible with other RFID tag localization techniques, some of which are described below and can be used by systems and in environments such as those shown in Figures 5A and 5B.

[0115] 11.1 Tracking RFID tag movement In one embodiment, RFID technology can be used in retail stores, especially omnichannel (also spelled omni-channel) stores. "Omnichannel" refers to a multichannel approach to sales that seeks to provide a seamless shopping experience to customers, whether they are shopping online from a desktop or mobile device, by phone, or in a brick-and-mortar store. What distinguishes an omnichannel customer experience from a multichannel customer experience is that there is true integration between channels on the backend. For example, when a store implements an omnichannel approach, a customer service representative at the store can immediately reference a customer's previous purchases and preferences just as easily as a customer service representative on the phone or a customer service web chat representative. Or, a customer can check the inventory by store on a company's website or app using a computer, tablet, or smartphone, and later purchase the item via the website or app and pick up the product at the customer's chosen location.

[0116] One problem retailers have with omni-channel ordering is detecting when an item has been selected through an omni-channel order. To address this problem, an RFID tag (called a tote tag) can be placed on a tote, shopping bag, shopping cart, or any other suitable container. Each item for sale also includes or has attached to it a separate RFID tag (called an item tag). An antenna array monitors the location of each tote tag and each item tag. If the distance between an item tag and a tote tag is below a threshold (e.g., less than the size of a tote), the system determines that the item corresponding to the item tag is a tote. To improve the reliability of detection, the system can further monitor the movement of the tote tag and the item tag. If they move together a distance above a threshold (e.g., greater than one meter), the system can determine that the item and tote are being carried by a customer.

[0117] The system can also monitor the customer's movements to determine if an item is selected by the customer. For example, if the item moves with the customer a distance above a threshold (e.g., greater than one meter), the system can determine that the item is being carried by the customer. For example, the customer can install a user app on their smartphone, and the system can detect the presence of the customer's smartphone through communication with the user app. The system can then track the movements of the smartphone (and, accordingly, the customer) using Bluetooth, WiFi, LTE, 3G, 4G, or any other wireless technology.

[0118] In some cases, the system may maintain a record of all smartphones that do not belong to customers (e.g., the store's own devices or employees' personal devices). Once the system detects a smartphone that is not in the record, the system may determine that the customer has entered the store and may track the customer's movements by tracking the smartphone.

[0119] The system can also track customer movements using facial recognition, gaiter recognition, or other recognition techniques. For example, a camera can be placed at the entrance of a store to recognize customers, and one or more cameras can be distributed throughout the store to monitor the entire store space. Each time a customer is captured and recognized by a camera, the location of the recognition can be recorded and compiled with previous locations to create a map of the customer's movements. The resolution of this monitoring (e.g., the distance between two recognitions of the same customer) can depend on the number of cameras in the store (e.g., the more cameras in a store, the higher the resolution can be). The system can then determine that an item is selected by a customer if they move together a distance greater than a threshold. Alternatively, or additionally, the system can determine that an item is selected by a customer if the items appear together in three or more locations. The system can also determine that an item is selected by a customer if the two locations where they appear together are more than one meter apart.

[0120] The system can further update the inventory when it determines that an item that was previously determined to be selected by a customer has been returned and is available for sale. The system can determine that an item has been returned if the item does not move for an extended period of time (e.g., more than five minutes). To improve reliability, the system can also check whether a customer is near the item while the item is not moving. If there are no customers remaining near the item, the system can determine that the item has been returned (e.g., because a customer who previously selected the item changed their mind and abandoned the item).

[0121] In another embodiment, RFID technology can be used to determine if an item is in the correct location in a store. In this case, one or more tags (called shelf tags) can be placed on each shelf that holds items for sale. Each shelf tag identifies a specific location on the shelf for a corresponding item. Each item also has an item tag. For example, the shelf tag can indicate the location of men's underwear and the item tag can be affixed to a pair of men's underwear. The system queries the locations of the shelf tag and the item tag to estimate the distance between them. If the distance is less than a threshold, the system can determine that the item is in the correct location. On the other hand, if the distance is greater than the threshold, the system can alert one or more store employees that the item is in the wrong location and should be moved to the correct location. The system can also provide instructions to the employees as to the actual location of the item and its appropriate location.

[0122] The system can also determine whether an item is in the correct location using tags attached to other retail fixtures (these tags are referred to as fixture tags). In general, each fixture tag can provide information regarding the identity of the fixture (e.g., shelf, table, counter, refrigerated display case, basket, grid, etc.), the location of the fixture, and the type and amount of items in the fixture. In some cases, the type and amount of items in the fixture can be determined based on industry standards. Alternatively, the type and amount of items in the fixture can be customized for each store.

[0123] Additionally, each employee may wear a tag (called an employee tag). In one embodiment, employees may wear a bracelet that includes an RFID tag. In another embodiment, the RFID tag may be sewn into the employee's uniform. In yet another embodiment, the RFID tag may be included on a badge worn by the employee. The system may use these tags to estimate and track the location of employees, for example, to manage inventory as described below.

[0124] In some cases, employee movements can be monitored by software without the use of an RFID tag attached to the employee. For example, the system can monitor employee movements by tracking the employee's smartphone. In these cases, the employee can install a user app to facilitate communication between the smartphone and the system. The system can recognize the employee, for example, from the employee's user account on the user app.

[0125] In another example, the system can track employees' wearable devices, such as smart watches, activity trackers (e.g., Fitbits), or smart glasses (e.g., eyeglasses with embedded electronics), among other devices. In this example, the system can maintain a record of the wearable devices owned by each employee to recognize the employee whenever a wearable device is detected. A system with a camera can also track employees via RFID tags on items that the camera recognizes as being held or carried by the employee.

[0126] For example, if the system determines that an item is misplaced or should be brought from back stock to the correct shelf, the system can estimate the locations of all employees using employee tags. It can then identify the employee closest to or heading toward the misplaced item. The system can alert that employee to place the item in the correct location. The system can also estimate and / or measure the time required for an employee to complete a task (e.g., time duration from alert to completion). This information can be used to review employee performance and identify changes to the store layout that can improve efficiency.

[0127] Several criteria can be used by the system to determine the appropriate employee to receive the alert. For example, the system can deliver the alert to an employee based on the employee's availability to receive and respond to the alert. In this example, the employee can communicate his availability (or unavailability) to the system via his employee device, e.g., a smartphone with a user app installed. The employee may indicate that he is in the middle of other tasks that should not be interrupted.

[0128] In another example, the system can send an alert to an employee based on their proximity to the item in question. For a misplaced item, the proximity can be quantified by the distance between the employee and the misplaced item. For an item being placed, the proximity can be quantified by the distance between the employee and the stockroom. In some cases, the proximity estimation takes into account the building or structure of the store. For example, instead of sending an alert to an employee on another floor, the system can send an alert to an employee, preferably on the same floor as the item in question.

[0129] In yet another embodiment, the system can send alerts to employees based on the employee's ability to complete a task. For example, if an item in the women's clothing department is found to be misplaced or abandoned in a woman's fitting room, the system can preferably send an alert to an employee in the women's clothing department instead of an employee in the grocery department.

[0130] The ability to complete a task can also be determined based on the current task the employee is handling. For example, if the employee has already handled several misplaced items, he may be more efficient at handling similar tasks. The system can also consult with a quality assurance system to determine the employee's ability. For example, the system can include a database of employee performance reviews for each task the employee has handled. If the system determines that the employee restocks the misplaced items with good efficiency, the system can preferably send an alert to the employee.

[0131] In yet another embodiment, the system can use a combination or weighted combination of the above criteria to determine the best employee to handle the problem. For example, the system can first locate available employees. Then, among these available employees, the system can locate those within a certain distance to the item in question. From these employees, the system can then determine the best employee based on the employee's ability to complete the task.

[0132] In some cases, the system can send the alert only to the most appropriate employee (as determined in any suitable manner). Alternatively, the system can send the alert to a group of appropriate employees, and each recipient can respond using his or her device (e.g., a smartphone). Once a recipient responds by indicating that he or she will attempt to process the task, the system can update the status of the issue, for example, to "in progress."

[0133] The system may also send the alert to the appropriate employee supervisor as determined by the system. Alternatively, or in addition, the system may copy the alert to quality assurance personnel to bring it to their attention and monitor the progress of the problem.

[0134] RFID technology can also be used to monitor inventory availability in a store in a real-time manner. In this case, the system can track the movement of an item that is picked up by a customer. As described above, the system can determine that an item is picked up by a customer if it is moving with the tote. More specifically, the system can use the movement / trajectory of the item's RFID tag and the movement / trajectory of the tote as determined from video data and / or data about the RFID tag on or in the tote to determine that the item is in the tote. Once the system determines that the item is picked up, the system subtracts the item from available inventory. Alternatively, the system can subtract the item from inventory until the item passes through a register where the item is checked out. In some cases, the system can also subtract an item, such as an article of clothing or a pair of shoes, if the customer is wearing the item.

[0135] RFID technology can also facilitate the verification of e-commerce orders, especially after the shipping box is sealed. Because RF signals can typically penetrate the shipping box, the RFID technology described above can be used to identify items with RFID tags in the shipping box. The identified items are then compared to the order corresponding to this shipment to determine if any items are missing or if any items should not be in the shipping box. If a missing item is identified, the system can check an inventory or other database to find if a replacement item is in the distribution center (DC) or a nearby store. The system can also prevent the shipping box from leaving the store and / or DC until the items are placed in the shipping box or located for a separate shipment.

[0136] In some cases, RFID technology can be used in changing rooms to track items being tried on by customers. The system can determine if an item left in a changing room has been in the changing room for longer than a threshold time (e.g., more than 15 minutes). Alternatively, the system can track the location of the item as well as the status of the changing room. For example, if the system determines that the item is in the changing room and the changing room is not in use, the system can determine that the item has been left in the changing room. In these cases, the system can alert an employee to pick up the item and place it back on the shelf for sale.

[0137] The system can determine the status of the fitting rooms by tracking the presence of the customer's mobile or wearable device in the fitting rooms. For example, the system can generate a map of the fitting rooms to display the mobile and wearable devices detected in each fitting room. If the device is not found in the fitting room, the system can indicate that the fitting room is likely unused. In this case, an employee can go to the fitting room to pick up the abandoned items.

[0138] The system can also determine the state of the fitting room using an RFID tag attached to the fitting room door (e.g., at the movable end of the door). In this case, the fitting room door can be designed to move away from the frame when it is unlocked. Thus, the RFID tag is in a first position when the door is closed or locked (i.e., when the fitting room is occupied) and in a second position when the door is open or unlocked (i.e., when the fitting room is unoccupied). The system can then determine the state of the fitting room based on the position of the RFID tag on the door. Similarly, another option is to install several reference tags in or on the fitting room curtain to detect when they move closer together or move further apart as a result of someone opening or closing the curtain.

[0139] Alternatively, each fitting room can use two RFID tags, one located at the movable end of the door and the other located in the door frame. Alternatively, the RFID tags can be located or incorporated in different parts of the fitting room door lock. The system can then determine the distance between these two tags. If they are within a threshold (e.g., about 10 cm), the system can determine that the door is closed or locked, otherwise the system can determine that the door is open or unlocked and the fitting room is unoccupied.

[0140] Additionally, the system can use a combination of amplitude variation and location of the RFID signal to determine whether clothing having an RFID tag is on a person. For example, if a garment is floating in the air in the middle of a fitting room, it is likely on a body. If the RFID tag localization system detects a significant drop in RSSI with an indication (e.g., from camera data) that the RFID tag is near the person's body, it can determine that an object / garment tagged with an RFID tag is likely on the person's body.

[0141] 11.2 Shelving of RFID tagged items Accurate tracking of items also enables the system to place items on shelves without human intervention using autonomous vehicles (e.g., robotic devices, drones, etc.). For example, an RFID tag can be attached to each item to provide information about the item's desired location within the store. The autonomous vehicle can include a tag reader that reads the RFID tag and can distribute the item to the desired location. The desired location (e.g., a designated shelf) can also be marked by an RFID tag (called a fixture tag). In some cases, the autonomous vehicle uses its built-in tag reader to locate the fixture tag, estimate the distance and direction to the fixture tag from its current location, and use the estimate to navigate toward the fixture.

[0142] In some cases, the system can monitor the location of the remotely controlled vehicle using an RFID tag on or embedded in the vehicle. If necessary, the system or a user can instruct the vehicle to proceed to a fixture. In these cases, the vehicle may not include any tag readers.

[0143] Alternatively, the RFID tag can contain the item's identification information (e.g., a serial number) rather than the item's desired or intended location. Instead, the identification information is associated with the desired location information in a database. An RFID tag reader (e.g., on an autonomous vehicle) can read the identification information and communicate with the database to retrieve the location information.

[0144] Automatic shelving by autonomous vehicles can be performed every night after the store closes and / or every morning before the store opens. In some cases, the shelving procedure is automatic such that it can be performed without human monitoring. Thus, shelving can be performed after hours to save on overtime costs.

[0145] In some cases, shelving can be performed on demand. For example, if the system determines that an item is on demand, the system can send a human or robot to a stockroom to select an item and deliver the item to the desired location. In some cases, a human or robot can be instructed to select a misplaced item and place it in the correct location. The system can direct the robot to the location of the misplaced item and to the desired location of the item. In some cases, RFID tag data and / or camera data can also reveal object orientation and other information such as weight, geometry and weight distribution to aid in complex problems such as grasping.

[0146] 11.3 Inventory monitoring of items with RFID tags The system can monitor inventory of items based on precise tracking of the location of items by RFID tags. As described above, the system can determine that an item has been selected or carried away by a customer, in which case the system can remove the item from the list of available items. The system can further place the item in a temporary list of items under consideration for purchase by the customer. Once the item is checked out by the customer (e.g., in the event the customer leaves the store with the item), the system can remove the item from the temporary list. However, if the customer changes their mind and returns the item (or simply abandons the item) before checking out, the system can return the item to the available list.

[0147] In some cases, once the system determines that an item is under consideration by a customer, the system may interrogate the RFID tag attached to the item at a frequency greater than 1 Hz to track the movement of the item. Once the item is returned to the shelf, the interrogation frequency may be reduced to reduce the computing load on the system.

[0148] Employees can participate in inventory monitoring by handling defective items. Defective items can be identified by employees or customers. In either case, employees can use an employee device to scan an RFID tag on the item and enter the item's status (e.g., "defective" or "damaged") into the system. The employee device can include a tag reader and an interactive interface (e.g., a touch screen) for the employee to update the inventory. In response to receiving the status, the system can remove the item from the available list and place the item on another list (e.g., a repair list or a return list). The system can also send one or more alerts to relevant personnel to handle the defective item.

[0149] 11.4 Employee and Product Location Using RFID Tag Location Systems The RFID tag location system can include multiple cameras, RFID tags, and a wireless communication system, such as Bluetooth or Wi-Fi, to track the exact location of employees and products in a store. The locations of employees and products can be obtained from RFID tag location data collected by the RFID tag location system and then displayed using a GUI on a smartphone or tablet. FIG. 6 shows an exemplary GUI displaying the locations of employees and some products on a floor plan of a store. Because the RFID tag location system can identify the exact locations of employees and products, the relative locations between employees and products can also be displayed, as shown in FIG. 6. In addition to showing the locations of products on the floor plan, the locations of products can also be displayed in a virtual tour of the store, in a three-dimensional view of the store, or in an online shopping function, as shown in FIG. 6.

[0150] 11.5 Product Selection Using the GUI The GUI, as described, may display one or more products on a store floor plan. The GUI may also allow a user, employee, or customer to interact with the displayed products to perform certain actions. User interaction with the GUI may be accomplished by several methods, including touch-based systems such as a pointing device, e.g., a mouse, a user's finger, or a stylus. For example, in a touch-based system, a user may select one or more products by drawing a shape with their finger around the products shown in the GUI. This process is illustrated in Figures 7A through 7D. Figure 7A shows a number of products displayed in the GUI. As shown in Figure 7B, a user may begin to draw a circular shape around the products using their finger until completing the circular shape as shown in Figure 7C. Thus, the products contained within the circular shape are selected. Prior to the user-specified action, information about the selected products may be displayed, such as the number of products selected, as shown in Figure 7D.

[0151] Once a product is selected by a user, the user can then specify a number of actions to be performed on the selected product. These actions include (1) listing product details such as style number, color, price, size, etc., (2) listing sales quantities or prices, (3) directing an RFID reader to read only the selected product, such as during receipt of a new shipment, during an inventory count, etc., (4) changing the floor display of a particular product, (5) choosing to receive price alerts for the selected product, (6) viewing information about similar products, (7) receiving recommendations for similar or newer models of the selected product, or (8) having the selected product delivered or picked for purchase. Some actions are available only to either employees or customers depending on their functionality.

[0152] 11.6 Inventory update process and automatic notification of new product shipments The RFID tag location system can also be used to facilitate inventory updates when new shipments of products arrive at the store. For example, a shipment may arrive at the store from a manufacturer, warehouse, distribution center, or another store. An RFID reader and user app can be used to verify the amount of product in the shipment. RFID readers can be optimally placed in store shipment process areas, such as stock rooms, sales floors, or other locations that the retailer can use for inbound shipment processes.

[0153] Products included in a shipment may or may not include RFID tags. For products that include RFID tags, employees can use an RFID reader and user app to verify that the quantity of product received matches the corresponding invoice for the product order. For products that do not include RFID tags, employees can add RFID tags to the products and code the tags with the appropriate product information using an RFID reader and user app. These products can then be added prior to verifying the quantity of product received in the shipment.

[0154] Once the amount of product received is confirmed, the RFID stock of product and the master corporate stock of product, which may include products with and without RFID tags across multiple stores, are updated to show the exact stock levels of product at the store where the shipment was received and at the corporate level across multiple stores. If a discrepancy exists between the amount of product received and the invoice, the RFID tag location system can facilitate resolving said discrepancy by determining whether the product arrived in the shipment, the product arrived without an RFID tag, or the product arrived with the wrong RFID tag.

[0155] Electronic notifications can also be sent automatically to customers informing them that a new shipment of products has been delivered to the store. Notifications can be sent using a variety of methods including email, text message, messaging applications such as WhatsApp, Facebook Messenger, geofencing applications, or other electronic messaging services integrated with the RFID tag location system. Notifications can be tailored to preferred products based on customer preferences, e.g., new products, best-selling products, or products selected by the customer to be notified. Notifications can also be sent to customers who have previously visited a particular store or who have subscribed to receive notifications from a particular retailer or store. Retargeting advertisements or electronic messages can also be sent to customers who have previously visited a store and were unable to purchase a particular product because it was not available, e.g., the preferred product was not in a desired dimension.

[0156] The RFID tag location system can also facilitate the discovery of products that have lost their RFID tags or products with incorrect RFID tags after delivery or during inventory checks. For example, an employee can inspect a stack of identical clothing and immediately discover that the stock level is zero, indicating an error in the RFID inventory due to a missing or incorrect RFID tag. In another example, an employee can be carrying a particular product and visually notice that the product is missing an RFID tag. When a product is discovered to have a missing or incorrect RFID tag after receipt and verification of the shipment, an employee can add or replace an RFID tag to the product and code the tag with the correct product information using an RFID reader and user app. Once the new RFID tag is coded, the product's RFID inventory is updated and an automatic electronic notification can be sent to the customer, as previously described.

[0157] 11.7 Automatic Product Movement and Retention Notification The RFID tag location system can also monitor product movements, for example, if a priority product is not moved to the appropriate location, such as the sales floor or storage area, e.g., the product is not placed on hold for a customer. Based on the RFID tag location data, if a product movement or product hold does not occur within a certain time threshold defined by the retailer, e.g., 30 minutes, an electronic notification can be sent to authorized employees, local management personnel, or corporate management personnel. The notification can be sent using a variety of methods including email, text message, messaging applications such as WhatsApp, Facebook Messenger, geofencing applications, or other electronic messaging services that are integrated with the RFID tag location system.

[0158] 11.8 Product Status Based on RFID Tags The RFID tag location system may also encode additional information into a product's RFID tag. For example, an RFID status tag may be used, which may contain various product status and tracking information. The RFID status tag may be distinct from an RFID tag, which may have the same product information as a set of RFID tags that correspond to the RFID status tag.

[0159] A number of product statuses can be coded into the RFID tag and may be based on categories including shipping, e-commerce order, and damage. In the shipping category, the product status may include (1) the product being sent from a first store to a second store, warehouse, or distribution center, (2) the date and time of the product status creation, (3) the type of shipping, e.g., moving to a different store, moving to a warehouse, moving to a distribution center, moving damaged or salvaged products, moving products where cleaning or rework services occur away from the store, (3) the origin of the product, e.g., a store number, (4) the destination of the product, e.g., a store number, a distribution center, a manufacturing facility number, or (5) a shipping number, e.g., a tracking number generated by the RFID tag location system or an existing legacy system.

[0160] In the e-commerce order category, the product status may include: (1) goods to be sent from the store to a third-party shipping address specified by the customer placing the order; (2) date and time of product status creation; (3) origin of the goods, e.g., a store number; (4) customer account number, e.g., an account created by the e-commerce system; (5) e-commerce order number, e.g., an order number created by an RFID tag location system or an existing legacy e-commerce system; or (6) e-commerce status, e.g., an "In Process" status for products that have been picked up and are currently awaiting packaging in a process area, and a "Packaged" status for products that have been selected and packaged for external shipment.

[0161] In the damage category, the product status can include (1) goods currently unavailable due to contamination, damage, or defects, (2) the date and time of product status creation, or (3) a damage shipment number, e.g., a reference number created by an RFID tag location system or an existing legacy system.

[0162] The use of RFID status tags can facilitate the assignment of product status in a particular area of ​​a store based on the location accuracy of the RFID tag location system or by product type. For example, an RFID status tag on a particular product can automatically assign the same status to other products in its vicinity, e.g., products within 4 inches of the product having an RFID tag. In another example, an RFID status tag on a particular product can assign the same status to a group of products throughout an entire store. Status changes for a group of products can be displayed in the GUI with different colors or symbols for those products. This visual indicator can help employees ascertain the status for the products.

[0163] The RFID tag location system can also automatically change product status based on the RFID stock of the product. For example, a product with an RFID tag that has an outgoing status, e.g., moving to another store, moving to a warehouse, or moving to a distribution center, can be considered as available stock for an e-commerce order to be fulfilled by the store sending the product, unless the outgoing process is confirmed by the store. Confirmation can include that the product is in a sealed box, that the transfer document is completed, etc.

[0164] 11.9 Product Arrival and Departure Route Tracking An RFID tag location system can include RFID readers, (depth) cameras and technology to precisely determine the location of an RFID tag and can be used to record the path of one or more RFID tagged products, for example products with RFID tags or RFID status tags grouped in a box, bag or cart, through the store as the products enter or exit the store. Using a user app, the path can then be displayed to the user in a GUI as an animation overlaid on top of a floor plan of the store, as shown in Figure 8.

[0165] The RFID tag location system can also play back recorded video feeds of the arrival or departure of RFID tagged products using the system's location technology and date, time and location data recorded by the cameras, as shown in Figure 8. In addition, the RFID tag location system can also identify individuals carrying RFID tagged products, personal Bluetooth or Wi-Fi enabled devices or user IDs based on face or gait recognition.

[0166] To ensure that the store is completely covered by the RFID tag location system, the components of the RFID tag location system can be mounted on the ceiling or walls in increments of 500 to 1000 square feet depending on the layout and environment of the store. This allows the RFID tag location system to track all RFID tagged products and Bluetooth or Wi-Fi ___33 enabled devices in the store. In addition, the system can also identify store boundaries, such as multiple floors, rooms, entrances, exits, etc. The store boundaries can be marked by RFID reference tags or other manual marking methods for detection. In particular, by identifying entrances and exits, the RFID tag location system can automatically register when products enter or exit the store.

[0167] 11.10 Smart, adaptive floor displays of product quantities The RFID tag location system may allow a user, e.g., an employee, to set a desired amount of a product in a particular area of ​​a store (e.g., 12 units of a product on a floor display). Additionally, the RFID tag location system may suggest to the user ideal placement of the product based on historical data on the performance of the product to maximize sales. For example, a particular product may have multiple variants, e.g., footwear, clothing, accessories, women's wedding attire, with different sizes. The RFID tag location system may suggest to the user the best performing product variant to place on the floor display for that particular store. Historical performance data may include historical sales, the number of times a product or product variant is viewed or tested by customers, or the conversion rate of the product or product variant, e.g., view to sales, customer test to sales, etc.

[0168] The RFID tag location system can also dynamically adapt to the quantity and location of products in the store in real time based on the stock inventory available in the store. For example, in Table 1, an ideal scenario is shown where M and L size products are the best performing variants, followed by S and XL size products. Based on the user-defined requirements on the total number of products, e.g., 12 in this example, the RFID tag location system automatically calculates the number of products to place in the floor display by size. In this case, more M and L size products are shown than S and XL size products because they perform better. [Table 1]

[0169] In another example, Table 2 illustrates an adaptive scenario in which there is insufficient stock of medium size products and therefore the ideal floor display previously shown in Table 1 cannot be achieved. In response, the RFID tag location system reallocates the number of product variants to place in the floor display based on the sub-optimal performing product variant. This does not require zeroing out the display of medium size products, rather the number of medium size products is reduced to accommodate available stock and customer demand. In this case, more large size products are displayed followed by small and extra large size products. [Table 2]

[0170] Table 3 shows yet another adaptation scenario where the M and L sizes of the product are sold out and the stock of other product variants is insufficient to meet the total number of products required. In this case, the RFID tag location system reallocates the number of product variants to best achieve the total number of products on display, while also prioritizing the best performing sizes. [Table 3]

[0171] The RFID tag location system also sets the quantity of the product on the floor display to zero if the floor quantity is set to zero. Additionally, a notification can be sent to an employee if at least one product is on the floor, but the product is not on the floor display. This is based on one possible retailer strategy where all products available on the floor must also be placed on the floor display. The RFID tag location system can also be configured to detect discrepancies in the quantities of products on the floor and on the floor display, among other things, to compensate for input errors into the system.

[0172] 11.11 Creating and Optimizing Picklists As previously discussed, an RFID tag location system can accurately track the amount of products that are in different areas of a store, such as the sales floor or stockroom, and therefore can determine what products or product variants may need to be moved to the sales floor in real time. For example, Table 4 shows the distribution of product variants in a store. As shown, based on the number of products shown in the floor displays, there are insufficient medium and XL size products available in a number of sales floors. As a result, two medium size and one XL size product that are sized must be moved from the stockroom to the sales floor. [Table 4]

[0173] To facilitate replenishment of products or product variants, the RFID tag location system can instantly compile a pick list, or list of requested products that need to be replenished in real time. The pick list can then be sent to a user, e.g., a stockroom employee, who then completes the request for all products by picking up and delivering the products to the sales floor.

[0174] The use of pick lists may also be applicable to fulfilling e-commerce orders where an RFID tag location system compiles a list of products required to be picked up in-store by an online customer. Products on a pick list may also be placed on hold by an employee on the customer's behalf, by the customer using the retailer's website or application, or by a customer variant in a user app. Pick lists may also be used in customer stock requests where products are requested from a stockroom by an in-store customer via a sales floor employee, or for misplaced products, where the product is in the wrong location on the sales floor or in the stockroom.

[0175] Since the RFID tag location system can track the location of a user, e.g., a stockroom employee, and the location of all products in a picklist, an optimized pick path (OPP) can be generated based on the shortest time or distance for the employee to pick up all products. The OPP can be displayed to the user in the GUI of the user app. In FIG. 9A, the OPP is displayed as a dotted line along with the picklist and the location of the products closest to the user. As the user moves, the OPP is updated, as shown in FIG. 9B. As the user starts picking up products in the picklist, the OPP continues to update and also shows the number of products to be picked up by the user, as shown in FIG. 9C and FIG. 9D. The next product to be picked up by the user is also shown in the GUI. The OPP can also be used for e-commerce orders, customer stock checks, and moving misplaced products in the stockroom or on the sales floor.

[0176] When a product on a first user's pick list is picked up and delivered to the sales floor by a second user, and the first user is still in the process of fulfilling the request and has not yet picked up the product, the RFID tag location system will specially mark the product on the first user's pick list to notify the first user that the product is no longer needed. This notification process can be performed in real time using the RFID tag location system.

[0177] 11.12 Picklist Filters The RFID tag location system may also allow the user to refine the pick list based on product attributes or location. For example, the user may filter the pick list according to women's wedding dresses, stock room 1, or women's wedding dresses in stock room 1. Additionally, the user may set a maximum amount of products to be included in the pick list (e.g., 10 units). The RFID tag location system then shows the pick list with the maximum 10 units. Based on the user filter and the maximum amount, the RFID tag location system may optimize the products on the pick list that generate the most sales for the store.

[0178] 11.13 Stray Products A lost product is a product that is misplaced in a store, e.g., a product is shown to be on the sales floor but instead is in a stock room. An RFID tag location system can actively and accurately track the location of units of a particular product, e.g., all of the units of men's black V-neck T-shirts are located on the sales floor or in a stock room. The combination of the location accuracy of an RFID tag location system and the ability to monitor all units of a particular product can enable automatic detection of lost products in a store. If a lost product is detected, a notification that a unit of the product is misplaced can be automatically sent to a user, e.g., an employee, immediately or can be sent after a user-defined time threshold longer than, e.g., 10 minutes.

[0179] Additionally, the user app can also generate a path in the GUI to direct the user to all lost products. This path generation feature can be used for non-lost products as well. For example, FIG. 10 shows a GUI in which a particular product is selected in a store. The units of the selected product may not be located in the same stockroom or in the same area of ​​a particular stockroom, for example, the units of the product may not be located within 6 feet of each other. In these examples, the GUI can display to the user the total number of locations the user must visit and retrieve all units of the product.

[0180] 11.14 Intelligent routing of product notifications to users The RFID tag location system, particularly the RFID reader, user app, and location tracking function of the system, can be used to accurately monitor the location of products or variants of products. By tracking all RFID tagged products in a store, the system can automatically notify a user, e.g., an employee, when a product needs to be replenished in a specific area of ​​the store in real time. The threshold or criteria for product replenishment can vary from user to user. For example, a product may need to have 10 units located on a floor display. If initially there are 10 units of a product on the floor display and a customer purchases one unit, a notification can be sent to an employee that the amount of the product on the floor display has decreased under the specified requirement, prompting the employee to move one unit of the product to the floor display.

[0181] An employee receiving a notification for a product restock can also send a request for the product to another employee, for example, a sales floor employee can request a product from a stockroom employee using an internal stock request. When a requested product is not in stock at a first store, the employee can instead request the product from a second store or warehouse using an external store request, and it will be delivered to the first store or to the customer's preferred address. This tracking feature can be used by a user using a user app on their mobile device to locate a particular product with an RFID tag in the store or in nearby stores.

[0182] The RFID tag location system can also intelligently route stock requests to specific employees or locations to minimize the time to deliver the requested stock to a particular area of ​​the store or to a customer. Internal stock requests can be routed to employees based on their proximity to an area of ​​the store, a customer or a stock room, and their ability to complete the task in the shortest amount of time. For example, employee A is working on five stock requests for other customers and they must prioritize completing those five stock requests. The RFID tag location system can then route additional requests to the closest available employee, e.g., employee B, to fulfill the stock request. Employees also have the option to turn off or mute notifications for stock requests if they are currently performing an unrelated task. The RFID tag location system can also monitor the time taken for an employee to complete a stock request by tracking the product and employee as they move through the store, from initial receipt of the stock request to delivery of the product to the customer or area of ​​the store.

[0183] For external stock requests, the RFID tag location system can actively monitor and update product availability at the chain stores. For example, if a customer at a second store has the requested product in their shopping cart, the RFID tag location system removes this product from the available stock at the second store to ensure that the customer at the first store has accurate information regarding product availability. The RFID tag location system can also be used to predict the time required for an externally requested product to be delivered to a store or customer's preferred address based on the distance between the dispatcher location, e.g., a second store or warehouse, and the destination, and data detailing the speed at which the stock request is being fulfilled and dispatched by the dispatcher location.

[0184] 11.15 Stock Request Fulfillment The RFID tag location system actively tracks the location of products available on the store and stockroom floor in real time and at all times. This active tracking can facilitate employees to complete stock requests for customers in a short time. For example, a customer can use the user app to request a check of the stock levels for a particular product in the store. FIG. 11 shows an example GUI in which a customer is requesting to identify a particular product variant, e.g., a medium size, black dress, from the stockroom to have the product delivered to them on the sales floor with options. This request can then be sent to an employee on the sales floor. The sales employee can then use the user app to request the customer requested product from the stockroom employee. The stockroom employee can then locate and pick up multiple requested products for different customers. To facilitate delivery of products to different customers on the sales floor, the stockroom employee can use the user app, which actively monitors the location of different customers in real time.

[0185] In addition to stock requests, there are cases where a product is misplaced or not immediately found by a customer, but is nevertheless present on the sales floor. The User App can present the location of the product, if present, in various areas of the store. For example, in FIG. 12, the GUI can show the customer the location of a selected product on the sales floor, in addition to the quantity available in the stockroom. When store employees do not move a misplaced product on the sales floor, the User App can also enable the customer to locate the misplaced product.

[0186] 11.16 Automatic Marking of Picklists A pick list may include a request for internal resupply, e-commerce orders, stock requests, misplaced products, or any other list of products that require the user to locate in relation to a list of products requested by the user. Products on the pick list may be automatically marked as picked up if the following conditions are met: (1) the user is engaged with the pick list using a user app, (2) the user is picking up products on the pick list, and (3) the RFID / computer vision item location system recognizes that products on the user's pick list are to be picked up by the user if the products are moving with the user based on the user's device or their RFID employee tag. Once these conditions are met, the RFID / computer vision item location system should automatically mark the products as picked up by the user. To improve the accuracy of the automatic marking of the pick list, a threshold value may be used to determine whether a product is picked up by the user, for example, at a time after the product is picked up or the distance the product has traveled.

[0187] 11.17 Tracking Customers with High Cart Value The RFID tag location system can also be used to actively track the amount and type of products in a customer's shopping cart in real time. Shopping carts can include baskets, bags, carts, etc. When a customer's shopping cart contains products that exceed a user-defined threshold, for example, five whole units or a value of $500, a notification can be automatically sent to an employee identifying these customers. In addition, certain products or product categories can also be flagged by an employee to be prioritized for tracking. This tracking function can have multiple functional uses in the store. For example, the tracking function can be used to prevent shoplifting by tracking customers who may have a significant amount or value of products in their shopping cart or who have selected a large number of flagged products. The tracking function can also be used to identify customers who may be willing to spend more money, which can notify employees to provide better customer service to these customers, increase sales for these customers, or recommend free products to these customers.

[0188] The RFID tag location data can be displayed to the user using the user app's GUI in a variety of formats. For example, an employee can see all customers in the store in the GUI and monitor their shopping carts based on the amount of products or by total value. To facilitate customer identification, if the customer uses the user app's customer variants, the RFID tag location system can associate a store person with the customer's configuration. Otherwise, customers may be further identified with the products in their shopping cart by tracking the product's movements and determining if the product is associated with a registered employee device.

[0189] 11.18 Automatic Notification of VIP Customers The RFID tag location system can also store data about customers. This data can include the number of visits a customer makes to a store or the amount of money a customer spends on a monthly or annual basis. Based on this data, VIP designations can be attributed to customers who exceed a user-defined threshold.

[0190] The RFID tag location system can then be used to detect and identify VIP customers and notify employees when a VIP enters a store or a particular section of a store. Identification of VIPs can be accomplished using a number of methods, including (1) detection of VIP status based on a customer profile stored in a user app on the customer's mobile device via Bluetooth or Wi-Fi, (2) recognition of a customer mobile device ID based on a user app on the customer's mobile device, or (3) identification based on face or gait recognition using the computer vision capabilities of the RFID tag location system.

[0191] 11.19 Identifying Potential Product Theft As mentioned above, the RFID tag location system can actively track the movement of products in a customer's shopping cart in real time. If an abnormal event occurs while monitoring the products, possible theft of the products can be detected. For example, if a customer removes the RFID tag from a product, the RFID tag location system can detect this removal and immediately notify an employee of this exact product and its last known location within the store. In addition, the RFID tag location system can also identify and retrieve video footage recorded by the system's camera or RFID reader to assist the employee in locating the customer or product. Once this information is provided to the employee, the employee can then approach the customer and provide assistance with the product without the RFID tag.

[0192] The RFID tag location system can also time-stamp and store anomalous events associated with RFID tagged products. This information can be used to indicate to employees in the GUI potential high theft zones within the store based on data such as the frequency of missing RFID tags. This data can be viewed in the GUI for a user-defined time frame, e.g., the previous 7 days, 30 days, 180 days. Additionally, the RFID tag location system can also identify and highlight zones of the store that could potentially be high theft zones by identifying the current location of RFID tags that tend to go missing within the store.

[0193] 11.20 Automatic fitting room monitoring Similar detection strategies to identifying potential product theft can also be used for automated monitoring of fitting rooms. An RFID tag location system can track products as they enter or leave the fitting room. Notifications can be sent to employees in real time about products entering and leaving the fitting room. If a product is left in the fitting room, the RFID tag location system can notify employees that a lost product is present in the fitting room and to return the product to its correct location in the store. If an RFID tag is removed, resulting in the loss of a product with the RFID tag location system, a notification can also be sent to employees that the product may have been lost and identifying the customer with whom the product was last associated. When the product in question is missing, the customer can also be identified by other products in their shopping cart.

[0194] 11.21 Capturing and Measuring Customer and Product Interactions The RFID tag location system can also be used to detect and measure data related to customer and product interactions within the store. For example, the system can track (1) how frequently products are picked up by customers, (2) how long products are looked at by customers, (3) which products are looked at together, (4) what products customers keep while looking at new products, (5) what products are taken away from the fitting room, (6) what products customers interact with before making a purchase, (7) products that may be tested by customers, e.g., customers trying on clothing, based on measured distortion of the RF signal due to its proximity to a body of moisture such as the human body, and (8) how long customers test products. For products that are tested by customers, information can also be collected about non-purchased products, e.g., clothing left in the fitting room, to evaluate manufacturing or finishing issues, e.g., a customer prefers the look of a garment but not its finish. This data can potentially be used to inform the store how to modify the manufacturing of products to improve sales.

[0195] To measure these parameters, an RFID tag location system can track objects in 3D space with high spatial and temporal resolution. For example, an RFID tag location system can detect whether a product is moved beyond a threshold distance, e.g., 4 inches, and held for more than a threshold period, e.g., 3 seconds. If such conditions are met, the product can be considered to have been picked up or viewed by a customer.

[0196] As previously mentioned, the RFID tag location system can track products and customer movements within the store. Customers can be identified either by (1) the customer using a user app on their mobile device, which is detected by Bluetooth, Wi-Fi or another wireless communication system or sensor, or (2) detecting customers based on individuals who do not have any tags or devices identifiable by the RFID tag location system, assuming that an employee has a tag or device.

[0197] The RFID tag location system can also collect product performance data according to product groups, such as product categories, sub-categories, products, colors, sizes, price ranges, any combination of the preceding types listed, and others. For these product groups, performance data that can be collected is as follows: (1) the product groups that are viewed the most or least, (2) the product groups that are viewed the longest or shortest, (3) the product groups that are taken to the fitting room the most or least, (4) the product groups that are tested or tried on the most or least, (5) the product groups that are tested or tried on for the longest or shortest period of time, (6) the product groups that have the best or worst conversion, defined as the amount of sales versus the other types of data listed. For example, if a product is viewed 100 times per day and sold 10 times per day, the conversion rate is 10%. In another example, if 100 products are tried on for 30 seconds or more and have 10 sales, the products that are tried on for 30 seconds or more have a conversion rate of 10%.

[0198] Based on product performance data collected by the RFID tag location system, improvements to store operations can be achieved by: (1) identifying the best performing sales areas within the store; (2) identifying areas with the most product interactions to improve staffing in that area; or (3) automatically calculating and recommending product assortments based on merchandising strategies, such as best-selling combinations (e.g., black jeans and white T-shirts perform best together) or identifying areas of the store that are best for a particular product type (e.g., dresses have the highest conversion in Zone A of the store).

[0199] Similarly, product performance data can improve the consumer shopping experience by: (1) understanding a customer's historical shopping preferences based on predefined product groups and notifying the customer of the arrival of new or restocked products that the customer previously searched for in the store or online; (2) personalizing the in-store shopping experience by highlighting products in the store or in zones of the store that may be of interest to the customer; (3) notifying the customer of possible in-store promotions; or (4) identifying the customer by detecting a customer's profile stored in a user app on the customer's mobile device via Bluetooth or WiFi, recognizing the customer mobile device id based on a user app on the customer's mobile device, or identifying the customer based on face or gait recognition using the computer vision capabilities of the RFID tag location system.

[0200] 11.22 Conclusion While various different embodiments have been described herein with reference to figures, various other means and / or structures for performing the functions and / or obtaining the results and / or one or more of the advantages described herein are possible. More generally, all parameters, dimensions, materials, and configurations described herein are exemplary, and the actual parameters, dimensions, materials, and / or configurations will depend on the particular application or applications in which the teachings of the disclosure are used. It should be understood that the foregoing embodiments are presented by way of example only, and that the embodiments may be practiced other than as specifically described and claimed. Embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. Moreover, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of the present disclosure, if such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent.

[0201] The above-described embodiments can be implemented in any of numerous ways. For example, embodiments of the technology disclosed herein may be implemented using hardware, software, or a combination thereof. If implemented in software, the software code may be executed on any suitable processor or collection of processors, whether provided on a single computer or distributed among multiple computers.

[0202] The various methods or processes outlined herein may be coded as software executable on one or more processors using any one of a variety of operating systems or platforms. Further, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine code or intermediate code that runs on a framework or virtual machine.

[0203] In this regard, the various inventive concepts may be embodied as a computer readable storage medium (or multiple computer readable storage media) (e.g., a computer memory, one or more floppy disks, compact disks, optical disks, magnetic tapes, flash memories, circuitry in field programmable gate arrays or other semiconductor devices, or other non-transitory or tangible computer storage media) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the invention described above. The computer readable medium or media may be portable, such that the program or programs stored thereon can be loaded into one or more different computers or other processors to implement various aspects of the invention, as described above.

[0204] The terms "program" or "software" are used herein generally to mean any type of computer code or set of computer-executable instructions that can be used to program a computer or other processor, as described above. It will further be appreciated that, according to one aspect, one or more computer programs implementing the methods of the present invention to be performed need not reside on a single computer or processor, but may be distributed in a modular fashion among a number of different computers or processors for implementing various aspects of the present invention.

[0205] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0206] Also, various disclosed concepts may be embodied as one or more methods, examples of which are provided. The operations performed as part of a method may be ordered in any suitable manner. Thus, embodiments can be created in which operations are performed in an order different from that illustrated, including performing some operations simultaneously even if shown as sequential operations in the exemplary embodiments.

[0207] All definitions and uses herein should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0208] As used in the specification and claims, the indefinite articles "a" and "an" should be understood to mean "at least one," unless clearly indicated otherwise.

[0209] As used in the specification and claims, the term "and / or" should be understood to mean "either or both" of the conjoined elements, i.e., elements that are conjunctive in some cases and disjunctive in other cases. Multiple elements listed with "and / or" should be construed in the same manner, i.e., "one or more" of the conjoined elements. Other elements, whether related or unrelated to the elements specifically identified, may optionally be present other than the elements identified by the "and / or" clause. Thus, as a non-limiting example, a reference to "A and / or B," when used in conjunction with open-ended language such as "comprising," can refer in one embodiment to only A (optionally including elements other than B), in another embodiment to only B (optionally including elements other than A), in yet another embodiment to both A and B (optionally including other elements), and so forth.

[0210] As used herein and in the claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" shall be interpreted as inclusive, i.e., including at least one, but also two or more, of a number of elements or a list of elements, and optionally additional items not listed. Only terms expressly indicated to the contrary, such as "only one of," or "exactly one of," or, when used in the claims, the term "consisting of," refer to the inclusion of exactly one element of a number of elements or a list of elements. In general, as used herein, the term "or" shall only be interpreted as indicating exclusive alternatives (i.e., "either or but not both") when preceded by terms of exclusivity, such as "either," "one of," "only one of," or "exactly one of." "Consisting essentially of," when used in the claims, shall have its ordinary meaning as used in the field of patent law.

[0211] As used herein, the terms "about" and "approximately" generally mean plus or minus 10% of the stated value.

[0212] As used in this specification and claims, the phrase "at least one" referring to a list of one or more elements should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each element specifically listed within the list of elements, or excluding any combination of elements in the list of elements. This definition also allows for elements other than the elements specifically identified within the list of elements to which the phrase "at least one" refers, whether related or unrelated to the elements specifically identified, may optionally be present. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B" or, equivalently, "at least one of A and / or B") can refer in one embodiment to at least one, optionally including two or more A's (and optionally including elements other than B) with no B present; in another embodiment to at least one, optionally including two or more B's (and optionally including elements other than A) with no A present; in yet another embodiment to at least one, optionally including two or more A's; and at least one, optionally including two or more B's (and optionally including other elements), etc.

[0213] In the claims, as well as in the above specification, all transitional terms such as "comprising," "including," "carrying," "having," "containing," "involving," "holding," "comprising," and the like, are to be understood as open-ended, i.e., meaning including but not limited to. Only the transitional terms "consisting of" and "consisting essentially of" shall be closed or semi-closed transitional terms, respectively, as set forth in the United States Patent and Trademark Office Guidelines, Section 2111.03.

Claims

1. 1. A method for locating a radio frequency identification (RFID) tag, comprising: receiving a plurality of first reference RFID signals from a first reference tag at a known location; receiving a plurality of second reference RFID signals from a second reference tag at a known location; receiving a plurality of first RFID signals from a first RFID tag at a first unknown location; designating the first RFID tag as a first virtual reference tag based on the plurality of first RFID signals; establishing an absolute position of the first virtual reference tag based on the plurality of first RFID signals, the plurality of first reference RFID signals, the plurality of second reference RFID signals, and the known positions of the first reference tag and the second reference tag; receiving a second plurality of RFID signals from a second RFID tag at a second unknown location; determining respective multipath signatures for the first reference tag, the second reference tag, the first virtual reference tag, and the second RFID tag based on the plurality of first reference RFID signals, the plurality of second reference RFID signals, the plurality of first RFID signals, and the plurality of second RFID signals; determining relative positions of each of the first reference tag, the second reference tag, the first virtual reference tag, and the second RFID tag based on a comparison of the respective multi-path signatures to the second RFID tag, the first reference tag, the second reference tag, and the first virtual reference tag, and locating the second RFID tag based on the respective known positions and respective relative positions of the first reference tag, the second reference tag, and the first virtual reference tag; A method comprising:

2. The method of claim 1 , wherein receiving the plurality of first RFID signals comprises receiving at least one first RFID signal from each of a plurality of angles of arrival.

3. 2. The method of claim 1, wherein designating the first RFID tag as the first virtual reference tag designates the first RFID tag as the first virtual reference tag in response to determining that the first RFID tag is stationary.

4. 2. The method of claim 1, wherein the multi-path signature for the second RFID tag represents RF power received by the first RFID tag reader along a line-of-sight (LOS) path and at least one non-line-of-sight (NLOS) path from the second RFID tag to the first RFID tag reader.

5. receiving a third plurality of RFID signals from a third RFID tag at a third unknown location; designating the third RFID tag as a second virtual reference tag based on the plurality of third RFID signals; determining a multipath signature for the second virtual reference tag, the multipath signature representing RF power received by the first RFID tag reader along a LOS path and at least one NLOS path from the second virtual reference tag to the first RFID tag reader; 5. The method of claim 4, further comprising: determining a location of the second RFID tag relative to the second virtual reference tag based on the multi-path signature for the second virtual reference tag based on the plurality of second RFID signals.

6. detecting a change in a plurality of RFID signals from the first virtual reference tag; detecting a change in a plurality of RFID signals from the second virtual reference tag; performing a comparison of the variances of the plurality of RFID signals from the first virtual reference tag and the variances of the plurality of RFID signals from the second virtual reference tag; determining that the first virtual reference tag and the second virtual reference tag are stationary based on the comparison; and in response to determining that the first virtual reference tag and the second virtual reference tag are stationary, maintaining the first virtual reference tag and the second virtual reference tag as virtual reference tags; The method of claim 5 further comprising:

7. 1. A system for locating a radio frequency identification (RFID) tag, comprising: a plurality of RFID tag readers that receive a plurality of first reference RFID signals from a first reference tag at a first known location, a plurality of second reference RFID signals from a second reference tag at a second known location, a plurality of first RFID signals from a first RFID tag at a first unknown location, and a plurality of second RFID signals from a second RFID tag at a second unknown location; a signal processor operatively coupled to the plurality of RFID tag readers; (i) designating the first RFID tag as a first virtual reference tag based on the plurality of first RFID signals; (ii) establishing an absolute location of the first virtual reference tag based on the plurality of first reference RFID signals, the plurality of second reference RFID signals, the plurality of first RFID signals, the first known location, and the second known location; (iii) determining respective multipath signatures for the first reference tag, the second reference tag, the first virtual reference tag, and the second RFID tag based on the plurality of first reference RFID signals, the plurality of second reference RFID signals, the plurality of first RFID signals, and the plurality of second RFID signals; (iv) determining relative positions of each of the first reference tag, the second reference tag, the first virtual reference tag, and the second RFID tag based on comparison of the respective multi-path signatures to the second RFID tag, the first reference tag, the second reference tag, and the first virtual reference tag, and locating the second RFID tag based on the respective known positions and respective relative positions of the first reference tag, the second reference tag, and the first virtual reference tag; Including, the system.

8. The system of claim 7 , wherein the plurality of RFID tag readers are configured to receive a plurality of first RFID signals from each of a plurality of angles of arrival.

9. the plurality of RFID tag readers are configured to receive a plurality of second RFID signals from a first angle of arrival of the plurality of angles of arrival; 9. The system of claim 8, wherein the processor is configured to determine the location of the second RFID tag by comparing a multipath signature of the second RFID tag from the first angle of arrival to a multipath signature of the first virtual reference tag at the first angle of arrival.

10. 8. The system of claim 7, wherein the processor is configured to designate the first RFID tag as the first virtual reference tag in response to determining that the first RFID tag is stationary.

11. a multipath signature for the second RFID tag representing RF power received by the first RFID tag reader in the plurality of RFID tag readers along a line-of-sight (LOS) path and at least one non-line-of-sight (NLOS) path from the second RFID tag to the first RFID tag reader; 8. The system of claim 7, wherein the multipath signature for the first virtual reference tag represents RF power received by the first RFID tag reader along a LOS path and at least one NLOS path from the first virtual reference tag to the first RFID tag reader.

12. the plurality of RFID readers are configured to receive a plurality of third RFID signals from a third RFID tag at a third unknown location; The processor, designating the third RFID tag as a second virtual reference tag based on the third plurality of RFID signals; determining a multipath signature for the second virtual reference tag, the multipath signature representing RF power received by the first RFID tag reader along a LOS path and at least one NLOS path from the second virtual reference tag to the first RFID tag reader; 12. The system of claim 11, configured to determine a location of the second RFID tag relative to the second virtual reference tag based on the multi-path signature for the second virtual reference tag based on the plurality of second RFID signals.

13. The processor, Detecting a change in a plurality of RFID signals from the first virtual reference tag; Detecting a change in a plurality of RFID signals from the second virtual reference tag; performing a comparison of the variances of the plurality of RFID signals from the first virtual reference tag and the variances of the plurality of RFID signals from the second virtual reference tag; determining that the first virtual reference tag and the second virtual reference tag are stationary based on the comparison; 13. The system of claim 12, configured to maintain the first virtual reference tag and the second virtual reference tag as virtual reference tags in response to determining that the first virtual reference tag and the second virtual reference tag are stationary.

Images