Methods and apparatus for locating RFID tags

Abstract

To distinguish ghosts by multipath.SOLUTION: A radio frequency identification (RFID) system includes an array of antennas to distinguish line-of-sight (LOS) paths from non-line-of-sight (NLOS) paths. A distance between adjacent antennas in the array of antennas is less than half of the wavelength of a radio frequency (RF) signal of the system. Each antenna in the antenna array is also digitally controlled to change a relative phase difference among the antennas, thereby allowing digital steering of the array of antennas across angles of arrival (AOAs) between 0 and π. The digital steering generates a plot of signal amplitudes as a function of AOAs. LOS paths are distinguished from NLOS paths on the basis of shapes (for example, depth, gradient, and the like) of extrema (for example, maxima or minima) in the plot.SELECTED DRAWING: Figure 3-2

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority under 35 U.S.C. §119(e) to U.S. patent application Ser. No. 62 / 577,530, filed Oct. 26, 2017, and U.S. patent application Ser. No. 62 / 477,796, filed Mar. 28, 2017, each of which is incorporated herein by reference in its entirety. [Background technology]

[0002] Radio frequency identification (RFID) technology has applications in many commercial areas, such as access control, animal tracking, security, and toll collection. A typical RFID system includes a tag (also called a transponder) and a reader (also called an interrogator). The reader includes an antenna for transmitting radio frequency (RF) signals and receiving RF signals reflected or emitted by the tag. The tag may also include an antenna and an application-specific integrated circuit (ASIC) or microchip. A unique electronic product code can be assigned to the tag to distinguish it from other tags.

[0003] RFID systems can use either active or passive tags. Active tags contain a transmitter to emit an RF signal to a reader and a power source (e.g., a battery) to power the transmitter. In contrast, passive tags do not contain a power source and derive power generated by the reader through induced current in the tag's antenna. In passive RFID systems, the reader transmits a signal using the reader antenna to excite the tag antenna. Once the tag is powered on (excited), it transmits stored data back to the reader.

[0004] A signal emitted or reflected by a tag can reach a reader via one or more paths. For example, the signal can travel along a straight line from the tag to the reader (called a line-of-sight or LOS path). The signal can also be reflected or scattered from obstacles (e.g., walls and other objects distributed throughout the environment) before reaching the reader. These paths are called non-line-of-sight (NLOS) paths. In some cases, a given signal can take multiple paths to the receiver, and several copies of the signal may arrive at the receiver. The reader sees each copy of the signal as originating from a different direction or angle of arrival. This phenomenon is called "multipath" in the field of RFID technology.

[0005] Multipath can cause unwanted interference and ghosts. If different copies of a signal overlap in time, they can interfere with each other. Destructive interference causes fading. If different copies of a signal do not overlap with each other, subsequent copies can appear as "ghosts." These ghosts can fool a receiver into thinking that extraneous RFID tags are present. Summary of the Invention

[0006] Embodiments of the present technology include methods and systems for locating radio frequency identification (RFID) tags. One example includes receiving a plurality of first RFID signals from a first RFID tag at a first unknown location using a system having one or more antennas or RFID tag readers. A processor coupled to the antenna designates the first RFID tag as a first virtual reference tag based on the plurality of first RFID signals. The antenna receives at least one second RFID signal from a second RFID tag at a second unknown location. and a processor for determining a location of the first RFID tag relative to the first virtual reference tag based on the at least one second RFID signal.

[0007] Another example of the present technology uses a first antenna to receive a first line-of-sight (LOS) signal from an RFID tag. A processor coupled to the first antenna estimates a first angle of arrival, a first phase difference, and a first frequency difference of the first LOS signal to determine a change in the first phase difference with respect to the first frequency difference. A second antenna receives a second line-of-sight (LOS) signal from the RFID tag. The processor estimates a second angle of arrival, a second phase difference, and a second frequency difference of the second LOS signal to determine a change in the second phase difference with respect to the second frequency difference. The processor then estimates the location of the RFID tag based on the first angle of arrival, the change in the first phase difference with respect to the first frequency difference, and the change in the second phase difference with respect to the second angle of arrival and second frequency difference.

[0008] Yet another example includes receiving, by a plurality of antennas, at least one RFID signal from at least one reference RFID tag. A processor operatively coupled to the antenna determines an estimated location of the reference RFID tag based on the RFID signal. It performs a comparison between the estimated location of the reference RFID tag and the actual location of the reference RFID tag. The processor is calibrated based on the comparison between the estimated location of the reference RFID tag and the actual location of the reference RFID tag. The antenna receives at least one RFID signal from an RFID tag at an unknown location. The processor then determines an estimated location of the RFID tag based on the RFID signal.

[0009] Yet another example includes receiving, by a plurality of antennas, reference RFID signals from respective reference RFID tags at respective known locations, the antennas also receiving at least one RFID signal from an RFID tag at an unknown location, and a processor coupled to the antennas determining the location of the RFID tag based on the RFID signals and the reference RFID signals.

[0010] Yet another example includes receiving a reference RFID signal from at least one reference RFID tag using an antenna array, a processor determining a receive sensitivity pattern of the antenna array based on the reference RFID signal, the antenna array receiving the RFID signal from the RFID tag at an unknown location, and the processor determining the location of the RFID tag based on the RFID signal and the receive sensitivity pattern of the antenna array.

[0011] Another example of the techniques of the present invention includes monitoring an RFID tag by receiving, with at least one antenna, a plurality of RFID signals from the RFID tag over a period of time. A processor coupled to the antenna estimates a plurality of possible trajectories of the RFID tag over a period of time based on the plurality of RFID signals. The processor then identifies a first trajectory of the plurality of possible trajectories as corresponding to a line-of-sight (LOS) path between the antenna and the RFID tag.

[0012] Another example method for locating an RFID tag includes receiving a signal from the RFID tag with multiple antennas. A processor generates a first digital representation of a response detected by a first antenna of the multiple antennas and a second digital representation of a response detected by a second antenna of the multiple antennas. The processor generates multiple sums of the first digital representation and the second digital representation, each sum being a relative phase difference representing a different angle of arrival for the signal from the RFID tag. The processor uses the sums to estimate the location of the RFID tag.

[0013] Embodiments of the present invention include an apparatus for locating a radio frequency identification (RFID) tag, The present invention includes a system and method for locating an RFID tag. In one embodiment, a method for locating an RFID tag includes detecting a signal from the RFID tag to a transmitter with a plurality of antennas. One or more analog-to-digital converters (ADCs) generate a first digital representation of a response detected by a first antenna of the plurality of antennas and a second digital representation of a response detected by a second antenna of the plurality of antennas. A processor coupled to the ADCs generates a plurality of sums of the first digital representation and the second digital representation, each sum being a relative phase difference representing a different angle of arrival for the signal from the RFID tag. The method also includes estimating a location of the RFID tag based on the plurality of sums.

[0014] All combinations of the foregoing concepts, and additional concepts discussed in more detail below, are considered to be part of the inventive subject matter disclosed herein (provided such concepts are not mutually inconsistent). In particular, all combinations of subject matter recited in the claims appearing at the end of this disclosure are considered to be part of the inventive subject matter disclosed herein. The terms expressly used in any disclosure incorporated by reference herein should be given a meaning most consistent with the specific concepts disclosed herein. [Brief explanation of the drawings]

[0015] Those skilled in the art will understand that the drawings are primarily for illustrative purposes and are not intended to limit the scope of the inventive subject matter described herein. The drawings are not necessarily to scale, and in some instances, various aspects of the inventive subject matter disclosed herein may be exaggerated or enlarged in the drawings to facilitate an understanding of different features. In the drawings, like reference characters generally refer to like features (e.g., functionally similar and / or structurally similar elements). [Figure 1-1] FIG. 1A illustrates an example system for locating radio frequency identification (RFID) tags in an environment having line-of-sight (LOS) and non-line-of-sight (NLOS) paths between the RFID tag and a receiver. [Figure 1-2] FIG. 1B illustrates an exemplary system for estimating the angle of arrival (AOA) of an incident signal. [Figure 1-3] FIG. 1C is a plot showing an example of composite signal amplitude versus angle of arrival / phase difference before (solid trace) and after (multiple lines) deconvolution or other modification according to an antenna receive sensitivity pattern. [Figure 1-4] FIG. 1D shows a plot of RFID tag signal amplitude versus angle and elevation angle for four different RFID tags, each at a different AOA relative to the antenna. [Figure 2] FIG. 2 is a block diagram of a transmitter and receiver suitable for use in the system of FIG. 1A. [Figure 3-1] FIG. 3A is a flow chart illustrating a method for locating an RFID tag using a system such as the one shown in FIG. 1A. [Figure 3-2] FIG. 3B illustrates locating an RFID tag using a virtual reference RFID tag, a reference RFID tag, and measurements by multiple readers from different AOAs. [Figure 3-3] 3C-3F are frames of video showing the measured location of an RFID tag, indicated by a circle, and a bounding box drawn around the object tagged with the RFID tag using neural networks or other computer vision techniques. [Figure 3-4] FIG. 3G is a flow chart illustrating a method for associating RFID tag measurements with image data. [Figure 4] FIG. 4 illustrates the LOS and NLOS signal paths from a moving RFID tag to a pair of antennas, as well as the corresponding real and "ghost" velocity vectors and trajectories acquired by the LOS and NLOS signals. [Figure 5-1] FIG. 5A shows a retail store and stockroom RFID tag location system. [Figure 5-2] FIG. 5B shows RFID tag transmitters and receivers on items for sale in the retail store of FIG. 5A. [Figure 6] FIG. 6 shows a graphical user interface (GUI) on a smartphone or tablet displaying employee and product locations derived from RFID tag location data. [Figure 7-1] 7A-7D show how the GUI can be used to select products or other items by RFID tag for a particular action. [Figure 7-2] Same as above. [Figure 8] FIG. 8 illustrates how a GUI can show real-time and / or historical activity of RFID tags in a store, stockroom, warehouse, or other environment monitored by an RFID tag location system. [Figure 9-1] 9A-9D show how the GUI can be used to pull items from a picklist based on RFID tag location data, for example, to plan and track pick paths in a stockroom or warehouse. [Figure 9-2] Same as above. [Figure 10] FIG. 10 shows how the GUI can be used to identify and locate items that have been passed along by RFID tag. [Figure 11] FIG. 11 shows how the GUI can be used to fulfill stock requests for items with RFID tags. [Figure 12] FIG. 12 illustrates how the GUI can be used to indicate the location of selected products tagged with RFID tags on the sales floor and / or in the stockroom. DETAILED DESCRIPTION OF THE INVENTION

[0016] To date, RFID location technology has not lived up to its hype. Combined with computer vision technology, the RFID location technology of the present invention offers unprecedented speed and accuracy. In fact, it can be over 300 times more accurate than conventional RFID location technology. For example, using the systems and methods disclosed below, RFID tags can be located to within 50 cm, 40 cm, 30 cm, 25 cm, 20 cm, 15 cm, 10 cm, 5 cm, or 2.5 cm of their actual location. This speed and accuracy allows it to be used to track RFID-tagged items in real time, down to the slightest movement. This level of speed and accuracy makes it possible to locate and restock items almost instantly and track interactions between RFID tags. For RFID tags on products in a store, this can yield data about customer interactions with products at an item-by-item level, enabling autonomous checkout.

[0017] Except where physically incompatible, all of the technologies disclosed herein can be used with each other. For example, an RFID tag location system can use multiple RFID tag readers to interrogate reference tags, virtual reference tags, and RFID tags from many angles of arrival and create (multipath) signatures based on the received signals. Such a system can locate tags in two or three dimensions relative to each other and / or relative to an absolute (known) location. The resulting locations can be correlated with video data to train neural networks or manage store or warehouse operations. Location information can also be displayed on smartphones, tablets, or other devices for purposes such as inventory and supply chain management, as described in more detail below.

[0018] 1. Multipath and RFID Signals To address the multipath problem in known radio frequency identification (RFID) technologies and accurately locate RFID tags, the systems, methods, and apparatus described herein use an array of antennas to distinguish RF signals traveling along a line-of-sight (LOS) path from RF signals traveling along a non-line-of-sight (NLOS) path. The distance between adjacent antennas in the antenna array can be less than half the wavelength of the system's radio frequency (RF) signals. Each antenna in the antenna array is also digitally controlled to vary its relative phase difference with respect to other antennas in the antenna array. Each distinct phase setting of the antenna array corresponds to a distinct angle of arrival (AOA) measured by the antenna array. As long as the array includes three or more antennas, this allows the antenna array to be digitally steered throughout an elevation angle AOA between 0 and π (i.e., between 0 and 180 degrees) and an azimuth angle AOA between 0 and 2π (i.e., between 0 and 360 degrees).

[0019] Digital steering then allows for the generation of a plot or other representation of signal amplitude as a function of AOA. LOS paths are distinguished from NLOS paths based on the extremes (e.g., maxima or minima) of the plot. For example, the highest (lower), steepest maximum (minimum) may be the AOA corresponding to the LOS path. Triangulation of the AOA against two or more distinct LOS paths yields the three-dimensional (3D) location of the RFID tag. Theoretically, this approach can locate items to perfect accuracy under perfect environmental conditions. Under realistic indoor conditions, location accuracy can be better than 50 cm using this technique.

[0020] The above estimated LOS paths can be used to determine the location of the RF tag via a triangulation scheme. A first antenna or group of antennas is used to estimate a first LOS path to the RF tag, and a second antenna or group of antennas is used to estimate a second LOS path to the same RF tag. Triangulating between the two LOS paths then provides an estimate of the RF tag's location in 3D.

[0021] The above approach utilizes digital steering of the antenna array and can be very cost-effective. Additionally, this approach can be conveniently scaled up to multiple antenna arrays. These antenna arrays can be distributed in a given space (e.g., in the ceiling of a store or warehouse) to ensure that at least two antenna arrays have a line-of-sight path with RFID tags in the space. This can be particularly advantageous in indoor environments where there may be multiple obstacles. Examples of indoor applications of this RFID approach include retail stores, libraries, and warehouses, among others (see more detail below).

[0022] Digital steering can also be used to locate other RF transceivers, including those found in smartphones, wearables, tablets, laptops, and other portable electronic devices equipped with WiFi, Bluetooth, or similar antennas. Similar to RFID tag location, described briefly above and in more detail below, a transmitter emits a trigger signal to a device with a WiFi, Bluetooth, or other RF transceiver. In response to this trigger signal, the device emits a response that is detected by two or more receivers via LOS and / or NLOS paths. A processor coupled to the receiver steers the receiver sensitivity pattern by digitally steering the AOA for different combinations of receivers (e.g., paired combinations of receivers) and searching for the strongest signal as a function of AOA.

[0023] 2. A system for distinguishing between LOS and NLOS paths FIG. 1A illustrates a NLOS path 13 connected to a device or item having an RF transceiver, such as an RFID tag 10, a smartphone, a wearable computing device, 1A illustrates a system 100 for distinguishing LOS paths 11 to a tablet or laptop. The system 100 includes an RFID reader (transmitter) 110 and two receivers 120a and 120b (collectively referred to as receivers 120, also referred to as receiver antennas 120). The reader 110 and receivers 120 are coupled to a processor 130. Two receivers 120 are shown in FIG. 1A for illustrative purposes. In practice, the system 100 may include three or more receivers 120. These receivers 120 may be arranged in a one-dimensional (1D) or two-dimensional (2D) array. In another example, the receivers 120 are randomly or irregularly distributed within a given space.

[0024] The receiver 120 may form (part of) a phased antenna array, in which case the distance d between the two receivers 120a and 120b is substantially less than half the carrier wavelength λ of the radio frequency (RF) signal used to interrogate the RFID tag 10, i.e., d≦λ / 2. The system 100 may be configured to operate at any one of a variety of carrier wavelengths (and therefore a variety of carrier frequencies).

[0025] For example, system 100 can use RF signals in the ultra-high frequency (UHF) region of the electromagnetic spectrum (e.g., about 850 MHz to about 960 MHz) or microwave signals (e.g., 2.45 GHz). The corresponding carrier wavelengths are about 31 cm to about 35 cm for UHF signals and about 12.2 cm for microwave signals. In this case, the distance d between two receivers 120a and 120b can be substantially less than 17.5 cm at UHF frequencies or less than 6.1 cm at microwave frequencies. In other applications, such as outdoor applications, the system can operate at corresponding longer wavelengths (e.g., 22 meters, 2400 meters, etc.) and lower frequencies (e.g., 13.56 MHz, 125 kHz, etc.). To locate Wi-Fi or Bluetooth devices, the system can operate in the unlicensed Industrial, Scientific, and Medical (ISM) band from 2.0 GHz to 2.4 GHz or any other suitable band (e.g., 5 GHz). Higher frequencies (shorter wavelengths) generally provide more accurate position estimates than lower frequencies (longer wavelengths).

[0026] The two receivers 120a and 120b each include an antenna 122a and 122b (collectively referred to as receiver antennas 122) to receive RF signals. The receiver antennas 122 can be digitally controlled to change the phase difference of the signals they receive from the RFID tag 10. This digital control allows for convenient steering of the two antennas 122 to different angles of arrival (AOAs).

[0027] In one embodiment, the reader 110 and receiver 120 can be located within a single enclosure to form an integrated device. The processor 130 can also be incorporated into the device. In another embodiment, the reader 110, receiver 120, and processor 130 can be distributed in different locations. For example, the receiver 120 can be located in a location with a clear view of the space being monitored for tags 10 (e.g., on the ceiling of a room), while the processor 130 is located in a location with better human access (e.g., in a control room). The reader 120 can be connected to the processor 130 via one or more wired connections or through a wireless link (e.g., a WiFi link).

[0028] In operation, reader 110 emits an RF signal toward RFID tag 10. In one embodiment, reader 110 transmits the RF signal throughout a given space (e.g., a room). In another embodiment, reader 110 radiates the RF signal with a smaller divergence to steer or sweep the RF signal throughout the space. In either case, when RFID tag 10 is within the given space, RFID tag 10 can emit a response signal as understood in the art of RFID tags.

[0029] Depending on the locations of the RFID tag 10 and the receiver 120, the response signal may propagate directly from the RFID tag 10 to the receiver 120 along the LOS path 11 without being reflected or scattered. The response signal can propagate in other directions as well. For example, the response signal may be reflected or scattered off a wall 12 (or any other obstacles distributed throughout a given space). In this case, the response signal reaches the receiver 120 along one or more NLOS paths 13. As mentioned above, this can create multipath problems and compromise the accuracy and reliability of the system 100.

[0030] The system 100 shown in FIG. 1 can distinguish signals along the LOS path 11 from signals along the NLOS path 13 based on their respective angles of arrival (AOA). The distinction can be made by determining the angles of arrival corresponding to extrema (e.g., maxima and minima) in the antenna's receive sensitivity pattern. For example, the system processor 130 can coherently sum signals received by adjacent antennas 122 at each of several phase differences, each of which corresponds to a different AOA. The coherent sum that produces a maximum corresponds to the AOA at which the LOS and NLOS signals arrive. In the absence of attenuation, the highest and steepest maximum generally corresponds to the LOS path 11, and the other maximum corresponds to the AOA for the NLOS path 13.

[0031] 1B shows a receiver 120 used to estimate an AOA θ based on the phase difference between signals received by two antennas 122a and 122b. RF signals 125a and 125b arriving at the two antennas 122a and 122b, respectively, can be considered to be substantially parallel to each other, provided that the distance d is sufficiently small compared to the distance between the receiver 120 and the RFID tag 10 (d<λ / 2). In this case, the signals 125a and 125b have the same AOA θ with respect to the antenna plane 15 defined by the two antennas 122a and 122b. Without being bound to any particular theory or mode of operation, the phase difference Δφ between the two signals 125a and 125b detected by the two antennas 122a and 122b, respectively, can be written as:

number

[0032] Equation (1) also represents the digital steering of antenna 122 toward a different AOA θ. In this case, the phase difference Δφ between two antennas 122a and 122b can be adjusted, for example, by applying a digital delay to one or both antennas 122a and 122b. This digital delay cancels the propagation delay Δ shown in FIG. 1B. Once the phase difference Δφ changes, the AOA θ changes accordingly, which means that antennas 122a and 122b are steered toward different AOA θ to receive signals 125a and 125b.

[0033] The phase difference Δφ can be varied over a range such that the corresponding AOAθ varies from 0 to π. At each AOAθ, the corresponding signal amplitude can be recorded. The signal amplitude can be the coherent sum of the signals detected by the two antennas 122a and 122b. Upon completing a scan of the AOAθ, a plot can be generated showing the signal amplitude as a function of the AOAθ to find the LOS path 11 (see, e.g., FIG. 1C and the discussion below).

[0034] In the system 100, the processor 130 controls the amount of delay applied to the antenna 122. The step size of the scan Δθ can be about π / 1000 to π / 10 (e.g., about π / 1000, about π / 500, about π / 200, about π / 100, about π / 50, about π / 20, or about π / 10, including any value and subrange therebetween).

[0035] Processor 130 can also take advantage of estimated, known, or measured symmetries to reduce scanning and / or processing time. For example, processor 130 can select and digitally calculate phase differences Δφ to steer antenna 122 at symmetric angles (e.g., ±45°) instead of asymmetric angles (e.g., −45° and +44°). Because the angles are symmetric, they yield antisymmetric results (e.g., results that have only a sign difference) and can therefore be calculated in approximately half the time as asymmetric angles.

[0036] Additionally, knowledge of the antenna pattern can be used to reduce the number of angles that need to be calculated for a given measurement accuracy. For example, the sensitivity of the antenna 122 may change rapidly around certain angles. At or near these angles, the scan Δθ step size can be reduced to sample more AOA and obtain more accurate results. In contrast, at angles where the sensitivity of the antenna 122 is relatively constant, the scan Δθ step size can be increased to sample fewer, thereby reducing scan and processing time.

[0037] FIG. 1C shows a plot 150 of conceptual signal amplitude A versus AOAθ, i.e., A(θ), for an RFID signal received by a pair of antennas 122 as shown in FIG. 1A. The top trace 151 represents a composite signal formed by digitally incrementing the phase difference between the signals received by the antennas. In this case, the composite signal includes a first maximum 155a near −3π / 8 and a second maximum 155b near +π / 4. The second maximum 155b is relatively high and narrow (sharp), while the first maximum 155a is relatively short and wide. In this case, the high and narrow second maximum 155b corresponds to a signal arriving at the receiver along a LOS path (e.g., path 11), while the wide and short first maximum 155a corresponds to a signal arriving at the receiver along a NLOS path (e.g., path 13).

[0038] The processor 130 can further be used to modify the antenna receive sensitivity pattern S(θ) from the signal amplitude A, thereby obtaining a line 152 shown along the horizontal axis. This modification can simplify the identification of angles of arrival corresponding to LOS and NLOS paths between the antenna and the RFID tag. Without being bound to any particular theory of operation mode, this modification can be performed by calibrating the antenna design to exclude the calibrated gain pattern or by modifying the measured signal amplitude A measured This can be done by effectively deconvolving the calibration pattern from (curve 151), which essentially means that the measured signal is the true signal A convolved with the antenna receive sensitivity pattern S(θ). true This is because it is a convolution of amplitudes.

number

[0039] The antenna receiving sensitivity pattern S(θ) is calculated based on a known radiation pattern (e.g., A true ) can be measured using a reference antenna with the , A measured Then, the receiver sensitivity pattern S(θ) can be calculated according to equation (2).

[0040] After deconvolution or other modification, amplitude curve 151 is transformed into two peaks 156a and 156b. The higher peak 156b corresponds to the LOS path, and the smaller peak 156a corresponds to the NLOS path between the antenna and the RFID tag. If desired, the processor can fit a curve (e.g., a Lorentzian or Gaussian) to peak 156 to generate a more accurate estimate of the AOA for the LOS and NLOS paths.

[0041] 3. Estimating the location of RFID tags Based on the AOA of the LOS paths, processor 130 can estimate the location of RFID tag 10 using a triangulation technique. More than one group of antenna arrays can be used. For example, a first antenna array, such as two antennas 122, is used to identify a first LOS path between RFID tag 10 and the first antenna array. A second antenna array (not shown) is used to identify a second LOS path between RFID tag 10 and the second antenna array. The location where the two LOS paths intersect each other (or where the error between them is minimized) is likely to be the location of RFID tag 10 in the plane of the LOS paths to the first and second antenna arrays.

[0042] Optionally, the processor can estimate the distance between each antenna and the RFID tag based on the amplitude or received signal strength indication (RSSI) of each LOS signal, or based on the slope of the phase difference over the frequency difference. With two or more distance estimates, the processor can trilaterate the location of the RFID tag in addition to, or instead of, triangulation based on AOA. These distance estimates can be used to more accurately or uniquely estimate the location of the RFID tag without AOA.

[0043] The slope of the phase difference over the frequency difference is related to a technique regularly used in radar and radar-like systems, where the phase of the received signal is directly compared to the phase of the transmitted signal. For an item (a tag in this example) at a given distance from the reader, this phase offset should vary in a predictable way with its carrier frequency. Thus, capturing this relative phase offset φ at multiple carrier frequencies, f, allows us to make an estimate of the distance from the reader as follows:

number

[0044] 4 Training and Operation of RFID Tag Location System Before becoming operational, an RFID tag location system can undergo a training phase. During this training phase, the RFID tag location system estimates the locations of reference RFID tags or other transceivers in known locations. The system calibrates itself by comparing the estimated locations of the reference RFID tags with their actual locations. Once training is complete, the system can locate unknown RFID tags, smartphones, and / or other devices. The system can repeat training periodically (e.g., at night, on weekends, etc.) or when desired (e.g., in response to a user command).

[0045] How an exemplary system (e.g., the system of FIG. 1A) performs LOS and NLOS To see how to determine and estimate the location of a tag, consider a reader that emits a continuous wave (cw) RF interrogation signal at a wavelength of λ. In a first (training) phase, the reader interrogates a set of tags whose locations are known. These tags are called reference tags. Each reference tag receives this interrogation signal and emits a response signal that is received in turn by each of k=1 to K antennas, each of which is located along a line segment of length D, where x k = kD / (K-1) is the lateral position of the kth antenna (other antenna arrangements are possible). Each antenna in the array detects the output of the tag and produces a complex baseband signal s representing the tag's output. k emits.

[0046] In the absence of multipath, the expected spatial response of each antenna to a tag with an arrival angle θ is:

number

[0047] (Since the system is not a beamforming system, it is not necessary to account for gain.) The power received across the antenna array in the θ direction can be calculated as follows:

number

[0048] B(θ) is also called the multipath profile of the antenna array because it (i.e., B(θ)) takes into account the radiated power from signals along the LOS and NLOS paths. The system measures the multipath profile of the antenna array for all of the reference tags and at each of several AOAs for one or more readers. Once the processor determines the AOAs for the LOS paths between the antenna and the reference tag, the processor can calculate the location of the reference tag using triangulation and / or trilateration, as described above.

[0049] The above techniques can be extended to 2D (or even 3D) antenna array topologies. For example, for a simple 2D array of 2x2 uniform rectangular arrays, assuming isotropic elements in the xy plane, the steering vector is given by:

number

number

[0050] The power experienced at each 3D angle (θ,φ) and therefore at the 3D multipath profile is calculated by:

number

[0051] FIG. 1D shows 3D multipath signatures measured for RFID tags at different positions and angles of arrival relative to a common receiver (antenna). Each plot shows RFID tag signal amplitude versus azimuth and elevation angles. The peaks represent LOS and NLOS paths between the tag and antenna, with the highest and steepest peak in each plot representing the LOS path. These multipath signatures can be compared to each other to determine the relative AOA and RFID tag location, as described in detail below.

[0052] After the system has completed training (measured all multipath profiles for the desired angles of arrival), it enters a second (operational) phase, which is performed in that environment (or a replicated environment). In the operational phase, the system interrogates non-reference tags (i.e., tags whose locations are unknown) and calculates a multipath profile for each unknown tag / reader combination. The system compares the multipath profile for each unknown tag with the multipath profile for the reference tag to determine the location of the unknown tag.

[0053] The system can estimate the location of an unknown tag by taking a weighted sum of three or more reference tag locations, where the weights depend on the distance between corresponding multipath profiles. For example, a reference tag location whose multipath profile more closely matches that of an unknown tag may be weighted more heavily than another reference tag location. The exact weighting can be determined using an appropriate distance metric, such as Euclidean distance or "metric learning," and it affects the locations of both the reference tag and the estimated locations of other unknown tags. Alternatively, or in addition, the system can cluster the reference tags and unknown tags according to a property (e.g., multipath profile) and determine a representative example of that property for weighting purposes.

[0054] The system can repeat the training phase, for example, periodically, to account for changes in the environment, such as changes in the number and location of reference tags and changes in the number, type, and location of obstacles that cause multipath effects. The system can also be tested in a third (post-training) phase, in which an unknown tag is moved through a series of known locations within the environment, for example, using a drone or robot. Operational Phase Similar to the system, the system measures the position of an unknown tag and compares it to the coordinates of a robot or drone to determine an optimal distance metric (metric learning) for weighting the reference tag positions.

[0055] In some cases, instead of computing a solution (e.g., AOA and LOS paths) from one antenna array and then overlaying another solution from another antenna array, raw data from both arrays can be obtained to generate a single composite solution. This allows for multiple solutions for arrays spaced farther apart than λ / 2. Solutions due to spurious signals can be ruled out by checking the validity of the resulting position estimate.

[0056] 5 Transmitters and receivers for LOS and NLOS determination FIG. 2 shows an RFID system 200 including multiple readers 210a-210n (collectively, RFID readers 210) and multiple receivers 220a-220n (collectively, receivers 220). The RFID system 200 also includes a processor 230, a common local oscillator (LO) 240, and analog front ends 250a-250n (collectively, front ends 250). Each reader 210 is grouped with a corresponding receiver 220 and a corresponding front end 250, as shown in FIG. 2. Other arrangements of the readers 210, receivers 220, and front ends 250 are possible. For example, one or more receivers 220 and / or one or more front ends 250 can share a common reader 210.

[0057] Each reader 210 includes a corresponding digital-to-analog converter (DAC) 218. The input of the DAC 218 is coupled to the processor 230, and the output of the DAC 218 is coupled to a low-pass filter 216. In operation, the DAC 218 generates an analog representation of the digital RFID tag interrogation signal generated by the processor 230. The filter 216 removes high-frequency spurs and noise from the analog RFID tag interrogation signal. The output of the filter 216 is coupled to an intermediate frequency (IF) input of the mixer 214. The LO input of the mixer 214 is coupled to the LO 240. The mixer 214 mixes the analog RFID tag interrogation signal with a high-frequency (e.g., 902-928 MHz) carrier from the LO 240 to generate an RF output, which is coupled to a power amplifier 212. The power amplifier 212 amplifies the RF output and couples it to a circulator 256, which transmits the amplified RF output to an antenna 252a via a band-pass filter 254a. Antenna 225a can be any suitable single antenna element. Circulator 256 substantially prevents the amplified RF power from propagating to or through receiver 220. Antenna 252 transmits the amplified RF power to the RFID tag, which responds with its own analog response signal.

[0058] Antenna 252 receives response signals from the RFID tags and couples them to bandpass filter 254, which filters the response signal and couples it to circulator 256. Circulator 256 then couples all or substantially all of the response signal to low-noise amplifier (LNA) 222. LNA 222 boosts the amplitude of the response signal and couples it to mixer 224, which mixes the response signal with an LO to create a downconverted RFID signal. Lowpass filter 226 removes high-frequency noise and spurs from the downconverted RFID signal, which is digitized by analog-to-digital converter (ADC) 228 and provided to processor 230.

[0059] 2 form an antenna array with a fixed or known phase difference between pairs of adjacent antennas 252. Components and connections between components of receiver 220 and front end 250 may be calibrated, adjusted, or lengthened. The phase relationship between adjacent antennas 252 may be adjusted by adjusting the phase difference between the signals received by the nearest adjacent antennas 252. ... adjacent antennas 252. The phase relationship between adjacent antennas

[0060] The system architecture shown in Figure 2 can be used to locate any wireless system, including Bluetooth and WiFi, which just operate at different frequencies. A system for locating RFID, Bluetooth, and / or WiFi devices can include multiple copies of the components shown in Figure 2, one copy for each type of device, operating at different frequency bands (e.g., 865-868 MHz or 902-928 MHz for RFID, 2400-2835.2 MHz for Bluetooth, and 2.4 GHz or 5 GHz for WiFi).

[0061] 6. RFID tag location estimation method FIG. 3A illustrates a method 300 for estimating the location of an RFID tag, smartphone, or other device having an RF transceiver using a system such as those shown in FIGS. 1A and 2. In step 302, a transmitter radiates or transmits an RFID tag interrogation signal to one or more RFID tags within a volume of interest, such as a store, stockroom, warehouse, or other environment, where the RFID tags are used. (Step 302 can be omitted when locating devices using active transmitters, such as cellular, WiFi, or Bluetooth transmitters.) The RFID tags respond to the RFID tag interrogation signal by radiating an analog RFID signal, called a response signal. Two or more antennas receive the response signals in step 304. One or more ADCs digitize the analog response signals in step 306. In addition, electronic components coupled to the antennas can also downconvert and filter the analog response signals to facilitate subsequent processing. The resulting digital RFID signals can be stored and processed in real time, post-processed, or both.

[0062] As described above, a processor coupled to the electronic components uses the digital RFID signal to identify the AOA of the signal relative to the antenna. For example, the processor can electronically steer the antenna's receive sensitivity pattern across one or more AOAs in step 308. In one embodiment, the AOAs can be selected a priori. For example, angles between 0 and π can be scanned using a uniform step size (e.g., about π / 1000 to about π / 10). Alternatively, or in addition, the AOAs can be selected based on previous measurements to reduce processing time. For example, at angles where the antenna sensitivity changes rapidly, the processor can use a smaller step size and take more samples. Additionally, the processor can use information about the RFID tag and the environment (including symmetry considerations) to select AOAs more likely to produce results to reduce processing time.

[0063] The processor can select a possible AOA based on a principal component analysis (PCA) of previously received signals. For example, the antenna can monitor the movement of a designated RFID tag. Between successive acquisitions of a response signal by the antenna, the RFID tag may move by a small amount ΔL, which can be much smaller than the distance between the RFID tag and the antenna. In this case, the AOA corresponding to the stronger signal of these adjacent measurements is selected. Since the AOAs obtained in the previous measurements may be substantially the same, the AOA estimated in the previous measurements may be used in the subsequent measurements.

[0064] Each candidate AOA corresponds to a particular phase offset (also called a phase setting) measured by the antenna. Thus, the processor can determine the signal strength for each AOA by digitally adjusting the phase difference between the digitized RFID signals from two or more antennas (e.g., nearest neighboring antennas) that have a known phase relationship and then coherently summing the digitized RFID signals in step 310. This steers the receive sensitivity pattern of the antenna through each corresponding AOA, which also results in the signal amplitude and phase detected by the antenna as a function of AOA (the phase difference between the antennas). The steering generates a plot of the signal amplitude as a function of AOA (see, for example, FIG. 1C).

[0065] In optional step 312, the processor can deconvolve or correct the antenna pattern from the signal amplitude plot (see peak 152 in FIG. 1C). This facilitates determining the LOS path by examining the height of the peaks. Generally, the highest peak corresponds to the signal traveling along the LOS path. For a more accurate estimation, the processor can fit a curve to the peaks, for example, using a polynomial or nonlinear regression, and estimate the AOA based on coefficients used to reduce or minimize errors associated with the curve fit.

[0066] After the antenna pattern is deconvolved or corrected from the signal amplitude and phase, the processor can look for minima (valleys) instead of maxima (peaks). In this case, the processor can identify the LOS and NLOS paths based on valley depth, valley width, valley slope, or some combination thereof. For example, the processor can identify the deepest, steepest valley in the representation of the signal amplitude relative to the AOA that corresponds to the AOA for the null along the LOS path to the RFID tag. Other valleys may correspond to angles relative to the null of other NLOS paths to the RFID tag.

[0067] In step 314, the processor compares the amplitude and phase with the AOA offset to determine the angle of arrival that most likely represents the LOS channel for the RFID tag. The processor can identify the LOS and NLOS paths based on the height of the maximum, the width of the maximum, the slope (rate of change) of the signal amplitude versus AOA, the curve fit coefficients, or a combination thereof. For example, the processor can look for the highest, steepest maximum in the signal amplitude versus AOA representation. This maximum represents the angle at which a peak in the antenna's receive sensitivity pattern appears along the RFID tag's LOS path. Other maxima can represent angles at which peaks appear along the RFID tag's NLOS path.

[0068] In some cases, the processor associates the response signal with an expected response from the RFID tag. This can be done, for example, in step 308 of method 300. In this case, the system (e.g., of FIG. 1A) builds or uses a library of expected responses from RFID tags. Each expected response corresponds to a distinct AOA. The processor compares the detected response signal to the expected responses to find the closest expected response. The AOA of the closest expected response is deemed to be the AOA of the response signal. This technique is similar to matched filtering and can increase the signal-to-noise ratio (SNR) by up to 20 dB or more.

[0069] In optional step 316, the processor estimates the location of the RFID tag within the environment using different angles of arrival from different pairs of antennas. For example, the processor may estimate the location of the RFID tag in at least two dimensions (e.g., in a plane parallel to the floor) from two or more estimated angles of arrival in that same plane. The estimated angles of arrival can be used to triangulate the location of the RFID tag. If the antennas are in different planes, the processor can estimate the location of the RFID tag in 3D space based on three or more estimated angles of arrival in different planes.

[0070] If more than one antenna is used in step 304 and the antennas are not all collinear, each RFID receiver can find its angle to the tag in 3D space, so that in optional step 316 the location of the tag can be determined without the constraints of antenna arrays in different planes.

[0071] In another optional step 318, the processor can track changes in the tag's location over time. More specifically, it can smoothly map various changes in the tag's location to a path in 2D or 3D space. To do this, the system measures the tag's location at multiple points in time, for example, once every second or every few seconds. It calculates the tag's location at each point in time and performs vector distance determination between successive positions to determine the tag's velocity. The processor can classify the tag's velocity by speed and direction and, based on the speed and direction, determine likely trajectories of the tag and which are likely to carry or move the RFID tag. For example, if the tag is moving at a walking pace toward an exit, the system can determine that a customer is taking an item with the tag to a checkout or store exit. Alternatively, if the tag is moving between stockrooms in a short period of time, the system can determine that an employee is restocking or shelving an item with the tag.

[0072] The system can also distinguish LOS signals from NLOS signals using measurements at multiple points in time. If the system detects an LOS signal and one or more NLOS signals, each appearing as a separate "ghost" tag, it can construct a trajectory for each signal. The trajectory for an LOS signal should vary smoothly, while the trajectory for an NLOS signal can change direction sharply as the tag moves relative to the antenna and obstacles that scatter or reflect the NLOS signal. More specifically, the processor can use the time-varying measurements of the LOS and NLOS signals to generate a principal vector of the tag's trajectory. Vectors that resolve to a smooth trajectory are likely to be LOS, while vectors that resolve to a rough trajectory or an impossible trajectory (e.g., due to some given or predetermined maximum speed of a human) are dismissed as multipath (NLOS) rays.

[0073] In systems with multiple pairs of antennas (i.e., three or more antennas), the processor can perform steps 302, 304, 306, 308, 310, 312, and 314 for different combinations of readers and antenna pairs to derive additional LOS and NLOS path information for one or more of the RFID tags in the environment. With a single reader and three or more antennas, for example, the processor can calculate the angle of arrival for the LOS path to each pair of adjacent antennas. If the midpoints of the line segments connecting different pairs of antennas are in different locations, each pair of antennas may have LOS paths to RFID tags with different angles of arrival.

[0074] The processor may also perform steps 302, 304, 306, 308, 310, 312, and 314 for a single pair of antennas and a combination of multiple readers. For example, the readers may emit signals synchronized in time and phase so that the readers interrogate RFID tags in a staggered or round-robin fashion. The processor uses information about the time and phase of the interrogation signals and the position of each reader relative to the antenna pair to determine angles of arrival for the LOS and NLOS paths. .

[0075] For systems with multiple readers, the processor can resolve the angle of arrival of each major component of the detected signal based on the location of the reader that triggered the signal. The processor can determine that those readers with matching locations / intersecting angles of arrival share the same line-of-sight path to the RFID tag. The processor can use this information to determine that other lines are the result of multipath (i.e., non-linear) rather than line-of-sight paths.

[0076] Another method is to map the tag's trajectory to the trajectory of a person within the field of view of one or more cameras. The cameras can be positioned to monitor the same volume that the antennas use to monitor RFID tags. This method can be used in conjunction with the above method to provide a single trajectory, as opposed to multiple trajectories with some vertical or horizontal shift. More specifically, the cameras can be used to detect groups of moving pixels (e.g., representing people or objects tagged with RFID tags). A processor coupled to the cameras determines the trajectories of the groups and assigns RFID tag locations to groups with matching trajectories. The processor can also segment and extract human bodies or perform pose estimation. For example, the processor can evaluate the difference between the trajectory of a bag being swung in a person's hand and the trajectory of the person.

[0077] The reader, antenna, and processor can repeatedly perform steps 302, 304, 306, 308, 310, 312, and 314. In one embodiment, steps 302-314 are performed at regular intervals. For example, the steps can have a repetition rate of about 0.1 Hz to about 100 Hz (e.g., about 0.1 Hz, about 0.2 Hz, about 0.5 Hz, about 1 Hz, about 2 Hz, about 5 Hz, about 10 Hz, about 20 Hz, about 50 Hz, or about 100 Hz, including any value and subrange therebetween).

[0078] In another embodiment, steps 302-314 can be executed at predetermined times in response to a command, a trigger event, or both. For example, steps 302-314 can be executed periodically (e.g., hourly, daily, during evening inventory, etc.). They can be executed in response to the arrival of a new shipment, a stocking or restocking event, a user command, or the detection of possible theft. For example, a store manager can invoke process 300 when the store opens in the morning and when it closes in the evening. Or, a processor can invoke process 300 automatically, for example, at predetermined times or in response to data from other sensors, including video cameras monitoring the same space as the RFID tag location system.

[0079] In yet another embodiment, steps 302-314 can be repeated more or less frequently in response to measured changes in the number or location of RFID tags. For example, the processor can determine that a first RFID tag is moving if its location or corresponding LOS path angle of arrival changes smoothly as a function of time. The processor can associate the movement of the first RFID tag with the person's movement from video or image data of the person, or information about a second RFID tag, smartphone, or other RF transceiver carried by or attached to the person. If the processor associates the movement of the first RFID tag with the person's movement, the processor can determine that the person also carries the first RFID tag.

[0080] The processor can use this information about the movement of the first RFID tag along with knowledge of the location of the first RFID tag to initiate other actions, such as detecting when the first RFID tag reaches or leaves a particular area or volume. When a person crosses a boundary surrounding the room, the processor can debit the person's account for the purchase of the item associated with the first RFID tag. The processor can also update product inventory to reflect the product movement or purchase, or sound an alarm if the movement of the first RFID tag is unauthorized.

[0081] 7 Virtual Reference Tags The system shown in Figures 1 and 2 and the process shown in Figure 3A can be used to identify a "virtual reference RFID tag" or "virtual reference tag," which is an RFID tag that can be used to generate accurate location estimates for other RFID tags. Using a virtual reference tag provides greater location accuracy for scenarios where there are multiple tags between the references; the denser the environment, the more accurate the location. Virtual reference tags also allow for the measurement of relative distances between items, even in the absence of non-virtual reference tags. For example, knowing that tag B is between tag A and tag C can be very helpful, even if the exact locations of tags A and C are unknown.

[0082] A simple method for viewing reference tags is shown in FIG. 3B, which shows a system with several RFID readers 320a-320c (collectively, RFID readers 320) interrogating RFID tags in a store or other environment. The RFID tag readers measure the LOS and NLOS signatures of RFID tags at different angles of arrival (AOAs). A plot from each reader shows the LOS signatures for a subset of the tags. A processor 328 wirelessly coupled to the RFID readers 320 compares these signatures with each other to obtain information about the relative locations of the RFID tags. This processor 328 may also be coupled via a smartphone, tablet, or computer to a remote server or a computer network, such as the Internet, for sharing and using information about the tag locations, as described in more detail below.

[0083] The processor 328 can determine the RFID tag location by fitting the signature to a curve representing the receiver sensitivity pattern of the RFID reader, determining the peaks (maximum values) of each curve, and interpolating between adjacent peaks to determine the Euclidean distance between the peaks. This Euclidean distance represents the error or deviation in the AOA for the corresponding RFID tag and RFID reader. For a pair of AOAs, if neither is known, the error represents the difference in the AOA (i.e., the relative AOA); if one AOA is known, the other can be estimated. Multiple relative (or absolute) AOAs for a single RFID tag can be used to estimate the relative (or absolute) location of the RFID tag. For example, collecting more data about the RFID tag by taking more measurements with more RFID readers over more AOAs improves location estimation accuracy beyond 50 cm, 40 cm, 30 cm, 25 cm, 20 cm, 15 cm, 10 cm, or 5 cm.

[0084] 3B illustrates how this can be used to locate an RFID tag 322 in an unknown position relative to other RFID tags and relative to one or more known locations. This shows a 1D diagram with reference tags 324a and 324b (collectively, reference tags 324) in known positions on opposite ends of a linear rack. Virtual reference tags 326a-326c and an unknown RFID tag 322 are in the rack between the reference tags 324.

[0085] The plots below the first and second RFID tag readers 320a, 320b show RFID tag signal amplitude versus angle of arrival for different RFID tags. These profiles exhibit multipath signatures similar to those described above, with the highest peaks representing the LOS path 323 between the tag and the RFID tag reader 320. (The symbols on each peak represent the corresponding (The symbol for the corresponding tag is matched.)

[0086] There are two plots for the first RFID tag reader 320a, the upper plot showing the multipath signature without any obstructions between the RFID tag reader 320a and the tag, and the lower plot showing the multipath signature with a person 321 between the RFID tag reader 320a and the tag on the left. Note that the person 321 attenuates / alteres the multipath signature for some tags but not others, and does not affect the multipath signatures received by the second and third RFID tag readers 320b, 320c.

[0087] The processor 328 can determine the relative positions of the tags by comparing their multipath signatures to one another. In this example, the tag signature for the RFID tag closest to any of the reference tags 324 (e.g., RFID tags 326a and 326b) has the lowest error when compared to the tag signature of the corresponding reference tag 324. The error metric used to compare the multipath signatures may be mean square error (MSE), dynamic temporal distortion (DTW), or any other metric that can be used to compare the similarity of signatures. Using the example of MSE, the lower the metric, the more similar the multipath signatures are. In a scenario where a multipath signature for RFID tag 322 is compared to multipath signatures for reference tags 324a and 324b, if the error between the multipath signature for RFID tag 322 and reference tag 324a is less than the error between the multipath signature for RFID tag 322 and reference tag 324b, processor 328 determines that RFID tag 322 is closer to reference tag 324a than to reference tag 324b. If the error between the multipath signature for RFID tag 322 and reference tag 324b is twice the error between the multipath signature for RFID tag 322 and reference tag 324a, RFID tag 322 may be twice as far from reference tag 324a as from reference tag 324b. Other nonlinear weightings may also be effective.

[0088] If repeated RFID signal measurements indicate that RFID tags 326a and 326b are not moving, they can be added as "virtual reference tags," even though their absolute locations are not known (at least to the same level of accuracy as the location of reference tag 324). This process can continue for other stationary RFID tags. For example, RFID tag 326c is closest to RFID tag 326a, so its RFID signature should be most similar to that of RFID tag 326a. If repeated measurements indicate that RFID tag 326c is also stationary, it may also be designated as a virtual reference tag. Continuing this process, the system can establish an ordering of RFID tags (and, therefore, items tagged by them). An error metric can be used as a surrogate for relative distance, and an estimate of absolute location can be established based on the relative distance and the known location of reference tag 324.

[0089] Because the above approach relies on the relative error of the tag signatures, it can be further improved by using signatures at multiple readers, calculating the error by reader, and summing (or combining) the errors at the different readers. This is where virtual reference tags can be used to reduce location measurement errors. Once the system has identified all of the stationary RFID tags, designated them as virtual reference tags, and located them with respect to at least their nearest neighbors, the processor 328 can locate the desired RFID tag 322 based on their multipath signatures with respect to one or more nearby virtual reference tags 326. Depending on the measurement error between different combinations of multipath signatures for the RFID tag 322 and the virtual reference tags 326, the processor 328 can increase the accuracy of its estimate of the RFID tag's actual location, for example, by 50°c of its actual location. The accuracy can be improved to within m, 40 cm, 30 cm, 25 cm, 20 cm, 15 cm, 10 cm, or 5 cm. Accuracy improves as the number of virtual reference tags 326 increases and the accuracy of each virtual reference tag's location improves.

[0090] The above 1D example can also be extended to 2D by placing a reference tag in 2D space (e.g., a wall or floor) and comparing the errors of the tag, reference tag, and virtual reference tag in this space. This example can be further extended to 3D by placing a reference tag in 3D space and comparing nearest neighbors.

[0091] This approach can be further improved by anything that changes the RF communication channel between the tag and reader. This can include moving the reader, changing the frequency at which the reader is operating, or even if a person (or object) moves into the space occupied by the reader and / or tag. For example, as shown in FIG. 3B, consider a person 321 walking between the virtual reference tag 326 and the first and second RFID readers 320a, 320b. The person attenuates or scatters the RFID signals from the virtual reference tag 326 (as well as the unknown RFID tag 322 and reference tag 324) that propagate toward the first and second RFID readers 320a, 320b. This creates a new set of signatures for the first and second RFID readers 320a, 320b that are completely uncorrelated with the set of signatures prior to the channel change and can therefore be used to reduce errors in the estimated locations of the virtual reference tag 326 and the unknown RFID tag 322.

[0092] The problem with using virtual reference tags tends to revolve around the amount of processing power used to compare every tag signature to every other tag signature. The amount of processing power can be reduced by first comparing the RFID tag's signature to its last signature. If the signature has not changed, no comparison needs to be performed. Other ways to reduce processing power include using sources of side information (e.g., video of the area around the RFID tag and previous location information) to limit comparisons to signatures of RFID tags known to be close enough to be of concern.

[0093] To identify or designate an RFID tag as a virtual reference tag, the system measures the RFID tag's location (e.g., using the methods described above) several times over a period of time as the environment around the RFID tag changes. These location estimates may be distributed over a region or volume whose size depends on noise and measurement uncertainty. As the number of measurements increases, the average location estimate may converge to a smaller region or volume whose size is limited by the underlying measurement uncertainty. Once the size of the region or volume reaches a predetermined threshold, the processor establishes an appropriate tag location (e.g., the centroid of the region or volume) and uses this tag location as a reference for similar tags. The system can repeat this process until a desired number or set of RFID tags has been added to the pool of virtual reference tags. Once a reference tag is established (its location is known), the method for calculating this reference tag location has proven reliable, and it can be used to estimate the locations of other tags.

[0094] If the location of an RFID tag changes, the system can remove it from the virtual reference tag pool unless similar RFID tags show similar changes (e.g., due to environmental changes such as human disturbance as shown in FIG. 3B). The system can identify changes in a group of RFID tags by looking at the signature of each RFID tag in the group relative to the signatures of each member of the group. The system can also look for changes (or lack of change) in the signatures received at other AOAs from the group. In FIG. 3B, for example, a person 321 reads a group of RFID tags using first and second RFID readers 320a, 320b. The correlated received LOS signature may change, but should not affect the LOS signature received by the third RFID reader 320c. The combination of correlated changes from a particular AOA and no changes in other AOAs between the group of RFID tags can indicate that the group is not moving.

[0095] The change in RFID position may be discounted in one or more of the following situations. For example, if there is no relative change between different RFID tags, it may be likely that all of the tags are blocked or moving together. In another example, the relative change is below a predetermined threshold (e.g., measurement uncertainty). In yet another example, the relative change is sufficiently short (e.g., within one, two, or three measurement cycles).

[0096] Information about the virtual reference tag can be combined with other information to improve RFID tag location accuracy. For example, an RFID tag location system can use the (estimated) positions of the virtual reference tag, product count data, and visual data from one or more cameras to determine the average density of products among virtual and / or real reference tags. In addition, the visual data can be used to determine whether a person is or was close enough to pick up or set down a product or RFID tag (pose / range estimation can be performed as well). If the visual data indicates that a virtual reference tag or a product bearing a virtual reference is or has been moved, the system can remove that virtual reference tag from the pool of virtual reference tags. Conversely, if the visual data indicates that a particular RFID tag has not moved for an extended period of time (e.g., hours or days), the system can designate that RFID tag as a virtual reference tag.

[0097] 8 Computer Vision Systems and Computer Vision Systems Training The RFID technology described above can be used to train a computer vision system to locate and / or identify different objects captured by a camera within or coupled to the computer vision system. For example, a computer vision system may include or be coupled to multiple cameras, which may be positioned to monitor a wide-angle area. Additionally, light sources emitting light of different wavelengths and / or intensities can be used to generate different environments to enhance the training of the computer vision system. Training images are acquired by the camera under different environments.

[0098] A combined computer vision / RFID tag location system can cross-reference the timestamp data of the scanned barcode / transaction, the items in the barcode / transaction, and camera data corresponding to the location of the register or checkout kiosk to pull frames containing those items. A processor performing object detection on those frames can draw a bounding box around the images to generate additional tagged images.

[0099] During training of a computer vision system, e.g., a processor running an artificial neural network, the RFID technology described above is used to locate and identify objects in training images. These objects are separated into separate images, which are fed into a training set for the computer vision system (e.g., an artificial neural network). If the computer vision system cannot distinguish between objects that are too close together, reinforcement learning can be used to filter out multiple objects. Additionally, training can train the computer vision system to recognize objects such as light fixtures, doors, and shopping carts that may be irrelevant during the use of the computer vision system, e.g., in a retail store. These objects can then be removed from the training data set. The frames containing images of these objects can then be marked as occluded frames. Because the RFID technology described above can automatically identify objects with RFID tags in an efficient manner, a large database of product images can be built without the need for a human to verify or inspect the contents of the database.

[0100] In certain cases, there may be errors in the tag's position relative to the actual object. In these situations, one can plot the tag's position from frame to frame against the human or object's position from frame to frame (e.g., achieving contiguous groups of pixels moving via optical flow or re-identification), average the distance from each set of contiguous pixel groups to each RFID tag across n frames, and group them based on which pairing resulted in the lowest error / average distance. Kalman filters work well to filter / group objects and / or objects. RFID tag first and second derivatives and pixel blob motion functions can also be incorporated to weight matches. If the goal is to attribute a product to a person, one can perform person detection in each frame and simply select pixels corresponding to the person. If the goal is to collect annotated images, one can use person detection to simply ignore pixels corresponding to the person in bounding box generation / pixel segmentation.

[0101] Figures 3C-3F show a sequence of video frames showing the movement of a shirt tagged with an RFID tag. The small circle represents the estimated location of the RFID tag attached to the shirt. The box is around a blob of pixels that the system correlates to the RFID tag movement obtained from the RFID signal received by the RFID tag reader. As shown in Figures 3C-3F, there is significant error in where the object actually is from frame to frame relative to where the system estimates the RFID tag is located, but the system is still able to provide an RFID location estimate for the pixel blob.

[0102] FIG. 3G illustrates a method 340 for correlating RFID tag location estimates with video data. This process 340 can be used to train a neural network to recognize items tagged with RFID tags or to correlate the motion of tagged and untagged objects (e.g., clothing and people with RFID tags). Process 340 begins by recognizing, segmenting out, and ignoring pixel blobs representing people in an image with a trained neural network (342). Next, the remaining pixel blobs in the image are assigned trackers that track their motion from frame to frame (344). Each pixel blob is then matched with the RFID tag that most closely follows its trajectory given a certain threshold (346). Once pixel groups have been paired to their respective RFID tags, object detection can be performed on each pixel group in each frame to discard images that clearly do not match the RFID tag description due to environmental occlusions such as bags, carts, jackets, people, and other such occlusion sources (348). Thinking of the tag data as being in pixel blobs also helps reduce or eliminate errors between tag location and item location.

[0103] Another note is that the RFID location provides constant feedback to the artificial neural network so that it is constantly learning what was right and wrong in each frame. This extends to autonomous checkout and human / product interaction, where process 340 can be used to teach the vision system what was right and wrong on a frame by frame basis.

[0104] 9. Tracking Moving RFID Tags 4 illustrates how the RFID systems and processes described above can be used to track RFID tags 402 moving in an obstacle-filled environment, such as a store, stockroom, or warehouse. In this example, a pair of RFID tag readers 410a and 410b (collectively, RFID tag readers 410) interrogate RFID tags 402 by transmitting RFID interrogation signals at regular intervals, for example, at a rate of about 0.01 Hz to about 1.0 Hz. RFID tag readers 410 may change their interrogation rate based on the signals received from RFID tags 410. If the RFID tag's response signal indicates that the RFID tag is moving rapidly, changing speed, or changing direction, RFID tag readers 410 can increase their interrogation rate to provide finer spatial and temporal resolution of the RFID tag's movement. Conversely, if the RFID tag's response signal indicates that the RFID tag is stationary or moving slowly, RFID tag readers 410 can decrease their interrogation rate to conserve energy. The RFID tag readers 410 can increase or decrease their interrogation rates, together or independently, depending on the relative movement of the RFID tags 402 .

[0105] The RFID tag reader 410 can broadcast interrogation signals over a wide range of angles, for example, via an isotropically radiating antenna, or can scan them over different angles with an antenna array as described above. A processor (not shown) wirelessly coupled to the RFID tag reader 410 uses the RFID signals from the RFID tag 402 to calculate a velocity vector 481 and trajectory 491 for the RFID tag 402. To calculate a given velocity vector 481, the processor can determine the RFID tag's position 471 at different moments in time and then determine a vector connecting those positions in 2D or 3D space. Scaling that vector by the time difference yields the velocity vector.

[0106] The processor can use the position measurements and / or velocity vector 481 to determine the RFID tag's trajectory 491. This can be a historical trajectory (i.e., where the RFID tag 402 has been) or a predicted trajectory (i.e., where the RFID tag 402 is going based on its estimated velocity). If desired, the RFID tag's current speed, recent trajectory, and / or predicted trajectory can be displayed on a smartphone, tablet, or other electronic device and used to initiate a transaction (e.g., a sale of an item associated with the RFID tag 402), prevent misplacement or theft, or track the item as it passes through a warehouse, as described in more detail below. If the velocity vector 481 and trajectory 491 indicate that the RFID tag 402 is not moving, the processor can select the RFID tag 402 as a virtual reference RFID tag, as described above.

[0107] The processor can also use the position measurements, velocity vector 481, and trajectory 491 to distinguish "real" RFID tags, such as RFID tag 410 in FIG. 4, from false signals or "ghost" RFID tags 482. In this case, ghost RFID tags 482 result from multipath effects. More specifically, FIG. 4 shows that RFID signals propagating between real RFID tag 402 and RFID tag readers 410a, 410b can take LOS paths 411a, 411b, resulting in accurate measurements of the RFID tag's position, velocity, and trajectory. These RFID signals can also take NLOS paths 413 between real RFID tag 402 and RFID tag reader 410. In this example, some portion of the RFID energy emitted by RFID tag 402 reflects or scatters off wall 412 toward first receiver 410a. Then, when the RFID tag 402 is in a specific position, this wall 412 transmits the RFID signal to the second receiver 410b. In essence, wall 412 causes first RFID tag reader 410a to receive spurious RFID signals and stops second RFID tag reader 410b from receiving any RFID signals when RFID tag 402 is between wall 412 and first RFID tag reader 412a.

[0108] In this case, processing the pseudo RFID signal simply results in the appearance of ghost RFID tag 482 shown in FIG. 4 , complete with ghost location 473, ghost velocity vector 483, and ghost trajectory 493. The processor can distinguish ghost tag 482 from the corresponding real tag 402 based on the discontinuity between ghost velocity vector 483 and ghost trajectory 493 and / or based on the similarity between ghost velocity vector 483 and real velocity vector 481, and between ghost trajectory 493 and real trajectory 491. In particular, the real and ghost velocities and trajectories appear mirror-symmetric about the line or plane defined by wall 412. The processor can use this mirror symmetry and the abrupt discontinuity at the beginning and end of ghost trajectory 493 (where wall 412 begins and ends) to distinguish between real RFID tag 402 and ghost RFID tag 482.

[0109] A camera 420 can similarly be used to track RFID tag 402. In FIG. 4, camera 420 takes a photograph (e.g., at video rate or in the vicinity) of person 401, who is carrying RFID tag 402. A processor can use an artificial neural network to recognize person 401 appearing in the image (e.g., as a general person, as an employee, or as a specific person) and correlate the person's movements with the movements of RFID tag 402. If RFID tag 402 is on a name tag, wristband, or ID card, this can be done as part of the process of training the neural network to recognize the person associated with RFID tag 402. If the neural network has already been trained, the processor can use the overlapping or matching behavior of the person and RFID tag 402 to track or trigger another action, such as selling an item carried by the person or to which RFID tag 402 is attached.

[0110] 10 RFID tag location system for retail spaces and stores with stockrooms 5A and 5B show different views of an RFID tag location system for a store 500. The RFID tag location system includes several RFID tag readers 510 distributed throughout the store's sales floor 590, stockroom 592, and changing rooms 580. The RFID tag readers 510 can be placed, for example, on or near the ceiling, as shown in FIG. 5B, to provide a clearer line of sight to the RFID tags 502 on the merchandise on the sales floor 590 and in the stockroom 592. Placing the RFID tag readers 510 above the RFID tags 502 also allows the 3D location of each tag to be determined using azimuth and elevation angle information obtained from the RFID signals received by the RFID tag readers.

[0111] RFID tags 502 may be dispersed throughout store 500, including items for sale such as clothing and other merchandise. RFID tags 502 may be embedded in the items or attached to the items by tags, clips, or stickers. Store 500 may also have other types of RFID tags, including reference RFID tags 504 in known locations such as fixed or movable clothing racks, walls, or tables. In addition, some of the movable RFID tags 502 may be designated as virtual reference RFID tags 506 if they remain stationary for a long enough period of time. Some RFID tags may also be attached to ID cards 508a and 508b (collectively, ID cards 508), key fobs, bracelets, or other items worn or carried by employees 503a-503c (collectively, employees 503) or customers 501. These ID cards 508 can identify specific employees and their locations. Similarly, there may be RFID tags 502 embedded or attached to shopping bags, baskets, or carts at the entrance 596 to the store.

[0112] RFID tag reader 510 communicates wirelessly with processor / controller 530 via wireless router 540 or other suitable device. A camera, shown disposed with RFID tag reader 510, similarly communicates with processor 530. Processor 530 can in turn communicate with tablets 530 and smartphones 540 carried by customers 501 and employees 503. It can also communicate via a suitable communications network, e.g., the Internet, with one or more servers, databases, or other remote devices that track store inventory and operations.

[0113] In operation, RFID tag reader 510 measures the location, velocity, and trajectory of RFID tags, as described above. This information is used to monitor inventory and trigger actions associated with items tagged with RFID tag 502. For example, if RFID tag reader 510 detects RFID tag 502 moving toward store exit 598 and camera 520 detects customer 501 following the same trajectory, processor 530 can trigger an automatic purchase of the associated item by customer 501. This allows customer 501 to skip checkout 582 and save time. Processor 530 can also guide customer 501 and employee 503 to specific items based on the precise RFID location estimates provided by RFID tag reader 520. This functionality can be used to guide customer 501 to desired items, such as shirts of a particular size or color, to control and re-shelve inventory, such as items left in a locker room, or to determine how inventory placement impacts sales. These and other uses are described in more detail below.

[0114] 11. Applications of High-Precision Object Location Using RFID Technology The RFID tag localization techniques described above provide fine spatial resolution and high accuracy, making them suitable for a wide number of applications, many of which are not possible with other RFID tag localization techniques. Some of these applications are described below and can be used by systems such as those shown in Figures 5A and 5B and in such environments.

[0115] 11.1 Tracking RFID tag movement In one example, RFID technology can be used in retail stores, particularly omnichannel (also spelled omni-channel) stores. "Omnichannel" refers to a multichannel approach to sales that seeks to provide customers with a seamless shopping experience, whether they are shopping online from a desktop or mobile device, by phone, or in a brick-and-mortar store. What distinguishes an omnichannel customer experience from a multichannel customer experience is that there is true integration between channels on the back end. For example, when a store implements an omnichannel approach, a customer service representative in the store can instantly reference a customer's previous purchases and preferences just as easily as a customer service representative on the phone or a customer service web chat representative. Or, a customer can use a computer, tablet, or smartphone to check inventory by store on a company's website or app, purchase items later via the website or app, and pick up the product at a location of the customer's choosing.

[0116] One problem retailers face with omnichannel ordering is detecting when an item is selected for an omnichannel order. To address this issue, RFI RFID tags (called tote tags) can be placed on totes, shopping bags, shopping carts, or any other suitable container. Each item for sale also includes or has attached to it a separate RFID tag (called an item tag). An antenna array monitors the location of each tote tag and each item tag. If the distance between an item tag and a tote tag is below a threshold (e.g., smaller than the size of the tote), the system determines that the item corresponding to the item tag is a tote. To improve detection reliability, the system can further monitor the movement of the tote tag and item tag. If they move together a distance above a threshold (e.g., greater than one meter), the system can determine that the item and tote are being carried by a customer.

[0117] The system can also monitor the customer's movements to determine whether an item is selected by the customer. For example, if the item travels with the customer a distance greater than a threshold (e.g., greater than one meter), the system can determine that the item is being carried by the customer. For example, the customer can install a user app on their smartphone, and the system can detect the presence of the customer's smartphone through communication with the user app. The system can then track the movements of the smartphone (and, accordingly, the customer) using Bluetooth, WiFi, LTE, 3G, 4G, or any other wireless technology.

[0118] In some cases, the system may maintain a record of all smartphones that do not belong to customers (e.g., the store's own devices or employees' personal devices). Once the system detects a smartphone that is not in the record, the system may determine that the customer has entered the store and may be able to track the customer's movements by tracking the smartphone.

[0119] The system can also track customer movements using facial recognition, gait recognition, or other recognition technologies. For example, a camera can be placed at the entrance of a store to recognize customers, and one or more cameras can be distributed throughout the store to monitor the entire store space. Each time a customer is captured and recognized by a camera, the location of the recognition is recorded and can be compiled with previous locations to create a map of the customer's movements. The resolution of this monitoring (e.g., the distance between two recognitions of the same customer) can depend on the number of cameras in the store (e.g., the greater the number in a store, the higher the resolution). The system can then determine that an item is selected by a customer if they move together a distance greater than a threshold. Alternatively, or additionally, the system can determine that an item is selected by a customer if the items appear together in three or more locations. The system can also determine that an item is selected by a customer if the two locations where they appear together are more than one meter apart.

[0120] The system can further update the inventory when it determines that an item that it previously determined would be selected by a customer has been returned and is available for sale. The system can determine that an item has been returned if the item has not moved for an extended period of time (e.g., more than five minutes). To improve reliability, the system can also check whether a customer is near the item while the item is not moving. If no customers remain near the item, the system can determine that the item has been returned (e.g., because a customer who previously selected the item changed their mind and abandoned the item).

[0121] In another embodiment, RFID technology is used to check whether an item is in the correct location in the store. In this case, one or more tags (called shelf tags) can be placed on each shelf that holds items for sale. Each shelf tag identifies a specific location on the shelf for the corresponding item. Each item also has an item tag. For example, a shelf tag can indicate the location of men's pants, and an item tag can be attached to a pair of men's pants. The system queries the locations of the shelf tag and the item tag to estimate the distance between them. If the distance is less than a threshold, the system can determine that the item is in the correct location. On the other hand, if the distance is greater than the threshold, the system can alert one or more store employees that the item is in the wrong location and should be moved to the correct location. The system can also provide employees with instructions about the actual location of the item and its appropriate location.

[0122] The system can also determine whether an item is in the correct location using tags attached to other retail fixtures (these tags are called fixture tags). Generally, each fixture tag can provide information about the identification of the fixture (e.g., shelf, table, counter, refrigerated display case, basket, grid, etc.), the location of the fixture, and the type and quantity of items in the fixture. In some cases, the type and quantity of items in the fixture can be determined based on industry standards. Alternatively, the type and quantity of items in the fixture can be customized for each store.

[0123] Additionally, each employee may wear a tag (called an employee tag). In one embodiment, employees may wear a bracelet containing an RFID tag. In another embodiment, the RFID tag may be sewn into the employee's uniform. In yet another embodiment, the RFID tag may be included on a badge worn by the employee. The system may use these tags to estimate and track the employee's location, for example, to manage inventory as described below.

[0124] In some cases, employee movements can be monitored by software without using RFID tags attached to the employee. For example, the system can monitor employee movements by tracking the employee's smartphone. In these cases, the employee can install a user app to facilitate communication between the smartphone and the system. The system can recognize the employee, for example, from the employee's user account on the user app.

[0125] In another example, the system can track employees' wearable devices, such as smartwatches, activity trackers (e.g., Fitbits), or smart glasses (e.g., eyeglasses with embedded electronics), among other devices. In this example, the system can maintain a record of the wearable devices owned by each employee so as to recognize the employee whenever a wearable device is detected. A system with a camera can also track employees via RFID tags on items that the camera recognizes as being held or carried by the employee.

[0126] For example, if the system determines that an item is misplaced or should be brought from back stock to the correct shelf, the system can estimate the locations of all employees using employee tags. It can then identify the employee closest to or heading toward the misplaced item. The system can alert that employee to place the item in the correct location. The system can also estimate and / or measure the time required for an employee to complete a task (e.g., time duration from alert to completion). This information can be used to review employee performance and identify changes to the store layout that can improve efficiency.

[0127] Several criteria can be used by the system to determine the appropriate employee to receive the alert. For example, the system can deliver the alert to an employee based on the employee's availability to receive and respond to the alert. In this example, the employee can communicate their availability (or unavailability) to the system via their employee device, e.g., a smartphone with a user app installed. The employee may also indicate that they are in the middle of other tasks that should not be interrupted.

[0128] In another example, the system can send an alert to an employee based on their proximity to the item in question. For a misplaced item, proximity can be quantified by the distance between the employee and the misplaced item. For an item being placed, proximity can be quantified by the distance between the employee and the stockroom. In some cases, the proximity estimation takes into account the store building or structure. For example, instead of sending an alert to an employee on another floor, the system can send an alert to an employee, preferably on the same floor as the item in question.

[0129] In yet another embodiment, the system can send alerts to employees based on the employee's ability to complete a task. For example, if an item in the women's clothing department is found to be misplaced or abandoned in a woman's changing room, the system can preferably send an alert to an employee in the women's clothing department instead of an employee in the grocery department.

[0130] The ability to complete a task can also be determined based on the current task the employee is handling. For example, if an employee has already handled several misplaced items, he may be more efficient at handling similar tasks. The system can also consult with a quality assurance system to determine an employee's ability. For example, the system can include a database of employee performance reviews for each task the employee has handled. If the system determines that the employee is efficient at restocking misplaced items, the system can preferably send an alert to the employee.

[0131] In yet another example, the system can use a combination or weighted combination of the above criteria to determine the best employee to handle the problem. For example, the system can first locate available employees. Then, among these available employees, the system can locate those within a certain distance to the item in question. From these employees, the system can then determine the best employee based on the employee's ability to complete the task.

[0132] In some cases, the system can send the alert only to the most appropriate employee (determined in any suitable manner). Alternatively, the system can send the alert to a group of appropriate employees, and each recipient can respond using their device (e.g., a smartphone). Once a recipient responds by indicating that they will attempt to process the task, the system can update the status of the issue, for example, to "in progress."

[0133] The system may also send the alert to the appropriate employee supervisor as determined by the system. Alternatively, or in addition, the system may copy the alert to quality assurance personnel to bring it to their attention and monitor the progress of the problem.

[0134] RFID technology can also be used to monitor store inventory availability in a real-time manner. In this case, the system can track the movement of items as they are picked up by customers. As mentioned above, the system can track items as they are moved with the tote. If the item is picked up, it can be determined that it has been picked up by a customer. More specifically, the system can use the movement / trajectory of the item's RFID tag and the movement / trajectory of the tote, as determined from video data and / or data about the RFID tag on or in the tote, to determine that the item is in the tote. Once the system determines that the item is picked up, the system deducts the item from available inventory. Alternatively, the system can deduct the item from inventory until the item passes through a register where it is checked out. In some cases, the system can also deduct an item, such as an article of clothing or a pair of shoes, if the customer is wearing the item.

[0135] RFID technology can also facilitate the verification of e-commerce orders, especially after the shipping box is sealed. Because RF signals can typically penetrate shipping boxes, the RFID technology described above can be used to identify items with RFID tags within the shipping box. The identified items are then compared to the order corresponding to this shipment to determine whether any items are missing or whether any items should not be in the shipping box. If a missing item is identified, the system can check an inventory or other database to find whether a replacement item is available at a distribution center (DC) or a nearby store. The system can also prevent the shipping box from leaving the store and / or DC until the items are placed in the shipping box or located for a separate shipment.

[0136] In some cases, RFID technology can be used in changing rooms to track items being tried on by customers. The system can determine if an item left in a changing room has been in the changing room for longer than a threshold time (e.g., more than 15 minutes). Alternatively, the system can track the location of the item as well as the status of the changing room. For example, if the system determines that the item is in the changing room and the changing room is not in use, the system can determine that the item has been left in the changing room. In these cases, the system can alert an employee to pick up the item and place it back on the shelf for sale.

[0137] The system can determine the status of a fitting room by tracking the presence of a customer's mobile or wearable device in the fitting room. For example, the system can generate a map of the fitting rooms to display the mobile and wearable devices detected in each fitting room. If a device is not found in a fitting room, the system can indicate that the fitting room is likely unused. In this case, an employee can go to the fitting room and pick up any abandoned items.

[0138] The system can also determine the state of the fitting room using an RFID tag attached to the fitting room door (e.g., on the movable end of the door). In this case, the fitting room door can be designed to move away from the frame when it is unlocked. Thus, the RFID tag is in a first position when the door is closed or locked (i.e., when the fitting room is occupied) and in a second position when the door is open or unlocked (i.e., when the fitting room is unoccupied). The system can then determine the state of the fitting room based on the position of the RFID tag on the door. Similarly, another option is to install several reference tags in or on the fitting room curtain to detect when they move closer together or move farther apart as a result of someone opening or closing the curtain.

[0139] Alternatively, each fitting room can use two RFID tags, one located on the movable end of the door and the other on the door frame. The tags can be placed on or incorporated into different parts of the lock on the fitting room door. The system can then determine the distance between these two tags. If they are within a threshold (e.g., about 10 cm), the system can determine that the door is closed or locked; if not, the system can determine that the door is open or unlocked and the fitting room is unoccupied.

[0140] Additionally, the system can use a combination of amplitude variations and location of the RFID signal to determine whether a garment bearing an RFID tag is on a person. For example, if a garment is floating in the air in the middle of a fitting room, it is likely on a body. If the RFID tag localization system detects a significant drop in RSSI with indications (e.g., from camera data) that the RFID tag is near a person's body, it can determine that the object / garment tagged with the RFID tag is likely on the person's body.

[0141] 11.2 Shelving items with RFID tags Accurate tracking of items also enables the system to use autonomous vehicles (e.g., robotic devices, drones, etc.) to place items on shelves without human intervention. For example, an RFID tag can be attached to each item to provide information about the item's desired location within the store. The autonomous vehicle can include a tag reader that reads the RFID tag and distributes the item to the desired location. The desired location (e.g., a designated shelf) can also be marked with an RFID tag (called a fixture tag). In some cases, the autonomous vehicle uses its built-in tag reader to locate the fixture tag, estimate the distance and direction to the fixture tag from its current location, and use the estimate to navigate toward the fixture.

[0142] In some cases, the system can monitor the location of the remotely controlled vehicle using an RFID tag on or embedded in the vehicle. If necessary, the system or user can instruct the vehicle to proceed to a fixture. In these cases, the vehicle may not include any tag readers.

[0143] Alternatively, the RFID tag can contain the item's identification information (e.g., a serial number) rather than the item's desired or intended location. Instead, the identification information is associated with the desired location information in a database. An RFID tag reader (e.g., on an autonomous vehicle) can read the identification information and communicate with the database to retrieve the location information.

[0144] Automated shelving by autonomous vehicles can be performed every night after the store closes and / or every morning before the store opens. In some cases, the shelving procedure is automatic so that it can be performed without human monitoring. Thus, shelving can be performed after hours to save on overtime costs.

[0145] In some cases, shelving can be performed on demand. For example, if the system determines that an item is on demand, the system can send a human or robot to a stockroom to select an item and deliver it to the desired location. In some cases, a human or robot can be instructed to select a misplaced item and place it in the correct location. The system can direct a robot to the location of the misplaced item and to the desired location of the item. In some cases, RFID tag data and / or camera data can be used to determine object orientation and other characteristics such as weight, geometry, and weight distribution to assist with complex problems such as grasping. It can also reveal information.

[0146] 11.3 Inventory monitoring of items with RFID tags The system can monitor inventory of items based on accurate tracking of item locations via RFID tags. As described above, the system can determine that an item has been selected or carried by a customer, in which case the system can remove the item from the list of available items. The system can further place the item on a temporary list of items under consideration for purchase by the customer. Once an item is checked out by a customer (e.g., in the event the customer leaves the store with the item), the system can remove the item from the temporary list. However, if the customer changes their mind and returns the item (or simply abandons the item) before checking out, the system can return the item to the available list.

[0147] In some cases, once the system determines that an item is under consideration by a customer, the system may interrogate the RFID tag attached to the item at a frequency greater than 1 Hz to track the item's movement. Once the item is returned to the shelf, the interrogation frequency may be reduced to reduce the computing load on the system.

[0148] Employees can participate in inventory monitoring by handling defective items. Defective items can be identified by employees or customers. In either case, employees can use an employee device to scan an RFID tag on the item and enter the item's status (e.g., "defective" or "damaged") into the system. The employee device can include a tag reader and an interactive interface (e.g., a touchscreen) for the employee to update inventory. In response to receiving the status, the system can remove the item from the available list and place the item on another list (e.g., a repair list or a return list). The system can also send one or more alerts to relevant personnel to handle the defective item.

[0149] 11.4 Employee and Product Location Using RFID Tag Location Systems The RFID tag location system can include multiple cameras, RFID tags, and a wireless communication system, such as Bluetooth or Wi-Fi, to track the precise locations of employees and products within a store. The locations of employees and products can be obtained from RFID tag location data collected by the RFID tag location system and then displayed using a GUI on a smartphone or tablet. FIG. 6 shows an exemplary GUI displaying the locations of employees and several products on a store floor plan. Because the RFID tag location system can identify the precise locations of employees and products, the relative locations between employees and products can also be displayed, as shown in FIG. 6. In addition to displaying the locations of products on the floor plan, the locations of products can also be displayed in a virtual tour of the store, in a three-dimensional view of the store, or in an online shopping function, as shown in FIG. 6.

[0150] 11.5 Product Selection Using the GUI The GUI, as described, may display one or more products on a store floor plan. The GUI may also allow a user, employee, or customer to interact with the displayed products to perform certain actions. User interaction with the GUI may be accomplished in several ways, including touch-based systems such as a pointing device, e.g., a mouse, a user's finger, or a stylus. For example, in a touch-based system, a user may select one or more products by drawing a shape with their finger around the products shown in the GUI. This process is illustrated in FIG. 7A through FIG. 7B. As illustrated in Figure 7D, Figure 7A shows a number of products displayed in a GUI. A user can begin to draw a circular shape around multiple products using their finger, as shown in Figure 7B, until completing the circular shape as shown in Figure 7C. In this way, the products contained within the circular shape are selected. Prior to the user-specified action, information about the selected products, such as the number of selected products, can be displayed as shown in Figure 7D.

[0151] Once a product is selected by a user, the user can then specify a number of actions to be performed on the selected product. These actions include (1) listing product details, such as style number, color, price, size, etc.; (2) listing sales quantity or price; (3) directing an RFID reader to read only the selected product, such as during receiving a new shipment or inventory count; (4) changing the floor display for a specific product; (5) choosing to receive price alerts for the selected product; (6) viewing information about similar products; (7) receiving recommendations for similar or newer models of the selected product; or (8) having the selected product delivered or selected for purchase. Some actions are available only to either employees or customers, depending on their function.

[0152] 11.6 Inventory Update Processing and Automatic Notification of New Product Shipments An RFID tag location system can also be used to facilitate inventory updates when new shipments of product arrive at a store. For example, a shipment may arrive at a store from a manufacturer, warehouse, distribution center, or another store. An RFID reader and user app can be used to verify the quantity of product in the shipment. RFID readers can be optimally placed in store shipment processing areas, such as stockrooms, sales floors, or other locations that the retailer can use for inbound shipment processing.

[0153] Products included in a shipment may or may not include RFID tags. For products that include RFID tags, employees can use an RFID reader and user app to verify that the quantity of product received matches the corresponding invoice for the product order. For products that do not include RFID tags, employees can add RFID tags to the products and encode the tags with the appropriate product information using an RFID reader and user app. These products can then be added before verifying the quantity of product received in the shipment.

[0154] Once the quantity of product received is confirmed, the RFID stock of product and the master corporate stock of product, which can include products with and without RFID tags across multiple stores, are updated to show the exact stock level of product at the store where the shipment was received and at the corporate level across multiple stores. If a discrepancy exists between the quantity of product received and the invoice, the RFID tag location system can facilitate resolving the discrepancy by determining whether the product arrived in the shipment, the product arrived without an RFID tag, or the product arrived with the wrong RFID tag.

[0155] Electronic notifications can also be automatically sent to customers informing them that a new shipment of products has been delivered to the store. Notifications can be sent using a variety of methods, including email, text message, messaging applications such as WhatsApp, Facebook Messenger, geofencing applications, or other electronic messaging services that are integrated with the RFID tag location system. Notifications can be tailored to priority products based on customer preferences, such as new products, best-selling products, or products selected by the customer for notification. Notifications can also be sent to customers who have previously visited a particular store or who have subscribed to receive notifications from a particular retailer or store. Retargeting advertisements or electronic messages can be sent to customers who have previously visited a store. This can also be sent to a customer who was unable to purchase a particular product because it was not available, for example, because the preferred product was not in the desired size.

[0156] The RFID tag location system can also facilitate the detection of products with missing RFID tags or incorrect RFID tags after delivery or during inventory checks. For example, an employee can inspect a stack of identical clothing and immediately discover that the stock level is zero, indicating an error in the RFID inventory due to a missing or incorrect RFID tag. In another example, an employee can be carrying a particular product and visually notice that the product is missing an RFID tag. When a product is discovered to have a missing or incorrect RFID tag after receipt and verification of the shipment, the employee can add or replace an RFID tag to the product and code the tag with the correct product information using an RFID reader and user app. Once the new RFID tag is coded, the product's RFID inventory is updated, and an automatic electronic notification can be sent to the customer, as described above.

[0157] 11.7 Automatic Product Movement and Retention Notification The RFID tag location system can also monitor product movements, for example, if a priority product is not moved to its appropriate location, such as a sales floor or storage area, or if the product is not placed on hold for a customer. Based on RFID tag location data, if a product movement or product hold does not occur within a specific time threshold defined by the retailer, e.g., 30 minutes, an electronic notification can be sent to an authorized employee, local management personnel, or corporate management personnel. The notification can be sent using a variety of methods, including email, text message, messaging applications such as WhatsApp, Facebook Messenger, geofencing applications, or other electronic messaging services integrated with the RFID tag location system.

[0158] 11.8 Product Status Based on RFID Tags The RFID tag location system can also encode additional information into a product's RFID tag. For example, an RFID status tag can be used, which can contain various product status and tracking information. The RFID status tag can be distinguished from the RFID tag itself, and a corresponding set of RFID tags can be assigned the same product information.

[0159] Multiple product statuses can be coded into the RFID tag and may be based on categories including shipping, e-commerce order, and damage. In the shipping category, the product status may include (1) product being sent from a first store to a second store, warehouse, or distribution center; (2) the date and time of product status creation; (3) the type of shipment, e.g., transfer to a different store, transfer to a warehouse, transfer to a distribution center, transfer of damaged or recalled product, or transfer of product where cleaning or rework service occurs away from the store; (4) the origin of the product, e.g., a store number; (5) the destination of the product, e.g., a store number, distribution center, or manufacturing facility number; or (6) a shipment number, e.g., a tracking number generated by an RFID tag location system or an existing legacy system.

[0160] In the e-commerce order category, the product status may be (1) the product being sent from the store to a third-party shipping address specified by the customer placing the order, (2) the date and time of product status creation, (3) the origin of the product, e.g., the store number, (4) the customer account number, e.g., the account created by the e-commerce system, (5) the e-commerce order number, e.g., the order number created by the RFID tag location system or the existing legacy e-commerce system, or (6) the e-commerce status, e.g., "In Process" status for products that have been picked up and are currently waiting to be packaged in the process area, and selected for external shipment. "Packaged" status for products packaged for shipping.

[0161] In the damage category, the product status can include (1) the product is currently unavailable due to contamination, damage, or defects, (2) the date and time of the product status creation, or (3) a damage shipment number, e.g., a reference number created by an RFID tag location system or an existing legacy system.

[0162] The use of RFID status tags can facilitate the assignment of product status in specific areas of a store based on the location accuracy of the RFID tag location system or by product type. For example, an RFID status tag on a specific product can automatically assign the same status to other products in its vicinity, e.g., products within 4 inches of the product with the RFID tag. In another example, an RFID status tag on a specific product can assign the same status to a group of products throughout the entire store. Status changes for a group of products can be displayed in the GUI using different colors or symbols for those products. This visual indicator can help employees verify the status for products.

[0163] The RFID tag location system can also automatically change product status based on the product's RFID inventory. For example, a product with an RFID tag that is in an outgoing status, e.g., moved to another store, moved to a warehouse, or moved to a distribution center, can be considered available inventory for e-commerce orders to be fulfilled by the store sending the product, unless the outgoing process is confirmed by the store. Confirmation can include that the product is in a sealed box, that transfer documentation is complete, etc.

[0164] 11.9 Product Arrival and Departure Route Tracking An RFID tag location system can include RFID readers, (depth) cameras, and technology to precisely determine the location of RFID tags, which can be used to record the path of one or more RFID-tagged products, for example, products with RFID tags or RFID status tags grouped in boxes, bags, or carts, through the store as the products enter or exit the store. Using a user app, the path can then be displayed to the user in a GUI as an animation overlaid on top of a store floor plan, as shown in Figure 8.

[0165] The RFID tag location system can also play back recorded video feeds of the arrival or departure of RFID-tagged products using the date, time, and location data recorded by the system's location technology and cameras, as shown in Figure 8. In addition, the RFID tag location system can also identify individuals carrying RFID-tagged products, personal Bluetooth or Wi-Fi enabled devices, or user IDs based on face or gait recognition.

[0166] To ensure that a store is completely covered by the RFID tag location system, RFID tag location system components can be mounted on the ceiling or walls in increments of 500 to 1,000 square feet (approximately 46 to 93 square meters), depending on the store's layout and environment. This allows the RFID tag location system to track all RFID-tagged products and Bluetooth or Wi-Fi 33-enabled devices within the store. In addition, the system can identify store boundaries, such as multiple floors, rooms, entrances, and exits. Store boundaries can be marked with RFID fiducial tags or other manual marking methods for detection. In particular, by identifying entrances and exits, the RFID tag location system can automatically register when products enter or exit the store. Cut.

[0167] 11.10 Smart, adaptive floor displays of product quantities The RFID tag location system may allow a user, e.g., an employee, to place a desired quantity of a product in a particular area of ​​a store (e.g., 12 units of product on a floor display). Additionally, the RFID tag location system may suggest ideal placement of the product to the user based on historical data on the product's performance to maximize sales. For example, a particular product may have multiple variants, e.g., footwear, clothing, accessories, and women's wedding attire, each with different sizes. The RFID tag location system may suggest to the user the best-performing product variant to place on the floor display for that particular store. Historical performance data may include historical sales, the number of times a product or product variant is viewed or tested by customers, or the conversion rate of the product or product variant, e.g., view to sales, customer testing to sales, etc.

[0168] The RFID tag location system can also dynamically adapt to the quantity and location of products in the store in real time based on the stock inventory available in the store. For example, Table 1 shows an ideal scenario in which medium and large products are the best performing variants, followed by small and extra large products. Based on user-defined requirements on the total number of products, e.g., 12 in this example, the RFID tag location system automatically calculates the number of products to place in the floor display by size. In this case, more medium and large products are shown than small and extra large products because they perform better. [Table 1]

[0169] In another example, Table 2 illustrates an adaptive scenario in which there is insufficient stock of medium-sized products and therefore the ideal floor display previously shown in Table 1 cannot be achieved. In response, the RFID tag location system reallocates the number of product variants to place in the floor display based on the suboptimal performing product variant. This does not require zeroing out the display of medium-sized products; rather, the number of medium-sized products is reduced to accommodate available stock and customer demand. In this case, more large-sized products are displayed, followed by small and extra-large-sized products. [Table 2]

[0170] Table 3 shows yet another adaptive scenario in which the M and L sizes of the product are sold out and the stock of other product variants is insufficient to meet the total number of products required. In this case, the RFID tag location system reallocates the number of product variants to best achieve the total number of products on display, while also prioritizing the best-performing sizes. [Table 3]

[0171] The RFID tag location system also sets the quantity of a product on a floor display to zero if the quantity on the sales floor is set to zero. Additionally, a notification can be sent to an employee if at least one product is on the sales floor, but the product is not on the floor display. This is based on one possible retailer strategy: all products available on the sales floor must also be placed on the floor display. The RFID tag location system can also be configured to detect discrepancies in the quantity of products on the sales floor and on the floor display, specifically to compensate for input errors into the system.

[0172] 11.11 Creating and Optimizing Picklists As previously mentioned, an RFID tag location system can accurately track the amount of product in different areas of a store, such as the sales floor or stockroom, and therefore determine what products or product variants may need to be moved to the sales floor in real time. For example, Table 4 shows the distribution of product variants in a store. As shown, based on the number of products shown in the floor displays, there are insufficient medium and extra-large products available in some sales areas. As a result, two medium and one extra-large sized product must be moved from the stockroom to the sales floor. [Table 4]

[0173] To facilitate the replenishment of products or product variants, the RFID tag location system can instantly compile a pick list, or list of requested products that need to be replenished in real time. The pick list can then be sent to a user, such as a stockroom employee, who then completes the request for all products by picking them up and delivering them to the sales floor.

[0174] The use of pick lists also exists in e-commerce note taking, where an RFID tag location system compiles a list of products requested by online customers to be picked up in store. may be applicable to satisfy the statement. Products on a picklist can be placed on hold by an employee on behalf of a customer, by a customer using the retailer's website or application, or by a customer variant in a user app. Picklists can also be used in customer stock requests, where products are requested from a stockroom by an in-store customer via a sales employee, or for misplaced products, where the product is placed in the wrong location on the sales floor or in the stockroom.

[0175] Because the RFID tag location system can track the location of a user, such as a stockroom employee, and the location of all products on a picklist, an optimized pick path (OPP) can be generated based on the shortest time or distance for the employee to pick up all products. The OPP can be displayed to the user in the user app GUI. In FIG. 9A, the OPP is displayed as a dotted line along with the picklist and the location of the products closest to the user. As the user moves, the OPP is updated, as shown in FIG. 9B. As the user begins to pick up products on the picklist, the OPP continues to update and indicates the number of products to be picked up by the user, as shown in FIGS. 9C and 9D. The next product to be picked up by the user is also indicated in the GUI. The OPP can also be used for e-commerce orders, customer stock checks, and moving misplaced products in the stockroom or on the sales floor.

[0176] When a product on a first user's pick list is picked up and delivered to the sales floor by a second user, and the first user is still fulfilling their request and has not yet picked up the product, the RFID tag location system will specifically mark the product on the first user's pick list to notify the first user that the product is no longer needed. This notification process can be performed in real time using the RFID tag location system.

[0177] 11.12 Picklist Filters The RFID tag location system may also allow users to refine pick lists based on product attributes or location. For example, a user can filter a pick list according to women's wedding dresses, stockroom 1, or women's wedding dresses in stockroom 1. Additionally, a user can set a maximum quantity of products to be included in the pick list (e.g., 10 units). The RFID tag location system then presents a pick list with a maximum of 10 units. Based on the user filter and the maximum quantity, the RFID tag location system can optimize the products on the pick list that generate the most sales for the store.

[0178] 11.13 Stray Products A stray product is a product that has been misplaced in a store; for example, a product is shown to be on the sales floor but is instead in a stockroom. An RFID tag location system can actively and accurately track the location of units of a particular product, for example, all units of men's black V-neck T-shirts are located on the sales floor or in a stockroom. The combination of the RFID tag location system's location accuracy and its ability to monitor all units of a particular product can enable automatic detection of stray products in a store. If a stray product is detected, a notification that a unit of the product is misplaced can be automatically sent to a user, e.g., an employee, immediately or can be sent after a user-defined time threshold, e.g., longer than 10 minutes.

[0179] Additionally, the user app can also generate a path in the GUI to direct the user to all lost products. This path generation feature can be used for non-lost products as well. For example, Figure 10 shows a GUI in which a specific product is selected in the store. Units of a product may not be located in the same stockroom or in the same area of ​​a particular stockroom, for example, units of a product may not be located within 6 feet of each other. In these examples, the GUI may display to the user the total number of locations the user must visit and retrieve all units of the product.

[0180] 11.14 Intelligent routing of product notifications to users The RFID tag location system, particularly the RFID reader, user app, and system's location tracking function, can be used to accurately monitor the location of products or product variants. By tracking all RFID-tagged products within a store, the system can automatically notify a user, such as an employee, when a product needs to be replenished in a specific area of ​​the store in real time. The threshold or criteria for product replenishment can vary by user. For example, a product may need to have 10 units located on a floor display. If 10 units of a product are initially on the floor display and a customer purchases one unit, a notification can be sent to an employee that the amount of product on the floor display has decreased under the specified requirement, prompting the employee to move one unit of the product to the floor display.

[0181] An employee receiving a notification for a product replenishment can also send a request for the product to another employee; for example, a sales floor employee can request a product from a stockroom employee using an internal stock request. If the requested product is not in stock at the first store, the employee can instead request the product from a second store or warehouse using an external store request, and have it delivered to the first store or to the customer's preferred address. This tracking feature can be used by users using a user app on their mobile device to locate a specific product with an RFID tag in the store or at nearby stores.

[0182] The RFID tag location system can also intelligently route stock requests to specific employees or locations to minimize the time it takes to deliver requested stock to a specific area of ​​the store or to a customer. Internal stock requests can be routed to employees based on their proximity to the store, customer, or stockroom, and their ability to complete the task in the shortest amount of time. For example, Employee A is working on five stock requests for other customers and must prioritize completing those five stock requests. The RFID tag location system can then route additional requests to the nearest available employee, such as Employee B, to fulfill the stock request. Employees also have the option to turn off or mute stock request notifications if they are currently performing an unrelated task. The RFID tag location system can also monitor the time it takes employees to complete a stock request by tracking the product and employee as they move through the store, from initial receipt of the stock request to delivery of the product to the customer or area of ​​the store.

[0183] For external stock requests, the RFID tag location system can actively monitor and update product availability at chain stores. For example, if a customer at a second store has the requested product in their shopping cart, the RFID tag location system removes this product from the available stock at the second store to ensure that customers at the first store have accurate information regarding product availability. The RFID tag location system can also be used to predict the time required for an externally requested product to be delivered to a store or customer's preferred address based on data detailing the distance between the dispatching location, e.g., a second store or warehouse, and the destination, and the rate at which stock requests are being fulfilled and sent by the dispatching location.

[0184] 11.15 Stock Request Fulfillment The RFID tag location system actively tracks the location of available products on the store and stockroom sales floors in real time and at all times. This active tracking can expedite employee completion of stock requests for customers. For example, a customer can use the user app to request a check of stock levels for a specific product in the store. FIG. 11 shows an example GUI in which a customer is requesting to identify a specific product variant, e.g., a size medium, black dress, from the stockroom to have the product delivered to them on the sales floor with options. This request can then be sent to an employee on the sales floor. The sales employee can then use the user app to request the customer-requested product from a stockroom employee. The stockroom employee can then locate and pick up multiple requested products for different customers. To facilitate delivery of products to different customers on the sales floor, the stockroom employee can use the user app, which actively monitors the locations of different customers in real time.

[0185] In addition to stock requests, there are cases where a product is misplaced or not immediately found by a customer, but is nevertheless present on the sales floor. The User App can present the location of the product, if present, in various areas of the store. For example, in FIG. 12, the GUI can show the customer the location of a selected product on the sales floor, in addition to the quantity available in the stockroom. When store employees do not remove a misplaced product on the sales floor, the User App can also allow the customer to locate the misplaced product.

[0186] 11.16 Picklist Auto-Marking A picklist relates to a list of products requested by a user, and can include requests for internal replenishment, e-commerce orders, stock requests, misplaced products, or any other list of products that require the user to locate. Products on a picklist can be automatically marked as picked up if the following conditions are met: (1) the user is engaged with the picklist using a user app, (2) the user is picking up a product on the picklist, and (3) the RFID / computer vision item location system recognizes products on the user's picklist as being picked up by the user if the product is traveling with the user based on the user's device or their RFID employee tag. Once these conditions are met, the RFID / computer vision item location system should automatically mark the product as picked up by the user. To improve the accuracy of automatic marking of picklists, a threshold can be used to determine whether a product is picked up by the user, for example, at the time the product is picked up or after the product has traveled a certain distance.

[0187] 11.17 Tracking Customers with High Cart Value The RFID tag location system can also be used to actively track the quantity and type of products in a customer's shopping cart in real time. A shopping cart can include a basket, bag, cart, etc. When a customer's shopping cart contains products that exceed a user-defined threshold, such as five whole units or a value of $500, a notification can be automatically sent to an employee identifying these customers. In addition, certain products or product categories can also be flagged by an employee to prioritize tracking. This tracking function can have multiple functional uses in the store. For example, the tracking function can be used to prevent shoplifting by tracking customers who may have a significant amount or value of products in their shopping cart or who have selected a large number of flagged products. The tracking function can also be used to identify customers who may be willing to spend more money, which can notify employees to provide better customer service to these customers, increase sales for these customers, or recommend free products to these customers.

[0188] RFID tag location data can be displayed to users using the user app's GUI in various formats. For example, an employee can see all customers in the store in the GUI and monitor their shopping carts based on product quantity or by total value. To facilitate customer identification, if a customer uses the user app's customer variants, the RFID tag location system can associate a store person with the customer's configuration. Otherwise, customers may be further identified with the products in their shopping cart by tracking the product's movement and determining whether the product is associated with a registered employee device.

[0189] 11.18 Automatic Notification of VIP Customers The RFID tag location system can also store data about customers, which can include the number of visits a customer makes to a store or the amount of money a customer spends on a monthly or annual basis. Based on this data, VIP designations can be attributed to customers who exceed a user-defined threshold.

[0190] The RFID tag location system can then be used to detect and identify VIP customers and notify employees when a VIP enters a store or a particular section of a store. Identifying VIPs can be accomplished using a number of methods, including (1) detecting VIP status based on a customer profile stored in a user app on the customer's mobile device via Bluetooth or Wi-Fi, (2) recognizing a customer's mobile device ID based on a user app on the customer's mobile device, or (3) identifying based on face or gait recognition using the computer vision capabilities of the RFID tag location system.

[0191] 11.19 Identifying Potential Product Theft As mentioned above, an RFID tag location system can actively track the movement of products in a customer's shopping cart in real time. If an abnormal event occurs while monitoring the products, possible product theft can be detected. For example, if a customer removes an RFID tag from a product, the RFID tag location system can detect this removal and immediately notify an employee of the exact product and its last known location within the store. Furthermore, the RFID tag location system can also identify and retrieve video footage recorded by the system's camera or RFID reader to assist the employee in locating the customer or product. Once this information is provided to the employee, the employee can then approach the customer and provide assistance with the product without the RFID tag.

[0192] The RFID tag location system can also timestamp and store anomalous events associated with RFID-tagged products. This information can be used to indicate potential high-theft zones within the store to employees in a GUI based on data such as the frequency of missing RFID tags. This data can be viewed in the GUI for a user-defined time frame, e.g., the previous 7 days, 30 days, or 180 days. Additionally, the RFID tag location system can identify and highlight zones of the store that may be high-theft zones by identifying the current locations of RFID tags that are potentially prone to going missing within the store.

[0193] 11.20 Automatic fitting room monitoring A similar detection strategy to identifying potential product theft can also be used for automated monitoring of fitting rooms. RFID tag location systems can track products as they enter or exit fitting rooms. Notifications can be sent in real time about products entering and exiting the fitting room. If a product is left in a fitting room, the RFID tag location system can notify an employee that a lost product is in the fitting room and return the product to its correct location in the store. If an RFID tag is removed, resulting in the product being lost in the RFID tag location system, a notification can also be sent to an employee that the product may be missing and identify the customer with whom the product was last associated. When the product in question is missing, the customer can also be identified by the other products in their shopping cart.

[0194] 11.21 Capturing and Measuring Customer and Product Interactions RFID tag location systems can also be used to detect and measure data related to customer and product interactions within a store. For example, the system can track (1) how frequently a product is picked up by a customer, (2) how long a product is viewed by a customer, (3) which products are viewed together, (4) what products customers hold while browsing new products, (5) what products are taken from the fitting room, (6) what products customers interact with before making a purchase, (7) products that may be being tested by customers, such as customers trying on clothing, based on measured distortion of the RF signal due to proximity to a body of moisture such as the human body, and (8) how long customers test products. For products tested by customers, information can also be collected about non-purchased products, such as clothing left in the fitting room, to evaluate manufacturing or finishing issues; for example, customers prefer the appearance of a garment rather than its finish. This data can potentially be used to inform stores about how to modify product manufacturing to improve sales.

[0195] To measure these parameters, an RFID tag location system can track objects in 3D space with high spatial and temporal resolution. For example, an RFID tag location system can detect whether a product is moved beyond a threshold distance, e.g., 4 inches, and held for a threshold period, e.g., 3 seconds. If these conditions are met, the product can be considered picked up or viewed by a customer.

[0196] As mentioned above, an RFID tag location system can track product and customer activity within a store. Customers can be identified either by (1) using a user app on their mobile device, which is detected by Bluetooth, Wi-Fi, or another wireless communication system or sensor, or (2) detecting customers based on individuals who do not have any tags or devices identifiable by the RFID tag location system, assuming an employee has a tag or device.

[0197] The RFID tag location system can also collect product performance data according to product groups, such as product category, subcategory, product, color, size, price range, any combination of the preceding types listed, and others. For these product groups, performance data that can be collected is as follows: (1) the product groups viewed the most or least, (2) the product groups viewed the longest or shortest, (3) the product groups taken to the fitting room the most or least, (4) the product groups tested or tried on the most or least, (5) the product groups tested or tried on for the longest or shortest period of time, and (6) the product groups with the best or worst conversion, defined as the amount of sales versus the other types of data listed. For example, if a product is viewed 100 times per day and sold 10 times per day, the conversion rate is 10%. In another example, if 100 products are tried on for 30 seconds or more and have 10 sales, the products that are tried on for 30 seconds or more have a 10% conversion rate. do.

[0198] Based on product performance data collected by the RFID tag location system, improvements to store operations can be achieved by: (1) identifying the highest performing sales areas within the store; (2) identifying areas with the most product interactions to improve staffing in that area; or (3) automatically calculating and recommending product assortments based on merchandising strategies, such as best-selling combinations (e.g., black jeans and white T-shirts perform best together) or identifying areas of the store that are best for a particular product type (e.g., dresses have the highest conversion in Zone A of the store).

[0199] Similarly, product performance data can improve the consumer shopping experience by: (1) understanding a customer's historical shopping preferences based on predefined product groups and notifying the customer of the arrival of new or restocked products that the customer previously searched for in-store or online; (2) personalizing the in-store shopping experience by highlighting products in the store or in zones of the store that may be of interest to the customer; (3) notifying the customer of possible in-store promotions; or (4) identifying the customer by detecting the customer's profile stored in a user app on the customer's mobile device via Bluetooth or WiFi, recognizing the customer's mobile device ID based on the user app on the customer's mobile device, or identifying the customer based on face or gait recognition using the computer vision capabilities of the RFID tag location system.

[0200] 11.22 Conclusion While various different embodiments have been described herein with reference to figures, various other means and / or structures are possible for performing the functions and / or obtaining the results and / or one or more of the advantages described herein. More generally, all parameters, dimensions, materials, and configurations described herein are exemplary, and the actual parameters, dimensions, materials, and / or configurations will depend on the particular application or applications in which the teachings of the present disclosure are used. It should be understood that the foregoing embodiments are presented by way of example only, and that the embodiments may be practiced other than as specifically described and claimed. Embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. Furthermore, any combination of two or more such features, systems, articles, materials, kits, and / or methods is within the scope of the present disclosure, provided that such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent.

[0201] The above-described embodiments can be implemented in any of numerous ways. For example, embodiments of the technology disclosed herein may be implemented using hardware, software, or a combination thereof. If implemented in software, the software code may be executed on any suitable processor or collection of processors, whether provided on a single computer or distributed among multiple computers.

[0202] The various methods or processes outlined herein may be coded as software executable on one or more processors using any one of a variety of operating systems or platforms. Further, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine code or intermediate code that runs on a framework or virtual machine.

[0203] In this regard, various inventive concepts may be embodied as a computer-readable storage medium (or multiple computer-readable storage media) (e.g., computer memory, one or more floppy disks, compact disks, optical disks, magnetic tapes, flash memory, circuitry in field programmable gate arrays or other semiconductor devices, or other non-transitory or tangible computer storage media) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the invention described above. The computer-readable medium or media may be portable, as described above, such that the program or programs stored thereon can be loaded into one or more different computers or other processors to implement various aspects of the invention.

[0204] The terms "program" or "software" are used herein to generally refer to any type of computer code or set of computer-executable instructions that can be used to program a computer or other processor, as described above. It will further be appreciated that, according to one aspect, one or more computer programs that perform the methods of the present invention to be performed need not reside on a single computer or processor, but may be distributed in a modular fashion among a number of different computers or processors for implementing various aspects of the present invention.

[0205] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0206] Also, various disclosed concepts may be embodied as one or more methods, examples of which are provided. The actions performed as part of a method may be ordered in any suitable manner. Thus, embodiments can be created in which actions are performed in an order different from that illustrated, including performing some actions simultaneously, even if shown as sequential actions in the exemplary embodiments.

[0207] All definitions defined and used herein should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0208] As used in this specification and claims, the indefinite articles "a" and "an" should be understood to mean "at least one," unless expressly indicated otherwise.

[0209] As used in this specification and claims, the term "and / or" should be understood to mean "either or both" of the conjoined elements, i.e., elements that are present conjunctively in some cases and disjunctively in other cases. Multiple elements listed with "and / or" should be construed in the same manner, i.e., "one or more" of the conjoined elements. Other elements, whether related or unrelated to the elements specifically identified, may optionally be present other than the elements identified by the "and / or" clause. Thus, as a non-limiting example, a reference to "A and / or B," when used in conjunction with open-ended language such as "comprising," can, in one embodiment, refer to A only (optionally including elements other than B); in another embodiment, refer to B only (optionally including elements other than A); in yet another embodiment, refer to both A and B (optionally including other elements); etc. What does that mean?

[0210] As used in the specification and claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" should be construed as inclusive, i.e., the inclusion of at least one, but also two or more, of a number of elements or list of elements, and optionally, additional items not listed. Only terms expressly indicated to the contrary, such as "only one of," or "exactly one of," or, when used in the claims, the term "consisting of," refer to the inclusion of exactly one element of a number of elements or list of elements. Generally, as used herein, the term "or" will only be construed to indicate exclusive alternatives (i.e., "either / or, but not both") when preceded by terms of exclusivity, such as "either," "one of," "only one of," or "exactly one of." "Consisting essentially of," when used in the claims, shall have its ordinary meaning as used in the field of patent law.

[0211] As used herein, the terms "about" and "approximately" generally mean plus or minus 10% of the stated value.

[0212] As used in this specification and claims, the phrase "at least one" referring to a list of one or more elements should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each element specifically listed within the list of elements, nor excluding any combination of elements in the list of elements. This definition also allows for elements other than those specifically identified within the list of elements to which the phrase "at least one" refers, whether related or unrelated to the elements specifically identified, may optionally be present. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B" or, equivalently, "at least one of A and / or B") can refer in one embodiment to at least one, optionally including two or more As (and optionally including elements other than B) with no B present; in another embodiment to at least one, optionally including two or more Bs (and optionally including elements other than A) with no A present; in yet another embodiment to at least one, optionally including two or more As, and at least one, optionally including two or more Bs (and optionally including other elements); and so on.

[0213] In the claims, as well as in the above specification, all transitional terms, such as "comprising," "including," "carrying," "having," "containing," "involving," "holding," "composed of," and the like, are to be understood as open-ended, i.e., meaning including but not limited to. Only the transitional terms "consisting of" and "consisting essentially of" shall be closed or semi-closed transitional terms, respectively, as set forth in the United States Patent and Trademark Office Guidelines, Section 2111.03.

Claims

1. A method for locating an RFID tag using a radio frequency identification (RFID) tag location system comprising an antenna array operably connected to a processor, comprising: During the training phase, measuring, for each of a plurality of reference RFID tags, with the antenna array, a reference signature indicative of radio frequency (RF) power received by the antenna array from the reference RFID tag, wherein each reference RFID tag is in a known location; In an operation phase after the training phase, measuring with the antenna array a signature indicative of RF power received by the antenna array from an RFID tag at an unknown location; comparing said signature to a previously measured signature of said RFID tag; in response to a signature not matching the previously measured signature, performing a comparison of the signature to the reference signature; determining an estimated location of the RFID tag relative to a known location of at least one reference RFID tag of the plurality of reference RFID tags based on the comparison; 16. A method for locating an RFID tag, comprising:

2. The method of claim 1, wherein the step of measuring a reference signature of at least one reference RFID tag includes a step of receiving at least a first portion of the RF power from the at least one reference RFID tag via a line-of-sight (LOS) path from the at least one reference RFID tag to the antenna array.

3. The method of claim 2, wherein the step of measuring a reference signature of at least one reference RFID tag further includes a step of receiving a second portion of the RF power from the at least one reference RFID tag via at least one non-line-of-sight (NLOS) path from the at least one reference RFID tag to the antenna array.

4. The method of claim 3, wherein the step of measuring a signature of the RFID tag includes the step of receiving the RF power from the RFID tag via a LOS path from the RFID tag to the antenna array and at least one NLOS path.

5. The method of claim 1, wherein the antenna array is a two-dimensional antenna array.

6. The method of claim 1, wherein the step of performing a comparison of the signature with the reference signature includes a step of clustering the RFID tag with a reference RFID tag of the plurality of reference RFID tags based on the reference signature.

7. The method of claim 1, wherein the step of estimating the location of the RFID tag includes a step of weighting an actual location of a reference RFID tag of the plurality of reference RFID tags according to the comparison.

8. The method of claim 7, wherein the step of weighting the actual location of the reference RFID tag is based on a distance metric.

9. The method of claim 1, wherein the step of performing a comparison of the signature with the reference signatures includes a step of determining an error between the signature and one of the reference signatures.

10. The method of claim 9, wherein the step of determining an estimated location of the RFID tag includes a step of estimating a distance between the RFID tag and at least one reference RFID tag of the plurality of reference RFID tags based on the error.

11. The method of claim 9, wherein the step of determining an estimated location of the RFID tag includes a step of nonlinearly weighting a distance between the RFID tag and at least one reference RFID tag of the plurality of reference RFID tags based on the error.

12. The method of claim 1, further comprising estimating an absolute position of the RFID tag to within 50 cm accuracy based on the estimated position of the RFID tag.

13. The method of claim 1, further comprising estimating an absolute position of the RFID tag to within 25 cm accuracy based on the estimated position of the RFID tag.

14. The method of claim 1, further comprising estimating an absolute position of the RFID tag to within 5 cm accuracy based on the estimated position of the RFID tag.

15. The method of claim 1, further comprising repeating the training phase after a change in an environment containing the plurality of reference RFID tags.

16. The method of claim 15, wherein the change in the environment includes at least one of a change in the number of reference RFID tags of the plurality of reference RFID tags, a change in the position of at least one reference RFID tag of the plurality of reference RFID tags, or a change in the number, type and position of obstacles that cause multipath effects.

17. The method of claim 1, further comprising the step of displaying an estimated location of the RFID tag to a user via a mobile device.

18. The estimated location is in at least one of a store, a stock room, or a warehouse; determining a path from the user to the RFID tag based on the estimated location; displaying to the user via the mobile device a path from the user to the RFID tag; 20. The method of claim 17, further comprising: determining a path for the user from the RFID tag to a destination location of the RFID tag based on the estimated location; displaying to the user via the mobile device a path from the RFID tag to a destination location of the RFID tag; 20. The method of claim 18, further comprising:

20. The RFID tag, wherein the RFID tag is a first RFID tag; determining a path for the user from the first RFID tag to an estimated location of a second RFID tag en route to the destination location; displaying to the user via the mobile device a path from the first RFID tag to an estimated location of the second RFID tag en route to the destination location; 20. The method of claim 19, further comprising:

21. The method of claim 17, further comprising the step of displaying information regarding an item associated with the RFID tag to the user via the mobile device.

22. An antenna array for measuring, during a training phase for each of a plurality of reference RFID tags, each in a known location, a reference signature indicative of radio frequency (RF) power received by the antenna array from the reference RFID tags, and for measuring, during an operational phase after the training phase, a signature indicative of RF power received by the antenna array from an RFID tag in an unknown location; a processor operatively connected to the antenna array for: performing a comparison of the signature to a previously measured signature of the RFID tag, and if the signature does not match the previously measured signature, performing a comparison of the signature to the reference signature, and determining an estimated location of the RFID tag relative to a known location of at least one reference RFID tag of the plurality of reference RFID tags based on the comparison of the signature to the reference signature; A radio frequency identification (RFID) tag location system comprising:

23. The RFID tag location system of claim 22, wherein the antenna array is configured to receive at least a first portion of the RF power from at least one reference RFID tag via a line-of-sight (LOS) path from the at least one reference RFID tag to the antenna array.

24. The RFID tag location system of claim 23, wherein the antenna array is configured to receive a second portion of the RF power from the at least one reference RFID tag via at least one non-line-of-sight (NLOS) path from the at least one reference RFID tag to the antenna array.

25. The RFID tag location system of claim 24, wherein the antenna array is configured to receive the RF power from the RFID tag via a LOS path from the RFID tag to the antenna array and at least one NLOS path.

26. The RFID tag location system of claim 22, wherein the antenna array is a two-dimensional antenna array.

27. ​​The RFID tag location system of claim 22, wherein the processor is configured to perform a comparison of the signature to the reference signature by clustering the RFID tag with a reference RFID tag of the plurality of reference RFID tags.

28. The RFID tag location system of claim 22, wherein the processor is configured to estimate a location of the RFID tag by weighting an actual location of a reference RFID tag of the plurality of reference RFID tags according to the comparison.

29. The RFID tag location system of claim 28, wherein the processor is configured to weight the actual location of the reference RFID tag based on a distance metric.

30. The RFID tag location system of claim 22, wherein the processor is configured to perform a comparison of the signature with one of the reference signatures by determining an error between the signature and one of the reference signatures.

31. The RFID tag location system of claim 30, wherein the processor is configured to estimate a distance between the RFID tag and at least one reference RFID tag of the plurality of reference RFID tags based on the error.

32. The RFID tag location system of claim 30, wherein the processor is configured to non-linearly weight a distance between the RFID tag and at least one reference RFID tag of the plurality of reference RFID tags based on the error.

33. The RFID tag location system of claim 22, wherein the processor is configured to estimate the absolute position of the RFID tag to within 50 cm accuracy based on the estimated position of the RFID tag.

34. The RFID tag location system of claim 22, wherein the processor is configured to estimate the absolute position of the RFID tag to within 25 cm accuracy based on the estimated position of the RFID tag.

35. The RFID tag location system of claim 22, wherein the processor is configured to estimate the absolute position of the RFID tag to within 5 cm accuracy based on the estimated position of the RFID tag.

36. The RFID tag location system of claim 22, wherein the processor is configured to transmit the estimated location of the RFID tag to a mobile device.

37. The RFID tag location system of claim 36, wherein the estimated location is in at least one of a store, a stockroom, or a warehouse, and wherein the mobile device is configured to display a path to the RFID tag.

38. The RFID tag location system of claim 37, wherein the mobile device is configured to display, for the RFID tag, a path from the RFID tag to a destination location.

39. The RFID tag is a first RFID tag; and 40. The RFID tag location system of claim 38, wherein the mobile device is further configured to display a path from the first RFID tag to an estimated location of a second RFID tag en route to the destination location.

40. The RFID tag location system of claim 36, wherein the mobile device is further configured to display information regarding an item associated with the RFID tag.

41. The method of claim 1, wherein the step of determining an estimated location of the RFID tag includes a step of associating the signature with a library of expected responses from the RFID tag.

42. In the training phase, The method of claim 1 , further comprising the step of calibrating the RFID tag location system based on the reference signatures and known locations of the plurality of reference RFID tags.

43. In the training phase, The method of claim 1 , further comprising the step of: for each of the plurality of reference RFID tags, measuring with the antenna array the RF power from the reference RFID tag at each of a plurality of angles of arrival.

Images