Method and apparatus for locating RFID tags

Abstract

A radio frequency identification (RFID) system includes an antenna array to distinguish line-of-sight (LOS) paths from non-line-of-sight (NLOS) paths. The distance between adjacent antennas in the antenna array is less than half the wavelength of the system's radio frequency (RF) signal. Each antenna in the antenna array is also digitally controlled to change the relative phase difference between the antennas, allowing the antenna array to be digitally steered through an angle of arrival (AOA) between 0 and π. The digital steering generates a graph of the signal amplitude according to the AOA. The LOS path is distinguished from the NLOS path based on the shape (e.g., depth, gradient, etc.) of the local extrema (e.g., maximum or minimum) in the graph.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority under 35 U.S.C. §119(e) to U.S. Application No. 62 / 577,530 filed on October 26, 2017 and U.S. Application No. 62 / 477,796 filed on March 28, 2017. Both of these applications are incorporated herein by reference in their entirety. Background Art

[0003] Radio frequency identification (RFID) technology has applications in many commercial areas, such as access control, animal tracking, security, and toll collection systems. A typical RFID system includes a tag (also called a transponder) and a reader (also called an interrogator). The reader includes an antenna for transmitting radio frequency (RF) signals and receiving RF signals reflected or transmitted by the tag. The tag may also include an antenna and an application specific integrated circuit (ASIC) or microchip. A unique electronic product code may be assigned to the tag to distinguish it from other tags.

[0004] RFID systems can use either active or passive tags. Active tags contain a transmitter that transmits an RF signal to a reader and a power source (e.g., a battery) to power the transmitter. In contrast, passive tags do not contain a power source, but instead draw power generated by the reader through an induced current in the tag's antenna. In a passive RFID system, the reader sends a signal using the reader antenna to excite the tag antenna. Once the tag is powered on (excited), the tag sends the stored data back to the reader.

[0005] Signals emitted or reflected by a tag can reach the reader via more than one path. For example, a signal can travel from the tag to the reader along a straight line (called a line-of-sight path or LOS path). The signal can also be reflected or scattered from obstacles (e.g., walls and other objects distributed in the environment) before reaching the reader. These paths are called non-line-of-sight (NLOS) paths. In some cases, a given signal can reach the receiver via multiple paths, causing several copies of the signal to reach the receiver. The reader senses each copy of the signal originating from a different direction or angle of arrival. This phenomenon is called "multipath" in the field of RFID technology.

[0006] Multipath can cause unwanted interference and ghosting. If different copies of a signal temporarily overlap, they can interfere with each other. Destructive interference results in attenuation. If different copies of a signal do not overlap each other, subsequent copies can appear as "ghosts." These ghosts can fool a receiver into determining that an additional RFID tag is present. Summary of the invention

[0007] Embodiments of the present technology include methods and systems for locating radio frequency identification (RFID) tags. One example includes using a system with one or more antennas or RFID tag readers to receive a plurality of first RFID signals from a first RFID tag at a first unknown location. A processor coupled to the antenna designates the first RFID tag as a first virtual reference tag based on the plurality of first RFID signals. The antenna receives at least one second RFID signal from a second RFID tag at a second unknown location. And the processor determines a position of the first RFID tag relative to the first virtual reference tag based on the at least one second RFID signal.

[0008] Another example of the present technology uses a first antenna to receive a first line of sight (LOS) signal from an RFID tag. A processor coupled to the first antenna estimates a first angle of arrival, a first phase difference, and a first frequency difference of the first LOS signal, and determines a change in the first phase difference relative to the first frequency difference. A second antenna receives a second line of sight (LOS) signal from the RFID tag. The processor estimates a second angle of arrival, a second phase difference, and a second frequency difference of the second LOS signal, and determines a change in the second phase difference relative to the second frequency difference. The processor then estimates a position of the RFID tag based on the first angle of arrival, the change in the first phase difference relative to the first frequency difference, the second angle of arrival, and the change in the second phase difference relative to the second frequency difference.

[0009] Yet another example involves receiving at least one RFID signal from at least one reference RFID tag with multiple antennas. A processor operably coupled to the antennas determines an estimated position of the reference RFID tag based on the RFID signal. The processor performs a comparison of the estimated position of the reference RFID tag with an actual position of the reference RFID tag. The processor is calibrated based on the comparison of the estimated position of the reference RFID tag with the actual position of the reference RFID tag. The antennas receive at least one RFID signal from an RFID tag at an unknown position. And the processor determines an estimated position of the RFID tag based on the RFID signal.

[0010] Another example includes receiving reference RFID signals from respective reference RFID tags at respective known locations with a plurality of antennas. The antennas also receive at least one RFID signal from an RFID tag at an unknown location. A processor coupled to the antennas determines the location of the RFID tag based on the RFID signals and the reference RFID signals.

[0011] Yet another example includes receiving a reference RFID signal from at least one reference RFID tag using an antenna array. A processor determines a reception pattern of the antenna array based on the reference RFID signal. The antenna array receives an RFID signal from an RFID tag at an unknown location, and the processor determines the location of the RFID tag based on the RFID signal and the reception pattern of the antenna array.

[0012] Another example of the present technology includes monitoring an RFID tag by receiving a plurality of RFID signals from the RFID tag over a period of time with at least one antenna. A processor coupled to the antenna estimates a plurality of possible trajectories of the RFID tag over the period of time based on the plurality of RFID signals. The processor then identifies a first trajectory of the plurality of possible trajectories that corresponds to a line of sight (LOS) path between the antenna and the RFID tag.

[0013] Another exemplary method of locating an RFID tag includes receiving a signal from an RFID tag with multiple antennas. A processor generates a first digital representation of a response detected by a first antenna of the multiple antennas and a second digital representation of a response detected by a second antenna of the multiple antennas. The processor generates multiple sums of the first digital representation and the second digital representation. Each of these sums is at a relative phase difference, and the relative phase difference represents a different angle of arrival of the signal from the RFID tag. The processor uses these sums to estimate the position of the RFID tag.

[0014] Embodiments of the present invention include devices, systems, and methods for locating radio frequency identification (RFID) tags. In one example, the method for locating an RFID tag includes sensing a signal from an RFID tag to the transmitter with multiple antennas. One or more analog-to-digital converters (ADCs) generate a first digital representation of a response detected by a first antenna in the multiple antennas, and a second digital representation of a response detected by a second antenna in the multiple antennas. A processor coupled to the ADC generates multiple sums of the first digital representation and the second digital representation. Each of the multiple sums is at a relative phase difference, and the relative phase difference represents a different angle of arrival of the signal from the RFID tag. The method also includes estimating the position of the RFID tag based on the multiple sums.

[0015] All combinations of the foregoing concepts and the additional concepts discussed in more detail below (provided that these concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of the claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein. Terms explicitly adopted herein may also appear in any disclosure incorporated by reference, and should be given the meaning that best fits the specific concepts disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The skilled artisan will appreciate that the drawings are primarily for illustrative purposes and are not intended to limit the scope of the inventive subject matter described herein. The drawings are not necessarily drawn to scale; in some cases, various aspects of the inventive subject matter disclosed herein may be shown exaggerated or enlarged in the drawings to facilitate understanding of different features. In the drawings, similar reference symbols generally refer to similar features (e.g., functionally similar and / or structurally similar elements).

[0017] Figure 1A An exemplary system for locating a radio frequency identification (RFID) tag in an environment having a line-of-sight (LOS) path and a non-line-of-sight (NLOS) path between the RFID tag and a receiver is shown.

[0018] Figure 1B An exemplary system for estimating the angle of arrival (AOA) of an incident signal is shown.

[0019] Figure 1C is a graph showing examples of composite signal amplitude versus angle of arrival / phase difference before (solid trace) and after (rays) deconvolution or otherwise correction of the antenna patch pattern.

[0020] Figure 1D Graphs of RFID tag signal amplitude versus angle and elevation are shown for four different RFID tags, each of which is at a different AOA relative to the antenna.

[0021] Figure 2 Is applicable to Figure 1A Block diagram of the transmitter and receiver of the system.

[0022] Figure 3A It is a graphic representation of the use of Figure 1A The system shown is a flow chart of a method for locating an RFID tag.

[0023] Figure 3B The use of a virtual reference RFID tag, a reference RFID tag, and multiple readers to locate an RFID tag from measurements at different AOAs is illustrated.

[0024] Figure 3C – 3F is a video frame showing the measured position of the RFID tag indicated by the circle and a bounding box drawn around the object tagged with the RFID tag using a neural network or other computer vision technology.

[0025] Figure 3G is a flow chart illustrating a method for associating RFID tag measurements with image data.

[0026] Figure 4The LOS and NLOS signal paths from a mobile RFID tag to a pair of antennas and the corresponding true and "ghost" velocity vectors and trajectories derived from the LOS and NLOS signals are illustrated.

[0027] Figure 5A An RFID tag locating system in a retail store and a warehouse is shown.

[0028] Figure 5B Shown in Figure 5A RFID tag transmitters and receivers are placed on items sold in retail stores.

[0029] Figure 6 A graphical user interface (GUI) for a smartphone or tablet is shown that displays employee locations and product locations derived from RFID tag location data.

[0030] Fig. 7A -7D shows how a GUI can be used to select products or other items with RFID tags for specific actions.

[0031] Figure 8 Shown is the manner in which a GUI may display real-time and / or historical movement of RFID tags in a store, backroom, warehouse, or other environment monitored by an RFID tag locating system.

[0032] Fig. 9A - 9D shows how a GUI may be used to plan and track, for example, a picking route in a stockroom or warehouse to pull items from a picking list based on RFID tag location data.

[0033] Fig.10 Shown is the manner in which a GUI may be used to identify and locate stray items with RFID tags.

[0034] Fig.11 Shown is the manner in which a GUI may be used to fulfill inventory requests for RFID-tagged items.

[0035] Fig.12 Shown is the manner in which a GUI may be used to display the location of selected products tagged with RFID tags on the sales floor and / or in the stockroom. DETAILED DESCRIPTION

[0036] So far, RFID positioning technology has not lived up to the expected expectations. Combined with computer vision technology, the RFID positioning technology of the present invention provides unprecedented speed and accuracy. In fact, it can be up to 300 times more accurate than traditional RFID positioning technology. For example, the system and method disclosed below can be used to locate an RFID tag within 50cm, 40cm, 30cm, 25cm, 20cm, 15cm, 10cm, 5cm or 2.5cm of its actual location. At this speed and accuracy, it can be used to track RFID-tagged items in real time, even the slightest movement. This level of speed and accuracy makes it possible to find and restock items almost instantly, and track interactions between RFID tags. For RFID tags on products in a store, this generates data about customer interactions with products item by item, and enables autonomous checkout.

[0037] Unless physically incompatible, all technologies disclosed herein can be used together with each other. For example, an RFID tag positioning system can use multiple RFID tag readers to interrogate reference tags, virtual reference tags, and RFID tags from multiple arrival angles and create (multi-path) signatures based on the received signals. Such a system can locate tags in two or three dimensions relative to each other and / or absolute (known) positions. The resulting positions can be associated with video data for training neural networks or managing the operation of stores or warehouses. The location information can also be displayed on a smartphone, tablet computer, or other device for inventory and supply chain management, etc., which is described in more detail below.

[0038] 1 Multipath and RFID Signals

[0039] To solve the multipath problem in known radio frequency identification (RFID) technology and accurately locate RFID tags, the systems, methods and devices described herein use antenna arrays to distinguish RF signals traveling along a line of sight (LOS) path from RF signals traveling along a non-line of sight (NLOS) path. The distance between adjacent antennas in the antenna array can be less than half the wavelength of the radio frequency (RF) signal of the system. Each antenna in the antenna array is also digitally controlled to change its relative phase difference relative to the other antennas in the antenna array. Each different phase setting of the antenna array corresponds to a different angle of arrival (AOA) measured by the antenna array. As long as the array includes three or more antennas, the antenna array can be digitally manipulated through an elevation angle AOA between 0 and π (i.e., between 0 and 180 degrees) and an azimuth angle AOA between 0 and 2π (i.e., between 0 and 360 degrees).

[0040] Digital manipulation then allows a graph or other representation of the signal amplitude to be generated based on the AOA. LOS paths are distinguished from NLOS paths based on local extrema (e.g., maxima or minima) in the graph. For example, the highest (lower) steepest maximum (minimum) may be at the AOA corresponding to the LOS path. Triangulating the AOA for two or more different LOS paths produces a three-dimensional (3D) position of the RFID tag. In theory, this method can locate an item to perfect accuracy under perfect environmental conditions. In realistic indoor conditions, this technique can achieve a position accuracy of better than 50 cm.

[0041] The LOS path estimated above can be used to determine the position of the RF tag via triangulation. The first antenna or antenna group is used to estimate the first LOS path to the RF tag, and the second antenna or antenna group is used to estimate the second LOS path to the same RF tag. Subsequently, the triangulation in the two LOS paths provides an estimate of the 3D position of the RF tag.

[0042] The above method utilizes digital manipulation of antenna arrays and can be cost-effective in practice. In addition, the method can be easily expanded to multiple antenna arrays. These antenna arrays can be distributed in a given space (for example, on the ceiling of a store or warehouse) to ensure that at least two antenna arrays have LOS paths for RFID tags in the space. This is particularly advantageous in indoor environments where multiple obstacles may exist. Examples of indoor applications of this RFID method include retail stores, libraries, warehouses, etc. (see more details below).

[0043] Digital manipulation can also be used to locate other RF transceivers, including those found in smartphones, wearable devices, tablets, laptops, and other portable electronic devices with WiFi, Bluetooth, or similar antennas. As with RFID tag positioning briefly described above and described in more detail below, a transmitter transmits a trigger signal to a device with a WiFi, Bluetooth, or other RF transceiver. In response to this trigger signal, the device sends a response that is detected by two or more receivers via a LOS and / or NLOS path. A processor coupled to the receiver manipulates the receiver's receive mode by digitally manipulating the AOA for different combinations of receivers (e.g., two-by-two combinations of receivers) and looking for the strongest signal based on the AOA.

[0044] 2 Systems for distinguishing LOS paths from NLOS paths

[0045] Figure 1AA system 100 is shown for distinguishing LOS paths 11 from NLOS paths 13 to a device or item, smartphone, wearable computing device, tablet, or laptop computer having an RF transceiver (e.g., RFID tag 10). System 100 includes an RFID reader (transmitter) 110 and two receivers 120a and 120b (collectively referred to as receivers 120, also referred to as receiver antennas 120). Reader 110 and receivers 120 are coupled to processor 130. For illustrative purposes, Figure 1A Two receivers 120 are shown in FIG. In practice, the system 100 may include more than two receivers 120. These receivers 120 may be arranged in a one-dimensional (1D) or two-dimensional (2D) array. In another example, the receivers 120 are randomly or irregularly dispersed in a given space.

[0046] The receiver 120 may form (part of) a phase-shifted antenna array. In this case, the distance d between the two receivers 120a, 120b is substantially equal to or less than half the carrier wavelength λ of the radio frequency (RF) signal used to interrogate the RFID tag 10, i.e., d≤λ / 2. The system 100 may be configured to operate at any of a variety of carrier wavelengths (and accordingly, a variety of carrier frequencies).

[0047] For example, the system 100 may use RF signals (e.g., about 850 MHz to about 960 MHz) or microwave signals (e.g., 2.45 GHz) in the ultra-high frequency (UHF) region of the electromagnetic spectrum. For UHF signals, the corresponding carrier wavelength is about 31 cm to about 35 cm, while for microwave signals, it is about 12.2 cm. In this case, the distance d between the two receivers 120a, 120b may be substantially equal to or less than 17.5 cm at UHF frequencies and may be less than 6.1 cm at microwave frequencies. In other applications, such as outdoor applications, the system may operate at lower frequencies (e.g., 13.56 MHz, 125 kHz, etc.) corresponding to longer wavelengths (e.g., 22 meters, 2400 meters, etc.). To locate WiFi or Bluetooth devices, the system may operate at 2.0 GHz to 2.4 GHz in the unlicensed Industrial, Scientific, and Medical (ISM) band, or at any other suitable frequency band (e.g., 5 GHz). Higher frequencies (shorter wavelengths) generally provide more accurate position estimates than lower frequencies (longer wavelengths).

[0048] The two receivers 120a, 120b include antennas 122a, 122b (collectively referred to as receiver antennas 122) to receive RF signals. The receiver antennas 122 can be digitally controlled to change the phase difference of the signals it receives from the RFID tag 10. This digital control allows the two antennas 122 to be conveniently steered toward different angles of arrival (AOA).

[0049] In one example, the reader 110 and the receiver 120 can be arranged in a single housing to form an integrated device. The processor 130 can also be integrated into the device. In another example, the reader 110, the receiver 120 and the processor 130 can be distributed in different locations. For example, the receiver 120 can be arranged at a location with a clear field of view of the space where the monitoring tag 10 is located (e.g., on the ceiling of a room), while the processor 130 is arranged at a location where personnel have better access (e.g., in a control room). The reader 120 can be connected to the processor 130 via one or more wired connections or by a wireless link (e.g., a WiFi link).

[0050] In operation, the reader 110 transmits an RF signal toward the RFID tag 10. In one example, the reader 110 transmits the RF signal in a given space, such as a room. In another example, the reader 110 transmits an RF signal with less divergence and steers or sweeps the RF signal spatially. In either case, if the RFID tag 10 is within the given space, the RFID tag 10 may transmit a response signal that is understood in the field of the RFID tag.

[0051] Depending on the location of the RFID tag 10 and the receiver 120, the response signal may propagate directly from the RFID tag 10 to the receiver 120 along the LOS path 11 without reflection or scattering. The response signal may also propagate in other directions. For example, the response signal may reflect or scatter from a wall 12 (or any other obstacle distributed throughout a given space). In this case, the response signal arrives at the receiver 120 along one or more NLOS paths 13. As described above, this may cause multipath problems and compromise the accuracy and reliability of the system 100.

[0052] The system 100 shown in FIG. 1 can distinguish signals along the LOS path 11 from signals along the NLOS path 13 based on the respective angles of arrival (AOA) of the signals. The distinction can be made by determining the angles of arrival corresponding to extreme values ​​(e.g., local maxima and minima) in the antenna reception pattern. For example, the system processor 130 can coherently sum the signals received by adjacent antennas 122 at each of several phase angles, each of the phase differences corresponding to a different AOA. The coherent sum that produces the maximum value corresponds to the AOA from which the LOS signal and the NLOS signal arrive. Without attenuation, the highest and steepest maximum value generally corresponds to the LOS path 11, while the other maxima correspond to the AOA of the NLOS path 13.

[0053] Figure 1BA receiver 120 is shown for estimating the AOAθ based on the phase difference of the signals received by the two antennas 122a, 122b. The RF signals 125a, 125b arriving at the two antennas 122a, 122b, respectively, can be considered to be substantially parallel to each other, provided that the distance d is sufficiently small (d<λ / 2) compared to the distance between the receiver 120 and the RFID tag 10. In this case, the signals 125a, 125b have the same AOAθ relative to the antenna plane 15 defined by the two antennas 122a, 122b. Without being bound by any particular theory or mode of operation, the phase difference Δφ between the two signals 125a, 125b detected by the two antennas 122a, 122b, respectively, can be written as:

[0054]

[0055] Where Δ is the length difference between the two paths taken by the two signals 125a, 125b. After determining the phase difference, AOAθ can be calculated according to equation (1).

[0056] Equation (1) also indicates that the antenna 122 may be digitally steered toward different AOAθ. In this case, the phase difference Δφ between the two antennas 122a, 122b may be adjusted by, for example, applying a digital delay to one or both antennas 122a, 122b. This digital delay cancels out Figure 1B Once the phase difference Δφ changes, the AOAθ changes accordingly, which means that the antennas 122a, 122b are steered toward different AOAθ to receive the signals 125a, 125b.

[0057] The phase difference Δφ may vary within a range such that the corresponding AOAθ varies from 0 to π. The corresponding signal amplitude may be recorded at each AOAθ. The signal amplitude may be the coherent sum of the signals detected by the two antennas 122a, 122b. After the scan of AOAθ is completed, a graph may be generated to display the signal amplitude according to AOAθ and find the LOS path 11 (see, e.g. Figure 1C and described below).

[0058] In the system 100, the processor 130 may be used to control the scanning of the AOA θ by controlling the amount of delay applied to the antenna 122. The step size Δθ of the scanning may be about π / 1000 to about π / 10 (e.g., about π / 1000, about π / 500, about π / 200, about π / 100, about π / 50, about π / 20, or about π / 10, including any values ​​and sub-ranges therebetween).

[0059] The processor 130 can also take advantage of estimated, known, or measured symmetries to reduce scanning and / or processing time. For example, the processor 130 can select and digitally calculate the phase difference Δφ to steer the antenna 122 at symmetric angles (e.g., ±45°) rather than asymmetric angles (e.g., −45° and +44°). Because the angles are symmetric, they produce antisymmetric results (e.g., the results have only a sign difference) and can therefore be calculated in about half the time of asymmetric angles.

[0060] In addition, knowledge of the antenna pattern can also be used to reduce the number of angles that need to be calculated for a given measurement accuracy. For example, the sensitivity of the antenna 122 can change rapidly around certain angles. At or near these angles, the step size Δθ of the scan can be reduced to sample more AOA and produce more accurate results. Conversely, at angles where the sensitivity of the antenna 122 remains relatively constant, the step size Δθ of the scan can be increased to sample less, thereby reducing scan time and processing time.

[0061] Figure 1C It shows that for Figure 1A 150 is a graph of nominal signal amplitude A relative to AOAθ, i.e., A(θ), for an RFID signal received by a pair of antennas 122 shown in FIG. The upper trace 151 represents a composite signal formed by digitally incrementing the phase difference between the signals received by the antennas. In this case, the composite signal includes a first maximum 155a close to –3π / 8 and a second maximum 155b close to +π / 4. The second maximum 155b is relatively high and narrow (steep), while the first maximum 155a is relatively short and wide. In this case, the high and narrow second maximum 155b corresponds to a signal arriving at the receiver along a LOS path (e.g., path 11), while the wide and short first maximum 155a corresponds to a signal arriving at the receiver along a NLOS path (e.g., path 13).

[0062] Processor 130 may further be used to correct the antenna receive pattern S(θ) from the signal amplitude A, thereby producing rays 152 shown along the horizontal axis. This correction may simplify the identification of the angles of arrival corresponding to LOS paths and NLOS paths between the antenna and the RFID tag. Without being bound by any particular theory of operating mode, such correction may be achieved by calibrating the antenna design and dividing the calibrated gain pattern or by calculating the angle of arrival corresponding to the LOS path and the NLOS path between the antenna and the RFID tag. 测得的 (Curve 151) effectively deconvolves the calibration pattern, since the measured signal is essentially the true signal A convolved with the antenna reception rate pattern S(θ) 真实 Convolution of magnitude:

[0063]

[0064] This deconvolution can be used to recover the true signal amplitude based on the AOAθ.

[0065] A known transmission pattern (e.g., A 真实 ) is used to measure the antenna receive pattern S(θ). Using this reference antenna as the illumination source, it is possible to record A 测得的 Then the receiving mode S(θ) can be calculated according to equation (2).

[0066] After deconvolution or other correction, the amplitude curve 151 is converted into two peaks 156a, 156b. The larger peak 156b corresponds to the LOS path between the antenna and the RFID tag, while the smaller peak 156a corresponds to the NLOS path between the antenna and the RFID tag. If desired, the processor can fit a curve (e.g., Lorentz or Gaussian) to the peaks 156 to generate a more accurate estimate of the AOA for the LOS path and the NLOS path.

[0067] 3. Estimating the location of RFID tags

[0068] Based on the AOA of the LOS path, the processor 130 can use triangulation to estimate the position of the RFID tag 10. Two or more antenna arrays can be used. For example, a first antenna array, such as two antennas 122, is used to identify a first LOS path between the RFID tag 10 and the first antenna array. A second antenna array (not shown) is used to identify a second LOS path between the RFID tag 10 and the second antenna array. The position where the two LOS paths intersect each other (or the error between them is minimized) is a possible position of the RFID tag 10 in the plane of the LOS path and the first and second antenna arrays.

[0069] If desired, the processor can estimate the distance between each antenna and the RFID tag based on the amplitude or received signal strength indication (RSSI) of each LOS signal or based on the slope of the phase difference relative to the frequency difference. With two or more distance estimates, the processor can trilaterate the position of the RFID tag in addition to or instead of AOA-based triangulation. These distance estimates can be used to more accurately or more uniquely estimate the position of the RFID tag without AOA.

[0070] The slope of the phase difference with respect to the frequency difference refers to a technique commonly used in radar and radar-like systems, where the phase of the received signal is directly compared to the phase of the transmitted signal. For an item (in this case, a tag) at a given distance from the reader, this phase offset should vary in a predictable manner with respect to its carrier frequency. Therefore, capturing this relative phase offset φ at multiple carrier frequencies f allows the distance to the reader to be estimated as:

[0071]

[0072] where c is the speed of light.

[0073] 4. Training and operation of RFID tag positioning system

[0074] The RFID tag locating system may go through a training phase before commencing operation. During this training phase, the RFID tag locating system estimates the locations of reference RFID tags or other transceivers at known locations. The system calibrates itself by comparing the estimated locations of the reference RFID tags with their actual locations. After training is complete, the system can locate unknown RFID tags, smartphones, and / or other devices. The system may repeat training periodically (e.g., at night, on weekends, etc.) or as needed (e.g., in response to user commands).

[0075] To clarify the exemplary system (e.g., Figure 1A How to determine and estimate LOS and NLOS paths in a system) Consider a reader that transmits a continuous wave (cw) RF interrogation signal of wavelength λ. In the first (training) phase, the reader interrogates a set of tags whose positions are known. These tags are called reference tags. Each reference tag receives this interrogation signal and emits a signal in response, which is received in turn by each of k=1…K antennas, each of which is located along a line segment of length D, where x k = kD / (K-1) is the lateral position of the kth antenna. (Other antenna arrangements are possible.) Each antenna in the array detects the tag output and emits a complex baseband signal s representing the tag output. k .

[0076] If there is no multipath, the expected spatial response for each antenna of the tag at the arrival angle θ is:

[0077]

[0078] (Since the system is not a beamforming system, there is no need to consider gain.) The power received in the entire antenna array in the θ direction can be calculated as:

[0079]

[0080] B(θ) is also referred to as the multipath profile of the antenna array because it (i.e., B(θ)) takes into account the incident power of the signal along both the LOS path and the NLOS path. The system measures the multipath profile of the antenna array at each of several AOAs for all reference tags and for one or more readers. Once the processor has determined the AOA of the LOS path between the antenna and the reference tag, the processor can calculate the position of the reference tag using triangulation and / or trilateration as described above.

[0081] The above technique can be extended to 2D (or even 3D) antenna array topologies. For example, for a simple 2D array in a 2×2 uniform rectangular array, assuming isotropy of the elements in the xy plane, the steering vector is given by:

[0082]

[0083] Among them, the position vector p x and p y is given by:

[0084] and

[0085] Here, d represents the spacing between rows and columns of the array.

[0086] The power received at each 3D angle (θ, φ), and hence the 3D multipath profile, is calculated as follows:

[0087]

[0088] Figure 1D 3D multipath signatures of RFID tags measured at different locations and different angles of arrival relative to a common receiver (antenna) are shown. Each graph shows the RFID tag signal amplitude relative to azimuth and elevation. The peaks represent the LOS path and NLOS path between the tag and the antenna, and the highest, steepest peak in each graph represents the LOS path. These multipath signatures can be compared to each other to determine the relative AOA and the location of the RFID tag, which is described in more detail below.

[0089] After the system has completed training (it has measured the multipath distribution for all expected angles of arrival), it enters the second (operational) phase, which is performed in the environment (or a replicated environment). In this operational phase, the system interrogates non-reference tags (i.e., tags whose positions are unknown) and calculates the multipath distribution for each unknown tag / reader combination. The system compares the multipath distribution of each unknown tag with the multipath distribution of the reference tags to determine the position of the unknown tag.

[0090] The system can estimate the position of the unknown tag by taking a weighted sum of three or more reference tag positions, where the weight depends on the distance between the corresponding multipath distributions. For example, the position of a reference tag whose multipath distribution more closely matches the multipath distribution of the unknown tag can be weighted more heavily than the position of another reference tag. The exact weighting can be determined using a suitable distance metric, such as Euclidean distance, or "metric learning", which utilizes the positions of the reference tags and the estimated positions of the other unknown tags. Alternatively or in addition, the system can cluster the reference tags and the unknown tags according to a property (e.g., multipath distribution) and define a representative example of the property for weighting.

[0091] The system may, for example, periodically repeat the training phase to account for changes in the environment, such as changes in the number and location of reference tags, and changes in the number, type, and location of obstacles that cause multipath effects. The system may also be tested in a third (post-training) phase, in which, for example, an unknown tag is moved through a series of known locations within the environment using a drone or robot. As in the operation phase, the system measures the location of the unknown tag and compares the measured location with the coordinates of the robot or drone to determine the best distance metric for weighting the reference tag locations (metric learning).

[0092] In some cases, instead of computing a solution (e.g., AOA and LOS path) from one antenna array and then superimposing it with another solution from another antenna array, the raw data from both arrays can be taken to produce a single composite solution. This can produce multiple solutions for arrays that are spaced farther apart than λ / 2. Aliased solutions can be ruled out by checking the plausibility of the resulting position estimate.

[0093] 5 Transmitters and receivers for LOS and NLOS determination

[0094] Figure 2 RFID system 200 is shown, which includes a plurality of readers 210a-210n (collectively referred to as RFID readers 210) and a plurality of receivers 220a-220n (collectively referred to as receivers 220). RFID system 200 also includes a processor 230, a common local oscillator (LO) 240, and analog front ends 250a-250n (collectively referred to as front ends 250). Each reader 210 is grouped with a corresponding receiver 220 and a corresponding front end 250, such as Figure 2 Other arrangements of the reader 210 , receiver 220 , and front end 250 are also possible. For example, more than one receiver 220 and / or more than one front end 250 may share a common reader 210 .

[0095] Each reader 210 includes a corresponding digital-to-analog converter (DAC) 218. The input of DAC 218 is coupled to processor 230, and the output of DAC 218 is coupled to low pass filter 216. In operation, DAC 218 generates an analog representation of the digital RFID tag interrogation signal generated by processor 230. Filter 216 removes high frequency glitches and noise from the analog RFID tag interrogation signal. The output of filter 216 is coupled to an intermediate frequency (IF) input of mixer 214. The LO input of mixer 214 is coupled to LO 240. Mixer 214 mixes the analog RFID tag interrogation signal with a high frequency (e.g., 902-928 MHz) carrier from LO 240 to produce an RF output, which is coupled to power amplifier 212. Power amplifier 212 amplifies the RF output and couples it to circulator 256, which transmits the amplified RF output to antenna 252a through bandpass filter 254a. Antenna 225a can be any suitable single antenna element. Circulator 256 substantially prevents the amplified RF output from propagating to or through receiver 220. Antenna 252 transmits the amplified RF output to the RFID tag, which responds with its own analog response signal.

[0096] Antenna 252 receives the response signal from the RFID tag and couples it to a bandpass filter 254, which filters the response signal and couples it to a circulator 256. Circulator 256 then couples all or substantially all of the response signal to a low noise amplifier (LNA) 222. LNA 222 increases the amplitude of the response signal and couples it to a mixer 224, which mixes the response signal with a LO to produce a down-converted RFID signal. Low pass filter 226 removes high frequency noise and glitches from the down-converted RFID signal, which is digitized by an analog-to-digital converter (ADC) 228 and fed to a processor 230.

[0097] Figure 2 The antennas 252 shown in the figure form an antenna array with a fixed or known phase difference between adjacent pairs of antennas 252. The components and connections between the components in the receiver 220 and the front end 250 can be calibrated, tuned, lengthened or trimmed to provide a known and stable phase relationship between the signals received by the nearest adjacent antennas 252. For example, at least one antenna in each pair of adjacent antennas can be coupled to a phase tuner (not shown) to set or adjust the phase relationship between the adjacent antennas 252. A processor (e.g., processor 230 or Figure 2 The relative phase relationship between adjacent antennas 252 is digitally measured and calibrated using a different processor (not shown) . Maintaining a fixed phase relationship between adjacent antennas 252 allows the reception patterns of the antennas to be digitally manipulated by digitally adjusting the phase difference between the signals.

[0098] Figure 2 The system architecture shown in can be used to locate any wireless system, including Bluetooth and WiFi; it will just operate at different frequencies. A system for locating RFID, Bluetooth and / or WiFi devices can include Figure 2 Multiple copies of the components shown in , one for each type of device and operating in a different frequency band (e.g., 865–868 MHz or 902–928 MHz for RFID, 2400–2835.2 MHz for Bluetooth, 2.4 GHz or 5 GHz for WiFi).

[0099] 6 Methods for estimating RFID tag location

[0100] Figure 3A Diagram showing the use and Figure 1A and Figure 2 A method 300 for estimating the location of an RFID tag, smartphone, or other device with an RF transceiver is provided by a system similar to the system shown in . In step 302, a transmitter issues or transmits an RFID tag interrogation signal to one or more RFID tags within a volume of interest, such as a store, warehouse, warehouse, or other environment in which RFID tags are used. (Step 302 may be omitted when locating a device with an active transmitter, such as a cellular, WiFi, or Bluetooth transmitter.) The RFID tag responds to the RFID tag interrogation signal by transmitting an analog RFID signal, which is referred to as a response signal. In step 304, two or more antennas receive the response signal. In step 306, one or more ADCs digitize the analog response signal. In addition, electronic components coupled to the antennas may also down-convert the analog response signal and filter it to facilitate subsequent processing. The resulting digital RFID signal may be stored and processed in real time, post-processed, or both.

[0101] As described above, a processor coupled to the electronic component uses the digital RFID signal to identify the AOA of the signal relative to the antenna. For example, in step 308, the processor can electronically manipulate the receive pattern of the antenna throughout one or more AOAs. In one example, the AOA can be selected in advance. For example, a uniform step size (e.g., about π / 1000 to about π / 10) can be used to scan angles between 0 and π. Alternatively or in addition, the AOA can be selected based on previous measurements to reduce processing time. For example, at corners where the sensitivity of the antenna changes rapidly, the processor can use a smaller step size to obtain more samples. In addition, the processor can use information about the RFID tag and the environment (including symmetry considerations) to select an AOA that is more likely to produce a result in order to reduce processing time.

[0102] The processor may select possible AOAs based on principal component analysis (PCA) of previously received signals. For example, the antenna may monitor the movement of a given RFID tag. Between successive acquisitions of response signals by the antenna, the RFID tag may move only a small amount ΔL, which may be much less than the distance between the RFID tag and the antenna. In this case, the AOAs corresponding to the stronger signals in these adjacent measurements may be substantially the same, and therefore, the AOA estimated in the previous measurement may be used in the subsequent measurement.

[0103] Each candidate AOA corresponds to a particular phase offset (also referred to as a phase setting) measured by the antenna. Thus, the processor can determine the signal strength of each AOA by digitally adjusting the phase difference between the digitized RFID signals from two or more antennas with a known phase relationship (e.g., nearest neighbor antennas), and then coherently summing the digitized RFID signals at step 310. This manipulates the receive pattern of the antenna through each of the corresponding AOAs. It also produces the amplitude and phase of the signal detected by the antenna as a function of the AOA (phase difference between the antennas). This manipulation generates a graph of the signal amplitude as a function of the AOA (e.g., see Figure 1C ).

[0104] In optional step 312, the processor may deconvolute or otherwise correct the antenna pattern from the signal amplitude pattern (see, e.g., Figure 1C 152). This facilitates determining the LOS path by examining the height of the peaks. Typically, the highest peak corresponds to a signal traveling along the LOS path. For a more accurate estimate, the processor can fit a curve to the peaks, for example using a polynomial or nonlinear regression, and estimate the AOA based on coefficients used to reduce or minimize the error associated with the curve fit.

[0105] After deconvolving or otherwise correcting the antenna pattern based on signal amplitude and phase, the processor may look for minima (valleys) rather than maxima (peaks). In this case, the processor may identify LOS and NLOS paths based on valley depth, valley width, valley slope, or some combination thereof. For example, the processor may identify the deepest, steepest valley in the representation of signal amplitude relative to AOA that corresponds to zero AOA along the LOS path to the RFID tag. Other valleys may correspond to zero angles for other NLOS paths to the RFID tag.

[0106] In step 314, the processor compares the amplitude and phase at the AOA offset to determine the angle of arrival that most likely represents the LOS channel of the RFID tag. The processor can identify LOS and NLOS paths based on the height of the maximum, the width of the maximum, the slope (rate of change) of the signal amplitude relative to the AOA, the curve fitting coefficients, or a combination thereof. For example, the processor can find the highest and steepest maximum in the representation of the signal amplitude relative to the AOA. This maximum represents the angle of the peak in the receive pattern pointing to the antenna along the LOS path at the RFID tag. Other maxima may represent angles pointing to the peak along the NLOS path at the RFID tag.

[0107] In some cases, the processor associates the response signal with an expected response from the RFID tag. This can be done, for example, at step 308 in method 300. In this case, the (e.g., Figure 1A A) system can build or use a library of expected responses from RFID tags. Each expected response corresponds to a different AOA. The processor compares the detected response signal to the expected response to find the closest expected response. The AOA closest to the expected response is considered the AOA of the response signal. This technique is similar to matched filtering and can improve the signal-to-noise ratio (SNR) by up to 20dB or more.

[0108] In optional step 316, the processor estimates the position of the RFID tag in the environment using different angles of arrival from different antenna pairs. For example, the processor may triangulate the position of the RFID tag in at least two dimensions (e.g., in a plane parallel to the floor) using two or more estimated angles of arrival in the same plane. If the antennas are located in different planes, the processor may estimate the position of the RFID tag in 3D space based on three or more estimated angles of arrival in different planes.

[0109] If more than two antennas are used in step 304, and the antennas are not all co-linear, each RFID receiver can find the angle with the tag in 3D space. With this, in optional step 316, the position of the tag can be determined without constraining the antenna arrays to be in different planes.

[0110] In another optional step 318, the processor can track the change of the position of the tag over time. More specifically, the processor can map the change of the stable change of the tag position to a path in 2D or 3D space. To this end, the system measures the position of the tag at many time points, for example, at a speed of once per second or every few seconds. It calculates the position of the tag at each time point, and then determines the speed of the tag by vector distance determination between consecutive positions. The processor can classify the speed of the tag by speed and direction, and determine the possible trajectory of the tag based on the speed and direction, and who (may) carry or move the RFID tag. For example, if the tag is moving towards the exit at a walking speed, the system can determine that the customer is taking the tagged item to the checkout or store exit. Alternatively, if the tag is moving quickly to the warehouse or moving out of the warehouse, the system can determine that the employee is stocking or putting up the tagged item.

[0111] The system may also use measurements at many points in time to distinguish LOS signals from NLOS signals. If the system detects a LOS signal and one or more NLOS signals, each of which appears as a separate "ghost" tag, it can build a trajectory for each signal. The trajectory of the LOS signal should change smoothly, while the trajectory of the NLOS signal may change direction dramatically when the tag moves relative to the antenna and obstacles that scatter or reflect the NLOS signal. More specifically, the processor can use time-varying measurements of the LOS signal and the NLOS signal to generate the main vector of the tag trajectory. The vector that solves the smooth trajectory may be the LOS, and the vector that solves the rough trajectory or the impossible trajectory (for example, due to some given or predetermined maximum speed of the person) is thrown as a multipath (NLOS) ray.

[0112] In a system with multiple pairs of antennas (i.e., three or more antennas), the processor may perform steps 302, 304, 306, 308, 310, 312, and 314 for different combinations of reader and antenna pairs, thereby deriving additional LOS and NLOS path information for one or more RFID tags in the environment. With a single reader and three or more antennas, for example, the processor may calculate the angle of arrival of the LOS path to each pair of adjacent antennas. If the midpoints of the line segments connecting different antenna pairs are at different locations, each pair of antennas may have a LOS path to the RFID tag with a different angle of arrival.

[0113] The processor can also perform steps 302, 304, 306, 308, 310, 312, and 314 for a combination of a single pair of antennas and multiple readers. For example, the readers can transmit signals that are synchronized in time and phase so that the readers interrogate the RFID tags in a staggered or polling schedule. The processor uses information about the time and phase of the interrogation signal and the position of each reader relative to the antenna pair to determine the angle of arrival of the LOS path and the NLOS path.

[0114] For systems with multiple readers, the processor can solve for the angle of arrival of each principal component of the signal based on the position of the reader that triggered the detected signal. The processor can determine that those readers with overlapping positions / intersecting angles of arrival share the same LOS path to the RFID tag. The processor can use this information to determine that other rays are the result of multipath (i.e., NLOS) rather than a LOS path.

[0115] Another approach is to map the trajectory of the tag to the trajectory of a person within the field of view of one or more cameras. The cameras can be set up to monitor the same volume in which the RFID tags are monitored using antennas. This method can be used in conjunction with the above method to provide a single trajectory, as opposed to multiple trajectories with some vertical or horizontal offset. More specifically, one or more cameras can be used to detect groups of moving pixels (e.g., representing people or objects marked with RFID tags). A processor coupled to the camera determines the trajectory of the group and assigns the RFID tag location to the group with a matching trajectory. The processor can also segment the human body or perform posture estimation. For example, the processor can evaluate the difference between the trajectory of a bag swinging in a person's hand and the trajectory of the person.

[0116] One or more readers, antennas, and processors may repeatedly perform steps 302, 304, 306, 308, 310, 312, and 314. In one example, steps 302 to 314 are performed at regular intervals. For example, the steps may have a repetition rate of about 0.1 Hz to about 100 Hz (e.g., about 0.1 Hz, about 0.2 Hz, about 0.5 Hz, about 1 Hz, about 2 Hz, about 5 Hz, about 10 Hz, about 20 Hz, about 50 Hz, or about 100 Hz, including any values ​​and subranges therebetween).

[0117] In another example, steps 302 through 314 may be performed at a predetermined time, in response to a command or a triggering event, or both. For example, steps 302 through 314 may be performed periodically (e.g., hourly, daily, nightly inventory counts, etc.). These steps may be performed in response to the arrival of a new shipment, inventory or restocking activity, user commands, or detection of a possible theft. For example, a store manager may trigger process 300 when the store opens in the morning and closes in the evening. Or a processor may automatically trigger process 300, for example, at a predetermined time or in response to data from other sensors, including cameras monitoring the same space as the RFID tag positioning system.

[0118] In yet another example, steps 302 to 314 may be repeated more or less frequently in response to changes in the number or location of measured RFID tags. For example, if the location of the first RFID tag or the corresponding LOS path arrival angle changes smoothly over time, the processor may determine that the first RFID tag is moving. The processor may associate the movement of the first RFID tag with the movement of the person based on video or image data of the person or information about a second RFID tag, a smartphone, or other RF transceiver carried by or attached to the person. If the processor associates the movement of the first RFID tag with the movement of the person, the processor may determine that the person is also carrying the first RFID tag.

[0119] The processor can use the information about the movement of the first RFID tag, along with the knowledge of the location of the first RFID tag, to trigger other actions. For example, if the first RFID tag reaches a particular area or volume or crosses a boundary around an area or volume, the processor can debit the person's account for the purchase of an item associated with the first RFID tag. The processor can also update product inventory to reflect the movement or purchase of the product, or issue an alarm if the movement of the first RFID tag is unauthorized.

[0120] 7 Virtual Reference Tags

[0121] Figure 1 and Figure 2 The system shown in Figure 3A The process shown in can be used to identify "virtual reference RFID tags" or "virtual reference tags", which are RFID tags that can be used to generate accurate position estimates for other RFID tags. For scenarios where there are multiple tags between references, using virtual reference tags provides greater position accuracy: the denser the environment, the more accurate the position. Virtual reference tags also enable relative distances between items to be measured even when non-virtual reference tags are not present. For example, even if the exact locations of tags A and C are unknown, knowing that tag B is between tag A and tag C can be very useful.

[0122] Figure 3BA simple way to view reference tags is shown in , which shows a system with several RFID readers 320a–320c (collectively, RFID readers 320) interrogating RFID tags in a store or other environment. The RFID tag readers measure the LOS and NLOS signatures of the RFID tags at different angles of arrival (AOA). A graph for each reader shows the LOS signatures for a subset of tags. A processor 328 wirelessly coupled to the RFID reader 320 compares these signatures to each other to generate information about the relative positions of the RFID tags. This processor 328 can also be coupled to a remote server or computer network, such as the Internet, to share and use information about the locations of the tags via a smartphone, tablet, or computer, which is described in more detail below.

[0123] The processor 328 can determine the RFID tag location by fitting the signature to a curve representing the reception pattern of the RFID reader, determining the peak (maximum value) of each curve, and interpolating between adjacent peaks to determine the Euclidean distance between the peaks. This Euclidean distance represents the error or deviation to the corresponding RFID tag and RFID reader AOA. For a pair of AOAs, if neither is known, the error represents the difference in AOAs (i.e., relative AOA); if one AOA is known, the other can be estimated. Multiple relative (or absolute) AOAs of a single RFID tag can be used to estimate the relative (or absolute) position of the RFID tag. For example, by collecting more data about the RFID tag through more measurements with more RFID readers at more AOAs, the accuracy of the position estimate can be improved, for example, to more than 50 cm, 40 cm, 30 cm, 25 cm, 20 cm, 15 cm, 10 cm, or 5 cm.

[0124] Figure 3B The manner in which this can be used to locate an RFID tag 322 at an unknown position relative to other RFID tags and relative to one or more known positions is illustrated. It shows a 1D view of the known positions of reference tags 324a and 324b (collectively referred to as reference tags 324) on either end of a linear rack. Virtual reference tags 326a-326c and unknown RFID tag 322 are located on the rack between reference tags 324.

[0125] The graphs below the first RFID tag reader 320a and the second RFID tag reader 320b show the RFID tag signal amplitude versus angle of arrival for different RFID tags. These distributions represent the multipath signature as described above, with the highest peak representing the LOS path 323 between the tag and the RFID tag reader 320. (The symbol above each peak matches the symbol of the corresponding tag on the rack.)

[0126] There are two graphs for the first RFID tag reader 320a: the upper graph shows the multipath signatures between the RFID tag reader 320a and the tags without any obstacles, and the lower graph shows the multipath signatures between the RFID tag reader 320a and the tags on the left by a person 321. Note that the person 321 attenuates / changes the multipath signatures of some tags but not others, and does not affect the multipath signatures received by the second RFID tag reader 320b and the third RFID tag reader 320c.

[0127] Processor 328 can determine the relative position of the tag by comparing the multipath signatures of the tag with each other. In this example, the tag signature of the RFID tag (e.g., RFID tag 326a, 326b) closest to any reference tag 324 has the lowest error compared to the tag signature of the corresponding reference tag 324. The error metric used to compare the multipath signatures can be mean square error (MSE), dynamic time warping (DTW), or any other metric that can be used to compare the similarity of the signatures. Using the example of MSE, the lower the metric, the more similar the multipath signatures. In the case of comparing the multipath signature of RFID tag 322 with the multipath signatures of reference tags 324a, 324b, if the error between the multipath signatures of RFID tag 322 and reference tag 324a is less than the error between the multipath signatures of RFID tag 322 and reference tag 324b, processor 328 determines that RFID tag 322 is closer to reference tag 324a than reference tag 324b. If the error between the multipath signatures of RFID tag 322 and reference tag 324b is twice the error between the multipath signatures of RFID tag 322 and reference tag 324a, then RFID tag 322 is twice as far away from reference tag 324b as from reference tag 324a. Other nonlinear weightings may also be applicable.

[0128] If repeated RFID signal measurements show that RFID tags 326a, 326b are not moving, then they can be added as "virtual reference tags" even if their absolute positions are unknown (at least to the same level of accuracy as the position of reference tag 324). This process can be continued for other fixed RFID tags. For example, RFID tag 326c is closest to RFID tag 326a, and therefore, its RFID signature should be most similar to the RFID signature of RFID tag 326a. If repeated measurements show that RFID tag 326c is also fixed, it can also be designated as a virtual reference tag. Continuing this process, the system can determine the order of RFID tags (and therefore the items tagged with RFID tags). The error metric can be used as a proxy for relative distance, and an estimate of absolute position can be established based on the relative distance and the known position of reference tag 324.

[0129] Because the above method relies on relative errors in tag signatures, the method can be further improved by using signatures at multiple readers, calculating errors by the readers, and summing (or otherwise combining) the errors at different readers. This is a case where virtual reference tags can be used to reduce position measurement errors. Once the system has identified all fixed RFID tags, designated them as virtual reference tags, and positioned these tags relative to at least the nearest neighbor tags, the processor 328 can position the desired RFID tag 322 relative to one or more nearby virtual reference tags 326 based on its multipath signature. By measuring the errors between different combinations of the multipath signatures of the RFID tag 322 and the virtual reference tag 326, the processor 328 can improve the accuracy of its estimate of the actual location of the RFID tag, for example, to within 50 cm, 40 cm, 30 cm, 25 cm, 20 cm, 15 cm, 10 cm, or 5 cm of its actual location. The accuracy becomes better as the number of virtual reference tags 326 increases, and the accuracy of the location of each virtual reference tag increases.

[0130] The 1D example above can also be extended to 2D by laying out reference labels in 2D space (e.g., a wall or floor) and comparing the errors of labels, reference labels, and virtual reference labels in that space. This example can be further extended to 3D by laying out reference labels in 3D space and comparing nearest neighbor labels.

[0131] This method can be further improved by any method that changes the RF communication channel between the tag and the reader. This can include moving the reader, changing the frequency at which the reader operates, or even moving a person (or object) within the space occupied by the reader and / or tag. For example, consider a person 321 walking between a virtual reference tag 326 and a first RFID reader 320a and a second RFID reader 320b, as shown in FIG. Figure 3B As shown. This person attenuates or scatters the RFID signals propagating from the virtual reference tag 326 (and the unknown RFID tag 322 and the reference tag 324) toward the first RFID reader 320a and the second RFID reader 320b. This creates a new set of signatures at the first RFID reader 320a and the second RFID reader 320b that are imperfectly associated with the set of signatures before the channel change, and thus can be used to reduce the error in the estimated position of the virtual reference tag 326 and the unknown RFID tag 322.

[0132] The problem with using virtual reference tags often revolves around the large amount of processing power used to compare each tag signature with every other tag signature. By first comparing the signature of an RFID tag with its last signature, the amount of processing power can be reduced. If the signatures have not changed, there is no need to compare. Other ways to reduce processing power include using auxiliary information sources (e.g., video and existing location information of the area around the RFID tag) to limit the comparison to the signatures of RFID tags that are known to be close enough to each other to be important.

[0133] In order to identify or designate an RFID tag as a virtual reference tag, the system measures the position of the RFID tag multiple times over a period of time as the environment around the RFID tag changes (e.g., using the method described above). These position estimates can be distributed over an area or volume whose size depends on noise and measurement uncertainty. As the number of measurements increases, the average position estimate can converge to a smaller area or volume whose size is limited by the basic measurement uncertainty. Once the size of the area or volume reaches a predetermined threshold, the processor sets an appropriate tag position (e.g., the center of mass of the area or volume) and uses the tag position as a reference for similar tags. The system can repeat this process until the desired number of RFID tags or a group of RFID tags have been added to the virtual reference tag pool. After setting the reference tag (with a known position), the method of calculating the position of this reference tag has proven to be reliable, and the method can be used to estimate the positions of other tags.

[0134] If the position of an RFID tag changes, the system can remove the RFID tag from the pool of virtual reference tags unless similar RFID tags exhibit similar changes (e.g., due to environmental changes, such as Figure 3B The system can identify changes in a queue of RFID tags by looking at the signature of each RFID tag in the queue relative to the signatures of its queue members. The system can also look for changes (or lack of changes) in the signatures received from other AOAs in the queue. Figure 3B For example, person 321 may change the LOS signatures received by first RFID reader 320a and second RFID reader 320b in a correlated manner, but should not affect the LOS signature received by third RFID reader 320c. The combination of correlated changes in some AOAs and no changes in other AOAs in a queue of RFID tags may indicate that the queue is not moving.

[0135] The change in RFID position may be reduced in one or more of the following cases. For example, if there is no relative change between different RFID tags, then all tags may be blocked or move together. In another example, the relative change is below a predetermined threshold (e.g., measurement uncertainty). In yet another example, the relative change is sufficiently short-lived (e.g., within one, two, or three measurement cycles).

[0136] Information about virtual reference tags can be combined with other information to increase RFID tag location accuracy. For example, an RFID tag location system can use the (estimated) position of virtual reference tags, product count data, and visual data from one or more cameras to determine the average density of products between virtual and / or real reference tags. In addition, visual data can be used to determine whether someone is or has been close enough to pick up or put down a product or RFID tag (gesture / reach estimation can also be performed). If the visual data shows that the virtual reference tag or the product with the virtual reference tag moves or has moved, the system can remove the virtual reference tag from the virtual reference tag pool. Conversely, if the visual data shows that a particular RFID tag has not moved for a long time (e.g., hours or days), the system can designate the RFID tag as a virtual reference tag.

[0137] 8Computer Vision Systems and Computer Vision System Training

[0138] The RFID technology described above can be used to train a computer vision system to locate and / or identify different objects captured by a camera in or coupled to the computer vision system. For example, a computer vision system can include or can be coupled to multiple cameras that can be configured to monitor a wide angle area. In addition, light sources that emit light of different wavelengths and / or intensities can also be used to generate different environments to enhance the training of the computer vision system. The training images are captured by the camera in different environments.

[0139] The combined computer vision / RFID tag location system can cross-reference the timestamp data of the scanned barcode / transaction, the items in the barcode / transaction, and the camera data corresponding to the location of the cashier or checkout counter to pull frames containing those items. A processor running object detection on those frames can draw a bounding box around the image to produce an additional labeled image.

[0140] During the training of a computer vision system (such as a processor executing an artificial neural network), the above-described RFID technology is used to locate and identify objects in training images. These objects are divided into discrete images, which are fed into the training set of the computer vision system (e.g., an artificial neural network). If the computer vision system does not distinguish between objects that are too close together, reinforcement learning can be used to filter multiple objects. In addition, such training can train the computer vision system to recognize objects, such as lamps, doors, and shopping carts, which may be irrelevant during the use of the computer vision system in a retail store, for example. These objects can then be removed from the training data set, and the frames containing images of these objects can be marked as closed frames. Because the above-described RFID technology can automatically identify objects with RFID tags in an efficient manner, a huge database of product images can be built without the need for human verification or verification of the database contents.

[0141] In some cases, there may be errors in the tag position relative to the actual object. In these cases, the tag position can be plotted frame by frame relative to the position of the person or object (e.g., continuous moving pixel groups are obtained via optical flow or re-identification), and the distance from each set of continuous pixel groups to each RFID tag over n frames is averaged, and then they are grouped based on which pairings produce the lowest error / average distance. A Kalman filter will work normally to filter / group objects and / or objects. The first and second derivatives of the RFID tag and pixel blob motion functions can also be combined to weight the matches. If the goal is to attribute products to people, person detection can be run in each frame to select only pixels corresponding to the figure of the person. If the goal is to collect labeled images, person detection can be used to simply ignore pixels corresponding to the figure of the person in bounding box generation / pixel segmentation.

[0142] Figure 3C –3F shows a series of video frames showing the movement of a shirt tagged with an RFID tag. The small circles represent the estimated positions of the RFID tags attached to the shirt. The boxes surround the pixel blobs to which the system associates the RFID tag motion derived from the RFID signals received by the RFID tag reader. Figure 3C As shown in Figure 3F, there is a significant error between the system's frame-by-frame estimate of where the RFID tag is relative to where the object actually came from, but the system is still able to associate the RFID location estimate with the pixel blob.

[0143] Figure 3GA process 340 for associating RFID tag position estimates with video data is illustrated. This process 340 can be used to train a neural network to identify items tagged with RFID tags or to associate the motion of tagged and untagged objects (e.g., clothing and people with RFID tags). The process 340 begins by identifying, segmenting, and ignoring pixel blobs in an image that represent people using a trained neural network (342). The remaining pixel blobs in the image are then assigned trackers that look at the motion of the pixel blobs frame by frame (344). Each pixel blob is then matched to the RFID tag that most closely follows its track given a certain threshold (346). After the pixel groups are paired with the corresponding RFID tags, object detection can be run on each pixel group in each frame to discard images that clearly do not match the RFID tag description due to environmental obstructions (such as bags, carts, coats, people, and other such obstruction sources) (348). Attributing tag data to pixel blobs is also used to reduce or eliminate errors between tag locations and item locations.

[0144] Said another way, the RFID location provides constant feedback to the artificial neural network so that it is always learning what it got right and what it got wrong in each frame. This extends to autonomous checkout and human / product interaction, where process 340 can be used to teach the vision system what is right and wrong, frame by frame.

[0145] 9. RFID Tags for Tracking Movement

[0146] Figure 4 The RFID system and process described above can be used to track an RFID tag 402 moving in an environment full of obstacles, such as a store, a warehouse, or a warehouse. In this example, a pair of RFID tag readers 410a, 410b (collectively referred to as RFID tag readers 410) interrogate the RFID tag 402 by transmitting an RFID interrogation signal at regular intervals (e.g., a rate of about 0.01 Hz to about 1.0 Hz). The RFID tag reader 410 can change the interrogation rate based on the signal received from the RFID tag 410. If the response signal of the RFID tag indicates that the RFID tag is moving at a high speed, a changing speed, or a changing direction, the RFID tag reader 410 can increase its interrogation rate to provide a finer spatial and temporal resolution of the movement of the RFID tag. On the contrary, if the response signal of the RFID tag indicates that the RFID tag is stationary or moving slowly, the RFID tag reader 410 can reduce its interrogation rate to save energy. The RFID tag reader 410 can increase or decrease its interrogation rate together or independently according to the relative movement of the RFID tag 402.

[0147] The RFID tag reader 410 can, for example, broadcast interrogation signals over a wide range of angles via co-directional transmitting antennas, or scan them at different angles with an antenna array as described above. A processor (not shown) wirelessly coupled to the RFID tag reader 410 uses the RFID signal from the RFID tag 402 to calculate the velocity vector 481 and trajectory 491 of the RFID tag 402. To calculate a given velocity vector 481, the processor can determine the location 471 of the RFID tag at different times, and then determine a vector connecting those locations in 2D or 3D space. Scaling this vector by the time difference produces a velocity vector.

[0148] The processor may use the position measurements and / or velocity vector 481 to determine a trajectory 491 of the RFID tag. This may be a historical trajectory (i.e., where the RFID tag 402 has been) or a predicted trajectory (i.e., where the RFID tag 402 is currently located based on its estimated velocity). If desired, the current velocity, recent trajectory, and / or predicted trajectory of the RFID tag may be displayed on a smartphone, tablet, or other electronic device and used to trigger transactions (e.g., sales of items associated with the RFID tag 402), prevent misplacement or theft, or track items as they are transferred from one warehouse to another, as described in more detail below. If the velocity vector 481 and trajectory 491 indicate that the RFID tag 402 is not moving, the processor may select the RFID tag 402 as a virtual reference RFID tag as described above.

[0149] The processor can also use the position measurements, velocity vector 481, and trajectory 491 to distinguish between "real" RFID tags (e.g., Figure 4 410) with an aliased or "ghost" RFID tag 482. In this case, the ghost RFID tag 482 is caused by a multipath effect. More specifically, Figure 4It is shown that the RFID signals propagating between the real RFID tag 402 and the RFID tag readers 410a, 410b can take LOS paths 411a, 411b, resulting in accurate measurement of the position, speed and trajectory of the RFID tag. These RFID signals can also take NLOS paths 413 between the real RFID tag 402 and the RFID tag reader 410. In this example, a portion of the RFID energy radiated by the RFID tag 402 is reflected or scattered from the wall 412 to the first receiver 410a. And when the RFID tag 402 is in certain positions, this wall 412 prevents it from sending RFID signals to the second receiver 410b. In short, the wall 412 causes the first RFID tag reader 410a to receive false RFID signals, and prevents the second RFID tag reader 410b from receiving any RFID signals when the RFID tag 402 is between the wall 412 and the first RFID tag reader 412a.

[0150] In this case, processing the fake RFID signal naively leads to Figure 4 The appearance of a ghost RFID tag 482 is shown, including a ghost position 473, a ghost velocity vector 483, and a ghost trajectory 493. The processor can distinguish the ghost tag 482 from the corresponding real tag 402 based on the discontinuity in the ghost velocity vector 483 and the ghost trajectory 493 and / or the similarity between the ghost velocity vector 483 and the real velocity vector 481 and between the ghost trajectory 493 and the real trajectory 491. In particular, the real speed and trajectory appear mirror-symmetrically with the ghost speed and trajectory about the line or plane defined by the wall 412. The processor can use this mirror symmetry and the abrupt discontinuity at the beginning and end of the ghost trajectory 493 (where the wall 412 begins and ends) to distinguish the real RFID tag 402 from the ghost RFID tag 482.

[0151] Camera 420 can also be used to track RFID tag 402. Figure 4 In the example of FIG. 4 , a camera 420 takes a picture of a person 401 carrying an RFID tag 402 (e.g., at or about video rate). A processor may use an artificial neural network to identify the person 401 that appears in the image (e.g., as a general person, as an employee, or as a specific person) and to correlate the person's movements with the movements of the RFID tag 402. If the RFID tag 402 is on a name tag, wristband, or ID card, this may be done as part of a process of training the neural network to recognize the person associated with the RFID tag 402. If the neural network has been trained, the processor may use the overlapping or coincident movements of the person and the RFID tag 402 to track or trigger another action, such as selling an item carried by the person or to which the RFID tag 402 is attached.

[0152] 10 RFID tag location system in stores with retail space and stockroom

[0153] Figure 5A and 5B 5 shows different views of an RFID tag location system in a store 500. The RFID tag location system includes a number of RFID tag readers 510 located throughout the store's sales floor 590, stockroom 592, and dressing rooms 580. The RFID tag readers 510 may be placed at or near the ceiling, e.g. Figure 5B 590 and in the backroom 592 to provide a clearer line of sight to the RFID tags 502 on the merchandise on the sales floor 590 and in the stockroom 592. Placing the RFID tag reader 510 above the RFID tags 502 also enables the 3D position of each tag to be measured using azimuth and elevation information derived from the RFID signals received by the RFID tag reader.

[0154] RFID tags 502 may be distributed throughout the store 500, including on sale items such as clothing and other merchandise. RFID tags 502 may be embedded in items or attached to items with tags, clips, or stickers. Other types of RFID tags may be present in the store 500, including reference RFID tags 504 at known locations (such as fixed or removable hangers, walls, or tables). In addition, if some of the removable RFID tags 502 remain stationary for a sufficiently long period of time, they may be designated as virtual reference RFID tags 506. And some RFID tags may be attached to ID cards 508a, 508b (collectively referred to as ID cards 508), key chains, bracelets, or other items worn or carried by employees 503a–503c (collectively referred to as employees 503) or customers 501. These ID cards 508 may identify specific employees and their locations. Similarly, at the entrance 596 of the store, RFID tags 502 may be embedded or attached to shopping bags, shopping baskets, or trolleys.

[0155] The RFID tag reader 510 communicates wirelessly with the processor / controller 530 via a wireless router 540 or other suitable device. The camera shown collocated with the RFID tag reader 510 is also in communication with the processor 530. The processor 530, in turn, can communicate with the tablet computer 530 and smartphone 540 carried by the customer 501 and the employee 503. The processor can also communicate with one or more servers, databases, or other remote devices that track store inventory and operations over a suitable communications network, such as the Internet.

[0156] In operation, the RFID tag reader 510 measures the position, speed, and trajectory of the RFID tag as described above. It uses this information to monitor inventory and trigger actions related to items tagged with the RFID tag 502. For example, if the RFID tag reader 510 detects that the RFID tag 502 is moving toward the exit 598 of the store, and the camera 520 detects that the customer 501 is moving along the same trajectory, the processor 530 can trigger the customer 501 to automatically purchase the relevant items. This enables the customer 501 to skip the checkout 582, saving time. The processor 530 can also guide the customer 501 and the employee 503 to specific items based on the accurate RFID position estimate of the RFID tag reader 520. This feature can be used to guide the customer 501 to the desired items, such as shirts of a specific size or color; control and re-shelve inventory, such as items left in the dressing room; or determine how the inventory settings affect sales. These and other applications are described in more detail below.

[0157] 11RFID technology is used for high-precision object positioning

[0158] The RFID tag positioning technology described above provides fine spatial resolution and high accuracy, making it suitable for a wide range of applications, many of which cannot be achieved using other RFID tag positioning technologies. Some of these applications are described below and can be used with Figure 5A and 5B The systems and environments shown are used in conjunction with similar systems and environments.

[0159] 11.1 Tracking RFID Tag Movement

[0160] In one example, RFID technology can be used in retail stores, particularly in omnichannel (also spelled omni-channel) stores. "Omnichannel" refers to a multi-channel sales approach that aims to provide customers with a seamless shopping experience, whether they are shopping online from a desktop or mobile device, or shopping in a physical store, via phone. What distinguishes an omnichannel customer experience from a multi-channel customer experience is the true integration between channels on the back end. For example, when a store has adopted an omnichannel approach, a customer service representative in the store can immediately reference a customer's previous purchases and preferences just as easily as a telephone customer service representative or a customer service web chat representative. Or a customer can use a computer, tablet, or smartphone to view the store's inventory on the company's website or app, then purchase the item through the online store or app and pick up the product at a location of the customer's choice.

[0161] One problem for retailers using omnichannel orders is detecting that items have been picked up through omnichannel orders. To solve this problem, RFID tags (called tote tags) can be placed on totes, shopping bags, shopping carts, or any other suitable containers. Each item for sale also includes or is attached to a separate RFID tag (called an item tag). The antenna array monitors the position of each tote tag and each item tag. If the distance between an item tag and a tote tag is below a threshold (e.g., less than the size of the tote), the system determines that the item corresponding to the item tag is in the tote. To improve the reliability of detection, the system can further monitor the movement of the tote tag and the item tag. If they move together for a distance exceeding a threshold (e.g., more than 1 meter), the system can determine that the item and the tote are being carried by the customer.

[0162] The system may also monitor a mobile customer to determine whether the customer has picked up an item. For example, if the item moves with the customer for a distance exceeding a threshold (e.g., more than 1 meter), the system may determine that the item is being carried by the customer. For example, the customer may install a user application on his / her smartphone, and the system may detect the presence of the customer's smartphone by communicating with the user application. The system may then track the movement of the smartphone (and therefore the customer) using Bluetooth, WiFi, LTE, 3G, 4G, or any other wireless technology.

[0163] In some cases, the system may maintain a record of all smartphones that do not belong to customers (e.g., the store's own devices or employees' personal devices). Once the system detects a smartphone that is not in the record, the system may determine that the customer has entered the store and may track the customer's movement by tracking the smartphone.

[0164] The system can also track the movement of customers using facial recognition, gait recognition, or other recognition technologies. For example, a camera may be placed at the entrance of a store to identify customers, and one or more cameras may be distributed throughout the store to monitor the entire store space. Each time a customer is captured and recognized by a camera, the recognition location can be recorded and compiled with the previous location to work out the customer's movement. The resolution of this monitoring (e.g., the distance between two recognitions of the same customer) can depend on the number of cameras in the store (e.g., a greater number of stores can increase the resolution). Then, if the distance moved together is greater than a threshold, the system can determine that the item is picked up by the customer. Alternatively or in addition, if they appear together at more than 3 locations, the system can determine that the item is picked up by the customer. If the two locations where they appear together are more than 1 meter apart, the system can also determine that the item is picked up by the customer.

[0165] The system may also update the inventory when it is determined that an item previously determined to be picked up by a customer is put back and available for sale. If the item has not been moved for a long period of time (e.g., longer than 5 minutes), the system may determine that the item is put back. To increase reliability, the system may also check whether there are any customers near the item when the item is not moved. When no customer is near the item, the system may determine that the item is put back (e.g., because the customer who previously picked up the item changed his mind and gave up the item).

[0166] In another example, RFID technology can be used to determine whether an item is in the correct location of a store. In this case, one or more tags (called shelf tags) can be placed on each shelf that holds items for sale. Each shelf tag identifies the specific location of the corresponding item on the shelf. Each item also has an item tag. For example, a shelf tag can indicate the location of men's pants, and the item tag can be attached to a pair of men's pants. The system interrogates the location of the shelf tags and the item tags to estimate the distance between them. If the distance is below a threshold, the system can determine that the item is in the correct location. On the other hand, if the distance is greater than a threshold, the system can warn one or more store employees that the item is in the wrong place and should be moved to the correct place. The system can also provide instructions to employees about the actual location of the item and its proper location.

[0167] The system can also use tags attached to any other retail fixtures (these tags are referred to as fixture tags) to determine whether the items are in the correct place. In general, each fixture tag can provide information about the identification of the fixture (e.g., shelf, table, counter, display case, basket, grid, etc.), the location of the fixture, and the type and quantity of items in the fixture. In some cases, the type and quantity of items in the fixture can be determined based on industry standards. Alternatively, the type and quantity of items in the fixture can be customized for each store.

[0168] In addition, each employee can wear a tag (referred to as an employee tag). In one example, an employee can wear a bracelet containing an RFID tag. In another example, an RFID tag can be sewn into an employee's uniform. In yet another example, an RFID tag can be included in a badge worn by an employee. The system can use these tags to estimate and track the location of employees, for example, for managing inventory, as described below.

[0169] In some cases, the movement of an employee can be monitored by software without using an RFID tag attached to the employee. For example, the system can monitor the movement of an employee by tracking the employee's smartphone. In these cases, the employee can install a user application to facilitate communication between the smartphone and the system. The system can identify the employee from his / her account on the user application, for example.

[0170] In another example, the system can track employees' wearable devices, such as smart watches, activity trackers (e.g., Fitbit), or smart glasses (e.g., glasses with embedded electronics), among other devices. In this example, the system can maintain a record of the wearable devices belonging to each employee so that the employee can be identified when a wearable device is detected. Systems with cameras can also track employees through RFID tags on items that the camera identifies as being held or carried by the employee.

[0171] For example, if the system determines that an item is misplaced, or should be taken from the back inventory to the correct shelf, the system can estimate the location of all employees using employee tags. The employee closest to or moving toward the misplaced item can then be identified. The system can alert the employee to put the item in the correct place. The system can also estimate and / or measure the time it takes for an employee to complete a task (e.g., time period from alert to completion). This information can be used to evaluate employee performance and identify changes in store layout that can improve efficiency.

[0172] The system may use several criteria to determine the appropriate employee to receive the alert. For example, the system may communicate the alert to the employee based on the employee's availability to receive and respond to the alert. In this example, the employee may communicate their availability (or unavailability) to the system via their employee device, such as a smartphone with a user application installed. The employee may indicate that they are in the middle of other tasks that cannot be interrupted.

[0173] In another example, the system may send an alert to an employee based on the employee's proximity to the problem item. For a misplaced item, proximity may be quantified by the distance between the employee and the misplaced item. For an item to be placed, proximity may be quantified by the distance between the employee and the stockroom. In some cases, the proximity estimate takes into account the building or structure of the store. For example, the system may prefer to send an alert to an employee on the same floor as the problem item, rather than to an employee on a different floor.

[0174] In yet another example, the system may send alerts to employees based on their ability to complete tasks. For example, if an item in the women's clothing department is misplaced, or an item is found lost in a women's fitting room, the system may preferably send an alert to an employee in the women's clothing department rather than an employee in the grocery department.

[0175] The ability to complete a task may also be determined based on the current task that the employee is working on. For example, if an employee is already working on certain misplaced items, it may be more efficient for that employee to work on a similar task. The system may also consult a quality assurance system to determine the employee's ability. For example, the system may include a database of employee performance reviews for each task that the employee has previously worked on. If the system determines that an employee is efficient at restocking misplaced items, the system may preferably send an alert to the employee.

[0176] In yet another example, the system may use a combination or weighted combination of the above criteria to determine the most appropriate employee to handle the problem. For example, the system may first find available employees. Then among these available employees, the system finds people who are within a certain distance of the problem item. Among these employees, the system may then determine the most appropriate employee based on the employee's ability to complete the task.

[0177] In some cases, the system may send an alert only to the most appropriate employee (determined by any suitable method). Alternatively, the system may send an alert to a group of appropriate employees, and each recipient may respond using his / her device (e.g., a smartphone). Once a recipient responds by indicating that he or she will handle the task, the system may update the status of the issue to, for example, "in progress."

[0178] The system may also send an alert to the supervisor of the appropriate employee determined by the system. Alternatively or in addition, the system may also copy the alert to personnel in the quality assurance department to alert them to monitor the progress of the problem.

[0179] RFID technology can also be used to monitor the availability of store inventory in a real-time manner. In this case, the system can track the movement of items picked up by customers. As described above, the system can determine that a customer has picked up an item when the item moves with the tote bag. More specifically, the system can determine that an item is in a tote bag using the movement / trajectory of the item's RFID tag and the movement / trajectory of the tote bag as determined from video data and / or data about the RFID tag on or in the tote bag. Once the system determines that an item is picked up, the system deducts the item from the available inventory. Alternatively, the system can deduct the item from inventory until the item passes through a cashier that checks it out. In some cases, the system can also deduct an item if the customer is wearing the item, such as a garment or a pair of shoes.

[0180] RFID technology can also facilitate verification of e-commerce orders, especially after the shipping box is sealed. RF signals can generally penetrate the shipping box, so the above-mentioned RFID technology can be used to identify the RFID-tagged items in the shipping box. The identified items are then compared with the order corresponding to this shipment to determine whether any items are missing or if any items should not be in the shipping box. If the missing items are confirmed, the system can check the inventory or other database to find out whether the replacement items are in the distribution center (DC) or a nearby store. The system can also prevent the shipping box from leaving the store and / or DC until the items are placed in the shipping box or are in a separate shipment.

[0181] In some cases, RFID technology can be used in dressing rooms to track items that customers try on. The system can determine if an item left in a dressing room has been in the dressing room for more than a threshold time (e.g., longer than 15 minutes). Alternatively, the system can track the location of the item and the status of the dressing room. For example, if the system determines that the item is in the dressing room and the dressing room is not in use, the system can determine that the item is left in the dressing room. In these cases, the system can alert an employee to pick up the item and put it back on the shelf for sale.

[0182] The system can determine the status of a fitting room by tracking the movement of customers or the presence of wearable devices in the fitting room. For example, the system can generate a map of the fitting rooms and display the movement and wearable devices detected in each fitting room. If no devices are found in the fitting room, the system can indicate that the fitting room may not be in use. In this case, an employee can enter the fitting room to pick up the abandoned items.

[0183] The system may also determine the state of the fitting room using an RFID tag attached to the fitting room door (e.g., on the movable edge of the door). In this case, the fitting room door may be designed to move away from the frame when unlocked. Thus, the RFID tag is in a first position when the door is closed or locked (i.e., when the fitting room is occupied) and in a second position when the door is open or unlocked (i.e., when the fitting room is not occupied). The system may then determine the state of the fitting room based on the position of the RFID tag on the door. Similarly, another option is to install several reference tags in or on the fitting room curtain and detect the movement of the reference tags closer together or further apart due to someone opening and closing the curtain.

[0184] Alternatively, two RFID tags can be used per fitting room: one placed on the active edge of the door and the other on the door frame. Alternatively, the RFID tags can be placed on or integrated into different parts of the lock on the fitting room door. The system can then determine the distance between the two tags. If they are within a threshold (e.g., about 10 cm), the system can determine that the door is closed or locked; otherwise, the system can determine that the door is open or unlocked and the fitting room is not occupied.

[0185] Additionally, the system can use a combination of position and amplitude displacement of the RFID signal to determine whether an RFID-tagged garment is on a person. For example, if a garment is dangling in the air in the middle of a fitting room, it is likely on a body. If the RFID tag location system detects a significant drop in RSSI, accompanied by an indication (e.g., from camera data) that the RFID tag is close to a person, it can be determined that the object / garment tagged with the RFID tag is likely on a person.

[0186] 11.2 Listing items with RFID tags

[0187] Precise tracking of items also allows the system to place items on shelves using autonomous vehicles (e.g., robotic devices, drones, etc.) without human intervention. For example, an RFID tag can be attached to each item, providing information about the desired location of the item in the store. The autonomous vehicle may include a tag reader to read the RFID tag and deliver the item to the desired location. The desired location (e.g., a designated shelf) may also be marked by an RFID tag (called a fixture tag). In some cases, the autonomous vehicle uses its internal tag reader to locate the fixture tag, and estimates the distance and direction from its current location to the fixture tag, and uses the estimated value to navigate toward the fixture.

[0188] In some cases, the system can monitor the location of the remote control vehicle using an RFID tag on or embedded in the vehicle. If necessary, the system or the user can guide the vehicle toward the fixture. In these cases, the vehicle may not include any tag reader.

[0189] Alternatively, the RFID tag may include identification information (e.g., a serial number) of the item, but not the expected or intended location of the item. Instead, the identification information is associated with the expected location information in the database. An RFID tag reader (e.g., on an autonomous vehicle) can read the identification information and communicate with the database to retrieve the location information.

[0190] Automated stocking can be performed by autonomous vehicles each evening after the store closes and / or each morning before the store opens. In some cases, the stocking process is automated so that it can be performed without human supervision. Thus, stocking can be performed after hours to save overtime costs.

[0191] In some cases, shelving can be performed on demand. For example, when the system determines that an item is in demand, the system can send a person or robot to a warehouse, pick up an item and deliver the item to the desired location. In some cases, a person or robot can also be directed to pick up misplaced items and place them in the correct location. The system can guide the robot to the location of the misplaced item and the desired location of the item. In some cases, RFID tag data and / or camera data can also reveal the orientation and other information of the object, such as weight, geometry, and weight distribution, to help solve complex problems such as gripping.

[0192] 11.3 Monitoring Inventory of Items with RFID Tags

[0193] The system can monitor the inventory of items based on accurately tracking the location of items with RFID tags. As described above, the system can determine that an item has been picked up or is being carried by a customer, in which case the system can remove the item from the list of available items. The system can also place the item on a temporary list of items that the customer is considering purchasing. Once the item is checked out by the customer (e.g., in the case where the customer leaves the store with the item), the system can remove the item from the temporary list. However, if the customer changes his mind and puts the item back before checking out (or simply abandons the item), the system can put the item back on the available list.

[0194] In some cases, once the system determines that an item is under consideration by a customer, the system may interrogate an RFID tag attached to the item at a frequency greater than 1 Hz to track the item's movement. After the item is returned to the shelf, the interrogation frequency may be reduced to reduce the computational burden on the system.

[0195] Employees may participate in inventory monitoring by handling defective items. Defective items may be identified by an employee or a customer. In either case, the employee may use an employee device to scan an RFID tag on the item and enter the status of the item (e.g., "defective" or "damaged") into the system. The employee device may include a tag reader and an interactive interface (e.g., a touch screen) for the employee to update the inventory. In response to receiving the status, the system may remove the item from the available list and place the item in another list (e.g., a repair list or a return list). The system may also send one or more alerts to relevant personnel to handle the defective item.

[0196] 11.4 Employee and Product Location Using RFID Tag Location System

[0197] The RFID tag location system may include multiple cameras, RFID tags, and wireless communication systems, such as Bluetooth or Wi-Fi, to track the precise location of employees and products within a store. The locations of employees and products may be derived from the RFID tag location data collected by the RFID tag location system and then displayed using a GUI on a smartphone or tablet. Figure 6 An exemplary GUI is shown that displays the location of an employee and several products on a floor plan of a store. Because the RFID tag location system can identify the exact location of the employee and the product, the relative location between the employee and the product can also be displayed, such as Figure 6 In addition to showing the location of the product on the floor plan, Figure 6 As shown in , the location of the product can also be shown in a virtual tour of the store, in a 3D view of the store, or in an online shopping feature.

[0198] 11.5 Selecting Products Using the GUI

[0199] As described, the GUI may display one or more products on a floor plan of a store. The GUI may also allow a user, employee, or customer to interact with the displayed products in order to perform specific actions. User interaction with the GUI may be accomplished by a number of methods, including a pointing device (such as a mouse), a touch-based system (such as a user's finger or a stylus), and the like. For example, in a touch-based system, a user may select one or more products by drawing a shape with his or her finger around the products displayed in the GUI. This process is described in detail in the accompanying drawings. FIG. 7A to FIG. 7D Shown in. Fig. 7A 1 shows a plurality of products displayed in a GUI. A user can use their finger to start drawing a circular shape around the plurality of products, such as Figure 7B As shown in Figure 7C The circular shape shown. Thus, the product contained within the circular shape is selected. Before the user-specified action, information about the selected product, such as the quantity of the selected product, can be displayed, such as Fig.7D shown.

[0200] Once the user has selected a product, the user can then specify a number of actions to be performed on the selected product. These actions may include the following: (1) listing product details, e.g., style number, color, price, size, etc., (2) listing the quantity or value sold, (3) instructing the RFID reader to read only the selected product, e.g., upon receipt of a new shipment, during an inventory count, etc., (4) changing the floor display of a particular product, (5) opting in to receive price alerts for the selected product, (6) displaying information about similar products, (7) receiving recommendations for similar products or newer models of the selected product, or (8) causing the selected product to be delivered or picked up for purchase. Certain actions may only be available to employees or customers, depending on their functionality.

[0201] 11.6 Automatic notifications of updated inventory and new product shipments

[0202] RFID tag location systems can also be used to facilitate updating inventory when new product shipments arrive at a store. For example, a shipment can be delivered to a store from a manufacturer, a warehouse, a distribution center, or another store. To verify the quantity of the shipped product, an RFID reader and a user application can be used. The RFID reader can be optimally located in a store shipment processing area, such as a stockroom, sales floor, or other location where a retailer can process inbound shipments.

[0203] The products included in a shipment may or may not include RFID tags. For products with RFID tags, employees can use an RFID reader and user application to verify that the quantity of products received matches the corresponding invoice for the product order. For products without RFID tags, employees can add RFID tags to the products and encode the appropriate product information to the tags using an RFID reader and user application. These products can then be added before confirming the quantity of products received in the shipment.

[0204] Once the quantity of product received is verified, the RFID inventory of the product and the main corporate inventory of the product (which may include products with and without RFID tags in multiple stores) will be updated to show the accurate inventory level of the product at the store where the shipment was received and at the corporate level of multiple stores. If there is a discrepancy between the quantity of product received and the quantity of product in the invoice, the RFID tag locating system can facilitate resolving the discrepancy by checking whether the product arrived in the shipment, whether the product arrived without an RFID tag, and whether the product arrived with an incorrect RFID tag.

[0205] Electronic notifications may also be automatically sent to customers notifying them of new shipments of products delivered to the store. Notifications may be sent using a variety of methods, including email, text messages, messaging applications, such as WhatsApp, Facebook Messenger, geo-fencing applications, or other electronic messaging services integrated with the RFID tag positioning system. Notifications may be customized for priority products based on customer preferences, such as new products, best-selling products, or products that customers choose to be notified of. Notifications may also be sent to customers who have previously visited a particular store, or who have subscribed to receive notifications from a particular retailer or retail location. Redirected advertisements or electronic messages may also be sent to customers who have previously visited a store but were unable to purchase a particular product due to lack of availability (e.g., the priority product does not have the desired size).

[0206] The RFID tag locating system can also facilitate the discovery of products that are missing RFID tags or products that have incorrect RFID tags after delivery or during inventory checks. For example, an employee may find that the inventory level of a piece of clothing is zero when checking a pile of identical clothing, indicating that the RFID inventory is in error due to a missing or incorrect RFID tag. In another example, an employee may be moving a particular product and visually notice that the product is missing an RFID tag. If a product is found to be missing an RFID tag or has an incorrect RFID tag after receipt and verification of the shipment, the employee can add an RFID tag to the product or replace the RFID tag of the product and encode the correct product information to the tag using an RFID reader and user application. After the new RFID tag is encoded, the RFID inventory of the product will be updated and an automatic electronic notification can be sent to the customer, as previously described.

[0207] 11.7 Automatic Notification of Product Moves and Holds

[0208] The RFID tag location system can also monitor product movement, for example, whether a priority product has not been moved to an appropriate location, such as a sales area, or monitor product retention, for example, a product has not been set up for customer reservation. Based on the RFID tag location data, if a product movement or product retention does not occur within a specific time threshold set by the retailer, for example, within 30 minutes, an electronic notification can be sent to an authorized employee, regional manager, or corporate manager. Notifications can be sent using a variety of methods, including email, text message, messaging applications, such as WhatsApp, Facebook messenger, geo-fencing applications, or other electronic messaging services integrated with the RFID tag location system.

[0209] 11.8 Product Status Based on RFID Tags

[0210] The RFID tag location system can also encode additional information in the RFID tag of the product. For example, an RFID status tag can be used, which can include various product status and tracking information. An RFID status tag is distinguished from an RFID tag, wherein an RFID status tag can assign the same product information to a group of RFID tags corresponding to the RFID status tag.

[0211] A variety of product statuses can be encoded into the RFID tag and can be based on various categories including Transfers Out, e-commerce orders, and damaged. In the Transfers Out category, the product status can include: (1) the item being sent from a first store to a second store, warehouse, or distribution center, (2) the time and date the product status was created, (3) the type of transfer, e.g., transfer to a different store, transfer to a warehouse, transfer to a distribution center, transfer of damaged or recalled products, transfer of products requiring cleaning or customization if such services are required outside of the store, (3) the source of the item, e.g., store number, (4) the destination of the item, e.g., store number, distribution center, manufacturing facility number, or (5) the transfer quantity, e.g., a tracking number created by an RFID tag location system or an existing legacy system.

[0212] In the e-commerce order category, the product status may include: (1) the product is being shipped from the store to the third-party shipping address specified by the customer when placing an order, (2) the time and date the product status was created, (3) the source of the product, such as a store number, (4) a customer account number, such as an account created by the e-commerce system, (5) an e-commerce order number, such as an order number created by an RFID tag positioning system or an existing traditional e-commerce system, or (6) an e-commerce status, such as In Progress - the product has been picked up and is currently in the processing area waiting to be packed, or Packed - has been picked up and packed for outbound shipment.

[0213] In the damage category, the product status may include: (1) merchandise that is currently unsaleable due to contamination, damage, or defects, (2) the time and date the product status was created, or (3) a damage transfer number, such as a reference number created by an RFID tag location system or an existing traditional system.

[0214] The use of RFID status tags can facilitate the assignment of product status in specific areas of a store based on the location accuracy of the RFID tag positioning system or based on product type. For example, an RFID status tag on a particular product can automatically assign the same status to other products in its immediate vicinity, such as products within 4 inches of the RFID-tagged product. In another example, an RFID status tag on a particular product can assign the same status to a group of products throughout the store. Different colors or symbols can be used for a group of products to display the status changes of these products in the GUI. This visual indicator can help employees verify the status of the product.

[0215] The RFID tag location system can also automatically change the status of a product based on the RFID inventory of the product. For example, a product with an RFID tag that has a transfer-out status (e.g., transferred to another store, transferred to a warehouse, or transferred to a distribution center) can be considered as available inventory to fulfill an e-commerce order by the store that sent the product, as long as the store has not confirmed the transfer-out process. Confirmation can include the product being in a sealed box, completion of transfer documents, etc.

[0216] 11.9 Tracking the arrival and departure routes of products

[0217] The RFID tag location system may include an RFID reader, a (depth) camera, and technology to accurately determine the location of an RFID tag, and the RFID tag location system may be used to record the path of one or more RFID tagged products (e.g., products grouped in a box, bag, or cart with an RFID tag or RFID status tag) through a store as the products enter or leave the store. Using a user application, the path may then be displayed to the user in a GUI as an animation overlaid on a floor plan of the store, such as Figure 8 shown.

[0218] RFID tag location systems can also use date, time, and system location technology and by Figure 8 The camera shown records the location data and plays back the recorded video feed of the arrival or departure of the RFID-tagged product. In addition, the RFID tag location system can also identify the individual accompanying the RFID-tagged product based on facial or gait recognition, the individual's Bluetooth or Wi-Fi-enabled device or user ID.

[0219] To ensure that the store is fully covered by the RFID tag positioning system, the components of the RFID tag positioning system can be installed on the ceiling or wall, depending on the layout and environment of the store, in increments of 500 to 1000 square feet. This enables the RFID tag positioning system to track all RFID-tagged products and Bluetooth or Wi-Fi-enabled devices within the store. In addition, the system can also identify the boundaries of the store, such as multiple floors, rooms, entrances, exits, etc. The boundaries of the store can be marked with RFID reference tags or other manual marking methods for detection. In particular, by identifying the entrances and exits, the RFID tag positioning system automatically registers when a product has entered or left the store.

[0220] 11.10 Intelligent and Adaptive Display of Product Quantity

[0221] The RFID tag locating system may enable a user, such as an employee, to set a desired number of products to be located in a particular area of ​​a store, e.g., 12 product units on a floor display. In addition, the RFID tag locating system may suggest to the user the ideal placement of products based on historical data about product performance in order to maximize sales. For example, a particular product may have multiple variations, e.g., footwear, clothing, accessories, women's wedding dresses in different sizes. The RFID tag locating system may suggest to the user the highest performing product variation to be placed on the floor display in that particular store. The historical performance data may include: historical sales, the number of times a product or product variation has been viewed or tested by a customer, or conversion rates for a product or product variation, e.g., views versus sales, customer testing versus sales, etc.

[0222] The RFID tag positioning system can also dynamically modify the quantity and location of products in the store in real time based on the inventory list available at the store. For example, in Table 1, an ideal scenario is shown where M and L size products are the best performing variants, followed by S and XL size products. Based on the user-defined requirement for the total number of products to be displayed, for example, 12 in this example, the RFID tag positioning system automatically calculates the number of products of each size to be placed on the floor display. In this case, since M and L size products perform better, more M and L size products are displayed than S and XL size products.

[0223] Table 1

[0224]

[0225]

[0226] In another example, Table 2 shows a modified scenario where the M-size product inventory is insufficient and therefore cannot satisfy the ideal floor display previously presented in Table 1. In response, the RFID tag positioning system reallocates the number of product variants to be placed on the floor display based on the next best performing product variant. This does not necessarily display zero M-size products, but rather reduces the number of M-size products to accommodate available inventory and customer demand. In this case, more L-size products are displayed, followed by S and XL-size products.

[0227] Table 2

[0228]

[0229] Table 3 shows another modified scenario, where both the M and L sizes are sold out, and the other product variants are not in sufficient stock to meet the total number of products required. In this case, the RFID tag positioning system will reallocate the quantity of product variants to meet the total number of products displayed as best as possible, while prioritizing the best performing sizes.

[0230] Table 3

[0231]

[0232] If the number of sales floors is set to zero, the RFID tag locating system sets the number of products on the floor display to zero as well. In addition, if at least one product is on the sales floor and no products are on the floor display, a notification may be sent to an employee. This is based on a possible retailer policy where all products available on the sales floor should also be placed on the floor display. The RFID tag locating system may also be configured to detect differences in the number of products on the sales floor and on the floor display, in particular to compensate for input errors of the system.

[0233] 11.11 Creation and Optimization of Pickup Lists

[0234] As previously described, the RFID tag positioning system can accurately track the number of products located in different areas of the store (e.g., the sales floor or the backroom), and therefore can determine which products or product variants may need to be moved to the sales floor in real time. For example, Table 4 shows the distribution of product variants in the store. As shown in the table, based on the number of products displayed on the floor display, the number of M and XL size products available on the sales floor is insufficient. As a result, two M size and one XL size products should be moved from the backroom to the sales floor.

[0235] Table 4

[0236]

[0237] To facilitate the replenishment of products or product variants, the RFID tag locating system can instantly compile a pick list or a list of requested products that need to be replenished in real time. The pick list can then be sent to a user, such as a warehouse employee, who then completes the request by picking up all requested products and delivering them to the sales floor.

[0238] The use of pick lists may also be applicable to fulfilling e-commerce orders, where the RFID tag locating system compiles a list of products requested by an online customer to be picked up in the store. Products in the pick list may also be placed on hold by an employee on behalf of the customer, by the customer using the retailer's website or application, or by a customer variant of the user application. Pick lists may also be used in customer inventory requests, where products are requested from the stockroom by an in-store customer via a sales floor employee, or for misplaced products, where products are placed in an incorrect location on the sales floor or in the stockroom.

[0239] Because the RFID tag location system can track the location of the user (e.g., warehouse employee) and the location of all products in the pick list, an optimized picking path (OPP) can be generated based on the shortest time or distance for the employee to pick up all products. The OPP can be displayed to the user in the GUI of the user application. Fig. 9A The OPP is shown as a dotted line, along with the nearest product and user's location on the pick list. The OPP will update as the user moves, such as Fig. 9B As the user starts picking up products on the pick list, the OPP will continue to update and will also display the number of products picked up by the user, as shown in Fig. 9C and Fig.9D Likewise, the next product to be picked up by the user will also be displayed in the GUI. OPP can also be used for e-commerce orders, customer inventory checks, and to move misplaced products in the stockroom or on the sales floor.

[0240] If a product on the first user's pick list is picked up by a second user and delivered to the sales area, and the first user is still in the process of fulfilling the request and before picking up the product, the RFID tag locating system will specifically mark the product on the first user's pick list to notify the first user that the product is no longer needed. This notification process can be performed in real time using the RFID tag locating system.

[0241] 11.12 Picklist Filters

[0242] The RFID tag locating system can also enable the user to refine the pick list based on product attributes or location. For example, the user can filter the pick list according to women's wedding dresses, warehouse 1, or women's wedding dresses in warehouse 1. In addition, the user can set a maximum number of products to be included in the pick list, for example, 10 units. The RFID tag locating system will then display the pick list with a maximum of 10 units. Based on the user filters and the maximum number, the RFID tag locating system can optimize the products on the pick list that generate the most sales for the store.

[0243] 11.13 Spurious Products

[0244] A stray product is a product that is placed in an incorrect location within a store, for example, a product that is designated as being on the sales floor, but is located in a stockroom. The RFID tag location system can actively and accurately track the location of a specific product unit, for example, all units of a men's black V-neck T-shirt are located on the sales floor or in a stockroom. The combination of the location accuracy of the RFID tag location system and the ability to monitor all units of a specific product can enable automatic detection of stray products in a store. If a stray product is detected, a notification that the product unit is in an incorrect location can be automatically sent to a user, such as an employee, immediately or after a user-defined time threshold, for example, greater than 10 minutes.

[0245] Additionally, the user application can generate a path within the GUI that leads the user to all stray products. This path generation feature can also be used for non-stray products. For example, Fig.10 A GUI is shown for selecting a particular product within a store. The units of the selected product may not be located in the same stockroom or in the same area of ​​a particular stockroom, e.g., the units of the product may not be located within six feet of each other. In these cases, the GUI may display to the user the total number of locations that the user must visit to retrieve all units of the product.

[0246] 11.14 Intelligently route product notifications to users

[0247] The RFID tag location system, in particular the RFID reader, user application, and location tracking features of the system can be used to accurately monitor the location of a product or a variation of a product. By tracking all RFID-tagged products within a store, the system can automatically notify users, such as employees, in real time if a product needs to be replenished in a specific area of ​​the store. The threshold or criteria for product replenishment can be determined by the user. For example, it may be required that 10 units of a product be located on a floor display. If initially there are 10 units of a product on a floor display, and a customer purchases 1 unit, a notification can be sent to the employee that the number of products on the floor display has dropped below the specified requirement, thereby prompting the employee to move 1 unit of the product to the floor display.

[0248] An employee who receives a product replenishment notification can also send a request for the product to another employee, for example, a sales floor employee can use an internal inventory request to request a product from an inventory employee. If the requested product is not in stock at the first store, the employee can instead use an external store request to request the product from a second store or warehouse for delivery to the first store or the customer's preferred address. This tracking feature can also be used by customers using a user application with their mobile device to locate a specific RFID-tagged product in the store or another nearby store.

[0249] The RFID tag positioning system can also intelligently route inventory requests to specific employees or locations to minimize the time to deliver the requested inventory to a specific area of ​​the store or customer. Internal inventory requests can be routed to employees based on their proximity to store areas, customers or warehouses and the ability to complete tasks in the shortest time. For example, employee A is processing 5 inventory requests for other customers, and it must give priority to completing these 5 inventory requests. The RFID tag positioning system can then route additional requests to the nearest available employee, for example, employee B, to fulfill the inventory request. If an unrelated task is currently being performed, the employee can also choose to turn off or mute notifications of inventory requests. The RFID tag positioning system can also monitor the time it takes for employees to complete inventory requests from the initial receipt of inventory requests for delivering products to customers or store areas by tracking them as they move through the store.

[0250] For external inventory requests, the RFID tag location system can actively monitor and update the availability of products at multiple stores. For example, if a customer at the second store has a requested product in their shopping cart, the RFID tag location system will remove the product from the available inventory at the second store to ensure that the customer at the first store has accurate information about product availability. The RFID tag location system can also be used to predict the time required for the externally requested product to be delivered to the store or customer's preferred address, based on the distance between the originating location (e.g., the second store or warehouse) and the destination, as well as data detailing the speed at which the inventory request is fulfilled and transported by the originating location.

[0251] 11.15 Inventory Request Fulfillment

[0252] The RFID tag location system actively tracks the location of products available on the store's sales floor and stockroom in real time and at all times. This active tracking facilitates employees to quickly complete customer inventory requests. For example, a customer can request a stock level check for a specific product in the store using the user application. Fig.11An exemplary GUI is shown in which a customer is requesting a product from the stockroom to be delivered to them on the sales floor with the option to specify a particular product variation, e.g., a medium-sized black dress. The request may then be sent to an employee on the sales floor. The employee on the sales floor may then use a user application to request the product requested by the customer from the stockroom employee. The stockroom employee may then locate and pick up a variety of requested products for different customers. To facilitate delivery of the product to different customers on the sales floor, the stockroom employee may use a user application that actively monitors the location of different customers in real time.

[0253] In addition to stock requests, there may be situations where a product is misplaced or not easily found by the customer, but still exists on the sales floor. The user application can provide the location of the product (if it exists) in various areas of the store. For example, Fig.12 In the example of FIG. 1 , the GUI can show the customer the location of the selected product on the sales floor, in addition to the quantity available in the stockroom. The user application can also enable the customer to find the misplaced product on the sales floor if the store's employees have not moved the misplaced product on the sales floor.

[0254] 11.16 Automatic marking of pick lists

[0255] A pick list refers to a list of products requested by a user and may include requests for internal replenishment, e-commerce orders, inventory requests, misplaced products, or any other list of products that a user needs to find. Products in a pick list may be automatically marked for pickup if the following conditions are met: (1) the user is working with the pick list using a user application, (2) the user picks up a product in the pick list, and (3) if the product on the user's pick list is moving with the user, the RFID / computer vision item locating system identifies the product as being picked up by the user based on the user's device or its RFID employee tag. Once these conditions are met, the RFID / computer vision item locating system should automatically mark the product when the user picks it up. To improve the accuracy of automatic picking list marking, thresholds may be used to determine whether a product is picked up by a user, such as the time since the product was picked up or the distance the product has moved.

[0256] 11.17 Tracking Customers’ High Cart Values

[0257] RFID tag location systems can also be used to actively track the number and type of products in a customer's shopping cart in real time. Shopping carts can include baskets, bags, trolleys, etc. If a customer's shopping cart contains products that exceed a user-defined threshold, for example, 5 total units or $500 in value, a notification can be automatically sent to employees who identify these customers. In addition, specific products or product categories can also be marked by employees for priority tracking. This tracking feature can have multiple functional uses in a store. For example, the tracking feature can be used to prevent shoplifting by tracking products that may have a large number or high value in their shopping carts or that may have selected many marked products. The tracking feature can also be used to identify customers who may be willing to spend more money, which can notify employees to provide customers with better customer service, to upsell these customers, or to recommend free products to these customers.

[0258] The RFID tag location data can be displayed to the user in various formats using a GUI in the user application. For example, an employee can view all customers in the store in the GUI and monitor their shopping carts based on the number of products or based on the total value. To facilitate identification of the customer, the RFID tag location system can associate the person in the store with a customer profile if the customer uses the customer variant of the user application. Otherwise, the customer who has the product in his or her shopping cart can still be identified by tracking the movement of the product and determining whether the product is associated with a registered employee device.

[0259] 11.18 Automatically notify VIP customers

[0260] The RFID tag location system can also store data about the customer. This data can include the number of visits a customer makes to a store or the amount of money a customer spends per month or year. Based on this data, a VIP title can be attributed to a customer who exceeds a user-defined threshold.

[0261] The RFID tag location system can then be used to detect and identify VIP customers and notify employees when the VIP enters the store or a specific section of the store. Identification of the VIP can be accomplished using a variety of methods, including (1) detecting VIP status based on the customer's profile stored in a user application on the customer's mobile device via Bluetooth or Wi-Fi, (2) identifying the customer's mobile device id based on the user application on the customer's mobile device, or (3) using the computer vision capabilities of the RFID tag location system to identify based on facial or gait recognition.

[0262] 11.19 Identifying Potential Product Theft

[0263] As mentioned previously, RFID tag locating systems can actively track the movement of products in a customer's shopping cart in real-time. If an unusual event occurs while monitoring a product, a potential theft of a product may be detected. For example, if a customer were to remove an RFID tag from a product, the RFID tag locating system can detect this removal and immediately notify an employee of this exact product and its last known location in the store. In addition, RFID tag locating systems can also identify and retrieve footage recorded by the system's cameras or RFID readers to assist employees in locating customers or products. Once this information is provided to an employee, the employee can then approach the customer to provide assistance with the product that is missing the RFID tag.

[0264] The RFID tag locating system can also time stamp and store unusual events associated with RFID tagged products. This information can be used to display potential high shoplifting areas to employees in the GUI based on data such as the frequency of disappearing RFID tags. This data can be viewed in the GUI in user-defined time frames, such as the previous 7 days, 30 days, 180 days, etc. In addition, the RFID tag locating system can also identify and highlight areas in the store that may become high shoplifting areas by identifying the current locations of RFID tags that are prone to disappearing within the store.

[0265] 11.20 Automatic monitoring dressing room

[0266] Similar detection strategies for identifying potential product theft can also be used to automatically monitor fitting rooms. RFID tag locating systems can track products as they enter or leave fitting rooms. Notifications of entering or leaving fitting rooms can be sent to employees in real time. If a product is left in a fitting room, the RFID tag locating system can notify employees that a stray product is in the fitting room and return the product to its correct location within the store. If an RFID tag were to be removed, causing the product to disappear from the RFID tag locating system, notifications can also be sent to employees that the product may have disappeared and identify the customer with whom the product was last associated. Customers may also be identified by other products in their shopping carts when the product in question disappears.

[0267] 11.21 Capturing and measuring customer and product interactions

[0268] The RFID tag location system can also be used to detect and measure data related to customer and product interactions within a store. For example, the system can track (1) how often customers pick up products, (2) how long customers view products, (3) which products are viewed together, (4) which product a customer holds when viewing a new product, (5) which products may be retrieved from a fitting room, (6) which products a customer interacts with before making a purchase, (7) based on distortion of the measured RF signal due to proximity to a body of water (e.g., a human body), products that may be tested by a customer, e.g., clothing that a customer tries on, and (8) how long a customer tests a product. For products tested by customers, information can also be collected on unpurchased products (e.g., clothing left in a fitting room) to assess manufacturing or fit issues, e.g., a customer prefers the look of a garment over the fit of a garment. This data can be used to inform the store on how to potentially modify the manufacturing of a product to increase sales.

[0269] To measure these parameters, the RFID tag positioning system is capable of tracking objects in 3D space with high spatial and temporal resolution. For example, the RFID tag positioning system can detect whether a product moves more than a threshold distance, e.g., 4 inches, and remains there for a threshold time period, e.g., more than 3 seconds. If such conditions are met, the product can be considered to be picked up or viewed by the customer.

[0270] As previously described, RFID tag location systems can track the movement of products and customers within a store. Customers can be identified by (1) using a user application on their mobile device that is detected by a Bluetooth, Wi-Fi, or other wireless communication system or sensor, or (2) by detecting a customer based on an individual not having a device or tag that can be identified by the RFID tag location system, assuming that an employee has the tag or device.

[0271] The RFID tag positioning system can also collect product performance data based on product groups, such as product categories, subcategories, products, colors, sizes, price ranges, any combination of the above types, and the like. For these product groups, the performance data that can be collected includes the following: (1) the product groups that are viewed the most or least, (2) the product groups that are viewed the longest or shortest, (3) the product groups that are taken to the fitting room the most or least, (4) the product groups that are tested or tried on the most or least, (5) the product groups that are tested or tried on for the longest or shortest period of time, and (6) the product groups with the best or worst conversions, where conversion is defined as the number of sales compared to the other types of data mentioned. For example, if a product is viewed 100 times a day and sold 10 times a day, the conversion rate is 10%. In another example, if 100 products are tried on for more than 30 seconds and 10 are sold, the products that are tried on for more than 30 seconds have a conversion rate of 10%.

[0272] Based on the product performance data collected by the RFID tag location system, improvements in store operations can be achieved through: (1) identifying the best performing sales areas in the store, (2) identifying the areas with the most product interactions to improve staffing in that area, or (3) marketing strategies, such as automatically calculating and recommending product categories based on best-selling combinations (e.g., black jeans and white t-shirts perform best together) or identifying store areas that are best suited for certain product types, for example, dresses in area A of the store have the highest conversion rate.

[0273] Similarly, product performance data can improve the customer shopping experience by: (1) understanding the customer's historical shopping preferences based on the previously defined product groups to notify the customer of new or restocked products that were previously searched for in the store or online, (2) personalizing the in-store shopping experience by highlighting products within the store or in areas of the store that the customer may be interested in, (3) notifying the customer of potential in-store promotions, or (4) identifying the customer by detecting the customer's profile stored in a user application on the customer's mobile device via Bluetooth or WiFi, identifying the customer's mobile device id based on the user application on the customer's mobile device, or identifying the customer based on facial or gait recognition using the computer vision capabilities of an RFID tag positioning system.

[0274] 11.22 Conclusion

[0275] Although various embodiments have been described and illustrated herein, a variety of other devices and / or structures for performing the functions and / or obtaining the results and / or one or more of the advantages described herein are possible. More generally, all parameters, dimensions, materials, and configurations described herein are meant to be exemplary, and actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications used for the disclosed teachings. It should be understood that the aforementioned embodiments are presented only by way of example, and embodiments may be practiced in other ways than those specifically described and claimed. Embodiments of the present disclosure relate to each individual feature, system, article, material, kit, and / or method described herein. In addition, if two or more of these features, systems, articles, materials, kits, and / or methods are not consistent with each other, any combination of such features, systems, articles, materials, kits, and / or methods is included within the scope of the present disclosure.

[0276] The above embodiments may be implemented in any of a variety of ways. For example, embodiments of the technology disclosed herein may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code may be executed on any suitable processor or set of processors, whether disposed in a single computer or distributed in multiple computers.

[0277] The various methods or processes outlined herein may be encoded as software that can be executed on one or more processors using any of a variety of operating systems or platforms. In addition, such software may be written using any of a variety of suitable programming languages ​​and / or programming or scripting tools, and such software may also be compiled into executable machine language code or intermediate code that is executed on a framework or virtual machine.

[0278] In this regard, various inventive concepts may be embodied as a computer-readable storage medium (or multiple computer-readable storage media) (e.g., computer memory, one or more floppy disks, compact disks, optical disks, tapes, flash memory, circuit configurations in field programmable gate arrays or other semiconductor devices, or other non-transitory media or tangible computer storage media) encoding one or more programs that, when executed on one or more computers or other processors, perform methods for implementing various embodiments of the present invention discussed above. The computer-readable medium or multiple computer-readable media may be removable so that the program or programs stored thereon may be loaded onto one or more different computers or other processors to implement various aspects of the present invention as discussed above.

[0279] The term "program" or "software" is used herein in a general sense to refer to any type of computer code or computer executable instruction set that can be used to program a computer or other processor to implement various aspects of the embodiments as discussed above. In addition, it should be understood that according to one aspect, one or more computer programs that perform the methods of the present invention when executed need not reside on a single computer or processor, but can be distributed in a modular manner between several different computers or processors to implement various aspects of the present invention.

[0280] Computer executable instructions can take many forms, such as program modules, which are executed by one or more computers or other devices. Typically, program modules include routines, programs, objects, components, data structures, etc., which perform specific tasks or implement specific abstract data types. Typically, in various embodiments, the functionality of program modules can be combined or distributed as desired.

[0281] Likewise, various disclosed concepts may be embodied as one or more methods, for which examples have been provided. The actions performed as part of a method may be ordered in any suitable manner. Thus, embodiments may be constructed that perform actions in an order different from that shown, which may include performing some actions simultaneously, even though shown as sequential actions in an illustrative embodiment.

[0282] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0283] The indefinite articles "a" and "an" as used in this specification and the claims, unless explicitly stated to the contrary, should be understood to mean "at least one".

[0284] The phrase "and / or" used in this specification and claims should be understood to mean "either or both" of the elements so connected, that is, the elements exist in combination in some cases and separately in other cases. Multiple elements listed with "and / or" should be interpreted in the same manner, that is, "one or more" of the elements so connected. In addition to the elements specifically identified by the "and / or" clause, other elements may optionally be present, whether related or unrelated to those specifically identified elements. Therefore, as a non-limiting example, a reference to "A and / or B", when used in conjunction with open language (such as "comprising"), in one embodiment, may refer only to A (optionally including elements other than B); in another embodiment, only to B (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements), and so on.

[0285] As used in this specification and claims, "or" should be understood to have the same meaning as "and / or" defined above. For example, when separating the items in the list, "or" or "and / or" should be interpreted as inclusive, that is, including at least one of many elements or element lists but also including more than one, and optionally including additional unlisted items. Only terms that are clearly indicated as opposite, such as "only one of ... " or "just one of ... ", or when used in the claims, "consisting of ... " will refer to including exactly one element in many elements or element lists. In general, the term "or" as used herein should only be interpreted as indicating exclusive alternatives (that is, "one or the other, but not both") when it is an exclusive term (such as "any one", "one of ... ", "only one of ... " or "just one of ... ") before. "Substantially consisting of ... "When used in the claims, it should have the common meaning used in the field of patent law.

[0286] As used herein, the terms "about" and "approximately" generally refer to plus or minus 10% of the stated value.

[0287] As used in this specification and claims, the phrase "at least one" referring to a list of one or more elements should be understood to refer to at least one element selected from any one or more elements in the list of elements, but not necessarily including at least one of each element specifically listed in the list of elements, and not excluding any combination of elements in the list of elements. This definition also allows elements other than the elements specifically identified in the list of elements to which the phrase "at least one" refers to to be optionally present, whether or not related to the specifically identified elements. Thus, as a non-limiting example, "at least one of A and B" (or equivalently "at least one of A or B", or equivalently "at least one of A and / or B") may refer to at least one (optionally including more than one) A, no B (and optionally including elements other than B) in one embodiment; to at least one (optionally including more than one) B, no A (and optionally including elements other than A) in another embodiment; to at least one (optionally including more than one) A, and at least one (optionally including more than one) B (and optionally including other elements) in yet another embodiment, and so on.

[0288] In the claims and the foregoing specification, all transitional phrases such as "comprising," "including," "carrying," "having," "containing," "involving," "maintaining," "consisting of," and the like shall be understood to be open-ended, i.e., meaning including but not limited to. Only the transitional phrases "consisting of" and "consisting essentially of" shall be closed or semi-closed transitional phrases, respectively, as set forth in Section 2111.03 of the U.S. Patent Office Manual of Patent Examining Procedures.

Claims

1. A method for locating a radio frequency identification (RFID) tag, the method comprising: receiving, at an RFID tag reader, a plurality of first RFID signals from a first RFID tag at a first unknown location; determining a multi-path signature of the first RFID tag based on the plurality of first RFID signals, the multi-path signature representing radio frequency (RF) power received by the RFID tag reader along a line-of-sight (LOS) path and at least one non-line-of-sight (NLOS) path from the first RFID tag to the RFID tag reader; determining that the first RFID tag is fixed based on the multi-path signature of the first RFID tag; In response to determining that the first RFID tag is stationary, designating the first RFID tag as a first virtual reference tag; receiving at least one second RFID signal from a second RFID tag at a second unknown location; determining a multi-path signature of the second RFID tag based on the at least one second RFID signal, the multi-path signature representing RF power received by an RFID tag reader along a LOS path and at least one NLOS path from the RFID tag to the RFID tag reader; comparing the multi-path signature of the second RFID tag with the multi-path signature of the first virtual reference tag; as well as Based on the comparison, a position of the second RFID tag relative to the first virtual reference tag is determined. 2 . The method of claim 1 , wherein receiving the plurality of first RFID signals comprises receiving at least one first RFID signal from each of a plurality of angles of arrival.

3. The method according to claim 1, further comprising: receiving a plurality of third RFID signals from a third RFID tag at a third unknown location; designating the third RFID tag as a second virtual reference tag based on the plurality of third RFID signals; as well as determining a multipath profile of the second virtual reference tag, the multipath profile representing RF power received by the RFID tag reader along a LOS path and at least one NLOS path from the second virtual reference tag to the RFID tag reader; as well as Based on the at least one second RFID signal, a position of the second RFID tag relative to the second virtual reference tag is determined according to a multipath distribution of the second virtual reference tag.

4. The method according to claim 3, further comprising: determining a first error between a multipath distribution of the second RFID tag and a multipath distribution of the first virtual reference tag; determining a second error between the multipath distribution of the second RFID tag and the multipath distribution of the second virtual reference tag, the second error being greater than the first error; Based on the second error being greater than the first error, it is determined that the RFID tag is closer to the first virtual reference tag than the second virtual reference tag.

5. The method according to claim 4, further comprising: detecting a change in the RFID signal from the first virtual reference tag; detecting a change in the RFID signal from the second virtual reference tag; performing a comparison of a change in the RFID signal from the first virtual reference tag and a change in the RFID signal from the second virtual reference tag; as well as Based on the comparison, it is determined that the first virtual reference tag and the second virtual reference tag are stationary.

6. The method according to claim 1, further comprising: determining, based on the plurality of first RFID signals, a position of the first virtual reference tag relative to a reference tag at a known position; as well as Based on the position of the first virtual reference tag relative to the reference tag and the position of the second RFID tag relative to the first virtual reference tag, the position of the second RFID tag relative to the reference tag is determined.

7. A system for locating a radio frequency identification (RFID) tag, the system comprising: an RFID tag reader configured to receive a plurality of first RFID signals from a first RFID tag at a first unknown location and to receive at least one second RFID signal from a second RFID tag at a second unknown location; as well as a processor operably coupled to the RFID tag reader to determine a multipath signature of the first RFID tag based on the plurality of first RFID signals, the multipath signature representing radio frequency (RF) power received by the RFID tag reader along a line-of-sight (LOS) path and at least one non-line-of-sight (NLOS) path from the first RFID tag to the RFID tag reader, to determine that the first RFID tag is stationary based on the multipath signature of the first RFID tag, to designate the first RFID tag as a first virtual reference tag in response to determining that the first RFID tag is stationary, to determine a multipath signature of the second RFID tag based on the at least one second RFID signal, the multipath signature representing RF power received by the RFID tag reader along the LOS path and at least one NLOS path from the second RFID tag to the RFID tag reader; to compare the multipath signature of the second RFID tag with the multipath signature of the first virtual reference tag, and to determine a position of the second RFID tag relative to the first virtual reference tag based on the comparison.

8. The system of claim 7, wherein the RFID tag reader is configured to receive at least one first RFID signal of the plurality of first RFID signals from each of a plurality of angles of arrival.

9. The system of claim 7, wherein: The RFID tag reader is configured to receive a plurality of third RFID signals from a third RFID tag at a third unknown location, and The processor is configured to: designating the third RFID tag as a second virtual reference tag based on the plurality of third RFID signals, determining a multipath profile of the second virtual reference tag, the multipath profile representing RF power received by the RFID tag reader along a LOS path and at least one NLOS path from the second virtual reference tag to the RFID tag reader; as well as Based on the at least one second RFID signal, a position of the second RFID tag relative to the second virtual reference tag is determined according to a multipath distribution of the second virtual reference tag.

10. The system of claim 9, wherein the processor is configured to: determining a first error between a multipath distribution of the second RFID tag and a multipath distribution of the first virtual reference tag; determining a second error between the multipath distribution of the second RFID tag and the multipath distribution of a second virtual reference tag, the second error being greater than the first error; and Based on the second error being greater than the first error, it is determined that the second RFID tag is closer to the first virtual reference tag than the second virtual reference tag.

11. The system of claim 9, wherein the processor is configured to: detecting a change in the RFID signal from the first virtual reference tag; detecting a change in the RFID signal from the second virtual reference tag; performing a comparison of a change in the RFID signal from the first virtual reference tag and a change in the RFID signal from the second virtual reference tag; and Based on the comparison, it is determined that the first virtual reference tag and the second virtual reference tag are stationary.

12. The system of claim 7 , wherein the processor is configured to determine a position of the first virtual reference tag relative to a reference tag at a known position based on the plurality of first RFID signals, and to determine a position of the second RFID tag relative to the reference tag based on a position of the first virtual reference tag relative to the reference tag and a position of the second RFID tag relative to the first virtual reference tag.

13. A method for locating a radio frequency identification (RFID) tag, the method comprising: receiving, at an RFID tag reader, a plurality of first RFID signals from a first RFID tag at a first unknown location; determining a multi-path signature of the first RFID tag based on the plurality of first RFID signals, the multi-path signature representing radio frequency (RF) power received by the RFID tag reader along a line-of-sight (LOS) path and at least one non-line-of-sight (NLOS) path from the first RFID tag to the RFID tag reader; designating the first RFID tag as a first virtual reference tag; receiving at least one second RFID signal from a second RFID tag at a second unknown location; determining a multi-path signature of the second RFID tag based on the at least one second RFID signal, the multi-path signature representing RF power received by an RFID tag reader along a LOS path and at least one NLOS path from the RFID tag to the RFID tag reader; comparing the multi-path signature of the second RFID tag with the multi-path signature of the first virtual reference tag; determining an error between the multi-path signature of the first RFID tag and the multi-path signature of the second RFID tag; as well as Based on the error, a position of the second RFID tag relative to the first virtual reference tag is determined.

14. The method of claim 13, wherein receiving the plurality of first RFID signals comprises receiving at least one first RFID signal from each of a plurality of angles of arrival.

15. The method according to claim 13, further comprising: receiving a plurality of third RFID signals from a third RFID tag at a third unknown location; designating the third RFID tag as a second virtual reference tag based on the plurality of third RFID signals; as well as determining a multipath profile of the second virtual reference tag, the multipath profile representing RF power received by the RFID tag reader along a LOS path and at least one NLOS path from the second virtual reference tag to the RFID tag reader; as well as Based on the at least one second RFID signal, a position of the second RFID tag relative to the second virtual reference tag is determined according to a multipath distribution of the second virtual reference tag.

16. The method according to claim 15, further comprising: determining a first error between a multipath distribution of the second RFID tag and a multipath distribution of the first virtual reference tag; determining a second error between the multipath distribution of the second RFID tag and the multipath distribution of the second virtual reference tag, the second error being greater than the first error; Based on the second error being greater than the first error, it is determined that the RFID tag is closer to the first virtual reference tag than the second virtual reference tag.

17. The method according to claim 16, further comprising: detecting a change in the RFID signal from the first virtual reference tag; detecting a change in the RFID signal from the second virtual reference tag; performing a comparison of a change in the RFID signal from the first virtual reference tag and a change in the RFID signal from the second virtual reference tag; as well as Based on the comparison, it is determined that the first virtual reference tag and the second virtual reference tag are stationary.

18. The method according to claim 13, further comprising: determining, based on the plurality of first RFID signals, a position of the first virtual reference tag relative to a reference tag at a known position; as well as Based on the position of the first virtual reference tag relative to the reference tag and the position of the second RFID tag relative to the first virtual reference tag, the position of the second RFID tag relative to the reference tag is determined.

Images