Method and locating device for locating radio tags, and method and assembly for configuring the locating device

Abstract

In order to configure a locating device (OG) for locating radio tags (RF), one or more first radio tags (RF) are conveyed through a specified region (B) multiple times under various conveyor conditions. In the process, the current respective position (P) of each first radio tag (RF) is detected by means of a locating sensor (C), a radio signal of the respective first radio tag (RF) is received, and a respective signal strength curve (TS1, TS2) of the received radio signal is detected. According to the invention, the detected current positions (P) and signal strength curves (TS1, TS2) are used to train a machine learning module (CNN) to reproduce a corresponding current position (P) using at least one signal strength curve (TS1, TS2). The locating device (OG) is designed to then feed signal strength curves (TS1, TS2) of received radio signals from second radio tags (RF) into the trained machine learning module (CNN) and ascertain the respective current position (PP) of a respective second radio tag (RF) using the resulting output signals of the machine learning module.

Description

[0001] Description

[0002] Method and locating device for locating radio tags and method and arrangement for configuring the locating device

[0003] Radio frequency identification (RFID) tags, such as transponders or RFID tags (RFID: radio-frequency identification), are often used to monitor or record goods, merchandise, or other objects. Such radio frequency tags can be read by appropriate readers using high-frequency radio waves. In many cases, the radio frequency tags are powered by the radio waves generated by the reader and therefore do not require their own power supply.

[0004] Radio frequency tags allow easy identification of appropriately labeled objects as well as the transmission of information between the radio frequency tag and the reader.

[0005] Using an RFID gate, objects equipped with an RFID tag can be read and / or identified within a range typically of a few meters. A typical RFID gate might, for example, comprise four antennas arranged in pairs on two separate side posts. However, different configurations or a larger number of antennas are also common.

[0006] Objects equipped with an RFID tag can generally be detected and read by an RFID gate with high reliability if they come sufficiently close to the RFID gate. However, it is considerably more difficult to locate an RFID tag more precisely because in many cases only the received signal strengths are available as a basis for measurement, which often only allows a very rough estimate of the respective distance of the RFID tag from the receiving antennas. The received signal strengths are often also referred to as RSSI values ​​(RSSI: Received Signal Strength Indicator). In addition, the detected signal strengths are often subject to strong noise and are strongly influenced by objects or building elements in the vicinity of the RFID gate. Last but not least, the detected signal strengths also depend on the alignment of the antennas or RFID tags as well as on the respective tagged object.

[0007] It is an object of the present invention to provide a method and a locating device for locating radio tags as well as a method and an arrangement for configuring the locating device, which allow a more precise and / or more robust localization of radio tags.

[0008] This object is achieved by methods having the features of patent claims 1 and 12, by an arrangement having the features of patent claim 14 and by a locating device having the features of patent claim 15.

[0009] To configure a tracking device for locating radio tags, one or more first radio tags are transported through a predetermined area numerous times under varying transport conditions. In each case, a current position of a respective first radio tag is detected using a localization sensor, a radio signal from the respective first radio tag is received, and a respective signal strength curve of the received radio signal is recorded. According to the invention, a machine learning module is trained based on the detected current positions and signal strength curves to reproduce a corresponding current position based on at least one signal strength curve.The tracking device is then configured to feed signal strength curves of received radio signals from second radio tags into the trained machine learning module and to determine the respective current position of a respective second radio tag based on its resulting output signals.

[0010] To carry out the above method, an arrangement for configuring a locating device for locating radio tags is provided. By carrying out the above method, a locating device for locating radio tags is set up.

[0011] To locate radio tags, it is intended to set up a locating device according to the above method and to determine a current position of a second radio tag using the configured locating device.

[0012] The methods according to the invention, the arrangement according to the invention, and the locating device according to the invention can be executed or implemented, for example, using one or more processors, computers, application-specific integrated circuits (ASICs), digital signal processors (DSPs), and / or so-called "field programmable gate arrays" (FPGAs). Furthermore, the methods according to the invention can be executed at least partially in a cloud and / or in an edge computing environment.

[0013] A key advantage of the invention is that radio tags can be located more precisely based on the radio signals they emit as intended. Since the machine learning module can often learn to distinguish radio signal patterns relevant to the position of radio tags from less relevant radio signal patterns, location according to the invention often proves to be relatively robust. Furthermore, the invention requires only a relatively low calibration effort, since calibration data for training the machine learning module only needs to be generated once, and the trained machine learning module can then be installed on a variety of tracking devices.

[0014] Advantageous embodiments and further developments of the invention are specified in the dependent claims.

[0015] According to an advantageous embodiment of the invention, a radio tag attached to an object can be moved with the object through the area. This allows training to be carried out under particularly realistic conditions. In particular, any potential influence of a tagged object on a signal strength curve can be taken into account during training.

[0016] Furthermore, to vary the conditions of carriage

[0017] - a movement trajectory, a movement direction, a movement speed, a movement pattern, an orientation, a type and / or a number of the at least one first radio tag,

[0018] - a movement trajectory, a movement direction, a movement speed, a movement pattern, an orientation, a type and / or a number of at least one object provided with a first radio tag,

[0019] - a position or orientation of an object influencing the radio signal,

[0020] - an alignment of an antenna of the tracking device,

[0021] - a transmission power, sensitivity or setting of the tracking device, and / or

[0022] - a hardware configuration of the tracking device can be varied. The hardware configuration can, for example, relate to the antenna model used, the reader model used, or the cable length between an antenna and a reader. Advantageously, the transport conditions can be varied so that they cover possible and / or permissible transport scenarios or types of use as representatively or completely as possible during operation of the configured tracking device.

[0023] According to an advantageous development of the invention, during the respective transport of one or more first radio tags through the area

[0024] - a movement trajectory, a movement direction, a movement speed, a movement pattern, an orientation, a type and / or a number of the at least one first radio tag or - a movement trajectory, a movement direction, a movement speed, a movement pattern, an orientation, a type and / or a number of at least one object provided with a first radio tag are recorded as a measurement result. The machine learning module can be trained accordingly to reproduce a corresponding measurement result based on at least one signal strength curve. In this way, the tracking device can be set up to determine other movement properties or transport conditions of second radio tags based on received radio signals in addition to the current position.

[0025] Advantageously, a respective radio signal can be received by multiple antennas. For each antenna, an antenna-specific signal strength curve of the received radio signal can be recorded and assigned to an antenna identifier that identifies the respective antenna. The respective signal strength curve can then be fed to the machine learning module along with a respective assigned antenna identifier. Using multiple antennas can generally significantly increase location accuracy.

[0026] Furthermore, the machine learning module can be fed a signal strength curve along with a received radio tag identifier. This allows specific properties of radio tags to be taken into account during training.

[0027] According to the invention, a reproduction error of the machine learning module, in particular an error in the reproduction of a current position, can be detected. This allows specific signal components to be searched for in the recorded signal strength curves, which, when fed into the machine learning module, reduce, in particular minimize, the reproduction error at least on average. Signal components found in this way can then be specifically extracted from the recorded signal strength curves and fed into the machine learning module as a preprocessed signal strength curve for training and / or for use by the machine learning module. In this way, specifically relevant signal components can be identified, which allow a conclusion to be drawn about a current position or movement of the radio tag, as independently as possible of environmental properties, antenna characteristics, or other influences not caused by the movement of a radio tag.

[0028] According to a further advantageous development of the invention, the trained machine learning module can be retrained at a location of the tracking device using position data recorded at the location. The position data used can include, in particular, localization data from an operator's object monitoring system or manually entered localization data, confirmation data, and / or correction data. Alternatively or additionally, a so-called auto-labeling process, e.g., via cluster analysis, can also be performed.

[0029] According to a further advantageous embodiment of the invention, a respective signal strength curve of received radio signals can be interpolated to a predetermined time frame. The interpolated signal strength curve can then be fed into the machine learning module. In this way, a quantity of data to be processed by the machine learning module can be adapted to the actual rate of change of the signal strengths. The time frame can typically be formed from points in time a few seconds apart.

[0030] Furthermore, an optical sensor, a camera, a radio wave-based sensor, a proximity sensor, and / or a laser scanner can be used as the localization sensor. In particular, the localization sensor can be implemented based on a lidar, radar, RTLS (Real-Time Locating System), Bluetooth, Wi-Fi, or 5G system. Preferably, several such localization sensors, in particular several cameras, can be used.

[0031] Advantageously, the machine learning module can comprise a recurrent neural network, an LSTM (Long Short-Term Memory) network, a convolutional neural network, and / or a temporal convolutional neural network. Such neural networks can be used to process time series data very efficiently.

[0032] According to a further advantageous development of the invention, multiple signal strength curves can be fed into the machine learning module, and in each case, an individual forecast regarding the position of a second radio tag can be output. The current position of the second radio tag can then be determined by combining the individual forecasts. In this way, positioning accuracy can generally be significantly increased. In particular, individual incorrect forecasts, e.g., due to a short-term signal failure or particularly severe interference, can generally be effectively compensated. Such aggregation of individual forecasts into an overall result can also be optimized through specific training of the machine learning module or another machine learning module.For this purpose, the way in which the individual forecasts are to be combined can be parameterized using setting parameters, whereby the setting parameters are set in such a way that a forecast error of the respective machine learning module is minimized. In this way, in particular, a hierarchical machine learning model can be implemented.

[0033] An embodiment of the invention is explained in more detail below with reference to the drawings, each of which illustrates in schematic form:

[0034] Figure 1 shows an arrangement for configuring a locating device, and Figure 2 shows a proper operation of the configured locating device.

[0035] Insofar as identical or corresponding reference symbols are used in the figures, these reference symbols designate identical or corresponding entities which can be implemented or configured in particular as described in connection with the relevant figure.

[0036] Figure 1 schematically illustrates an arrangement for configuring a tracking device OG. The tracking device OG can, in particular, be an RFID gate or a part thereof. The tracking device OG serves to monitor a predetermined area B through which objects OBJ, e.g., parcels, packaging, products, product components, robotic vehicles, etc., move. The area B can, for example, be a transport area or a conveyor belt in a warehouse, a logistics center, a warehouse, or a production facility. For identification and / or location, the objects OBJ are each provided with a radio frequency (RF) tag, e.g., an RFID tag.

[0037] According to the invention, the locating device OG is to be configured in such a way that it can locate radio tags RF moving through the area B as accurately and / or robustly as possible based on the radio signals received by them.

[0038] For this purpose, the locating device OG has a reader R for radio tags, several antennas A1 and A2 coupled to the reader R, and a CNN machine learning module. In the present embodiment, the CNN machine learning module comprises a convolutional neural network, in particular a temporally convolutional neural network. Such a convolutional neural network is often also referred to as a "convolutional neural network." Alternatively or additionally, the CNN machine learning module can comprise a recurrent neural network, an LSTM network, or another neural network. The CNN machine learning module is to be trained to predict the current positions of radio tags RF.

[0039] Training is generally understood as the optimization of a mapping of the input signals of a machine learning module, in this case a CNN, to its output signals. This mapping is optimized according to one or more predefined criteria during a training phase. A prediction error can be used as a criterion, particularly in predictive models. Through training, for example, the network structures of neurons in a neural network and / or the weights of connections between the neurons can be adjusted or optimized so that the predefined criteria are met as closely as possible. Training can thus be viewed as an optimization problem.For such optimization problems in the field of machine learning, a variety of efficient optimization methods are available, in particular gradient-based optimization methods, gradient-free optimization methods, backpropagation methods, particle swarm optimizations, genetic optimization methods and / or population-based optimization methods.

[0040] In particular, artificial neural networks, recurrent neural networks, convolutional neural networks, LSTM networks, deep learning architectures, support vector machines, Bayesian neural networks, autoencoders, Gaussian processes, data-driven regression models, K-nearest neighbor classifiers, physical models or decision trees can be trained in this way.

[0041] In the present exemplary embodiment, to train the CNN machine learning module, objects OBJ provided with first radio tags RF are moved through area B many times under varied transport conditions, in particular on different movement trajectories TR. Here and below, radio tags used for training are referred to as first radio tags, while radio tags that are to be located during normal operation are referred to as second radio tags. This distinction merely serves to distinguish the training phase from an application phase and, in particular, does not imply any structural differences between first and second radio tags. In fact, (first) radio tags that are identical or similar to the (second) radio tags to be located during normal operation are preferably used for training.Accordingly, the first and second radio tags are identified by the same reference numeral in the figures.

[0042] The transport conditions are varied during the training phase so that they cover the transport scenarios, types of use or influencing factors possible and / or permissible in the intended operation of the configured locating device OG as completely and / or representatively as possible. In particular, the movement trajectories TR are preferably varied over the entire area B and / or with regard to a direction of movement, a speed of movement or a movement pattern. Furthermore, an orientation, a type and / or a number of objects OBJ moved through area B can be varied. In addition, a position or orientation of an object influencing the radio signal in area B or in its surroundings, an alignment of the antennas A1, A2 and / or a transmission power, sensitivity, setting or hardware configuration of the locating device OG can be varied.

[0043] To locate the objects OBJ and thus the first radio tags RF, several cameras C are aimed at area B. These cameras continuously record images of area B from different directions, for example, in top and profile views. The cameras C are coupled to an optical evaluation device OPT, to which the recorded images are continuously transmitted. The optical evaluation device OPT serves to continuously detect and localize the objects OBJ. Alternatively or in addition to the cameras C, other localization sensors can be provided to localize the objects OBJ, such as radio wave-based sensors, proximity sensors, or laser scanners.

[0044] In the present exemplary embodiment, the optical evaluation device OPT continuously determines a respective current position P of a respective object OBJ and thus of a respective radio tag RF over time based on the transmitted images. The current position P can be determined, in particular, as a position vector, a motion vector, and / or an indication of the zone of area B in which a respective object OBJ is currently located. For the latter purpose, area B can be divided, for example, into zones to the left and right of a central axis of an RFID gate and / or into zones within and outside the radio range of the RFID gate.

[0045] If a discrete zone value is determined as the current position P, the CNN machine learning module can be trained using known, relevant methods to perform a discrete classification. If a quasi-continuous position or movement value is determined as the current position P, the CNN machine learning module can be trained using similarly relevant methods to output a quasi-continuous forecast. Training of the CNN machine learning module is explained in more detail below.

[0046] Preferably, the optical evaluation device OPT can also determine additional data about the movements of the objects OBJ in a time-resolved manner as measurement results, such as a respective orientation, a respective direction of movement, a respective speed of movement, a respective movement pattern, a respective type, and / or a number of objects OBJ. The measurement results can then be used to train the CNN machine learning module to reproduce not only a current position but also the measurement results. A variety of known optical pattern or object recognition methods are available for evaluating the images by the optical evaluation device OPT.

[0047] Furthermore, the reader R transmits radio waves into area B via the antennas A1 and A2. These radio waves are received by a first radio tag RF located therein, temporarily supplying it with energy. As a result, the respective first radio tag RF transmits a radio signal as it moves through area B, which is received by the antennas A1 and A2 and forwarded to the reader R. The radio signal contains, among other things, a radio tag identifier (not shown) that identifies this first radio tag RF.

[0048] While the respective first radio tag RF is moved through area B, the reader R records a respective temporal profile TS1 of a signal strength of the radio signal received by the antenna A1 as well as a respective temporal profile TS2 of a signal strength of the radio signal received by the antenna A2. The antenna-specific signal strength profiles TS1 and TS1 are each recorded in the form of a temporal sequence, i.e. in the form of a time series of so-called RSSI values. If necessary, the signal strength profiles TS1 and TS2 are each interpolated to a predetermined time grid with a time interval of, for example, 1 s, 2 s, 5 s, or 10 s and further processed in the interpolated form.

[0049] To increase the amount of training data available for training, it can optionally be provided to mirror the determined positions P and, in parallel, the signal strength curves TS1 and TS2 at one or more symmetry planes or symmetry axes of the antenna arrangement, here A1 and A2. In the present exemplary embodiment, according to Figure 1, a mirroring at the symmetry plane between the antennas A1 and A2 would correspond to a mirroring of the positions P at this symmetry plane and an interchange of the signal strength curves TS1 and TS2. The mirrored training data can then be added to the measured, unreflected positions P and signal strength curves TS1 and TS2. By expanding the training data in this way, the robustness of the predictions of the CNN machine learning module can be significantly increased in many cases. In particular, the mere memorization of the specific geometry of the antenna arrangement can be reduced to a certain extent.

[0050] To train the CNN machine learning module, both the respective signal strength curve TS1, associated with an antenna identifier ID1 identifying antenna A1, and the respective signal strength curve TS2, associated with an antenna identifier ID2 identifying antenna A2, are fed from the reader R into the CNN machine learning module as time-resolved input signals. Optionally, the received radio tag identifier can also be fed to the CNN machine learning module.

[0051] The goal of the training is for the CNN machine learning module to reproduce an actually detected current position, here P, as accurately as possible based on input antenna-specific signal strength curves, here TS1 and TS2. This means that an output signal PP of the CNN machine learning module matches the actually determined current position P as closely as possible, at least on average.

[0052] For this purpose, a deviation D is determined between the respective output signal PP and the respective detected current position P. The deviation D represents a reproduction or prediction error of the CNN machine learning module. The deviation D can be calculated, in particular, as the square or absolute value of a difference, in particular a vector difference, according to D = (P - PP) 2or D = |P - PP|. The deviation D is determined individually for each of the multiple transports of the first radio tags RF through area B. The deviations D determined during the training phase are fed back to the CNN machine learning module, as indicated by a dashed arrow in Figure 1. Based on the fed-back deviations D, the CNN machine learning module is trained to minimize these deviations D and thus the reproduction error, at least on average. As already indicated above, a variety of well-known optimization methods are available to minimize the deviations D, such as gradient descent methods, particle swarm optimization, or genetic optimization methods.In this way, the machine learning module CNN is trained by a supervised learning process to predict the current position of a respective object OBJ and thus of the respective radio tag RF relatively accurately based on fed-in, antenna-specific signal strength curves.

[0053] Optionally, to improve the forecast quality, specific signal components can be searched for in the recorded signal strength curves that, when fed into the CNN machine learning module, reduce the reproduction error D, at least on average. Signal components found in this way can then be extracted from the recorded signal strength curves and used as input signals for training the CNN machine learning module. Alternatively or additionally, signal components that have a negative influence on the reproduction error D can be searched for. Such signal components can then be removed from the relevant signal strength curves before they are used to train the CNN machine learning module.

[0054] Figure 2 schematically illustrates the intended operation of the tracking device OG, which has been configured by training the machine learning module CNN as described above. In addition to the tracking device OG used for training, the trained machine learning module CNN can also be used by other identical or similar tracking devices as described below. During intended operation, the configured tracking device OG is intended to locate objects OBJ moving through area B, each of which is provided with a second radio tag RF, based on received radio signals.

[0055] For this purpose, the reader R transmits radio waves into area B via the antennas A1 and A2. These waves are received by a second radio tag RF located therein and attached to an object OBJ, temporarily supplying it with energy. As a result, the respective second radio tag RF emits a radio signal as it moves through area B, which is received by the antennas A1 and A2 and forwarded to the reader R. As described above, the reader R detects a respective antenna-specific signal strength curve TS1 or TS2 of the radio signal and feeds this, together with the associated antenna identifier ID1 or ID2, into the trained machine learning module CNN as an input signal. If necessary, the signal strength curves e can be interpolated beforehand to a specified time grid.Furthermore, as indicated above, signal components with a positive influence on forecast quality can be preferentially fed into the trained CNN machine learning module. Complementarily, signal components with a negative influence on forecast quality can be removed from the relevant signal strength curves before they are fed into the system.

[0056] Based on the input, antenna-specific signal strength curves TS1 and TS2, the trained CNN machine learning module then predicts the current position PP of the tagged object OBJ or radio tag RF, as well as, if necessary, additional data about the movement of the object OBJ. The predicted current position PP and, if necessary, the additional movement data are finally output as intended by the configured tracking device OG.

Claims

Patent claims 1. A method for configuring a locating device (OG) for locating radio tags (RF), wherein a) one or more first radio tags (RF) are transported many times under varied transport conditions through a predetermined area (B), wherein in each case - a current position (P) of a respective first radio tag (RF) is detected by means of a localization sensor (C), and - a radio signal of the respective first radio tag (RF) is received and a respective signal strength curve (TS1, TS2) of the received radio signal is recorded, b) a machine learning module (CNN) is trained on the basis of the recorded current positions (P) and signal strength curves (TS1, TS2) to reproduce a corresponding current position based on at least one signal strength curve, and c) the locating device (OG) is configured to feed signal strength curves (TS1, TS2) of received radio signals from second radio tags (RF) into the trained machine learning module (CNN) and to determine a respective current position of a respective second radio tag (RF) based on its resulting output signals (PP), characterized in that a reproduction error (D) of the machine learning module (CNN) is recorded, that specific signal components are searched for in recorded signal strength curves (TS1, TS2),which, when fed into the machine learning module (CNN), reduce the reproduction error (D) at least on average, and that signal components found in this way are specifically extracted from recorded signal strength curves (TS1, TS2) and fed into the machine learning module (CNN) for training and / or use of the machine learning module (CNN).

2. Method according to claim 1, characterized in that a radio tag (RF) attached to an object (OBJ) is moved with the object (OBJ) through the area.

3. Method according to one of the preceding claims, characterized in that for varying the transport conditions - a movement trajectory (TR), a movement direction, a movement speed, a movement pattern, an orientation, a type and / or a number of the at least one first radio tag (RF), - a movement trajectory (TR), a movement direction, a movement speed, a movement pattern, an orientation, a type and / or a number of at least one object (OBJ) provided with a first radio tag (RF), - a position or orientation of an object influencing the radio signal, - an alignment of an antenna (Al, A2 ) of the locating device (OG), - a transmission power, sensitivity or setting of the tracking device (OG), and / or - a hardware configuration of the tracking device (OG) is varied.

4. Method according to one of the preceding claims, characterized in that during the respective conveyance of one or more first radio tags (RF) through the area (B) - a movement trajectory (TR), a movement direction, a movement speed, a movement pattern, an orientation, a type and / or a number of the at least one first radio tag (RF) or - a movement trajectory (TR), a movement direction, a movement speed, a movement pattern, an orientation, a type and / or a number of at least one object (OBJ) provided with a first radio tag (RF) is recorded as a measurement result, and that the machine learning module (CNN) is trained to reproduce a corresponding measurement result based on at least one signal strength curve (TS1, TS2).

5. Method according to one of the preceding claims, characterized in that a respective radio signal is received by a plurality of antennas (Al, A2), that for a respective antenna (Al, A2) an antenna-specific signal strength curve (TS1, TS2) of the received radio signal is detected and assigned to an antenna identifier (ID1, ID2) identifying the respective antenna (Al, A2), and that a respective signal strength curve (TS1, TS2) together with a respectively assigned antenna identifier (ID1, ID2) is fed to the machine learning module (CNN).

6. Method according to one of the preceding claims, characterized in that a respective signal strength curve (TS1, TS2) is supplied to the machine learning module (CNN) together with a respectively received radio tag identifier.

7. Method according to one of the preceding claims, characterized in that the trained machine learning module (CNN) is retrained at a location of use of the locating device (OG) using position data recorded at the location.

8. Method according to one of the preceding claims, characterized in that a respective signal strength curve (TS1, TS2) of received radio signals is interpolated to a predetermined time grid, and that the respectively interpolated signal strength curve is fed into the machine learning module (CNN).

9. Method according to one of the preceding claims, characterized in that an optical sensor, a camera, a radio wave-based sensor, a proximity sensor and / or a laser scanner is used as the localization sensor (C).

10. Method according to one of the preceding claims, characterized in that the machine learning module (CNN) comprises a recurrent neural network, an LSTM network, a convolutional neural network and / or a temporal convolutional neural network.

11. A method for locating radio tags (RF), wherein a locating device (OG) is set up according to one of the preceding claims, and a current position (PP) of a second radio tag (RF) is determined by means of the configured locating device (OG).

12. The method according to claim 11, characterized in that a plurality of signal strength curves (TS1, TS2) are fed into the machine learning module (CNN) and in each case an individual forecast is output regarding a position of the second radio tag (RF), and that the current position is determined by a combination of the individual forecasts.

13. Arrangement for configuring a locating device (OG) for locating radio tags (RF), configured to carry out the method steps of a method according to one of claims 1 to 10.

14. Locating device (OG) for locating radio tags (RF), configured by carrying out a method according to one of claims 1 to 10.

Images