Tracking Method, Tracking System, and Machine-Readable Medium

Abstract

Systems and methods are disclosed for providing context tracking information to a tracking sensor system to provide accurate and efficient object tracking. Context data of a first tracking sensor system is used to identify an object being tracked by a second tracking sensor system.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims the benefit and priority of U.S. Provisional Application No. 62 / 828,198, entitled "Tracking Aggregation and Alignment", filed on April 2, 2019, which is hereby incorporated by reference in its entirety for all purposes. Background of the Invention

[0003] The present disclosure generally relates to tracking systems. More specifically, certain embodiments of the present disclosure relate to the aggregation and switching of tracking system data between tracking systems to facilitate more efficient and effective tracking of objects within an environment.

[0004] In the digital age, with the increase in digital sensors, object tracking has become increasingly desirable. Unfortunately, in large / open environments, user tracking is a very challenging prospect, especially when accurate location and activity tracking are desired. As used herein, an open environment refers to an area that allows a tracked object to move in multiple directions with relatively few restrictions. For example, such an environment may include an amusement park, an airport, a shopping mall, or other relatively large - scale environments that may have multiple tracking coverage areas. Accurately tracking unique individuals is challenging, especially in open environments and in situations where crowd density poses an obstruction (where one individual may block another).

[0005] This section is intended to introduce the reader to various aspects of the art that may be related to the various aspects of the technology described and / or claimed below. This discussion is considered to be helpful in providing background information to the reader to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light and not as an admission of prior art. Summary of the Invention

[0006] Certain embodiments commensurate in scope with the originally claimed subject matter are outlined below. These embodiments are not intended to limit the scope of the present disclosure, but rather these embodiments are only intended to provide a brief overview of certain disclosed embodiments. In fact, the present disclosure may include a variety of forms that may be similar to or different from the embodiments set forth below.

[0007] The embodiments described herein relate to a tracking system that efficiently aggregates and / or conveys tracking data between tracking systems such that the context of one tracking sensor can enhance the tracking of other tracking sensors. More specifically, context information (e.g., location, time, identity of the tracked object) determined by one tracking sensor can be used to facilitate more efficient and / or effective tracking of other sensors. For example, such context information can result in an increased confidence in the identity of the tracked object, can result in an efficient filtering of possible identities attributable to the tracked object, etc. This can lead to increased processing efficiency and can also enable tracking of objects in a more granular open environment.

[0008] For example, in a first embodiment, a tangible, non-transitory, machine-readable medium includes machine-readable instructions that, when executed by one or more processors of a machine, cause the machine to: receive a tracked target context of a first tracked object from a first tracking sensor system; provide the tracked target context from the first tracking sensor system to a second tracking sensor system different from the first tracking sensor system; and cause an identification of a newly observed tracked object by the second tracking sensor system based on the tracked target context from the first tracking sensor system.

[0009] In a second embodiment, a computer-implemented method includes: receiving a tracked target context of a first tracked object from a first tracking sensor system; providing the tracked target context from the first tracking sensor system to a second tracking sensor system different from the first tracking sensor system; and causing an identification of a newly observed tracked object by the second tracking sensor system based on the tracked target context from the first tracking sensor system.

[0010] In a third embodiment, a system includes: a first tracking sensor system, a second tracking sensor system, and a context tracking sensor system. The first tracking sensor system tracks a first tracked object in a first coverage area. The second tracking sensor system tracks a second tracked object in a second coverage area. The context tracking system receives a tracked target context of the first tracked object from the first tracking sensor system; provides the tracked target context from the first tracking sensor system to a second tracking sensor system different from the first tracking sensor system; and causes an identification of the second tracked object based on the tracked target context from the first tracking sensor system. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] These and other features, aspects, and advantages of the present disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings, where like characters represent like parts throughout the drawings, wherein:

[0012] Figure 1 is a schematic diagram illustrating a multi - sensor tracking component with a context tracking system according to an embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram illustrating an open environment of a system using Figure 1 according to an embodiment of the present disclosure;

[0014] Figure 3 is a flowchart illustrating a process for identifying a tracking context according to an embodiment;

[0015] Figure 4 is a flowchart illustrating a process for using the acquired context to identify the context at a subsequent tracking sensor according to an embodiment;

[0016] Figure 5 is a flowchart illustrating a process for using a confidence interval to determine sufficient context for target identification according to an embodiment;

[0017] Figure 6 is a schematic diagram illustrating an example transfer for increasing tracking input according to an embodiment;

[0018] Figure 7 is a flowchart illustrating a process for filtering possible identification predictions based on provided sensor context according to an embodiment;

[0019] Figure 8 is a schematic diagram illustrating an example control action based on a tracked identity according to an embodiment;

[0020] Figure 9 is a flowchart for classifying individuals into groups using machine learning techniques according to an embodiment;

[0021] Figure 10 is an illustration of grouped and ungrouped individuals according to an embodiment;

[0022] Figure 11 is an illustration of a use case for changing an interactive environment based on compliance rules according to an embodiment; and

[0023] Figure 12 is a schematic diagram illustrating an example control action based on a tracked identity according to an embodiment. Detailed Description

[0024] One or more specific embodiments of the present disclosure will be described below. These described embodiments are merely examples of the techniques of the present disclosure. Further, in the process of endeavoring to provide a brief description of these embodiments, all features of the actual implementation may not be described in the specification. It should be understood that in the development of any such actual implementation, as in any engineering or design project, many implementation-specific decisions must be made to achieve the specific goals of the developer, such as in accordance with system-related and business-related constraints, which may vary from implementation to implementation. Additionally, it should be understood that such development work may be complex and time-consuming, but for those of ordinary skill in the art who benefit from the present disclosure, this may still be a routine task of design, fabrication, and manufacture.

[0025] When introducing elements of various embodiments of the present disclosure, the articles "a", "an", and "the" are intended to mean that there is one or more elements. The terms "comprising", "including", and "having" are intended to be inclusive and mean that there may be additional elements in addition to the listed elements. Further, it should be understood that the reference to "one embodiment" or "an embodiment" of the present disclosure is not intended to be construed as excluding the existence of additional embodiments that also incorporate the recited features.

[0026] The present disclosure generally relates to a tracking system that accumulates and / or switches context information for efficient and effective tracking processing. By using the context of other previously determined tracking sensors, an independent tracking sensor can more effectively determine the identity of the object being tracked. With this in mind, Figure 1 is a schematic diagram illustrating a multi-sensor tracking system 100 with a context tracking system 102 according to an embodiment of the present disclosure. As illustrated, the multi-sensor tracking system 100 includes a plurality of tracking sensors, such as one or more light detection and ranging (LIDAR) systems 104, one or more radio frequency identification (RFID) reader systems 106, one or more time-of-flight (ToF) systems 107, one or more computer vision systems 108, and / or one or more millimeter wave (mmWave) systems 109.

[0027] The LIDAR system 104 can track individuals, objects, and / or groups of individuals or objects by illuminating a target with pulsed light and measuring the reflected pulses. The wavelength and time differences between the pulsed light and the reflected pulses can be used to generate a spatial indication of the position of the target individual, group, and / or object. The LIDAR system 104 is capable of covering a large area of space while relatively easily and effectively detecting objects. However, the LIDAR system 104 may not be effective in actually identifying the object being tracked, but can be best used to identify the presence and location of an object independent of the identification of the object.

[0028] The RFID reader system 106 can read digital data encoded in RFID tags (e.g., worn by an individual or placed on a specific object to track the individual or object). When an RFID tag enters the vicinity of the RFID reader, the RFID reader can provide an excitation signal to the RFID tag such that the RFID tag emits radiation that can be judged by the RFID reader to identify a specific RFID tag. Since each RFID tag has its own unique identification information, the RFID reader system 106 can efficiently and effectively identify the target being tracked. However, the RFID reader system 106 requires the RFID tag to be placed relatively closely to the RFID reader, resulting in a smaller coverage for implementing multiple RFID readers and / or significant hardware costs.

[0029] Similar to the LIDAR system 104, the ToF system 107 (e.g., a three-dimensional time-of-flight sensor system) can track individuals, groups, and / or objects by illuminating the target with pulsed light and measuring the characteristics of the reflected pulses. Specifically, the ToF system 107 can emit infrared light pulses and measure the time corresponding to the return of the pulses. The ToF system 107 can also map textures (e.g., skin texture) to identify individuals, groups, and / or objects. Thus, the ToF system 107 can obtain the three-dimensional positioning and texture attributes of the tracked identity. Another benefit is that since the ToF system 107 may not rely on visible lighting conditions, the ToF system 107 may not cause lighting condition limitations, which may be a characteristic of some camera-based vision acquisition systems. However, a redundant system may still be useful because the ToF system 107 may be less effective and less accurate in certain environmental conditions (such as on a rainy day).

[0030] The computer vision system 108 can receive camera data (e.g., still images and / or videos) for context analysis. For example, the camera data can be analyzed to perform object recognition and / or tracking. Using facial recognition, the computer vision system 108 can identify the target being tracked. Unfortunately, however, the computer vision system 108 can be quite expensive while typically covering a limited tracking area. In addition, the computer vision system 108 may take a considerable amount of time to analyze computer images.

[0031] A millimeter wave (mmWave) system 109 (e.g., a millimeter wave radar sensor system) can provide a large bandwidth to verify the presence of an identity being tracked. Specifically, the millimeter wave system 109 can allow for the transmission of high-rate data with low latency. For example, the millimeter wave system 109 can include devices that emit and / or receive millimeter waves to communicate with one or more computing devices (e.g., wearable devices) associated with an individual in order to quickly identify the individual (or verify an identity proposed by the individual). Additionally, the millimeter wave system 109 may be able to maintain tracking of the identity being tracked even through surfaces (e.g., physical barriers that are transmissive to radio waves, such as windows and walls). However, the millimeter wave system 109 may utilize components that are relatively close to the identity being tracked, resulting in a smaller coverage area.

[0032] As can be understood, each tracking system has its trade-offs. Therefore, it may be desirable to use a combination of tracking systems such that the benefits of each of the various tracking systems can be used in conjunction with one another. To this end, the task of the context tracking system 102 is to maintain and / or trade tracking data from each of the various tracking systems. For example, the context tracking system 102 can receive positioning information from the LIDAR system 104, object identifiers obtained from RFID tags in the vicinity of one or more RFID reader systems 106, and / or object identity and / or positioning from the computer vision system 108.

[0033] Using this information, object tracking within an open environment can be obtained more efficiently and effectively. Figure 2 is a schematic diagram of an open environment 200 of a system using Figure 1 in accordance with an embodiment of the present disclosure. Although Figure 2 illustrates a theme park environment, the current discussion is not intended to limit the application of the context tracking system to such an embodiment. In fact, the current technology can be used in a variety of environmental applications.

[0034] As illustrated, the open environment 200 can have many individual coverage areas 202A, 202B, 202C, and 202D, each of which is tracked by one or more sensor systems. For example, the coverage area 202A is a parking lot tracked by the computer vision system 204. The coverage area 202B is the area between the entrance gate 206 and the attraction 208. The coverage area 202B has a LIDAR system 210, an RFID reader system 212, and a millimeter wave system 213. The coverage area 202C is tracked by the second LIDAR system 214 and the ToF system 215. The coverage area 202D is tracked using the second millimeter wave system 217.

[0035] The context tracking system 216 is communicatively coupled to the various tracking systems within the open environment 200. Similar to Figure 1For the context tracking system 102, the context tracking system 216 can maintain and / or trade tracking data between tracking systems. For example, the computer vision system 204 can detect a specific vehicle 219 in the coverage area 202A. The computer vision system 204 can analyze the visual image of the vehicle 219 for the identification of the vehicle 219. For example, the computer vision system 204 can identify the alphanumeric characters of the license plate of the vehicle 219 for the identification of the vehicle 219.

[0036] The identified object can be used to identify other objects in the context tracking system 216. For example, the identification of the vehicle 219 can be provided to the context tracking system 216. For example, the vehicle 219 can be identified as corresponding to one or more persons (e.g., a group of persons) whom the computer vision system 204 has detected leaving the vehicle 219 at a specific time. Based on this information, the context tracking system 216 can determine that one or more persons are likely to be in the coverage area 202A. In addition, the identity of the person or group based on the computer vision system 204 can be used to determine the identity of the person or group in another coverage area. In particular, the context tracking system 216 can record the unique characteristics of one or more persons identified in the coverage area 202A and can use the recorded characteristics to determine the identity of the tracked objects in other coverage areas (such as the coverage area 202B and the coverage area 202D).

[0037] By providing such context to the context tracking system 216 and / or other tracking sensors in the coverage area, the tracking analysis can better understand the possible candidate objects that will approach. For example, if the computer vision system 204 provides an indication that vehicle A is in the parking lot (or that person A or group A associated with vehicle A is likely to be in the parking lot [e.g., based on the presence of vehicle A in the parking lot]) to track the sensors in the adjacent coverage areas, the tracking sensors in these adjacent coverage areas can "pre-heat" using the data identifying the possible object identifiers. Thus, in some embodiments, at least in part depending on the identification provided by the computer vision system 204, a larger coverage system (such as the LIDAR system 210) with less object identification capabilities can be used to track the location of the user. In addition, as briefly pointed out above, the identified object can be used to identify other objects, such as a group of persons. In fact, it may be beneficial for the context tracking system 216 to track groups. For example, the vehicle 219 can be identified as corresponding to a group of persons (e.g., group A). In this case, the context tracking system 216 can determine that all the persons leaving the vehicle 219 are associated with each other and thus can include group A. As will be discussed later, other methods of identifying and determining groups are possible.

[0038] If only one user is associated with Vehicle A, then the single identified object leaving the vehicle is likely the associated user. However, in some instances, the context provided from a previous tracking system may not meet a threshold level of likelihood. In such cases, additional tracking can be selectively enabled to identify a specific object / user. For example, another computer vision system 218 at the entry door 206 is used to identify the object / user. Since the entry door 206 is a funnel-in location with direct viewing access to the desired object / user, this location can be the primary location for the computer vision system 218. In such an embodiment, in the case where an individual object / user is detected by the computer vision system 218, the object recognition analysis performed by the computer vision system 218 may be greatly influenced by the context data provided by the computer vision system 204 via the context tracking system 216. For example, possible candidates for identification can be filtered out based on the data provided by the computer vision system 204, resulting in a more efficient and faster processing of object / user identification. Additionally, the millimeter wave system 213 in the coverage area 202B can also be used to identify or verify the presence of an individual object / user. In fact, the millimeter wave system 213 can work together with the computer vision system 218 in identifying an object. For example, in the case of using the context data provided by the computer vision system 204 via the context tracking system 216, the millimeter wave system 213 can be used as a second mechanism for filtering possible candidates to increase the accuracy and processing speed of object identification.

[0039] In some embodiments, the context tracking system 216 can use location information to infer / predict the identification of an object / user. For example, if the RFID reader system 212 indicates that Person A enters the scenic spot 208 at 12:00 noon and the scenic spot 208 has an exit point that is typically reached within 10 minutes, then the second LIDAR system 214 can infer / predict that the object reaching the exit point at 12:10 is likely to be Person A. Thus, a tracking system that provides detailed identification of the object / user may not be necessary at the exit of the scenic spot 208, resulting in a more efficient use of resources. Additionally, the tracking system does not necessarily need to be in the line of sight of the object / user. In fact, as illustrated in the coverage area 202D, the second millimeter wave system 217, although not in the line of sight of the object located in the restaurant 221, is positioned to track the object in the restaurant 221. The second millimeter wave system 217 can be capable of this type of communication because certain materials associated with the restaurant 221 and / or the object being tracked (e.g., fabric, fiberglass reinforced plastic, etc.) may be transmissive (e.g., radio frequency transmissive) to the radiation emanating from the second millimeter wave system 217.

[0040] As will be discussed in more detail below, tracking information (e.g., identification and location of an object / user) can be used for many different purposes. For example, in one embodiment, kiosk 220 can provide specific information useful to the identified object / user when a particular object / user is tracked to kiosk 220. Additionally, in some embodiments, such tracking can help theme park personnel understand object / user interests based on location (where the object / user is tracked to such location (e.g., attractions, restaurants, etc.)).

[0041] The basic utility of the context tracking system has been discussed, Figure 3 FIG. 5 is a flow chart illustrating a process 300 for identifying and maintaining a tracking context in accordance with an embodiment. Process 300 begins with selecting a tracking target (block 302). The tracking target can be selected based on one or more criteria of an object observed in the environment and can vary by coverage area. For example, for the open environment 200, coverage area 202A may be particularly interested in tracking vehicles in a parking lot. Thus, the tracking target can be selected based on a range of object motion speeds, object sizes, object shapes, etc. attributable to a vehicle. In contrast, it can be assumed that there are no vehicles in coverage area 202B, but rather people associated with the vehicles. Thus, the criteria for selecting a tracking object in coverage area 202B can be a range of object motion speeds, object sizes, object shapes, etc. attributable to a person or group of people.

[0042] Process 300 continues by identifying the tracking target identity (block 304). For example, as mentioned above, the identity can be determined based on computer vision analysis of the object (e.g., at entry gate 206), based on other identified objects (where the relationship between the other identified objects and the tracked target is recorded (e.g., in a tangible storage device of context tracking system 216, etc.)).

[0043] Once the identity is determined, the location and / or position of the tracking target is identified (block 306). This information in combination with the identity can be useful to other tracking sensor systems, enabling the tracking sensor systems to filter in or filter out a particular identity (e.g., a subset of identities) as a possible identity of the objects they track. For example, as mentioned above with respect to Figure 2 the identity and location indicating that person A is entering an attraction along with a duration estimate of the length of the attraction from start to end can enable the tracking system at the exit covering the attraction to filter in person A as a possible identity of an object that may be detected at or near the end of the duration estimate at the exit point of the attraction. Additionally, if person A is in the attraction, then person A is clearly not in the parking lot either. Thus, the tracking system sensors in the parking lot can filter out person A as a candidate identity, resulting in a faster response time for the identification analysis.

[0044] As can be appreciated, the tracking target context is maintained and / or processed (block 308) by the context tracking system. As mentioned herein, the tracking target context can include the observation activities of a particular tracking sensor system, such as tracking time, tracking location, tracking identity, tracking identities associated with the object being tracked, and the like. In some embodiments, the context tracking system can maintain context data in some embodiments and also filter in or filter out candidates based on the context data, and thus provide the filtering result to the tracking system for efficient identification analysis. In other embodiments, the context tracking system can provide context to the tracking system, enabling the tracking system to perform filtering and filtering-based identification.

[0045] Figure 4 FIG. 400 is a flow chart illustrating a process 400 for using the acquired context to identify the context at a subsequent tracking sensor. Process 400 begins with receiving the tracking target context (or filtering information if the tracking target context is maintained at the context tracking system) (block 402). Once received, the tracking sensor can use the filtering in and / or filtering out techniques discussed above to determine the identity of the object (block 404).

[0046] Perform a target control action based on the subsequent tracking identifier (block 406). For example, as mentioned above, a kiosk can be controlled to display specific information useful to the identified person. In other embodiments, access to a specific restricted portion of the environment can be granted based on determining that the identity is authorized to access the restricted portion of the environment. Additionally, in some embodiments, metric tracking can be maintained for subsequent business analysis, research, and / or reporting, such as access to a specific area, recorded user activities, and the like.

[0047] As mentioned above, sometimes a specific confidence threshold level may be required for a positive match of the identity with the object being tracked. Figure 5 FIG. 500 is a flow chart illustrating a process 500 for using a confidence interval to determine sufficient context for target identification. Process 500 begins with collecting tracking sensor input (e.g., context) (block 502). As mentioned above, the context tracking system can receive tracking sensor input that provides identity, location, and / or other information that provides possible context for the object of observation of other tracking sensors.

[0048] Based on these tracking inputs, one or more tracking identities of the observed object can be predicted by other tracking sensors (block 504). For example, as mentioned above, context information indicating that a specific person A has entered a scenic spot can be used to assume that person A will likely leave the scenic spot at a predicted time based on the attributes of the person and / or the attributes of the scenic spot. For example, if the person is observed by the tracking system to have a below-average or above-average movement speed, the average dwell time of the scenic spot can be adjusted downwards or upwards accordingly. This adjusted dwell time can be used to determine when person A is likely to leave the scenic spot, enabling the tracking system at the exit of the scenic spot to predict that the person leaving the scenic spot at the adjusted dwell time is person A.

[0049] In some embodiments, a prediction confidence score can be calculated. The prediction confidence score can indicate the likelihood that the tracked object has a specific identity. For example, if based on known information, the target is very likely to be person A, the prediction confidence score may be greater than the prediction confidence score when the target is only somewhat likely or unlikely to be person A (based on known information). The prediction confidence score can vary based on multiple factors. In some embodiments, redundant information (e.g., similar identifications using two or more sets of data) can increase the prediction confidence score. Additionally, observable characteristics of the tracked object can be used to influence the prediction confidence score. For example, a known size associated with an identity can be compared to the size of the tracked target. The prediction confidence score can increase based on the proximity of the known size and the size of the target observed by the tracking sensor.

[0050] After obtaining the prediction confidence score, a determination is made as to whether the prediction confidence score meets a score threshold (decision block 508). The score threshold can indicate the minimum score that can result in an identity being associated with the tracked target.

[0051] If the prediction confidence score does not meet the threshold, additional tracking inputs can be obtained (block 510). To obtain additional tracking inputs, tracking sensors in an open environment can continue to accumulate tracking information and / or obtain context data of the tracked target. In some embodiments, the tracked target can be encouraged to move towards a specific sensor to obtain new tracking inputs about the tracked target. For example, Figure 6FIG. 0 is a schematic diagram illustrating an example transfer scenario 600 for increasing tracking input according to an embodiment. In scenario 600, the predicted confidence score of the tracked target 602 is less than a threshold, as indicated by balloon 604. Accordingly, the electronic display 606 can be controlled to display a message that guides the target towards a location (here, pizza store 608) where additional tracking input can be obtained. Here, the target is encouraged to move towards pizza store 608 by providing an electronic notification via an electronic billboard that guides the target to pizza store 608. At the entrance to pizza store 608, there are additional tracking sensors (e.g., RFID reader 610). When the target 602 moves near pizza store 608, the tracking sensors obtain additional tracking input of the target 602.

[0052] Return Figure 5 , when additional tracking input is collected, an additional prediction of the identity of the target is made (block 504), a new predicted confidence score using the newly collected additional tracking input is determined (block 506), and an additional determination as to whether the predicted confidence threshold is met is made (decision block 508). The process can continue until the predicted confidence threshold is met.

[0053] Once the predicted confidence threshold is met, an identifier can be attributed to the tracked object (block 512). For example, return Figure 6 , the additional tracking information received via RFID reader 610 causes the predicted confidence score to increase to a level greater than or equal to the threshold, as indicated by balloon 612. Accordingly, the tracked target 602 (e.g., the tracked object) is attributed to identifier 614 (e.g., Person A).

[0054] As mentioned above, the context provided by one tracking sensor system can be useful for the analysis by other tracking sensor systems. For example, candidate predicted identities can be "filtered in" (e.g., whitelisted) or "filtered out" (e.g., blacklisted) based on the context provided by another tracking sensor system. Figure 7 FIG. 15 is a flow chart illustrating a process 700 for filtering possible identification predictions based on provided sensor context according to an embodiment.

[0055] Process 700 begins by receiving context data from other sensor systems (block 702). For example, the context data can include the identity of the tracked object, the location of the tracked object, and a timestamp indicating when the tracked object was at a particular location.

[0056] Context data can be used to supplement the tracking functionality of a tracking sensor. For example, the possible candidate identities of newly observed tracked objects can be filtered (e.g., filtered in or filtered out) based on context information indicating the positions of the identified tracked objects provided by other tracking sensors (block 704). For example, if a parking lot tracking sensor indicates that a target object with the identity of Person A was tracked in the parking lot five minutes ago, and it takes 20 minutes to travel from the parking lot to the coverage area associated with the tracking sensor that observed the new target object, then the identity of Person A can be filtered out as a possible candidate identity for the new target object because it would be infeasible for Person A to reach the coverage area from the parking lot in 5 minutes. Additionally and / or alternatively, the filtering in of candidate identities can occur by obtaining context data that indicates which identities may have reached the coverage area at the time the new target object was observed. These identities can be whitelisted as possible candidate identities for the newly observed target object in the coverage area. This can save processing time and improve the operation of the computer system employed according to the disclosed embodiments.

[0057] Once the candidate identities are filtered, the tracking system can predict the identity of the new tracked object from the filtered candidate identities (block 706). For example, the identity can be selected from a list of unfiltered identities (e.g., not on a blacklist) and / or can be selected from a list of filtered-in identities (e.g., on a whitelist). As can be appreciated, this can enable more efficient object tracking because the context information can reduce the number of candidate identities to be considered for identifying the tracked object. Additionally, in some instances, identity tracking may not be possible without such context. For example, certain LIDAR systems may not be able to perform the identification of target objects without such context. Thus, the current techniques provide more efficient and enhanced tracking capabilities than existing tracking systems.

[0058] Enhanced context tracking techniques have been discussed, Figure 8is a schematic diagram illustrating an example control action 800 according to an embodiment, which can be implemented based on an identity tracked by means of the context tracking techniques provided herein. In one embodiment, the kiosk 802 or other device may include a display 804. The display 804 may present a graphical user interface (GUI) 806 that provides information about the open environment. As illustrated, the GUI 806 may be personalized based on the tracked identity. For example, as illustrated by the status balloon 808, the context tracking system may identify that the tracked object is "James" based on the context data trade-off between various target sensors. In response to the tracked object approaching the display 804, a control action command may be presented to the display 804 (e.g., the underlying hardware controlling the display 804), which instructs the GUI 806 to display information related to the identified person. For example, a personalized waiting time associated with James is provided here. As can be understood, each identity may be associated with a unique identifier and other information (e.g., waiting time, demographic information, and / or other personalized data) to enable personalized control actions to be performed. Additionally, as will be discussed with respect to Figure 12 the control actions may be implemented based on identities corresponding to groups.

[0059] In another embodiment, the wearable device 810 may be disposed on the tracked object 812. Based on the interaction with the open environment feature 814 (here a virtual game mystery box), the personalized data associated with the identity of the tracked object may be updated. For example, the game database 816 here may be updated by means of a control action command to reflect the updated game state based on the interaction with the open environment feature 814. Thus, as illustrated in the database record 818, an additional 5 points are provided to the identity 123 associated with James in the database. Additionally, the display 820 of the wearable device 810 may be controlled by means of a control action command to display an indication 822 that confirms the state change based on the interaction of the tracked object 812 with the open environment feature 814.

[0060] As mentioned above, the context tracking system may identify and track groups. Although specific data patterns indicating the identification and / or tracking of groups are discussed below, these are merely examples. As can be understood, using machine learning, additional data patterns indicating grouping may be observed by the machine learning systems described herein. Figure 9 is a flowchart of a method 900 for grouping individuals using machine learning techniques. One or more steps of the method 900 may be performed by Figure 1 the context tracking system 102 or Figure 2executed by one or more components of the context tracking system 216. In fact, a grouping system (e.g., a machine learning system) can be integrated into Figure 1 the context tracking system 102 or Figure 2 the context tracking system 216. In particular, the grouping system can receive training data (block 902) indicating the (one or more) patterns of an active group. For example, the patterns indicating an active group can include the proximity level between one or more individuals within a certain amount of time, an indication of the association of a vehicle with one or more individuals, etc. In some embodiments, the (one or more) patterns can include an indication of an association, such as a presumed child and one or more individuals (e.g., an adult) based on the height of the tracked individual. Based on the received training data, at block 904, the grouping system can become proficient in detecting the presence of an active group. In particular, the grouping system can identify the (one or more) patterns in the raw data indicating an active group in the environment (e.g., the open environment 200).

[0061] In some embodiments, time series data can be mined for patterns to identify and track groups. The time series data can provide data over a period of time, which can help machine learning logic identify patterns of people over time. These patterns can be used to identify and / or track groups based on the activities / patterns observed over time. For example, in an identification example, when a group of users participates in a specific common activity or a common activity for a threshold amount of time, they can be identified as part of a group, which can be observed as a pattern from the time series data.

[0062] In another example, the machine learning system can also be equipped to recognize that not all members of a group may be within a certain proximity to each other. For example, one or more individuals of an active group may go to the restroom while one or more individuals of the same active group do not. In this case, the grouping system can retain the identities of all members of the active group and / or determine subgroups of the active group, which can correspond to one or more individuals within the active group. In fact, consistent with the example above, the first subgroup of the active group can be the members of the active group who go to the restroom, while the second subgroup can be the members of the active group who do not go to the restroom while the first subgroup is in the restroom. Additionally, an active group can correspond to a group that is retained in the database of detected groups for a specific period of time. For example, as long as one or more members of the group are present in Figure 2 the open environment 200 for a certain duration, the active group can remain active.

[0063] At block 906, the machine learning system can determine whether a person or group (e.g., a subgroup) is at least part of an active group based at least on training data and the identified pattern(s) indicating the active group. For example, the machine learning system can determine that two individuals who have spent the last hour together are a group. As another example, the grouping system can group individuals who show affection for each other (such as holding each other's hands). In fact, the grouping system can employ weighted probabilities to determine whether one or more individuals should be grouped. Specifically, one or more behavioral patterns, characteristics, and / or attributes observed for certain tracked identities can be weighted with probabilities and used to determine groups. For example, the grouping system can attach a weighted probability (e.g., a value between zero and one) of being in a group to an individual of small stature because typically an infant or toddler may not be without adult supervision. However, the grouping system can employ weighted probabilities for other attributes and / or behaviors of an individual (such as common location, proximity, association with the same vehicle, etc.). The grouping system can sum the weighted probability values of being in a group and can separately determine whether the individual(s) associated with the weighted probability value(s) is part of the group or forms a group. In some embodiments, one of the weighted probability values may be sufficient to exceed a threshold to separately determine whether the individual(s) associated with the weighted probability value(s) is part of the group or forms a group.

[0064] At block 908, the grouping system can update the database including the active groups. When a newly identified group is determined, the database including the active groups can be updated to allow tracking. Thus, the machine learning system can maintain a record of who is grouped. In some embodiments, the grouping may affect downstream processing. For example, as will be discussed below, group tracking can even be utilized in a gaming experience.

[0065] As mentioned herein, the grouping system can identify groups of individuals based on activities / patterns observed in time series data (e.g., as captured by sensors of a sensor system). Any number of patterns can be identified by a machine learning algorithm, which can implement supervised machine learning to train the grouping system on how to identify groups. For the sake of discussion, Figure 10Is a schematic diagram 1000, which provides an example of a set of individuals that a grouping system using the techniques described herein may consider likely to be a group and / or unlikely to be a group. Specifically, as shown in illustration 1002, individuals leaving or entering the same vehicle may be considered likely to be grouped by the grouping system. In contrast, as shown in illustration 1004, individuals leaving or entering different vehicles (with a relatively long distance between the vehicles) may be considered unlikely to be grouped. Additionally, as shown in illustration 1006, individuals who are close to each other for a significant amount of time may be grouped. However, individuals who are spaced apart in a queue at an attraction and may exceed a specific threshold distance may be considered less likely to be grouped. For example, in illustration 1008, each person 1011 represents 20 individuals lined up in a row. As shown in illustration 1008, individual 1010 is separated from another individual 1012 in the row at the attraction by one hundred people. Therefore, the grouping system may determine that individual 1010 and individual 1012 should not be grouped. However, in some cases, the grouping system may determine that individuals are likely to be grouped even if they are spaced apart. As shown in illustration 1014, even though person 1016 is separated from persons 1018, 1020, persons 1016, 1018, and 1020 may be considered likely to be grouped. In this case, persons 1018, 1020 will be walking towards the restroom 1022, while person 1016 is waiting outside. Even though person 1016 may be alone for some time, the grouping system may determine that persons 1016, 1018, 1020 are a group. Additionally, the grouping system may also group separated individuals, but the time they spend together exceeds a certain threshold time period. For example, people in a group may choose to sit in a rest area rather than wait in the queue at an attraction with another person in the same group. Therefore, the grouping system may keep the group active for a period of time even if one or more members of the group are separated.

[0066] As can be understood, once a group is identified, additional functionality can be implemented. For example, Figure 11FIG. 1100 illustrates how groups identified by a grouping system can change an interactive environment of an interactive game. In FIG. 1100, it is necessary to comply with the rule 1102 on the display 1103 to unlock the door 1104 (e.g., a portal in a virtual and / or augmented reality experience, etc.). The rule 1102 stating that "all team members must enter together" must be satisfied to enter the door 1104. Here, the grouping system has identified persons 1114 and 1112 as part of the group / team with person 1110. However, these persons 1114 and 1112 are not present with person 1110. Since not all team members were present with team member 110 at time 1111, the door was not unlocked. Later, at time 1113, all team members (e.g., persons 1110, 1112, 1114) appear at the door 1104. In response to the presence of the entire group, an effect manifests from the interactive environment (e.g., the door 1104 unlocks). Thus, by tracking the group, one or more components of the interactive environment can be controlled based on group aspects (e.g., the location of team members, the cumulative activities of team members, etc.), thereby providing an interesting and interactive experience for guests.

[0067] Figure 12 FIG. 1200 is a schematic diagram illustrating an example control action according to an embodiment, which example control action 1200 can be implemented based on a group identity tracked by means of the context tracking techniques provided herein. Similar to Figure 8 , Figure 12 it includes a kiosk 1202 or another device that may include a display 1204. The display 1204 can present a graphical user interface (GUI) 1206 that provides information about the open environment. As illustrated, the GUI 1206 can be personalized based on the tracked identity. For example, as illustrated by the status balloon 1208, the context tracking system can identify that the tracked object is part of the tracked group "Group 1" based on the context data trade-off between various target sensors. In response to the tracked object approaching the display 1204, a control action command can be presented to the display 1204 (e.g., controlling the underlying hardware of the display 1204), which control action command instructs the GUI 1206 to display information related to the identified group. For example, here instructions associated with its identity are provided to Group 1. Specifically, the GUI 1209 displays "Hello Group 1, complete the task in less than 10 minutes." As can be understood, each group identity can be associated with a unique identifier and other information (e.g., wait time, instructions, demographic information, and / or other personalized data), thereby enabling personalized control actions to be performed.

[0068] In another embodiment, each member of the tracked group 1212 (i.e., Group 1) may have a wearable device 1210 communicatively coupled to one or more components of the context tracking system. Based on the interaction with the open environment feature 1214 (here, a virtual game mystery box), the personalized data associated with the identity of the tracked group may be updated. For example, here the game database 1216 may be updated by means of a control action command to reflect the updated game state based on the interaction with the open environment feature 1214. Thus, as illustrated in database record 1218, an additional 15 points are provided to the identity 456 associated with Group 1 in the database. Additionally, the display 1220 of the wearable device 1210 on each of the group members may be controlled by means of a control action command to display an indication 1222 that confirms a state change based on the interaction of each member of the tracked group 1212 with the open environment feature 1214.

[0069] The technologies presented and claimed herein are cited and applied to substantial objects and specific examples of a practical nature, which demonstrably improve the art, and are thus not abstract, intangible, or purely theoretical. Additionally, if any claim appended to the end of this specification contains one or more elements designated as "means for [performing]... [function]" or "step for [performing]... [function]", such elements are intended to be interpreted in accordance with 35 U.S.C. 112(f). However, for any claim that contains elements designated in any other way, such elements are not intended to be interpreted in accordance with 35 U.S.C. 112(f).

Claims

1. A tangible non-transitory machine-readable medium comprising machine-readable instructions that, when executed by one or more processors of a machine, cause the machine to: Receive a tracked target context of a first tracked object from a first tracking sensor system that tracks a first coverage area; Provide the tracked target context from the first tracking sensor system to a second tracking sensor system that tracks a second coverage area different from the first coverage area, wherein the second tracking sensor system is different from the first tracking sensor system; And Cause the second tracking sensor system to identify a newly observed tracked target based on the tracked target context from the first tracking sensor system by: Determining a subset of candidate identities observed at a previous location outside an identified position range predicted to be reachable from a set of candidate identities based on a time difference between a time of observing the set of candidate identities at the previous location and a time of observing the newly observed tracked target by the second tracking sensor system, and filtering out from the set of candidate identities the subset of candidate identities that the newly observed tracked target can be identified as based on the tracked target context from the first tracking sensor system; And Provide the set of candidate identities without the subset to the second tracking sensor system.

2. The machine-readable medium of claim 1, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: Filter out a first subset of the set of candidate identities that the newly observed tracked target cannot be identified as from the set of candidate identities based on the tracked target context from the first tracking sensor system; and Identify the identity of the newly observed tracked target from the set of candidate identities without the first subset via the second tracking sensor system.

3. The machine-readable medium of claim 1, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: Filter in a second subset of the set of candidate identities that the newly observed tracked target can be identified as based on the tracked target context from the first tracking sensor system; and Provide the second subset to the second tracking sensor system.

4. The machine-readable medium of claim 3, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: Determine the second subset as part of the set of candidate identities tracked at the previous location based on a time difference between a time of tracking the set of candidate identities at a previous location within the identified position range and a time of observing the newly observed tracked target by the second tracking sensor system.

5. The machine-readable medium according to claim 3, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: generate a whitelist based on the second subset; and provide the whitelist to the second tracking sensor system.

6. The machine-readable medium according to claim 1, wherein the first tracking sensor system, the second tracking sensor system, or both comprise: a light detection and ranging (LIDAR) system, a radio frequency identification (RFID) system, a computer vision system, a time-of-flight (ToF) system, a millimeter wave (mmWave) system, or any combination thereof.

7. The machine-readable medium according to claim 6, wherein the second tracking sensor system includes the LIDAR system.

8. The machine-readable medium according to claim 1, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: determine a predicted confidence score indicative of a confidence level of the identified newly observed tracked target; and collect additional tracking sensor system inputs for another identification of the newly observed tracked target in response to the predicted confidence score failing to meet a confidence threshold.

9. The machine-readable medium according to claim 8, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to collect the additional tracking sensor system inputs by providing a direction for the newly observed tracked target to move forward into a tracking sensor system coverage area.

10. The machine-readable medium according to claim 9, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to provide encouragement by motivating the newly observed tracked target to move forward into the tracking sensor system coverage area.

11. The machine-readable medium according to claim 1, wherein the first tracked object includes an object type different from the newly observed tracked target.

12. The machine-readable medium according to claim 11, wherein the first tracked object includes a vehicle and the newly observed tracked target includes one or more persons.

13. The machine-readable medium according to claim 1, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: receive training data indicative of a group of persons and associated attributes; identify patterns in the associated attributes to identify grouping attributes indicative of persons that should be grouped; determine whether the first tracked object is associated with the pattern; and in response to determining that the first tracked object is associated with the pattern, identify the first tracked object as part of an active group.

14. The machine-readable medium according to claim 13, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: determine whether a second tracked object is a member of the active group based on the pattern by: Determine the amount of time that the first tracked object and the second tracked object have spent in a threshold proximity to each other; and When the amount of time exceeds a threshold, associate the second tracked object with the active group.

15. The machine-readable medium according to claim 13, comprising machine-readable instructions that, when executed by one or more processors of the machine, cause the machine to: Based on the pattern, determine whether a second tracked object is a member of the active group by: Associate a first weighted probability with the second tracked object, where the first weighted probability represents the likelihood that the second tracked object is a member of the active group; Associate a second weighted probability with the second tracked object, where the second weighted probability represents a second likelihood that the second tracked object is a member of the active group; and When the sum of the first weighted probability and the second weighted probability exceeds a threshold, associate the second tracked object with the active group.

16. The machine-readable medium according to claim 15, wherein the first weighted probability is a value representing the amount of time that the second tracked object has spent near the first tracked object.

17. A computer-implemented tracking method, comprising: Receive tracking sensor system input from a first tracking sensor system that tracks a first coverage area, the tracking sensor system input including the tracked target context of a first tracked object; Provide the tracked target context from the first tracking sensor system to a second tracking sensor system that tracks a second coverage area different from the first coverage area, where the second tracking sensor system is different from the first tracking sensor system; And Cause the second tracking sensor system to identify a newly observed tracked target based on the tracked target context from the first tracking sensor system by: Determine a subset of candidate identities observed at a previous location outside the predicted reachable identification location range based on the time difference between the time of observing a set of candidate identities at the previous location and the time of observing the newly observed tracked target by the second tracking sensor system, and filter out from the set of candidate identities the subset of candidate identities that the newly observed tracked target can be identified as based on the tracked target context from the first tracking sensor system; And Provide the set of candidate identities without the subset to the second tracking sensor system.

18. The computer-implemented method according to claim 17, comprising: Determine a predicted confidence score indicating the confidence level of the identification of the newly observed tracked target; And Collect additional tracking sensor system input for another identification of the newly observed tracked target in response to the predicted confidence score failing to meet a threshold.

19. The computer-implemented method according to claim 17, wherein the first tracked object and the newly observed tracked target comprise a first group of individuals.

20. A tracking system, comprising: a first tracking sensor system configured to track a first tracked object in a first coverage area; a second tracking sensor system configured to track a second tracked object in a second coverage area different from the first coverage area; and a context tracking system configured to: receive a tracked target context of the first tracked object from the first tracking sensor system; provide the tracked target context from the first tracking sensor system to the second tracking sensor system different from the first tracking sensor system; and cause the identification of the second tracked object based on the tracked target context from the first tracking sensor system, wherein the second tracking sensor system is different from the first tracking sensor system, by: determining a subset of candidate identities observed at a previous location outside a range of identified locations predicted to be reachable from the set of candidate identities based on the time difference between the time of observing the set of candidate identities at the previous location and the time of observing the second tracked object by the second tracking sensor system, and filtering from the set of candidate identities the subset of candidate identities that the second tracked object can be identified as, based on the tracked target context from the first tracking sensor system; and providing the set of candidate identities without the subset to the second tracking sensor system.

21. The system according to claim 20, wherein the first tracking sensor system comprises a different type of sensor than the second tracking sensor system.

Images