Tracking aggregation and alignment

Abstract

To provide systems and methods that provide contextual tracking information to tracking sensor systems to provide accurate and efficient object tracking.SOLUTION: In an open environment 200 that uses a multi-sensor tracking system, contextual data provided by a computer vision system 204 via a contextual tracking system 216 in a coverage zone 202A is used to identify tracked objects in different coverage zones 202B, 202C, and 202D.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and the benefit of U.S. Provisional Application No. 62 / 828,198, entitled "Track Aggregation and Alignment," filed April 2, 2019, which is incorporated herein by reference in its entirety for all purposes.

[0002] The present disclosure relates generally to tracking systems, and more particularly, to the aggregation and handoff of tracking system data between tracking systems to facilitate more efficient and effective tracking of objects within an environment. [Background technology]

[0003] In the digital age, with the proliferation of digital sensors, object tracking has become increasingly desirable. Unfortunately, in large / open environments, tracking users is an extremely challenging prospect, especially when precise location and activity tracking is desired. As used herein, an open environment refers to an area in which tracked objects can move in multiple directions with relatively little confinement. For example, such environments can include amusement parks, airports, shopping malls, or other relatively large environments that may have multiple tracking coverage zones. Accurately tracking unique individuals is difficult, especially in open environments, as well as situations where crowd density presents obstacle challenges and one individual may block another.

[0004] This section is intended to introduce the reader to various aspects of art that may be related to various aspects of the present technology, which are described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art. Summary of the Invention

[0005] Certain embodiments within the scope of the originally claimed invention are summarized below. These embodiments are not intended to limit the scope of the disclosure; rather, these embodiments are intended only to provide a brief summary of certain disclosed embodiments. Indeed, the disclosure may encompass a variety of forms that may be similar to or different from the embodiments set forth below.

[0006] Embodiments described herein relate to tracking systems that efficiently aggregate and / or communicate tracking data between tracking sensors, allowing the context of one tracking sensor to enhance the tracking of other tracking sensors. More specifically, context information (e.g., location, time, identity of tracked object) determined by one tracking sensor can be used to facilitate more efficient and / or effective tracking of other sensors. For example, such context information can increase the confidence in tracked object identity and provide efficient filtering of possible identities that may be attributed to tracked objects. This can improve processing efficiency and enable more fine-grained tracking of objects in open environments.

[0007] By way of example, in a first embodiment, a tangible, non-transitory machine-readable medium includes machine-readable instructions that, when executed by one or more processors of the machine, cause the machine to receive tracking target context of a first tracked object from a first tracking sensor system; provide tracking target context from the first tracking sensor system to a second tracking sensor system different from the first tracking sensor system; and provide identification of a newly observed tracking target by the second tracking sensor system based on the tracking target context from the first tracking sensor system.

[0008] In a second embodiment, a computer-implemented method includes receiving tracking target context of a first tracked object from a first tracking sensor system, providing the tracking target context from the first tracking sensor system to a second tracking sensor system different from the first tracking sensor system, and providing an identification of a newly observed tracking target by the second tracking sensor system based on the tracking target context from the first tracking sensor system.

[0009] In a third embodiment, a system includes a first tracking sensor system, a second tracking sensor system, and a context tracking sensor system. The first tracking sensor system tracks a first tracking object in a first coverage area. The second tracking sensor system tracks a second tracking object in a second coverage area. The context tracking system receives tracking target context for the first tracking object from the first tracking sensor system, provides tracking target context from the first tracking sensor system to a second tracking sensor system different from the first tracking sensor system, and provides identification of the second tracking object based on the tracking target context from the first tracking sensor system. [Brief explanation of the drawings]

[0010] These and other features, aspects, and advantages of the present disclosure will become better understood from the following detailed description when taken in conjunction with the accompanying drawings, in which like reference numerals refer to like elements throughout.

[0011] [Figure 1] FIG. 1 is a schematic diagram illustrating a multi-sensor tracking component with a context tracking system according to one embodiment of the present disclosure.

[0012] [Figure 2] FIG. 2 is a schematic diagram illustrating an open environment using the system of FIG. 1 according to one embodiment of the present disclosure.

[0013] [Figure 3] 1 is a flowchart illustrating a process for identifying a tracking context, according to one embodiment.

[0014] [Figure 4] 10 is a flowchart illustrating a process for identifying a context in a subsequent tracking sensor using the acquired context, according to one embodiment.

[0015] [Figure 5] 1 is a flowchart illustrating a process for using confidence intervals to determine sufficient context for target identification, according to one embodiment.

[0016] [Figure 6] FIG. 1 is a schematic diagram illustrating an exemplary diversion for increasing tracking input, according to one embodiment.

[0017] [Figure 7] 10 is a flowchart illustrating a process for filtering possible discrimination predictions based on a provided sensor context, according to one embodiment.

[0018] [Figure 8] FIG. 1 is a schematic diagram illustrating an exemplary control action based on tracking identity, according to one embodiment.

[0019] [Figure 9] 1 is a flowchart for grouping individuals into groups using machine learning techniques, according to one embodiment.

[0020] [Figure 10] FIG. 1 is a schematic diagram of grouped and ungrouped individuals, according to one embodiment.

[0021] [Figure 11]FIG. 1 is a schematic diagram of a use case for modifying an interaction environment based on compliance rules, according to one embodiment.

[0022] [Figure 12] FIG. 1 is a schematic diagram illustrating an exemplary control action based on tracking identity, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0023] One or more specific embodiments of the present disclosure are described below. These described embodiments are merely examples of the techniques of the present disclosure. Additionally, in an effort to provide a concise description of these embodiments, not all features of an actual implementation may be described herein. It will be appreciated that, as with any engineering or design project, the development of any such actual implementation will require numerous implementation-specific decisions to be made to achieve the developer's particular goals, which may vary from implementation to implementation, including compliance with system-related and business-related constraints. Moreover, it will be appreciated that such a development effort may be complex and time-consuming, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill in the art having the benefit of this disclosure.

[0024] When introducing elements of various embodiments of the present disclosure, the articles "a," "an," and "the" are intended to mean that there are one or more of the elements. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements other than the listed elements. Furthermore, it should be understood that references to "one embodiment" or "embodiments" of the present disclosure are not intended to exclude the existence of additional embodiments that also incorporate the listed features.

[0025] The present disclosure generally relates to tracking systems that accumulate and / or hand off context information for efficient and effective tracking processing. By using the previously determined context of other tracking sensors, an independent tracking sensor can more efficiently determine the identity of a tracked object. With this in mind, FIG. 1 is a schematic diagram illustrating a multi-sensor tracking system 100 with a context tracking system 102, according to one embodiment of the present disclosure. As shown, the multi-sensor tracking system 100 includes multiple tracking sensors, such as one or more light detection and ranging (LIDAR) systems 104, one or more radio frequency identification (RFID) reader systems 106, one or more time-of-flight (ToF) systems 107, one or more computer vision systems 108, and / or one or more millimeter wave (mmWave) systems 109.

[0026] The LIDAR system 104 can track individuals, objects, and / or groups of individuals or objects by illuminating the target with pulsed light and measuring the reflected pulse. The wavelength and time difference between the pulsed light and the reflected pulse can be used to generate a spatial representation of the location of the target individuals, groups, and / or objects. The LIDAR system 104 can cover a large spatial area while detecting objects relatively easily and efficiently. However, the LIDAR system 104 may not be effective at actually identifying the tracked objects and instead may best be used to identify the presence and location of objects independently of the object's identity.

[0027] The RFID reader system 106 can read digital data encoded on RFID tags (e.g., worn by individuals or placed on specific objects to track individuals or objects). When an RFID tag comes into proximity of the RFID reader, the RFID reader can provide an energizing signal to the RFID tag, causing the RFID tag to emit radiant energy that can be interpreted by the RFID reader to identify the specific RFID tag. Because each RFID tag has unique identification information, the RFID reader system 106 can effectively and efficiently identify tracking targets. However, the RFID reader system 106 requires that RFID tags be placed relatively close to the RFID reader, which reduces coverage and / or significantly increases the hardware costs of implementing a large number of RFID readers.

[0028] Similar to the LIDAR system 104, the ToF system 107 (e.g., a three-dimensional time-of-flight sensor system) can track individuals, groups, and / or objects by illuminating a target with pulsed light and measuring the characteristics of the reflected pulse. Specifically, the ToF system 107 can emit a pulse of infrared light and measure the time corresponding to the return pulse. The ToF system 107 can also map texture (e.g., skin texture) to identify individuals, groups, and / or objects. Thus, the ToF system 107 can obtain three-dimensional location and texture attributes of tracked identities. Another advantage is that the ToF system 107 may not be dependent on visible lighting conditions, and therefore is not subject to lighting condition limitations that may characterize some camera-based vision acquisition systems. However, the ToF system 107 may be less effective and accurate in certain environmental conditions, such as on rainy days, so a redundant system may still be useful.

[0029] The computer vision system 108 can receive camera data (e.g., still images and / or video) for context analysis. For example, the camera data can be analyzed to perform object recognition and / or tracking. Using facial recognition, the computer vision system 108 can identify tracking targets. Unfortunately, however, computer vision systems 108 can be quite expensive while often covering a limited tracking area. Furthermore, the computer vision system 108 can take a considerable amount of time to analyze the computer images.

[0030] A millimeter wave (mmWave) system 190 (e.g., a millimeter wave radar sensor system) provides high bandwidth and can authenticate the presence of a tracked identity. Specifically, the mmWave system 190 can enable high-rate data transfer with low latency. For example, the mmWave system 190 can include devices that emit and / or receive millimeter waves to communicate with one or more computing devices (e.g., wearable devices) associated with an individual to quickly identify the individual (or authenticate an identity proposed by the individual). Furthermore, the mmWave system 190 can maintain track of the tracked identity even through surfaces (e.g., radio wave-permeable physical barriers such as windows and walls). However, the mmWave system 109 can utilize components relatively close to the tracked identity, resulting in reduced coverage.

[0031] As can be appreciated, each tracking system has its trade-offs. Therefore, it may be desirable to use a combination of tracking systems, allowing the benefits of the various tracking systems to be used in combination with each other. To do this, the context tracking system 102 is responsible for maintaining and / or trading tracking data from each of the various tracking systems. For example, the context tracking system 102 may receive location information from the LIDAR system 104, object identifiers obtained from RFID tags in proximity to one or more of the RFID reader systems 106, and / or object identities and / or locations from the computer vision system 108.

[0032] This information can be used to obtain more effective and efficient object tracking within the open environment. Figure 2 is a schematic diagram illustrating an open environment 200 using the system of Figure 1, according to one embodiment of the present disclosure. Although Figure 2 illustrates a theme park environment, the present discussion is not intended to limit the application of the context tracking system to such an embodiment. Indeed, the present technology can be used in a variety of environmental applications.

[0033] As shown, open environment 200 can have many separate coverage zones 202A, 202B, 202C, and 202D, each tracked by one or more sensor systems. For example, coverage zone 202A is a parking lot tracked by computer vision system 204. Coverage zone 202B is the area between entrance gate 206 and attraction 208. Coverage zone 202B has a LIDAR system 210, an RFID reader system 212, and an mmWave system 213. Coverage zone 202C is tracked by a second LIDAR system 214 and a ToF system 215. Coverage zone 202D is tracked using a second mmWave system 217.

[0034] The context tracking system 216 is communicatively coupled to various tracking systems in the open environment 200. Similar to the context tracking system 102 of FIG. 1, the context tracking system 216 can maintain and / or exchange tracking data between the tracking systems. For example, the computer vision system 204 can detect a particular car 219 within the coverage zone 202A. The computer vision system 204 can analyze a visual image of the car 219 to identify the car 219. For example, the computer vision system 204 can identify alphanumeric characters on the license plate of the car 219 to identify the car 219.

[0035] The identified object can be used to identify other objects in the context tracking system 216. For example, the identification of a car 219 can be provided to the context tracking system 216. The car 219 can be identified, for example, as corresponding to one or more people (e.g., a group of people) that the computer vision system 204 detected exiting the car 219 at a particular time. Based on this information, the context tracking system 216 can determine that one or more people are likely to be in the coverage zone 202A. Furthermore, the identity of an individual or group based on the computer vision system 204 can be used in determining the identity of an individual or group in another coverage zone. In particular, the context tracking system 216 can record characteristics unique to one or more people identified in the coverage zone 202A and use the recorded characteristics in determining the identity of tracked objects in other coverage zones, such as the coverage zone 202B and the coverage zone 202D.

[0036] By providing such context to the context tracking system 216 and / or other tracking sensors in the coverage zone, tracking analytics can better recognize potentially approaching candidate objects. For example, if the computer vision system 204 provides notification to tracking sensors in nearby coverage zones that Car A is in a parking lot (or that Person A or Group A associated with Car A is likely to be in the parking lot [e.g., based on Car A's presence in the parking lot]), the tracking sensors in these nearby coverage zones can be "preheated" with data identifying possible object identifiers. Thus, in some embodiments, a larger coverage system with fewer object identification capabilities, such as the LIDAR system 210, can be used to track a user's location, relying at least in part on the identification provided by the computer vision system 204. Furthermore, as briefly discussed above, identified objects can be used to identify other objects, such as groups of people. Indeed, it can be beneficial for the context tracking system 216 to track groups. For example, Car 219 can be identified as corresponding to a group of people (e.g., Group A). In this case, the context tracking system 216 may determine that all people exiting the car 219 are related to one another and therefore may comprise group A. As will be explained later, other methods of identifying and determining groups are possible.

[0037] If only one user is associated with car A, a single identified object exiting the car may have a high probability of being the associated user. However, in some cases, the context provided by the preceding tracking system may not meet the threshold level of probability. In such cases, additional tracking may be selectively enabled to identify a specific object / user. For example, another computer vision system 218 at the entrance gate 206 for identifying the object / user. Because the entrance gate 206 is a funnel-shaped location with desirable direct line-of-sight access to the object / user, this location may be the primary location for the computer vision system 218. In such an embodiment in which individual objects / users are detected by the computer vision system 218, the object identification analysis performed by the computer vision system 218 may be significantly influenced by the contextual data provided by the computer vision system 204 via the context tracking system 216. For example, viable candidates for identification may be filtered based on the data provided by the computer vision system 204, resulting in more efficient and faster processing of object / user identification. Furthermore, the mmWave system 213 may also be used in the coverage zone 202B to identify or authenticate the presence of individual objects / users. Indeed, the mmWave system 213 can work in conjunction with the computer vision system 218 in identifying objects. For example, the mmWave system 213 can use contextual data provided by the computer vision system 204 via the context tracking system 216 to act as a second mechanism for filtering potential candidates to improve the accuracy and speed of object identification.

[0038] In some embodiments, the context tracking system 216 can use location information to infer / predict the identity of an object / user. For example, if the RFID reader system 212 indicates that person A enters the attraction 208 at 12:00 noon and that the attraction 208 has an exit point that is typically reached in 10 minutes, the second LIDAR system 214 can infer / predict that an object arriving at the exit point at 12:10 is likely person A. Thus, a tracking system that provides detailed identification of the object / user may not be needed at the exit of the attraction 208, resulting in a more efficient use of resources. Furthermore, the tracking system does not necessarily need to be within line of sight of the object / user. Indeed, as shown in coverage zone 202D, the second mmWave system 217 is positioned to track objects within the restaurant 221, even though it is not within line of sight of the objects located in the restaurant 221. The second mmWave system 217 may be able to enable this type of communication because certain materials (e.g., fabrics, fiberglass reinforced plastic, etc.) associated with the restaurant 221 and / or tracked objects may be transparent to radiation (e.g., transparent to radio frequencies) emitted from the second mmWave system 217.

[0039] As discussed in more detail below, tracking information (e.g., object / user identification and location) can be used for many different purposes. For example, in one embodiment, kiosk 220 can provide specific information useful to an identified object / user in tracking the particular object / user to kiosk 220. Furthermore, in some embodiments, this tracking can help theme park personnel understand the interests of an object / user based on where the object / user is tracked (e.g., an attraction, a restaurant, etc.).

[0040] Having discussed the basic utility of a context tracking system, FIG. 3 is a flowchart illustrating a process 300 for identifying and maintaining a tracking context according to one embodiment. Process 300 begins with selecting a tracking target (block 302). The tracking target can be selected based on one or more criteria of objects observed in the environment and may vary for each coverage zone. For example, with respect to open environment 200, coverage zone 202A may be particularly interested in tracking vehicles in a parking lot. Thus, tracking targets can be selected based on the range of object movement speeds, object sizes, object shapes, etc., that are attributed to vehicles. In contrast, in coverage area 202B, it can be assumed that no vehicles are present, but instead that there are people associated with vehicles. Thus, the criteria for selecting tracking objects within coverage zone 202B can be the range of object movement speeds, object sizes, object shapes, etc., that are attributed to a person or group of people.

[0041] Process 300 proceeds to identify the tracked target identity (block 304). For example, as described above, the identity may be determined based on other identified objects, based on computer vision analysis of the object (e.g., at entrance gate 206), where the relationship between the other identified objects and the tracked target is recorded (e.g., in tangible storage, such as context tracking system 216).

[0042] Once the identity is determined, the position and / or location of the tracked target is identified (block 306). This information, combined with the identity, can be useful to other tracking sensor systems, allowing them to filter in or out certain identities (e.g., a subset of identities) as viable identities of the objects they track. For example, as discussed above with respect to FIG. 2, the identity and location indicating person A entering an attraction, along with a duration estimate of the length of the attraction from start to finish, can enable a tracking system covering that exit of the attraction to filter in person A as a possible identity of the object detected at the exit point of the attraction at or near the end of the duration estimate. Furthermore, if person A is in the attraction, person A is clearly not in the parking lot. Thus, the tracking system sensor in the parking lot can filter out person A as a candidate identity, allowing for faster response times for the discriminant analysis.

[0043] As can be appreciated, the tracking target context is maintained and / or traded by the context tracking system (block 308). As discussed herein, the tracking target context can include the observed activity of a particular tracking sensor system, such as tracking time, tracking location, tracking identity, and tracking identity associated with the tracked object. In some embodiments, the context tracking system can maintain the context data and filter in or out candidates based on the context data, providing the filtered results to the tracking system for efficient discrimination analysis. In other embodiments, the context tracking system can provide the context to the tracking system, enabling the tracking system to perform filtering and discrimination based on the filtering.

[0044] 4 is a flowchart illustrating a process 400 for identifying a context at a subsequent tracking sensor using the acquired context, according to one embodiment. Process 400 begins with receiving a tracked target context (or filtering information, if the tracked target context is maintained in a context tracking system) (block 402). Once received, the tracking sensor can determine the identity of the object using the filtering-in and / or filtering-out techniques described above (block 404).

[0045] Targeted control actions are performed based on the subsequent tracking identifier (block 406). For example, as described above, a kiosk can be controlled to display specific information useful to the identified person. In other embodiments, access to specific restricted portions of the environment can be granted based on a determination that the identity is authorized for access to the restricted portion of the environment. Additionally, in some embodiments, metric tracking, such as visits to specific areas, recorded activity of users, etc., can be maintained for subsequent business analysis, research, and / or reporting.

[0046] As mentioned above, sometimes a certain threshold level of confidence may be required to make a positive match of an identity to a tracked object. Figure 5 is a flowchart illustrating a process 500 for using a confidence interval to determine sufficient context for target identification, according to one embodiment. Process 500 begins with collecting tracking sensor input (e.g., context) (block 502). As mentioned above, a context tracking system can receive tracking sensor input that provides identity, location, and / or other information that provides actionable context for observed objects of other tracked sensors.

[0047] Based on these tracking inputs, one or more tracking identities of the observed object can be predicted by other tracking sensors (block 504). For example, as described above, contextual information indicating that a particular person A has entered an attraction can be used to assume that person A will eventually exit the attraction at a predicted time, perhaps based on the person's attributes and / or the attraction's attributes. For example, if the tracking system observes that the person has a below-average or above-average movement speed, the average duration of the attraction can be adjusted upward or downward accordingly. This adjusted duration is used to determine the time that person A is likely to exit the attraction, allowing the tracking system at the attraction's exit to predict that the person leaving the attraction at the adjusted duration is person A.

[0048] In some embodiments, a predicted confidence score may be calculated. The predicted confidence score may indicate the likelihood that the tracked object has a particular identity. For example, if the target is highly likely to be Person A based on known information, the predicted confidence score may be larger than if the target is somewhat likely or unlikely based on known information. The predicted confidence score may vary depending on several factors. In some embodiments, redundant information (e.g., similar identifications using two or more data sets) may increase the predicted confidence score. Additionally, observable characteristics of the tracked object may be used to influence the predicted confidence score. For example, a known size associated with the ID may be compared to the size of the tracked target. The predicted confidence score may be increased based on the proximity of the known size and the size of the target observed by the tracking sensor.

[0049] After the predicted confidence score is obtained, a determination is made as to whether the predicted confidence score meets a score threshold (decision block 508). The score threshold may indicate a minimum score that can result in an identity associated with the tracked target.

[0050] If the predicted confidence score does not meet the threshold, additional tracking input can be acquired (block 510). To acquire additional tracking input, tracking sensors in the open environment can continue to accumulate tracking information and / or acquire contextual data for the tracked target. In some embodiments, the tracked target can be prompted to move toward a particular sensor to acquire new tracking input for the tracked target. For example, FIG. 6 is a schematic diagram illustrating an exemplary scenario 600 for increasing tracking input, according to one embodiment. In scenario 600, the predicted confidence score for tracked target 602 is less than the threshold, as indicated by speech bubble 604. Thus, electronic display 606 can be controlled to display a message directing the target to a location where additional tracking input can be acquired, here, pizzeria 608. Here, the target is prompted to move toward pizzeria 608 by providing an electronic notification via electronic signage directing the target to pizzeria 608. At the entrance to pizzeria 608, there is an additional tracking sensor (e.g., RFID reader 610). The tracking sensor obtains additional tracking input of the target 602 as the target 602 moves into the vicinity of the pizzeria 608 .

[0051] Returning to Figure 5, as additional tracking input is collected, additional predictions of the target's identity are made (block 504), a new prediction confidence score is determined using the newly collected additional tracking input (block 506), and an additional determination is made as to whether the prediction confidence threshold has been met (decision block 508). This process can continue until the prediction confidence threshold is met.

[0052] Once the predicted confidence threshold is met, the identifier can be attributed to the tracked object (block 512). For example, returning to Figure 6, additional tracking information received via RFID reader 610 results in an increased predicted confidence score above the threshold, as indicated by bubble 612. Thus, tracked target 602 (e.g., tracked object) is attributed to identifier 614 (e.g., Person A).

[0053] As previously mentioned, context provided by one tracking sensor system can be useful for analysis by other tracking sensor systems. For example, candidate predicted identities can be "filtered in" (e.g., whitelisted) or "filtered out" (e.g., blacklisted) based on context provided from another tracking sensor system. Figure 7 is a flowchart illustrating a process 700 for filtering possible identity predictions based on provided sensor context, according to one embodiment.

[0054] Process 700 begins with receiving context data from other sensor systems (block 702). For example, the context data may include tracked object identity, tracked object location, and a timestamp indicating that the tracked object was at a particular location.

[0055] Contextual data can be used to supplement the tracking capabilities of the tracking sensors. For example, viable candidate identities for a newly observed tracked object can be filtered (e.g., filtered in or out) based on contextual information indicating the location of the identified tracked object provided by other tracking sensors (block 704). For example, if a target object with the identity of person A was tracked in a parking lot five minutes ago and the parking lot tracking sensor indicates that it would take 20 minutes to travel from the parking lot to the coverage area associated with the tracking sensor that observed the new target object, then person A's identity can be filtered out as a viable candidate identity for the new target object because it is impossible for person A to reach the coverage area from the parking lot in five minutes. Additionally and / or alternatively, filtering of candidate identities can occur by obtaining contextual data indicating which identities were able to reach the coverage area at the time the new target object was observed. These identities can be whitelisted as viable candidate identities for new target objects observed in the coverage area. This can save processing time and improve operation of computer systems employed in accordance with disclosed embodiments.

[0056] Once the candidate identities have been filtered, the tracking system can predict the identity of a new tracked object from the filtered candidate identities (block 706). For example, an identity can be chosen from a list of identities that have not been filtered out (e.g., not on a blacklist) and / or can be selected from a list of identities that have been filtered in (e.g., on a whitelist). As can be appreciated, contextual information can reduce the number of candidate identities considered to identify a tracked object, thereby enabling more efficient object tracking. Furthermore, in some cases, the absence of such context can render identity tracking impossible. For example, certain LIDAR systems may be unable to perform target object identification without such context. Thus, the present technology provides more efficient and improved tracking capabilities than conventional tracking systems.

[0057] Having discussed enhanced context tracking techniques, FIG. 8 is a schematic diagram illustrating an example control operation 800 that can be achieved based on an identity tracked via the context tracking techniques provided herein, according to one embodiment. In one embodiment, a kiosk 802 or other device can include a display 804. The display 804 can present a graphical user interface (GUI) 806 that provides information about the open environment. As shown, the GUI 806 can be personalized based on the tracked identity. For example, as indicated by a status bubble 808, the context tracking system can identify a tracked object as “James” based on context data trade-offs between various target sensors. In response to a tracked object approaching the display 804, a control action command can be presented on the display 804 (e.g., the underlying hardware controlling the display 804) to instruct the GUI 806 to display information related to the identified person. Here, for example, James is provided with a personalized wait time associated with his identity. As can be appreciated, each identity can be associated with a unique identifier and other information (e.g., wait time, demographic information, and / or other personalized data) to perform personalized control actions. Additionally, as discussed with respect to FIG. 3, control actions can be achieved based on identities corresponding to groups, as shown in FIG.

[0058] In another embodiment, a wearable device 810 can be placed on a tracked object 812. Based on interaction with an open environment feature 814, here a virtual game mystery box, personalized data associated with the tracked object's identity can be updated. For example, here, a game database 816 can be updated via a control action command to reflect an updated game status based on interaction with the open environment feature 814. Thus, as shown in database record 818, the identity 123 associated with James is provided with five additional points in the database. Additionally, a display 820 of the wearable device 810 can be controlled via a control action command to display a notification 822 confirming the status change based on interaction between the tracked object 812 and the open environment feature 814.

[0059] As described above, the context tracking system can identify and track groups. Particular patterns of data indicative of group identification and / or tracking are discussed below, but these are merely examples. As can be appreciated, using machine learning, additional data patterns indicative of grouping can be observed by the machine learning systems described herein. FIG. 9 is a flowchart of a method 900 for grouping individuals using machine learning techniques. One or more steps of method 900 can be performed by one or more components of the context tracking system 102 of FIG. 1 or the context tracking system 216 of FIG. 2. Indeed, the grouping system (e.g., a machine learning system) can be integrated into the context tracking system 102 of FIG. 1 or the context tracking system 216 of FIG. 2. In particular, the grouping system can receive training data for patterns indicative of an active group (block 902). For example, patterns indicative of an active group can include proximity levels of one or more persons at a particular time, indications of vehicle associations with one or more persons, etc. In some embodiments, the patterns can include indications of associations, such as one or more persons (e.g., adults) and presumed children, based on the heights of the tracked individuals. Based on the received training data, the grouping system can be trained to detect the presence of active groups, in block 904. In particular, the grouping system can identify patterns in the raw data that are indicative of active groups in an environment (e.g., open environment 200).

[0060] In some embodiments, time series data can be mined for patterns to identify and track groups. Time series data can provide data for a period of time, which can help machine learning logic identify patterns of people over time. These patterns can be used to identify and / or track groups based on observed activities / patterns over time. For example, in an example of identification, a group of users can be identified as part of a group when they participate in a particular common activity or multiple common activities at a threshold time, which can be observed as a pattern from the time series data.

[0061] In another example, the machine learning system can also be equipped to recognize that not all members of a group can be within a certain proximity of one another. For example, one or more members of an active group may go to the bathroom, while one or more members of the same active group do not. In this case, the grouping system can retain the identities of all members of the active group and / or determine subgroups of the active group, where the subgroups can correspond to one or more members within the active group. Indeed, following the above example, a first subgroup of the active group can be members of the active group who go to the bathroom, and a second subgroup can be members of the active group who do not go to the bathroom while the first subgroup is in the bathroom. Furthermore, an active group can correspond to a group that remains in a database of detected groups for a specific time period. For example, an active group can remain active as long as one or more members of the group are present in the open environment 200 of FIG. 2 within a specific time period.

[0062] In block 906, the machine learning system may determine whether an individual or group (e.g., a subgroup) is at least part of an active group based on at least the training data and the identified patterns indicative of the active group. For example, the machine learning system may determine that two individuals who spent the past hour together are a group. As another example, the grouping system may group individuals who show affection for each other, such as holding each other's hands. Indeed, the grouping system may use weighted probabilities in determining whether one or more individuals should be grouped. Specifically, one or more behavioral patterns, characteristics, and / or attributes observed for a particular tracked identity may be weighted with a probability and utilized to determine the group. For example, because infants and young children are typically unlikely to be left alone with adult supervision, the grouping system may attach a weighted probability (e.g., a value between 0 and 1) of belonging to a group to an individual of short stature. Nevertheless, the grouping system may use weighted probabilities for other attributes and / or behaviors of the individuals (e.g., common locations, proximity, association with the same car, etc.). The grouping system can sum the weighted probability values ​​of being in a group and determine that each individual with which a weighted probability value is associated is part of or forms a group. In one embodiment, one of the weighted probability values ​​may be sufficient to exceed a threshold to determine that each individual with which the weighted probability value is associated is part of or forms a group.

[0063] In block 908, the grouping system can update the database containing active groups. As newly identified groups are determined, the database containing active groups can be updated and tracked. Thus, the machine learning system can keep track of who has been grouped. In some embodiments, grouping can affect downstream processing. For example, as discussed below, group tracking can be utilized in gaming experiences.

[0064] As described herein, a grouping system can identify groups of individuals based on activity / patterns observed in time-series data (e.g., as captured by sensors in a sensor system). Machine learning algorithms can identify any number of patterns, and the algorithms can implement supervised machine learning to train the grouping system on how to identify groups. For discussion purposes, FIG. 10 is a schematic diagram 1000 providing an example of a set of individuals deemed likely to be a group and / or unlikely to be a group by a grouping system using the techniques described herein. Specifically, as shown in diagram 1002, individuals entering and exiting the same vehicle may be deemed likely to be grouped by the grouping system. In contrast, as shown in diagram 1004, individuals entering and exiting different vehicles with a relatively large distance between them may be deemed unlikely to be grouped. Furthermore, as shown in diagram 1006, individuals who are in close proximity to each other for a significant amount of time are likely to be grouped. However, widely spaced individuals who are able to pass a certain threshold distance in an attraction's queue may be deemed unlikely to be grouped. For example, in diagram 1008, each person 1011 represents 20 individuals in a line. As shown in diagram 1008, person 1010 is separated from another person 1012 in a line for a 100-person attraction, and therefore, the grouping system may determine that person 1010 and person 1012 should not be grouped. However, in some cases, the grouping system may determine that individuals are likely to be grouped together even though they are spaced apart. As shown in diagram 1014, person 1016, person 1018, and person 1020 may be likely to be grouped together even though person 1016 is separated from people 1018 and 1020. In this case, people 1018 and 1020 are heading to the restroom 1022, while person 1016 is waiting outside.The grouping system can determine that the persons 1016, 1018, 1020 are a group even though the person 1016 may be alone for a period of time. Additionally, the grouping system can also group individuals who are separated but who have spent time together for more than a certain threshold period of time. For example, a person in a group may choose to sit in a rest area rather than wait in a queue for an attraction with other people from the same group. Thus, the grouping system can keep a group active for a period of time even if one or more members of the group are separated.

[0065] As can be appreciated, once a group is identified, additional functionality can be implemented. For example, FIG. 11 is a diagram 1100 illustrating how a group recognized by a grouping system can modify the interactive environment of an interactive game. In diagram 1100, compliance with rule 1102 on display 1103 is required to unlock door 1104 (e.g., a portal in a virtual and / or augmented reality experience). To enter door 1104, rule 1102 that "all team members must enter together" must be satisfied. Here, the grouping system has identified persons 1114 and 1112 as part of person 1110's group / team. However, these persons 1114 and 1112 are not present with person 1110. Because not all team members are with team member 1110 at one time, the door is not unlocked. Later, at time 1113, all team members (e.g., persons 1110, 1112, 1114) are present at door 1104. Effects occur from the interactive environment (e.g., door 1104 unlocks) in response to the presence of the entire group. Thus, by tracking the group, one or more components of the interactive environment can be controlled based on aspects of the group (e.g., team member locations, team member accumulated activity, etc.) to provide an enjoyable interactive experience for guests.

[0066] FIG. 12 is a schematic diagram illustrating an example control action 1200 that can be achieved based on group identity tracked via the context tracking technology provided herein, according to one embodiment. Similar to FIG. 8, FIG. 12 includes a kiosk 1202, or another device, can include a display 1204. The display 1204 can present a graphical user interface (GUI) 1206 that provides information about the open environment. As shown, the GUI 1206 can be personalized based on the tracked identity. For example, as indicated by the status bubble 1208, the context tracking system can identify a tracked object as part of a tracking group “Group 1” based on context data trade-offs between various target sensors. In response to a tracked object approaching the display 1204, a control action command can be presented on the display 1204 (e.g., the underlying hardware controlling the display 1204) instructing the GUI 1206 to display information related to the identified group. Here, for example, Group 1 is provided with instructions associated with its identity. Specifically, the GUI 1209 reads, “Hello Group 1, complete your task in under 10 minutes.” As can be appreciated, each group identity can be associated with a unique identifier and other information (e.g., wait times, instructions, demographic information, and / or other personalized data), allowing personalized control actions to be performed.

[0067] In another embodiment, each member of the tracked group 1212 (i.e., Group 1) can have a wearable device 1210 communicatively coupled to one or more components of the context tracking system. Based on interactions with the open environment feature 1214, here a virtual game mystery box, personalized data associated with the identities of the tracked group can be updated. For example, here, the game database 1216 can be updated via control action commands to reflect an updated game status based on interactions with the open environment feature 1214. Thus, as shown in database record 1218, the identity 456 associated with Group 1 has 15 points added in the database. Additionally, the display 1220 of each of the group members' wearable devices 1210 can be controlled via control action commands to display a notification 1222 confirming the change in status based on each member of the tracked group 1212's interaction with the open environment feature 1214.

[0068] The technology presented and claimed herein refers to and applies to specific examples of a tangible and practical nature that clearly improve the art of the present invention, and as such is not abstract, intangible, or purely theoretical. Furthermore, to the extent any claim appended hereto includes one or more elements designated as "means for performing a function" or "steps for performing a function," such elements are intended to be construed under 35 U.S.C. §112(6). However, for any claim including elements designated in any other manner, such elements shall not be construed under 35 U.S.C. §112(f).

Claims

1. a first tracking sensor system configured to track a first tracking object within a first coverage area; a second tracking sensor system configured to track a second tracked object in a second coverage area different from the first coverage area; one or more processors, receiving a tracking target context for the first tracked object from the first tracking sensor system; providing the tracking target context from the first tracking sensor system to the second tracking sensor system; identifying the second tracked object by the second tracking sensor system based on the tracked target context from the first tracking sensor system; receiving a control action associated with the open environment feature based on the identification of the second tracked object and the second tracked object being within a threshold distance from the open environment feature; Instructing the implementation of the control action; one or more processors configured to A system comprising:

2. The one or more processors: filtering out candidate identities from a set of candidate identities that the second tracked object can identify based on the tracked target context from the first tracked sensor system; providing the set of candidate identities that does not include the candidate identity to the second tracking sensor system; The system of claim 1 , configured to:

3. the one or more processors are configured to determine that the candidate identity is tracked at a location outside of an identified location that the candidate identity is predicted to reach during a time difference, the time difference being a time difference between a first time that the candidate identity is tracked at the location and a second time that the second tracked object is tracked by the second tracking sensor system; The system of claim 2 .

4. determining that the candidate identity is tracked at a location outside of an identified location where the candidate identity is expected to arrive during the time difference includes including including an approximate time the candidate identity will remain at the location, an approximate speed of travel associated with the candidate identity, or both; The system of claim 3.

5. At least the first tracking sensor system or the second tracking sensor system includes at least a Light Detection and Ranging (LIDAR) system, a Radio Frequency Identification (RFID) system, a computer vision system, a Time of Flight (ToF) system, or a Millimeter Wave (mmWave) system; The system of claim 1 .

6. the open environment feature includes access to a restricted attraction, and the control action is to enable permission to access the restricted attraction for the second tracked object based on identifying that the second tracked object is associated with permission to access the restricted attraction. The system of claim 1 .

7. the open environment functionality includes a display, and the control action presents information associated with the second tracked object via the display based on the identification of the second tracked object and the second tracked object being within the threshold distance from the display. The system of claim 1 .

8. the second tracking object is associated with a wearable device; The one or more processors: updating information associated with the second tracked object based on an interaction of the second tracked object with the open environment feature; instructing the wearable device to display an indication that the information associated with the second tracking object has been updated; The system of claim 1 , configured to:

9. the information is associated with a virtual game, and the interaction with the open environment feature is associated with a change in a virtual game status associated with the second tracked object. The system of claim 8.

10. the change includes adding points to the virtual game status. The system of claim 9.

11. the one or more processors are configured to determine, based on the tracked target context, that the second tracked object is associated with an identified group, the open environment functionality including access to a restricted attraction, and the control action enabling permission to access the restricted attraction for the identified group based on the identified group being associated with permission to access the restricted attraction and a respective number of the identified group within the threshold distance from the restricted attraction. The system of claim 1 .

12. the one or more processors are configured to determine, based on the tracking target context, that the second tracking object is associated with an identified group, the open environment functionality includes a display, and the control action is to present information associated with the identified group via the display based on the identified group being within the threshold distance from the display. The system of claim 1 .

13. The one or more processors: determining, based on the tracking target context, that the second tracking object is associated with an identified group associated with a plurality of wearable devices; updating information associated with the identified groups based on interactions with the open environment functionality by at least one of the identified groups; instructing at least one wearable device of the plurality of wearable devices to display an indication that the information associated with the identified group has been updated; The system of claim 1 , configured to:

14. the information is associated with a virtual game, and the interaction with the open environment feature is associated with a change in a virtual game status associated with the identified group. The system of claim 13.

15. the change includes adding points to the virtual game status. The system of claim 14.

16. A tangible, non-transitory, machine-readable medium, comprising: When executed by one or more processors of a machine, receiving a tracking target context for a first tracked object from a first tracking sensor system that tracks a first coverage area; providing the tracking target context from the first tracking sensor system to a second tracking sensor system tracking a second coverage area different from the first coverage area; receiving a control action associated with the open environment feature based on an identification of a second tracked object by the second tracking sensor system based on the tracked target context from the first tracking sensor system and the second tracked object being within a threshold distance from the open environment feature; causing the realization of said control action; a machine-readable medium comprising machine-readable instructions for causing said machine to perform the steps of:

17. the open environment functionality includes a display, and the control action presents information associated with the second tracked object via the display based on the identification of the second tracked object and the second tracked object being within the threshold distance from the display.

17. The machine-readable medium of claim 16.

18. the second tracking object is associated with a wearable device; The one or more processors: updating information associated with the second tracked object based on an interaction of the second tracked object with the open environment feature; instructing the wearable device to display an indication that the information associated with the second tracking object has been updated; 17. The machine-readable medium of claim 16, configured to:

19. the information is associated with a virtual game, and the interaction with the open environment feature results in points being added to a virtual game status associated with the second tracked object.

20. The machine-readable medium of claim 18.

20. receiving a tracking sensor system input from a first tracking sensor system, the tracking sensor system input including a tracking target context of the first tracked object; providing the tracking target context from the first tracking sensor system to a second tracking sensor system; identifying a second tracked object by the second tracking sensor system based on the tracked target context from the first tracking sensor system; receiving a control action associated with the open environment feature based on the identification of the second tracked object and the second tracked object being within a threshold distance from the open environment feature; causing the realization of said control action; 11. A computer-implemented method comprising:

Images