Tracking aggregation and alignment

Abstract

Systems and methods for providing contextual tracking information to a tracking sensor system to provide accurate and efficient object tracking are disclosed. Contextual data of the first tracking sensor system is used to identify a tracked object of the second tracking sensor system.

Description

This application is a divisional application of the patent application with application number 202080026171.5, application date September 29, 2021, and name “Tracking Aggregation and Alignment”.

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and the benefit of U.S. Provisional Application No. 62 / 828,198, filed April 2, 2019, entitled “Tracking Aggregation and Alignment,” which is hereby incorporated by reference in its entirety for all purposes. Background Art

[0002] The present disclosure relates generally to tracking systems. More specifically, certain embodiments of the present disclosure relate to the aggregation and switching of tracking system data between tracking systems to facilitate more efficient and effective tracking of objects within an environment.

[0003] In the digital age, with the increase of digital sensors, object tracking has become more and more desirable. Unfortunately, in large / open environments, user tracking is a very challenging prospect, especially when accurate location and activity tracking is desired. As used herein, an open environment refers to an area that allows tracked objects to move in multiple directions with relatively few restrictions. For example, such an environment may include an amusement park, an airport, a shopping mall or other relatively large-scale environment that may have multiple tracking coverage areas. Accurately tracking a unique individual is challenging, especially in an open environment and when there is an obstruction (where one individual may block another individual) problem in the crowd density.

[0004] This section is intended to introduce the reader to various aspects of the art that may be relevant to various aspects of the present technology described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Therefore, it should be understood that these statements are to be read in this light and not as admissions of prior art. Summary of the Invention

[0005] Certain embodiments commensurate in scope with the originally claimed subject matter are summarized below. These embodiments are not intended to limit the scope of the present disclosure, but rather these embodiments are intended only to provide a brief overview of certain disclosed embodiments. In fact, the present disclosure may include various forms that may be similar or different from the embodiments set forth below.

[0006] Embodiments described herein relate to a tracking system that efficiently aggregates and / or transmits tracking data between tracking systems so that the context of one tracking sensor can enhance the tracking of other tracking sensors. More specifically, context information (e.g., location, time, identity of the tracked object) determined by one tracking sensor can be used to facilitate more efficient and / or more effective tracking of other sensors. For example, such context information can lead to increased confidence in the identity of the tracked object, can lead to efficient filtering of possible identities attributable to the tracked object, etc. This can result in increased processing efficiency and can also enable more granular tracking of objects in open environments.

[0007] By way of example, in a first embodiment, a tangible, non-transitory, machine-readable medium includes machine-readable instructions that, when executed by one or more processors of a machine, cause the machine to: receive a tracked target context of a first tracked object from a first tracking sensor system; provide the tracked target context from the first tracking sensor system to a second tracking sensor system that is different from the first tracking sensor system; and cause recognition of a newly observed tracked target by the second tracking sensor system based on the tracked target context from the first tracking sensor system.

[0008] In a second embodiment, a computer-implemented method includes receiving a tracked target context of a first tracked object from a first tracking sensor system; providing the tracked target context from the first tracking sensor system to a second tracking sensor system that is different from the first tracking sensor system; and causing recognition of a newly observed tracked target by the second tracking sensor system based on the tracked target context from the first tracking sensor system.

[0009] In a third embodiment, a system includes a first tracking sensor system, a second tracking sensor system, and a context tracking sensor system. The first tracking sensor system tracks a first tracked object in a first coverage area. The second tracking sensor system tracks a second tracked object in a second coverage area. The context tracking system receives a tracked target context of the first tracked object from the first tracking sensor system; provides the tracked target context from the first tracking sensor system to a second tracking sensor system different from the first tracking sensor system; and causes recognition of the second tracked object based on the tracked target context from the first tracking sensor system. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] These and other features, aspects, and advantages of the present disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings, in which like characters represent like parts throughout, and in which: Figure 1 is a schematic diagram illustrating a multi-sensor tracking component with a context tracking system according to an embodiment of the present disclosure; Figure 2 It illustrates the use of an embodiment according to the present disclosure Figure 1 A schematic diagram of the open environment of the system; Figure 3 is a flow chart illustrating a process for identifying a tracking context according to an embodiment; Figure 4 is a flow chart illustrating a process for using acquired context to identify context at a subsequent tracking sensor according to an embodiment; Figure 5 is a flow chart illustrating a process for determining sufficient context for target recognition using confidence intervals according to an embodiment; Figure 6 is a diagram illustrating an example transition for increasing tracking input according to an embodiment; Figure 7 is a flow chart illustrating a process for filtering possible recognition predictions based on provided sensor context, according to an embodiment; Figure 8 is a schematic diagram illustrating example control actions based on a tracked identity according to an embodiment; Figure 9 is a flow chart for classifying individuals into groups using machine learning techniques according to an embodiment; Figure 10 is a diagram of grouped and ungrouped individuals according to an embodiment; Figure 11 is an illustration of a use case for changing an interactive environment based on compliance rules according to an embodiment; and Figure 12 is a schematic diagram illustrating example control actions based on a tracked identity according to an embodiment. DETAILED DESCRIPTION

[0011] One or more specific embodiments of the present disclosure are described below. These described embodiments are merely examples of the techniques of the present disclosure. Furthermore, in an effort to provide a brief description of these embodiments, not all features of an actual implementation may be described in the specification. It should be understood that in the development of any such actual implementation, as in any engineering or design project, many implementation-specific decisions must be made to achieve the developer's specific goals, such as in accordance with system-related and business-related constraints, which may vary from implementation to implementation. Furthermore, it should be understood that such development work may be complex and time-consuming, but for a person of ordinary skill having the benefit of this disclosure, it may still be a routine task of design, fabrication, and manufacturing.

[0012] When introducing elements of various embodiments of the present disclosure, the articles "a," "an," and "said" are intended to mean that there are one or more of the elements. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements in addition to the listed elements. Furthermore, it should be understood that references to "one embodiment" or "an embodiment" of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features.

[0013] The present disclosure generally relates to a tracking system that accumulates and / or switches contextual information for efficient and effective tracking processing. By using the context previously determined by other tracking sensors, an independent tracking sensor can more effectively determine the identity of the tracked object. With this in mind, Figure 1 is a schematic diagram illustrating a multi-sensor tracking system 100 with a context tracking system 102 according to an embodiment of the present disclosure. As illustrated, the multi-sensor tracking system 100 includes multiple tracking sensors, such as one or more light detection and ranging (LIDAR) systems 104, one or more radio frequency identification (RFID) reader systems 106, one or more time-of-flight (ToF) systems 107, one or more computer vision systems 108, and / or one or more millimeter wave (mmWave) systems 109.

[0014] The LIDAR system 104 can track individuals, objects, and / or groups of individuals or objects by illuminating a target with pulsed light and measuring the reflected pulses. The wavelength and time differences between the pulsed light and the reflected pulses can be used to generate a spatial indication of the location of the target individual, group, and / or object. The LIDAR system 104 is capable of covering a large area of ​​space while detecting objects relatively easily and efficiently. However, the LIDAR system 104 may not be effective in actually identifying the tracked object, but may be best used to identify the presence and location of the object independently of the object's identification.

[0015] The RFID reader system 106 can read digital data encoded in RFID tags (e.g., worn by an individual or placed on a specific object to track the individual or object). When an RFID tag comes into proximity with the RFID reader, the RFID reader can provide an excitation signal to the RFID tag, causing the RFID tag to emit radiation that can be interpreted by the RFID reader to identify the specific RFID tag. Because each RFID tag has its own unique identification information, the RFID reader system 106 can efficiently and effectively identify the tracked target. However, the RFID reader system 106 requires that the RFID tags be placed in relatively close proximity to the RFID reader, resulting in limited coverage and / or significant hardware costs for implementing multiple RFID readers.

[0016] Similar to the LIDAR system 104, the ToF system 107 (e.g., a three-dimensional time-of-flight sensor system) can track individuals, groups, and / or objects by illuminating a target with pulsed light and measuring the characteristics of the reflected pulses. Specifically, the ToF system 107 can emit pulses of infrared light and measure the time corresponding to the return of the pulses. The ToF system 107 can also map textures (e.g., skin texture) to identify individuals, groups, and / or objects. Thus, the ToF system 107 can obtain three-dimensional positioning and texture properties of the tracked identity. Another benefit is that because the ToF system 107 may not rely on visible lighting conditions, the ToF system 107 may not incur lighting condition limitations, which may be a characteristic of some camera-based vision acquisition systems. However, redundant systems may still be useful because the ToF system 107 may be less effective and less accurate in certain environmental conditions (such as on rainy days).

[0017] The computer vision system 108 can receive camera data (e.g., still images and / or video) for contextual analysis. For example, the camera data can be analyzed to perform object recognition and / or tracking. Using facial recognition, the computer vision system 108 can identify the tracked target. Unfortunately, however, computer vision systems 108 can be quite expensive and typically cover a limited tracking area. Furthermore, the computer vision system 108 can take considerable time to analyze the computer image.

[0018] A millimeter wave (mmWave) system 109 (e.g., a millimeter wave radar sensor system) can provide large bandwidth to verify the presence of a tracked identity. Specifically, the millimeter wave system 109 can allow for the transmission of high rates of data with low latency. For example, the millimeter wave system 109 can include devices that emit and / or receive millimeter waves to communicate with one or more computing devices (e.g., wearable devices) associated with an individual in order to quickly identify the individual (or verify an identity proposed by the individual). Furthermore, the millimeter wave system 109 may be able to maintain tracking of the tracked identity even through surfaces (e.g., physical barriers that are transparent to radio waves, such as windows and walls). However, the millimeter wave system 109 may utilize components that are relatively close to the tracked identity, resulting in less coverage.

[0019] As can be appreciated, each tracking system has its trade-offs. Therefore, it may be desirable to use a combination of tracking systems so that the benefits of each of the various tracking systems can be used in conjunction with one another. To this end, the context tracking system 102 is tasked with maintaining and / or facilitating tracking data from each of the various tracking systems. For example, the context tracking system 102 may receive positioning information from the LIDAR system 104, object identifiers obtained from RFID tags in proximity to one or more RFID reader systems 106, and / or object identities and / or positioning from the computer vision system 108.

[0020] Using this information, object tracking within open environments can be achieved more efficiently and effectively. Figure 2 It illustrates the use of an embodiment according to the present disclosure Figure 1 Schematic diagram of the open environment 200 of the system. Figure 2 A theme park environment is illustrated, but the current discussion is not intended to limit the application of the context tracking system to such an embodiment. In fact, the current technology can be used in a variety of environmental applications.

[0021] As shown, open environment 200 may have a number of separate coverage zones 202A, 202B, 202C, and 202D, each tracked by one or more sensor systems. For example, coverage zone 202A is a parking lot tracked by computer vision system 204. Coverage zone 202B is the area between entry gate 206 and attraction 208. Coverage zone 202B has a LIDAR system 210, an RFID reader system 212, and a millimeter wave system 213. Coverage zone 202C is tracked by a second LIDAR system 214 and a ToF system 215. Coverage zone 202D is tracked using a second millimeter wave system 217.

[0022] The context tracking system 216 is communicatively coupled to various tracking systems in the open environment 200. Similar to Figure 1 In the context tracking system 102, the context tracking system 216 can maintain and / or exchange tracking data between tracking systems. For example, the computer vision system 204 can detect a particular car 219 in the coverage area 202A. The computer vision system 204 can analyze the visual image of the car 219 to identify the car 219. For example, the computer vision system 204 can recognize alphanumeric characters of the license plate of the car 219 to identify the car 219.

[0023] The identified objects can be used to identify other objects in the context tracking system 216. For example, the identification of car 219 can be provided to the context tracking system 216. For example, car 219 can be identified as corresponding to one or more people (e.g., a group of people) that the computer vision system 204 has detected leaving car 219 at a particular time. Based on this information, the context tracking system 216 can determine that the one or more people are likely in coverage zone 202A. In addition, the identity of the person or group based on the computer vision system 204 can be used to determine the identity of the person or group in another coverage zone. In particular, the context tracking system 216 can record characteristics unique to the one or more people identified in coverage zone 202A and can use the recorded characteristics to determine the identity of tracked objects in other coverage zones, such as coverage zone 202B and coverage zone 202D.

[0024] By providing such context to the context tracking system 216 and / or other tracking sensors in the coverage area, tracking analysis can be more informed about possible candidate objects that are approaching. For example, if the computer vision system 204 provides an indication that car A is in a parking lot (or that person A or group A associated with car A is likely in the parking lot [e.g., based on car A's presence in the parking lot]) to tracking sensors in adjacent coverage areas, the tracking sensors in these adjacent coverage areas can be "pre-heated" with data identifying possible object identifiers. Thus, in some embodiments, a larger coverage system with less object recognition capabilities (such as the LIDAR system 210) can be used to track the user's location, at least in part based on the recognition provided by the computer vision system 204. Furthermore, as briefly noted above, the recognized objects can be used to identify other objects, such as groups of people. In fact, it may be beneficial for the context tracking system 216 to track groups. For example, car 219 may be identified as corresponding to a group of people (e.g., group A). ​​In this case, the context tracking system 216 can determine that all people exiting car 219 are associated with each other and therefore may include group A. As will be discussed later, other methods of identifying and determining groups are possible.

[0025] If only one user is associated with car A, then the single identified object exiting the car is likely the associated user. However, in some instances, the context provided by the previous tracking system may not meet the threshold level of likelihood. In such cases, additional tracking can be selectively enabled to identify specific objects / users. For example, another computer vision system 218 at entry gate 206 can be used to identify objects / users. Since entry gate 206 is a funnel-in location with direct viewing access to the desired object / user, this location can be the primary location for computer vision system 218. In such an embodiment, when individual objects / users are detected by computer vision system 218, the object recognition analysis performed by computer vision system 218 can be significantly influenced by the contextual data provided by computer vision system 204 via context tracking system 216. For example, possible candidates for identification can be filtered out based on the data provided by computer vision system 204, resulting in more efficient and faster processing of object / user identification. Furthermore, millimeter wave system 213 in coverage area 202B can also be used to identify or verify the presence of individual objects / users. In practice, mmWave system 213 can work in conjunction with computer vision system 218 in identifying objects. For example, using contextual data provided by computer vision system 204 via context tracking system 216, mmWave system 213 can serve as a secondary mechanism for filtering possible candidates to increase the accuracy and processing speed of object identification.

[0026] In some embodiments, contextual tracking system 216 can use positioning information to infer / predict the identity of an object / user. For example, if RFID reader system 212 indicates that person A entered attraction 208 at 12:00 PM, and attraction 208 has an exit point typically reached within 10 minutes, second LIDAR system 214 can infer / predict that the person arriving at the exit point at 12:10 PM is likely person A. Consequently, a tracking system that provides detailed identification of the object / user may not be necessary at the exit of attraction 208, resulting in more efficient use of resources. Furthermore, a tracking system need not necessarily be within the line of sight of the object / user. In fact, as illustrated in coverage area 202D, second mmWave system 217, while not within the line of sight of an object located in restaurant 221, is positioned to track the object in restaurant 221. Second mmWave system 217 may be capable of this type of communication because certain materials associated with restaurant 221 and / or the tracked object (e.g., fabric, fiberglass reinforced plastic, etc.) may be transmissive (e.g., radio frequency transparent) to radiation emerging from second mmWave system 217.

[0027] As will be discussed in more detail below, tracking information (e.g., identification and location of an object / user) can be used for many different purposes. For example, in one embodiment, the kiosk 220 can provide specific information useful to the identified object / user when tracking a specific object / user at the kiosk 220. Furthermore, in some embodiments, such tracking can help theme park personnel understand the interests of the object / user based on the location to which the object / user was tracked (e.g., an attraction, a restaurant, etc.).

[0028] Having discussed the basic utility of context tracking systems, Figure 3 3 is a flow diagram illustrating a process 300 for identifying and maintaining tracking contexts according to an embodiment. Process 300 begins by selecting a tracking target (block 302). The tracking target can be selected based on one or more criteria of objects observed in the environment and can vary by coverage area. For example, for the open environment 200, coverage area 202A can be particularly interested in tracking vehicles in a parking lot. Therefore, tracking targets can be selected based on ranges of object motion speed, object size, object shape, etc. that can be attributed to vehicles. In contrast, it can be assumed that there are no vehicles in coverage area 202B, but rather people associated with vehicles. Therefore, the criteria for selecting tracking objects in coverage area 202B can be ranges of object motion speed, object size, object shape, etc. that can be attributed to a person or a group of people.

[0029] The process 300 continues by identifying the identity of the tracked target (block 304). For example, as mentioned above, the identity can be determined based on computer vision analysis of the object (e.g., at the entry gate 206), based on other identified objects (where the relationship between the other identified objects and the tracked target is recorded (e.g., in tangible storage such as the context tracking system 216)).

[0030] Once the identity is determined, the location and / or position of the tracking target is identified (block 306). This information, combined with the identity, may be useful to other tracking sensor systems, enabling the tracking sensor system to filter in or filter out specific identities (e.g., a subset of identities) as possible identities of the objects it is tracking. For example, as described above with respect to Figure 2 As mentioned, the identity and location indicating that person A is entering an attraction, along with a duration estimate for the attraction's length from start to finish, can enable a tracking system covering the attraction's exit to filter person A as a possible identity detected at or near the attraction's exit point at the end of the duration estimate. Furthermore, if person A is within the attraction, then person A is clearly not also in the parking lot. Therefore, the tracking system sensors in the parking lot can filter person A out as a candidate identity, thereby achieving a faster response time for the identification analysis.

[0031] As can be appreciated, the tracking target context is maintained and / or traded by the context tracking system (block 308). As mentioned herein, the tracking target context can include observation activity of a particular tracking sensor system, such as tracking time, tracking location, tracking identity, tracking identity associated with the tracked object, and the like. In some embodiments, the context tracking system can maintain context data in some embodiments and also filter in or out candidates based on the context data, thereby providing the filtering results to the tracking system for efficient recognition analysis. In other embodiments, the context tracking system can provide the context to the tracking system, thereby enabling the tracking system to perform filtering and filter-based recognition.

[0032] Figure 4 4 is a flow chart illustrating a process 400 for using acquired context to identify context at a subsequent tracking sensor according to an embodiment. The process 400 begins by receiving a tracking target context (or filtering information, if the tracking target context is maintained at the context tracking system) (block 402). Once received, the tracking sensor can use the filtering and / or filtering techniques discussed above to determine the identity of the object (block 404).

[0033] Based on the subsequent tracking identifier, a targeted control action is performed (block 406). For example, as mentioned above, the kiosk can be controlled to display specific information useful to the identified person. In other embodiments, access to a specific restricted portion of the environment can be granted based on determining that the identity is authorized to access the restricted portion of the environment. Furthermore, in some embodiments, a metric track can be maintained for subsequent business analysis, research, and / or reporting, such as visits to specific areas, recorded user activity, etc.

[0034] As mentioned above, sometimes a certain confidence threshold level may be required to make a positive match of an identity to a tracked object. Figure 5 is a flow chart illustrating a process 500 for determining sufficient context for target recognition using confidence intervals, according to an embodiment. The process 500 begins by collecting tracking sensor input (e.g., context) (block 502). As mentioned above, the context tracking system may receive tracking sensor input that provides identity, location, and / or other information that provides possible context for objects observed by other tracking sensors.

[0035] Based on these tracking inputs, one or more tracked identities of the observed object can be predicted by other tracking sensors (block 504). For example, as mentioned above, contextual information indicating that a particular person A has entered a tourist attraction can be used to assume that person A will likely eventually leave the attraction at a predicted time based on the person's attributes and / or the attributes of the attraction. For example, if the tracking system observes that the person has a below-average or above-average movement speed, the average duration of the attraction can be adjusted downward or upward accordingly. This adjusted duration can be used to determine when person A is likely to leave the attraction, thereby enabling the tracking system at the attraction's exit to predict that the person leaving the attraction at the adjusted duration is person A.

[0036] In some embodiments, a prediction confidence score may be calculated. The prediction confidence score may indicate the likelihood that the tracked object has a particular identity. For example, if, based on known information, the target is very likely to be person A, the prediction confidence score may be greater than the prediction confidence score when (based on known information) the target is only somewhat likely or unlikely to be person A. The prediction confidence score may vary based on a variety of factors. In some embodiments, redundant information (e.g., similar identification using two or more sets of data) may increase the prediction confidence score. Additionally, observable characteristics of the tracked object may be used to influence the prediction confidence score. For example, a known size associated with an identity may be compared to the size of the tracked target. The prediction confidence score may be increased based on the proximity of the known size to the target size observed by the tracking sensor.

[0037] After obtaining the prediction confidence score, a determination is made as to whether the prediction confidence score satisfies a score threshold (decision block 508). The score threshold may indicate a minimum score that may result in an identity being associated with the tracked target.

[0038] If the prediction confidence score does not meet the threshold, additional tracking input may be obtained (block 510). To obtain additional tracking input, the tracking sensors in the open environment may continue to accumulate tracking information and / or obtain contextual data about the tracked target. In some embodiments, the tracked target may be encouraged to move toward a particular sensor to obtain new tracking input about the tracked target. For example, Figure 66 is a schematic diagram illustrating an example transfer scenario 600 for increasing tracking input according to an embodiment. In scenario 600, the prediction confidence score of tracked target 602 is less than a threshold, as indicated by balloon 604. Therefore, electronic display 606 can be controlled to display a message directing the target toward a location (here, pizzeria 608) where additional tracking input can be obtained. Here, the target is encouraged to move toward pizzeria 608 by providing an electronic notification via an electronic billboard, which directs the target to pizzeria 608. At the entrance to pizzeria 608, there is an additional tracking sensor (e.g., RFID reader 610). As target 602 moves near pizzeria 608, the tracking sensor obtains additional tracking input for target 602.

[0039] return Figure 5 As additional tracking input is collected, additional predictions are made about the target's identity (block 504), new prediction confidence scores using the newly collected additional tracking input are determined (block 506), and additional determinations are made as to whether a prediction confidence threshold is met (decision block 508). This process may continue until the prediction confidence threshold is met.

[0040] Once the prediction confidence threshold is met, an identifier may be attributed to the tracked object (block 512). Figure 6 , additional tracking information received via RFID reader 610 causes the prediction confidence score to increase to a level greater than or equal to the threshold, as indicated by balloon 612. Thus, tracked target 602 (e.g., tracked object) is attributed to identifier 614 (e.g., person A).

[0041] As mentioned above, the context provided by one tracking sensor system may be useful for analysis by other tracking sensor systems. For example, candidate predicted identities may be "filtered in" (e.g., whitelisted) or "filtered out" (e.g., blacklisted) based on the context provided by another tracking sensor system. Figure 7 is a flow diagram illustrating a process 700 for filtering possible recognition predictions based on provided sensor context, according to an embodiment.

[0042] Process 700 begins by receiving context data from other sensor systems (block 702). For example, the context data may include the identity of a tracked object, the location of the tracked object, and a timestamp indicating when the tracked object was at a particular location.

[0043] Contextual data can be used to supplement the tracking functionality of the tracking sensor. For example, possible candidate identities for newly observed tracked objects can be filtered (e.g., filtered in or filtered out) based on contextual information indicating the location of the identified tracked object provided by other tracking sensors (block 704). For example, if a parking lot tracking sensor indicates that a target object with the identity of person A was tracked in a parking lot five minutes ago, and it takes 20 minutes to travel from the parking lot to the coverage area associated with the tracking sensor that observed the new target object, the identity of person A can be filtered out as a possible candidate identity for the new target object because it would be infeasible for person A to reach the coverage area from the parking lot within 5 minutes. Additionally and / or alternatively, filtering in candidate identities can occur by obtaining contextual data indicating which identities may have already reached the coverage area when the new target object was observed. These identities can be whitelisted as possible candidate identities for the observed new target object in the coverage area. This can save processing time and improve the operation of computer systems employed in accordance with the disclosed embodiments.

[0044] Once the candidate identities are filtered, the tracking system can predict the identity of a new tracked object from the filtered candidate identities (block 706). For example, the identity can be selected from a list of unfiltered identities (e.g., not in a blacklist) and / or can be selected from a list of filtered identities (e.g., in a whitelist). As can be appreciated, this can enable more efficient object tracking because the contextual information can reduce the number of candidate identities to be considered for identifying the tracked object. Furthermore, in some instances, identity tracking may not be possible without such context. For example, some LIDAR systems may not be able to perform identification of a target object without such context. Thus, the current technology provides more efficient and enhanced tracking capabilities than existing tracking systems.

[0045] Enhanced context tracking techniques have been discussed, Figure 8is a schematic diagram illustrating an example control action 800 according to an embodiment that can be implemented based on an identity tracked with the aid of context tracking techniques provided herein. In one embodiment, a kiosk 802 or other device may include a display 804. The display 804 may present a graphical user interface (GUI) 806 that provides information about an open environment. As illustrated, the GUI 806 may be personalized based on the tracked identity. For example, as illustrated by the status balloon 808, the context tracking system may identify that the tracked object is "James" based on a weighing of context data between various target sensors. In response to the tracked object approaching the display 804, a control action command may be presented to the display 804 (e.g., the underlying hardware controlling the display 804) that instructs the GUI 806 to display information related to the identified person. For example, James is provided here with a personalized wait time associated with his identity. As can be appreciated, each identity can be associated with a unique identifier and other information (e.g., wait time, demographic information and / or other personalized data) to enable personalized control actions to be performed. In addition, as will be described with respect to Figure 12 As discussed, control actions may be implemented based on identities corresponding to groups.

[0046] In another embodiment, a wearable device 810 may be provided on a tracked object 812. Based on an interaction with an open environmental feature 814 (here, a virtual game mystery box), personalized data associated with the identity of the tracked object may be updated. For example, the game database 816 may be updated by means of a control action command to reflect the updated game state based on the interaction with the open environmental feature 814. Thus, as illustrated in database record 818, an additional 5 points are provided in the database to the identity 123 associated with James. In addition, the display 820 of the wearable device 810 may be controlled by means of a control action command to display an indication 822 that confirms the state change based on the interaction of the tracked object 812 with the open environmental feature 814.

[0047] As mentioned above, the context tracking system can identify and track groups. Although specific data patterns that indicate the identification and / or tracking of groups are discussed below, these are merely examples. As can be appreciated, using machine learning, additional data patterns that indicate grouping can be observed by the machine learning system described herein. Figure 9 is a flow chart of a method 900 for grouping individuals using machine learning techniques. One or more steps of the method 900 may be performed by Figure 1 The context tracking system 102 or Figure 2In practice, a grouping system (e.g., a machine learning system) may be integrated into Figure 1 The context tracking system 102 or Figure 2 904 . In particular, the grouping system may receive training data indicating pattern(s) of active groups (block 902 ). For example, the pattern(s) indicating an active group may include a level of proximity between one or more people over a certain amount of time, an indication of an association of a vehicle with one or more people, etc. In some embodiments, the pattern(s) may include an indication of an association, such as a presumed child with one or more people (e.g., an adult) based on the height of the tracked individual. Based on the received training data, at block 904 , the grouping system may become adept at detecting the presence of active groups. In particular, the grouping system may recognize pattern(s) in the raw data indicating active groups in an environment (e.g., the open environment 200 ).

[0048] In some embodiments, time series data can be mined for patterns to identify and track groups. Time series data can provide data over a period of time, which can help machine learning logic identify patterns of people over time. These patterns can be used to identify and / or track groups based on activities / patterns observed over time. For example, in an identification example, when a group of users participates in a particular common activity or a common activity for a threshold amount of time, they can be identified as part of a group, which can be observed as a pattern from the time series data.

[0049] In another example, the machine learning system may also be equipped to recognize that not all members of a group may be present within a certain proximity to each other. For example, one or more persons of an activity group may go to the restroom while one or more persons of the same activity group do not go to the restroom. In this case, the grouping system may retain the identities of all members of the activity group and / or determine subgroups of the activity group, which may correspond to one or more persons within the activity group. Indeed, consistent with the example above, a first subgroup of the activity group may be members of the activity group that go to the restroom while a second subgroup may be members of the activity group that do not go to the restroom while the first subgroup is in the restroom. Furthermore, activity groups may correspond to groups that remain in the database of detected groups for a particular time period. For example, as long as one or more members of the group are present in the database for a certain duration of time, the group may be present in the database for a particular time period. Figure 2 In the open environment 200, the activity group can remain active.

[0050] At block 906, the machine learning system may determine whether a person or group (e.g., a subgroup) is at least part of an activity group based at least on the training data and the identified pattern(s) indicative of activity groups. For example, the machine learning system may determine that two individuals who have spent the past hour together are a group. As another example, the grouping system may group individuals that display affection for one another (such as holding each other's hands). In practice, the grouping system may employ weighted probabilities to determine whether one or more individuals should be grouped. Specifically, one or more behavioral patterns, characteristics, and / or attributes observed using certain tracked identities may be weighted using probabilities and used to determine groups. For example, the grouping system may assign a weighted probability (e.g., a value between zero and one) to individuals of small stature, as infants or toddlers may not typically be left unsupervised by adults. However, the grouping system may employ weighted probabilities for other attributes and / or behaviors of individuals (e.g., common location, proximity, association with the same vehicle, etc.). The grouping system may sum the weighted probability values ​​for being in a group and may separately determine whether the individual(s) associated with the weighted probability value are part of a group or are a group. In some embodiments, one of the weighted probability values ​​may be sufficient to exceed a threshold value to determine that the individual(s) with which the weighted probability value is associated are part of a group or form a group, respectively.

[0051] At block 908, the grouping system can update a database containing active groups. As newly identified groups are identified, the database containing active groups can be updated to allow for tracking. Thus, the machine learning system can maintain a record of who has been grouped. In some embodiments, grouping can impact downstream processing. For example, as discussed below, group tracking can even be utilized within a gaming experience.

[0052] As mentioned herein, a grouping system can identify groups of individuals based on activities / patterns observed in time series data (e.g., as captured by sensors of a sensor system). Any number of patterns can be identified by a machine learning algorithm that can implement supervised machine learning to train the grouping system how to identify groups. For the sake of discussion, Figure 101000, which provides examples of sets of individuals that may be considered as groups and / or may not be groups by the grouping system using the technology described herein. Specifically, as shown in diagram 1002, individuals leaving or entering the same vehicle may be considered as potentially grouped by the grouping system. In contrast, as shown in diagram 1004, individuals leaving or entering different vehicles (wherein there is a relatively long distance between the vehicles) may be considered as potentially grouped. In addition, as shown in diagram 1006, individuals that are close to each other for a significant amount of time may be grouped. However, individuals spaced apart that may exceed a specific threshold distance in a queue at an attraction may be considered as unlikely to be grouped. For example, in diagram 1008, each person 1011 represents 20 individuals in a row. As shown in diagram 1008, individual 1010 is separated from another individual 1012 in the row at the attraction by 100 people. Therefore, the grouping system can determine that individual 1010 and individual 1012 should not be grouped. However, in some cases, the grouping system can determine that even if individuals are spaced apart, they may be considered as potentially grouped. As shown in diagram 1014, even if person 1016 is separated from people 1018 and 1020, people 1016, 1018, and 1020 may still be considered to be grouped. In this case, people 1018 and 1020 will walk toward restroom 1022, while person 1016 waits outside. Even if person 1016 may be alone for a period of time, the grouping system may determine that people 1016, 1018, and 1020 are a group. In addition, the grouping system may also group separate individuals, but the time they spend together exceeds a certain threshold time period. For example, a person in a group may choose to sit in a rest area rather than wait in a queue at an attraction with another person in the same group. Therefore, the grouping system may keep a group in an active state for a period of time, even if one or more members of the group are separated.

[0053] As can be appreciated, once a group is identified, additional functionality can be implemented. For example, Figure 111100 is an illustration of how a group identified by a grouping system can change the interactive environment of an interactive game. In illustration 1100, it is necessary to comply with rule 1102 on display 1103 to unlock door 1104 (e.g., a portal in a virtual and / or augmented reality experience, etc.). Rule 1102, which indicates that "all team members must enter together," must be met to enter door 1104. Here, the grouping system has identified persons 1114 and 1112 as part of a group / team with person 1110. However, these persons 1114 and 1112 are not present with person 1110. Since not all team members were present with team member 110 at time 1111, the door is not unlocked. Later, at time 1113, all team members (e.g., persons 1110, 1112, 1114) appear at door 1104. In response to the presence of the entire group, an effect appears from the interactive environment (e.g., door 1104 unlocks). Thus, by tracking groups, one or more components of the interactive environment can be controlled based on aspects of the group (e.g., location of team members, cumulative activity of team members, etc.) to provide guests with an entertaining and interactive experience.

[0054] Figure 12 is a diagram illustrating an example control action 1200 that may be implemented based on group identities tracked with the context tracking techniques provided herein, according to an embodiment. Figure 8 , Figure 12 A kiosk 1202 or another device may include a display 1204. Display 1204 may present a graphical user interface (GUI) 1206 that provides information about the open environment. As illustrated, GUI 1206 may be personalized based on the tracked identity. For example, as illustrated by status balloon 1208, the context tracking system may identify the tracked object as part of a tracked group "Group 1" based on a balance of contextual data between various target sensors. In response to the tracked object approaching display 1204, a control action command may be presented to display 1204 (e.g., the underlying hardware controlling display 1204) that instructs GUI 1206 to display information related to the identified group. For example, Group 1 is provided with instructions associated with its identity. Specifically, GUI 1209 displays "Hello Group 1, complete the task in less than 10 minutes." As will be appreciated, each group identity may be associated with a unique identifier and other information (e.g., wait time, instructions, demographic information, and / or other personalized data) to enable personalized control actions to be performed.

[0055] In another embodiment, each member of a tracked group 1212 (i.e., Group 1) may have a wearable device 1210 communicatively coupled to one or more components of a contextual tracking system. Based on interactions with an open environment feature 1214 (here, a virtual game mystery box), personalized data associated with the identity of the tracked group may be updated. For example, a game database 1216 may be updated via a control action command to reflect the updated game state based on the interaction with the open environment feature 1214. Thus, as illustrated in database record 1218, an additional 15 points are provided in the database to the identity 456 associated with Group 1. Furthermore, a display 1220 of the wearable device 1210 on each of the group members may be controlled via a control action command to display an indication 1222 confirming the state change based on each member of the tracked group 1212's interaction with the open environment feature 1214.

[0056] The technology presented and claimed herein is cited and applied to substantial objects and specific examples of a practical nature that demonstrably advance the art and, therefore, is not abstract, intangible, or purely theoretical. Furthermore, if any claim appended to the end of this specification contains one or more elements designated as "means for [performing] ... [function]" or "steps for [performing] ... [function]," it is intended that such elements be construed under 35 U.S.C. § 112(f). However, for any claim containing elements designated in any other manner, it is intended that such elements not be construed under 35 U.S.C. § 112(f).

Claims

1. A tangible, non-transitory, machine-readable medium comprising machine-readable instructions that, when executed by one or more processors of a machine, cause the machine to: receiving a tracked target context of a first tracked object from a first tracking sensor system; providing the tracked target context from the first tracking sensor system to a second tracking sensor system different from the first tracking sensor system; as well as Recognition of a newly observed tracked target by the second tracking sensor system is caused based on the tracked target context from the first tracking sensor system.

2. The machine-readable medium of claim 1 , comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: filtering out a subset of a set of candidate identities that the newly observed tracked target can be identified as based on the context from the first tracking sensor system; and The set of candidate identities without the subset is provided to the second tracking sensor system.

3. The machine-readable medium of claim 2, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: The subset is determined to be part of the set of candidate identities tracked at a previous location outside the identified location range based on a time difference between a time when the set of candidate identities was tracked at the previous location outside the identified location range and a time when the newly observed tracked target was observed by the second tracking sensor system.

4. The machine-readable medium of claim 2, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: generating a blacklist based on the subset of the set of candidate identities; and providing the blacklist to the second tracking sensor system.

5. The machine-readable medium of claim 2, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: generating a blacklist based on the subset of the set of candidate identities; and providing the set of candidate identities without the blacklist to the second tracking sensor system.

6. The machine-readable medium of claim 1 , comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: filtering into a subset of a set of candidate identities that the newly observed tracked target can be identified as based on the context from the first tracking sensor system; and The subset is provided to the second tracking sensor system.

7. The machine-readable medium of claim 6, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: The subset is determined to be part of the set of candidate identities tracked at a previous location within the identified location range based on a time difference between a time when the set of candidate identities was tracked at the previous location and a time when the newly observed tracked target was observed by the second tracking sensor system.

8. The machine-readable medium of claim 6, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: generating a whitelist based on the subset of the set of candidate identities; and providing the whitelist to the second tracking sensor system.

9. The machine-readable medium of claim 1 , wherein the first tracking sensor system, the second tracking sensor system, or both comprise: Light detection and ranging (LIDAR) systems, radio frequency identification (RFID) systems, computer vision systems, time of flight (ToF) systems, millimeter wave (mmWave) systems, or any combination thereof.

10. The machine-readable medium of claim 9, wherein the second tracking sensor system comprises the LIDAR system.

11. The machine-readable medium of claim 1 , comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: determining a predicted confidence score indicative of a confidence level of the identification of the newly observed tracked target; Additional tracking sensor system input is collected for another identification of the newly observed tracked target in response to the predicted confidence score failing to meet a confidence threshold.

12. The machine-readable medium of claim 11 , comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to collect the additional tracking sensor system input by providing directions to the newly observed tracked target to proceed into a tracking sensor system coverage area.

13. The machine-readable medium of claim 12, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to provide encouragement by incentivizing the newly observed tracked target to proceed into the tracking sensor system coverage area.

14. The machine-readable medium of claim 1, wherein the first tracked object comprises a different object type than the newly observed tracked target.

15. The machine-readable medium of claim 14, wherein the first tracked object comprises a vehicle and the newly observed tracked objects comprise one or more persons.

16. The machine-readable medium of claim 1 , comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: receiving training data indicating a set of persons and associated attributes; identifying patterns in the associated attributes to identify a grouping attribute indicating persons who should be grouped; determining whether the first tracked object is associated with the pattern; as well as In response to determining that the first tracked object is associated with the pattern, the first tracked object is identified as part of an active group.

17. The machine-readable medium of claim 16, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: Based on the pattern, determining whether the second tracked object is a member of the activity group is performed by: determining an amount of time the first tracked object and the second tracked object have spent in a threshold proximity to each other; and When the amount of time exceeds a threshold, the second tracked object is associated with the activity group.

18. The machine-readable medium of claim 16, comprising machine-readable instructions that, when executed by the one or more processors of the machine, cause the machine to: Based on the pattern, determining whether the second tracked object is a member of the active group is performed by: associating a first weighted probability with the second tracked object, wherein the first weighted probability represents a likelihood that the second tracked object is a member of the activity group; associating a second weighted probability with the second tracked object, wherein the second weighted probability represents a second likelihood that the second tracked object is a member of the activity group; and When a sum of the first weighted probability and the second weighted probability exceeds a threshold, the second tracked object is associated with the activity group.

19. The machine-readable medium of claim 18, wherein the first weighted probability is a value representing an amount of time that the second tracked object has spent in the vicinity of the first tracked object.

20. A computer-implemented method comprising: receiving a tracking sensor system input from a first tracking sensor system, the tracking sensor system input comprising a tracked target context of a first tracked object; providing the tracked target context from the first tracking sensor system to a second tracking sensor system different from the first tracking sensor system; as well as Recognition of a newly observed tracked target by the second tracking sensor system is caused based on the tracked target context from the first tracking sensor system.

21. The computer-implemented method of claim 20, comprising: A portion of candidate identifiers of the newly observed tracked target is filtered in, filtered out, or both filtered in and out based on the tracked target context.

22. The computer-implemented method of claim 20, comprising: determining a predicted confidence score indicative of a confidence level of the identification of the newly observed tracked target; as well as Additional tracking sensor system input is collected for another identification of the newly observed tracked target in response to the prediction confidence score failing to meet a threshold.

23. The computer-implemented method of claim 20, wherein the first tracked object and the newly observed tracked target comprise a first group of individuals.

24. A system comprising: a first tracking sensor system configured to track a first tracked object in a first coverage area; a second tracking sensor system configured to track a second tracked object in a second coverage area; as well as A contextual tracking system that is configured to: receiving a tracked target context of the first tracked object from the first tracking sensor system; providing the tracked target context from the first tracking sensor system to a second tracking sensor system that is different from the first tracking sensor system; as well as Recognition of the second tracked object is caused based on the tracked target context from the first tracking sensor system, wherein the second tracking sensor system is different from the first tracking sensor system.

25. The system of claim 24, wherein the first tracking sensor system comprises a different sensor type than the second tracking sensor system.

Images