Apparatus and method for determining potential paths of an object - Patents.com

Abstract

The present disclosure provides methods, apparatus, and systems for determining potential paths of an object, the method of determining potential paths of an object includes determining whether a first location at which a first object is detected at a detection time along a first path of a first object is within a ground distance of a second location at which the second object is detected after the detection time along a second path of a second object, and in response to determining that the first location is within the ground distance from the second location, combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object.

Description

[Technical field]

[0001] The present disclosure relates to methods, devices and systems for detecting or determining the path of an object, and more particularly, to methods, devices and systems for determining potential paths of an object. [Background technology]

[0002] There is an increasing need to track people in high resolution for various reasons, such as monitoring patients wandering around a hospital ward, understanding shopper movement patterns in a shopping mall, tracking persons of interest for law enforcement and security purposes, etc. Among all the methods currently in use, one method is the use of facial recognition (FR) via video cameras, but this method requires the deployment of many cameras to achieve fine-grained tracking. Summary of the Invention [Problem to be solved by the invention]

[0003] The use of multi-mode sensors, combining the use of cameras for FR with the use of sensors of different modes such as Wi-Fi and LiDAR (light detection and ranging), is becoming increasingly common to achieve fine-grained tracking. However, some sensors, such as LiDAR, only work well when there is a line-of-sight from the sensor to the object being tracked, and do not work well and accurately when there are occlusions caused by other objects coming in between. When occlusions occur, or when two or more objects are too close to each other, tracking of the object may be interrupted, and the sensor may become confused and confuse or lose track of the object.

[0004] Therefore, there is a need to address the above problems and limitations of multi-modal sensing techniques / systems that combine FR with fine-grained object tracking methods such as LiDAR sensing to develop methods, apparatus and systems for determining potential paths of an object to achieve high-resolution tracking over a large area, i.e., across multiple cameras / sensors, within or lack of line-of-sight conditions.

[0005] Furthermore, other desirable features and characteristics will become apparent from the following detailed description and the appended claims, taken in conjunction with the accompanying drawings and background of this disclosure. [Means for solving the problem]

[0006] In a first aspect, the present disclosure provides a method for determining a potential path of an object, the path of the object including a plurality of locations where the object is detected at a corresponding plurality of detection times, the method including: determining whether a first location along a first path of the first object at a detection time is within a ground distance of a second location along a second path of the second object at which a second object is detected after the detection time, and, in response to determining that the first location is within a ground distance of the second location, combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object.

[0007] In a second aspect, the present disclosure provides an apparatus for determining a potential path of an object, the path including a plurality of locations where an object is detected at a corresponding plurality of detection times. The apparatus includes at least one processor and at least one memory including computer program code configured to cause a server, in conjunction with the at least one processor, to at least: determine whether a first location along a first path of the first object at a detection time is within a ground distance from a second location along a second path of the second object at a detection time where a second object is detected after the detection time; and, in response to a determination that the first location is within a ground distance from the second location, combine at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object.

[0008] In a third aspect, the present disclosure provides a system for determining potential paths of an object, comprising an apparatus of the second aspect above, an object tracking apparatus, and an imaging apparatus.

[0009] Additional benefits and advantages of the disclosed embodiments will become apparent from the specification and drawings, in which the benefits and / or advantages may be obtained individually from various embodiments and features of the specification and drawings, and not all need be provided to obtain one or more of such benefits and / or advantages. [Brief description of the drawings]

[0010] Embodiments of the present disclosure will be better understood and readily apparent to those skilled in the art from the following specification, merely by way of example and in conjunction with the drawings in which:

[0011] [Figure 1] FIG. 1 is a schematic diagram showing a floor plan of a room and a system installed therein for determining potential paths for a person 102, according to an embodiment of the present disclosure.

[0012] [Diagram 2] FIG. 2 is a schematic diagram showing a floor plan of a room and a system installed therein for determining potential paths for a person 202a, according to an embodiment of the present disclosure.

[0013] [Diagram 3] FIG. 2 is a schematic diagram showing a floor plan of a room and a system installed therein for determining potential paths for a person 202a, according to another embodiment of the present disclosure.

[0014] [Figure 4] 1 shows a block diagram of a system for determining potential paths of an object.

[0015] [Diagram 5] 1 shows a flowchart illustrating a method for determining potential paths of an object according to various embodiments of the present disclosure.

[0016] [Figure 6] FIG. 1 is a block diagram illustrating a system for determining potential paths of an object, according to various embodiments of the present disclosure.

[0017] [Figure 7] FIG. 1 is a schematic diagram illustrating a system for determining potential paths for a person according to a first embodiment of the present disclosure.

[0018] [Figure 8A] FIG. 8 illustrates potential paths of an object determined by the system of FIG. 7. [Figure 8B] FIG. 8 illustrates potential paths of an object determined by the system of FIG. 7.

[0019] [Figure 9] 1 shows a schematic diagram illustrating exemplary decision area angles, according to an embodiment of the present disclosure;

[0020] [Figure 10]1 shows a schematic diagram illustrating an exemplary decision region length, according to an embodiment of the present disclosure.

[0021] [Figure 11A] 1 illustrates a person tracking scenario in which two people (target person and person A) at different heights and distances from the tracking sensor are tracked according to an embodiment of the present disclosure. [Figure 11B] 1 illustrates a person tracking scenario in which two people (target person and person A) at different heights and distances from the tracking sensor are tracked according to an embodiment of the present disclosure. [Figure 11C] 1 illustrates a person tracking scenario in which two people (target person and person A) at different heights and distances from the tracking sensor are tracked according to an embodiment of the present disclosure.

[0022] [Figure 12] FIG. 2 illustrates a schematic diagram showing an object tracking subsystem for determining potential paths for a person, according to various embodiments of the present disclosure.

[0023] [Figure 13] FIG. 1 illustrates a diagram illustrating an object tracking system for determining potential paths for a person, according to various embodiments of the present disclosure.

[0024] [Figure 14] FIG. 13 illustrates a reliability score calculation table according to an embodiment of the present disclosure.

[0025] [Figure 15A] 1 illustrates an exemplary target object track group table in accordance with an embodiment of the present disclosure.

[0026] [Figure 15B] FIG. 1 shows a schematic diagram illustrating pathways at different association levels according to an embodiment of the present disclosure.

[0027] [Figure 16]1 shows a flowchart illustrating a process for forming track groups for an object according to an embodiment of the present disclosure.

[0028] [Figure 17] FIG. 13 shows a schematic diagram illustrating a system including an object tracking subsystem for determining potential paths of a person according to a second embodiment of the present disclosure.

[0029] [Figure 18] 13 illustrates a track group table for an exemplary target object according to a second embodiment of the present disclosure.

[0030] [Figure 19A] 13 shows a flowchart illustrating a process of forming a track group of a target person according to a second embodiment of the present disclosure. [Figure 19B] 13 shows a flowchart illustrating a process of forming a track group of a target person according to a second embodiment of the present disclosure. [Figure 19C] 13 shows a flowchart illustrating a process of forming a track group of a target person according to a second embodiment of the present disclosure.

[0031] [Figure 20A] 1 shows a flowchart illustrating a process for displaying potential paths of an object under multiple sensor settings according to an embodiment of the present disclosure. [Figure 20B] 1 shows a flowchart illustrating a process for displaying potential paths of an object under multiple sensor settings according to an embodiment of the present disclosure. [Figure 20C] 1 shows a flowchart illustrating a process for displaying potential paths of an object under multiple sensor settings according to an embodiment of the present disclosure. [Figure 20D] 1 shows a flowchart illustrating a process for displaying potential paths of an object under multiple sensor settings according to an embodiment of the present disclosure.

[0032] [Figure 21A]FIG. 2 illustrates an exemplary target person track group table and associated tracks, according to an embodiment of the present disclosure.

[0033] [Figure 21B] FIG. 2 illustrates a diagram showing potential paths for an object, according to an embodiment of the present disclosure.

[0034] [Figure 22A] FIG. 1 shows a schematic diagram illustrating a system including two object tracking sensors for determining potential paths of an object, according to an embodiment of the present disclosure. [Figure 22B] FIG. 1 shows a schematic diagram illustrating a system including two object tracking sensors for determining potential paths of an object, according to an embodiment of the present disclosure.

[0035] [Figure 23] FIG. 1 shows a schematic diagram illustrating a system including an object tracking subsystem for determining a vicinity trajectory and potential contacts of a target person, according to an embodiment of the present disclosure.

[0036] [Figure 24] FIG. 1 shows a schematic diagram illustrating a system including an object tracking subsystem for determining a vicinity trajectory and potential contacts of a target person according to another embodiment of the present disclosure.

[0037] [Diagram 25] 1 shows a flow diagram illustrating a process of determining potential paths of an object and modifying the potential paths and their data entries in a database (DB) for communication to a sensor and / or human monitoring server according to an embodiment of the present disclosure.

[0038] [Figure 26] 5 and 6. FIG. 6 illustrates a schematic diagram of an exemplary computing device suitable for use in performing the method of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0039] Terminology Object - An object can be a person, a pet, a vehicle, an object, an item, a device, a pole, a piece of furniture, or any matter, fixed or moving. An object can be animate or non-animate. For animate objects such as people and pets, the object can typically be detected based on facial features, body parts, characteristic, and object motion, or a combination thereof. Examples of facial features of an object (person) include the relative position, size, shape, and / or contour of the eyes, nose, cheekbones, jaw, and chin, as well as iris pattern, skin color, hair color, or a combination thereof. Characteristics include physical characteristics such as height, body size, body proportions, limb length, hair color, skin color, clothing, possessions, and other similar features or combinations. Motion may include behavioral characteristics such as body movement, limb position, direction of movement, speed of movement, walking pattern, the way the object stands, moves, speaks, or changes in physical characteristics due to interaction with other objects, other similar characteristics or combinations. For non-living objects, the object may typically be based on shape, color, speed of movement, movement characteristics / patterns, and changes in physical characteristics due to interaction with other objects.

[0040] In various embodiments of the present disclosure, an object can be detected by an image capture device such as a camera by capturing an image of the object and identifying the object based on the facial features, body parts, characteristics and / or movement of the object in the image. Additionally or alternatively, the object can be detected by an object tracking device such as a LiDAR. An object tracking device such as a LiDAR detects an array of light pulses emitted by the object tracking device and reflected back from surrounding objects, measures the distance (e.g., angular distance and axial distance) between the object and the object tracking device based on the detected light pulses, and optionally generates a 3D map of all objects around the object tracking device based on their respective distances, tracks the change in the object's distance to the object tracking device over time, and identifies its path. In general, an object tracking device such as a LiDAR can provide a wide range of fine-grained tracking compared to an imaging device, whose detection is limited to a field of view. In various embodiments below, the term "sensor" can be used to refer to one or both of the imaging device and the object tracking device.

[0041] In various embodiments, an object identifier is assigned to the object upon detection and to data acquired by the sensor and used in and associated with the object's identification and detection for subsequent identification and tracking of the object. If the object is subsequently identified based on the same or other facial features, body parts, characteristics, movements, or combinations thereof of the object, the same object ID is assigned.

[0042] In various embodiments below, when a potential path determination is performed for an object, the object may be referred to as a target object.

[0043] Location - A location is a position or spot in space where an object is detected by a sensor. The space may be an open space, a closed space such as a room, or may correspond to the field of view of the sensor that detects the object. Note that sensors that detect and track the path of an object are typically mounted on a wall or ceiling and located at a distance from the object.

[0044] In one embodiment, XY coordinates referenced to a sensor or a point in space (e.g., a corner or a center) can be used to specify a location and quantify the ground distance between two locations (e.g., the distance between two objects or the distance between an object and a sensor). In another embodiment, polar coordinates referenced to the sensor using a pan angle (or radians), a tilt angle (radians), and / or a focal length of the sensor can be used to specify a location and quantify the ground distance. For example, a sensor may be pre-configured with a reference line / plane in space, an object may be detected at a specific angular distance (or angle) away from the reference line / plane or with a reference position / location in space, and an axial distance between the object and the reference position / location of the sensor may be calculated based on the tilt angle (radians), a known or pre-configured height of the object (e.g., the average height of a human), and / or a focal length of the sensor to focus on the object.

[0045] In various embodiments, the term “location” may be referred to as a “detection point.” Object location data and its corresponding time data associated with the detection of the object are recorded for processing and for determining the path of the object.

[0046] Detection time - The detection time is the timestamp when an object is detected by a sensor. In various embodiments, such time data of the object and its corresponding location data related to the detection of the object are recorded for processing and determining the path of the object.

[0047] Pathway - A pathway is a passage or path that connects multiple locations (detection points) of an object with the same object ID over consecutive times. More specifically, there are multiple instances of an object with the same object ID being detected by a sensor over consecutive times in space (e.g., the field of view of the sensor), and by concatenating / joining each detection point (location) from the multiple detection instances according to their chronological order (detection time), the path of the object can be determined.

[0048] Potential pathway - A potential pathway is a pathway that could potentially represent the exact path of an object. This potential pathway may differ from the path of the current object. A potential pathway for such an object (target object) is determined when the path tracking of the current object is unreliable or when the system or device that detects the path of the current object is unreliable and no longer accurately represents the exact path of the object. For example, the confidence level of an object's path may be affected if there is a blockage of line of sight at one detection point along the object's path, if there is another object moving close to the object around the detection point, if there is an abrupt change in the object's path, if the object suddenly disappears from the detection area, or if a new object is detected near the point where the object was last detected.

[0049] Ground distance - Ground distance is the distance extended from the detection point of the target object, thereby forming a region centered on the detection point of the target object. Such distance or region is used to search and determine if another object was also detected at the same time or shortly after the detection point, which is then used to determine if the tracking of the target object is reliable and potential (alternate) paths for the target object.

[0050] In one embodiment, such ground distances are fixed and extend outward in all directions from the detection point of the object, thereby forming a circular area having a fixed radius centered on the detection point of the object.

[0051] In another embodiment, the ground distance may extend from the detection point relative to the sensor, for example, the ground distance may extend from the detection point an axial distance towards and away from the sensor's reference position / location and an angular distance towards and away from the sensor's reference line / plane, thereby forming a rectangular area centered on the detection point of the object.

[0052] Exemplary embodiments Embodiments of the present disclosure will now be described, by way of example only, with reference to the drawings in which like reference numbers and letters refer to similar elements or equivalents.

[0053] Some portions of the description which follow are explicitly or implicitly presented in terms of algorithms and functional or symbolic representations of operations on data within a computer memory. These algorithmic descriptions and functional or symbolic representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is conceived to be a self-consistent sequence of steps leading to a desired result. These steps are steps requiring physical manipulations of physical quantities, such as electrical, magnetic or optical signals capable of being stored, transferred, combined, compared, and otherwise manipulated.

[0054] Unless otherwise noted, and as will become apparent below, throughout this specification discussions using terms such as "receiving," "calculating," "determining," "updating," "generating," "initializing," "outputting," "retrieving," "identifying," "dispersing," "authenticating," and the like, will be understood to refer to operations and processes of a computer system or similar electronic device that manipulates and transforms data, represented as physical quantities within the computer system, into other data, similarly represented as physical quantities in the computer system or other information storage, transmission, or display device.

[0055] This specification also discloses an apparatus for performing the operations of the method. Such an apparatus may be specially constructed for the required purposes, or may comprise a computer or other device selectively activated or reconfigured by a computer program stored in the computer. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various machines can be used with programs in accordance with the teachings of this specification. Alternatively, the construction of more specialized apparatus to perform the required method steps may be appropriate. The structure of the computer will become apparent from the following description.

[0056] Furthermore, this specification also implicitly discloses a computer program in that it is clear to those skilled in the art that the individual steps of the methods described herein may be implemented by computer code. The computer program is not intended to be limited to a specific programming language and its implementation. It will be understood that a variety of programming languages ​​and their coding may be used to implement the teachings of the disclosure contained herein. Furthermore, the computer program is not intended to be limited to a specific control flow. There are many other variations of the computer program that can use different control flows without departing from the spirit or scope of the present disclosure.

[0057] Furthermore, one or more steps of a computer program may be executed in parallel rather than sequentially. Such a computer program may be stored on any computer-readable medium. The computer-readable medium may include a storage device such as a magnetic or optical disk, a memory chip, or other storage device suitable for interfacing with a computer. The computer-readable medium may also include a hard-wired medium, as exemplified by the Internet system, or a wireless medium, as exemplified by the GSM mobile phone system, the Long Term Evolution (LTE) system, and the 5G mobile network system. When loaded and executed on such a computer, the computer program effectively produces an apparatus for performing the steps of the preferred method.

[0058] Various embodiments of the present disclosure relate to methods and apparatus for determining potential paths of an object. It will be appreciated by those skilled in the art that such apparatus and image capturing devices may be implemented as part of a system to provide the same technical effect.

[0059] FIG. 1 shows a schematic diagram 100 illustrating a system for determining a floor plan of a room and potential paths of a person 102 located therein, according to an embodiment of the present disclosure. The system may include a person identification sub-system 104 and a high-resolution object tracking sub-system 106 operating in sync. The person identification sub-system 104 includes an imaging device, such as a camera, configured to capture an image of the entrance of the room and any person appearing at the entrance. The person identification sub-system 104 is also configured to identify a person ID (or object ID) of the person based on facial features in the image, for example, by comparing the facial features with all facial features in a database in communication with the person identification sub-system 104. The object tracking sub-system 106 may be configured to scan the entire room to detect all objects in the room within its line-of-sight, including objects (people) at the entrance, at regular intervals. The object tracking subsystem 106 associates the object at the entrance with the person at the entrance identified by the person identification subsystem 104 and assigns the same person ID (or object ID) to the object. In this manner, subsequent detection of the object by the object tracking subsystem 106 and the detection data of the object (e.g., object shape / contour data, location data, and time data) are assigned to the same person ID (object ID) and stored in the database or people monitoring server 108 for subsequent object track search, identification and monitoring purposes.

[0060] A person walking around the room over a period spanning multiple detection times / intervals of the object tracking subsystem 106 will accordingly cause the object tracking subsystem 106 to generate multiple detection points for the person under the same person ID.

[0061] In this embodiment, the person 102 enters the room and the person identification subsystem 104 captures an image 104 of the person 102 and identifies the person ID of the person 102 (e.g., person ID "Bob"). Separately, the object tracking subsystem 106 may detect multiple objects in the room, including an object (in this case, the person 102) at the entrance point 111, and associate the object 102 with the person ID "Bob". Then, as the person 102 moves around the room, the object tracking subsystem 106, which scans all objects in the entire room at regular intervals, detects the object under person ID "Bob" at multiple detection points in the room, as shown with points 111-120. Such detection data and associated person IDs are then stored in a database in communication with the person monitoring server 108. The multiple detection points of the object can then be linked / combined according to a time sequence to determine the path 122 of the object in the room, which is displayed on the client device or server 110.

[0062] Such object tracking methods using multiple modes of sensors work well when there is a line-of-sight from the sensor to the person being tracked, but when there are occlusions caused by other people or objects, e.g., pillars, the track ID (e.g., object ID or person ID) may be mistakenly swapped for another person or object, or the track ID may become corrupted because the system is no longer confident and a new track ID is assigned to the person as if a new person or object had been detected.

[0063] As mentioned above, person tracking may be unreliable when there is line of sight obstruction. Figures 2 and 3 show schematic diagrams 200 and 300, respectively, illustrating a floor plan of a room and a system for determining potential paths of a person 202a installed therein, according to an embodiment of the present disclosure. A person 202a enters the room, and a person identification subsystem 204 captures an image 205 of the person 202a and identifies a person ID (e.g., person ID "Bob") of the person 202a. Separately, an object tracking subsystem 206 may detect the person at point 211 and associate the object 202a with person ID "Bob" as well as other people under different person IDs in the room, including people 202b, 202c, and 202d. As the persons 202a-d move around in the room, the object tracking subsystem 206, which scans all objects in the room at regular intervals, detects the person 202a and other persons b-d at multiple detection points in the room, associates their detection data (e.g., object shape / outline data, position data and time data) under their respective person IDs and stores them in the surveillance server 208.

[0064] However, in one example, as shown in Figure 2, when person 202a walks to point 217, person 202b may stand between person 202a and object tracking subsystem 204, blocking the line of sight (indicated by the dashed line in Figure 2) from object tracking subsystem 206 to person 202a. Therefore, object tracking subsystem 206 may not be able to detect the location of person 202a (Bob) and may instead identify person 202b standing at point 227 as person 202a and person 202a at point 217 as person 202b. Their person identities may be switched before and after the block. Thus, the subsequent movements of person 202b and detection of person 202b at points 227-230 may be characterized under person 202a with person ID "Bob" (Track 1), and the subsequent movements of person 202a and detection of person 202a at points 217-220 may be characterized under person 202b with person ID "Bob" (Track 2). As a result, if a user requests to view Bob's activities by searching Track 1, Bob's incorrect path 240 may be displayed on the screen of the client device or server 210.

[0065] In another example, as shown in Fig. 3, when person 202a walks to point 317, person 202b may stand between person 202a and object tracking subsystem 306, blocking the line of sight (indicated by the dashed line in Fig. 3) from object tracking subsystem 306 to person 202a, and object tracking subsystem 306 cannot detect the position of person 202a. As a result, object tracking subsystem 306 may lose track of person 202a after point 316, and the subsequent movement of person 202a and detection of person 202a at points 317-320 may be characterized as that of new person 202e under a new person ID (track 5), while the subsequent movement of person 202b and detection of person 202b at points 227-230 may be characterized under person 202a with person ID "Bob" (track 1). As a result, an incorrect or incomplete route 340 of person 202a ending at point 316 is displayed on the screen of the client device or server 210.

[0066] Therefore, there is a need to provide a method and apparatus capable of determining potential paths of an object to address the problems and limitations discussed above.

[0067] FIG. 4 shows a block diagram of a system 400 for determining potential paths of an object.

[0068] System 400 includes a requestor device 402, an object pathway determination server 408, a coordination server 440, hosts 450A-N, and sensors 442A-N.

[0069] The requester device 402 communicates with the object routing server 408 and / or the coordination server 440 via connections 416 and 421, respectively. The connections 416 and 421 may be wireless (e.g., via NFC communication, Wi-Fi communication, Bluetooth, etc.) or may be over a network (e.g., the Internet). The connections 416 and 421 may be network (e.g., the Internet) connections.

[0070] The object routing server 408 further communicates with a coordination server 440 via a connection 420. The connection 420 may be over a network (e.g., a local area network, a wide area network, the Internet, etc.). In one configuration, the object routing server 408 and the coordination server 440 are combined and the connection 420 may be an interconnected bus.

[0071] The linking server 440 in turn communicates with the hosts 450A-422N via respective connections 422A-422N. The connections 422A-422N may be a network (eg, the Internet).

[0072] The hosts 450A to 450N are servers. In this specification, the term host is used to distinguish the hosts 450A to 450N from the linked server 440. In this specification, the hosts 450A to 450N are collectively referred to as the host 450, and the host 450 refers to any one of the multiple hosts 450. The host 450 may be combined with the linked server 440.

[0073] As an example, the hosts 450 are managed by security personnel of an organization, and the coordination server 440 is a central server that coordinates the hosts 450 and determines which hosts 450 will transfer data and acquire data such as image input.

[0074] The sensors 442A-442N are connected to the cooperation server 440 or the object path determination server 408 via respective connections 444A-444N or connections 446A-446N. The sensors 442A-442N are collectively referred to as sensors 442 in this specification. The connections 444A-444N are collectively referred to as connections 444 in this specification, and the connection 444 refers to one of the multiple connections 444. Similarly, the connections 446A-446N are collectively referred to as connections 446 in this specification, and the connection 446 refers to one of the multiple connections 446. The connections 444 and 446 may be wireless (e.g., via NFC communication, Wi-Fi communication, Bluetooth, etc.) or may be over a network (e.g., the Internet). The sensor 442 may be one of an image capturing device, an object tracking device, a video capturing device, a motion sensor, and a temperature sensor, and may be configured to send input according to its type to at least one of the object path determination servers 408.

[0075] In an exemplary embodiment, each of the devices 402 and 442 and servers 408, 440, and 450 provides an interface to enable communication with the other connected devices 402 and 442 and / or servers 408, 440, and 450. Such communication is facilitated by application programming interfaces ("APIs"). Such APIs may be programmatic interfaces, such as graphical user interfaces (GUIs), Web-based interfaces, application programming interfaces (APIs), and / or portions of user interfaces, which may include a set of interface elements, messaging interfaces corresponding to messages of a communication protocol, and / or remote procedure calls (RPCs) corresponding to suitable combinations thereof.

[0076] Use of the term "server" herein may refer to a single computing device or to multiple interconnected computing devices that cooperate to perform a particular function, i.e., a server may be contained within a single hardware unit or may be distributed among several or many different hardware units.

[0077] Collaboration server 440

[0078] The coordination server 440 is associated with an entity (e.g., a business, an organization, a moderator of a service). In one configuration, the coordination server 440 is owned and operated by the entity that operates the server 408. In such a configuration, the coordination server 440 may be implemented as part of the server 408 (e.g., a computer program module, a computing device, etc.).

[0079] The coordination server 440 may be configured to manage the registration of users. A registered user has an object tracking account that contains the user's details. The registration step is called onboarding. A user can perform onboarding to the coordination server 440 using any of the requester devices 402.

[0080] It is not necessary to have an object tracking account with the coordination server 440 to access the functionality of the coordination server 440. However, there are features available to registered users. For example, it is possible to customize the ground distance or area around the path of an object to increase or decrease the number of objects that may be associated with a target object, and to graphically display the association level of the target object with other objects as well as the paths of other objects that have a high association level with the target object. These additional features are described below.

[0081] The user onboarding process is performed by the user via one of the requester devices 402. In one configuration, the user downloads an app (including an API for interacting with the coordination server 440) to the sensor 442. In another configuration, the user accesses a website (including an API for interacting with the coordination server 440) on the requester device 402.

[0082] The registration details may include, for example, the user's user identifier (ID) or face photo, the user's address, contact details, or other important information, and the sensors 442 authorized to update the object tracking account.

[0083] Once onboarded, users have an object tracking account that stores all the details.

[0084] Requester Device 402

[0085] The requester device 402 is associated with a subject (or requester) who is a party to an object path determination request originating from the requester device 402. The requester may be an interested member of the public or a security officer of an entity who is assisting in obtaining data necessary to obtain a graphical representation of an object's path within the entity. The requester device 402 may be a computing device such as a desktop computer, an interactive voice response (IVR) system, a smartphone, a laptop computer, a personal digital assistant computer (PDA), a mobile computer, a tablet computer, etc.

[0086] In one example configuration, requester device 402 is a computing device in a watch or similar wearable and is equipped with a wireless communication interface.

[0087] Object Routing Server 408

[0088] The object routing server 408 is as described in the glossary section.

[0089] The object path determination server 408 is configured to determine potential paths for an object.

[0090] Host 450

[0091] Host 450 is a server associated with an entity (such as a company or organization) that manages (e.g., establishes, administers) object information, such as detection data and object paths or potential paths.

[0092] In one configuration, the entity is a bank. Thus, each entity operates a host 450 to manage resources by that entity. In one configuration, the host 450 receives a warning signal that the path of an object or its detection point within a detection space or detection period has a low confidence level. The host 450 can then be configured to transmit resources to process detection data acquired by the sensor 442 about the object and its associated objects or within the detection space or around the detection period. For example, the host 450 may be configured to acquire relevant video or image input for processing.

[0093] Sensor 442

[0094] The sensor 442 is associated with a user associated with the requester device 402. The sensor 442 may be one of an imaging device, an object tracking device, a photographing device, a motion sensor, and a temperature sensor, and may be configured to send input depending on its type to at least one of the object path determination servers 408. Details of how the sensors may be utilized to determine potential paths for an object are provided below.

[0095] 5 illustrates a flowchart 500 illustrating a method for determining a potential path of an object according to various embodiments of the present disclosure. In step 502, a step is performed in which a first location at which a first object is detected upon detection along a first path of the first object is within a ground distance from a second location at which a second object is detected along a second path of the second object since said detection. In step 504, in response to determining in step 502 that the first location is within a ground distance from the second location, a step is performed in which at least a portion of the first path detected prior to said detection, at least one of the first location and the second location, and at least a portion of the second path detected since said detection are combined to form a continuous path of the first object.

[0096] FIG. 6 shows a block diagram illustrating a system 600 for determining potential paths of an object according to various embodiments of the present disclosure.

[0097] In one example, management of image input and signal input is performed by at least imaging device 602, object tracking device 603, and device 604. System 600 includes imaging device 602 and object tracking device 604 in communication with device 604. In one embodiment, device 604 can be generally described as a physical device including at least one processor 606 and at least one memory 608 including computer program code. The at least one memory 608 and computer program code are configured by at least one processor 606 to cause the physical device to perform the operations described in FIG. 5. Processor 606 is configured to receive input images from imaging device 602 and input signals (e.g., light pulses, object position) from object tracking device 603, or retrieve input images and input signals from a database. Alternatively or additionally, the multiple input images captured by the imaging device 602 and the multiple input signals (e.g., light pulses, object position) from the object tracking device 603 are stored in a database 610, and the processor 606 is configured to retrieve the multiple input images and the multiple input signals from the database 610.

[0098] The imaging device 602 may be a device such as a closed-circuit television (CCTV) that detects appearance and provides various data (camera data) including appearance data that may be used by the system to identify a person or object under a person / object ID. In one embodiment, data obtained from the imaging device 602 may be stored in a memory 608 of the device 604 or in a database 610 accessible by the device 604. The appearance data may include: (i) facial feature data, such as the relative position, size, shape and / or contour of the eyes, nose, cheekbones, jaw and chin, as well as iris pattern, skin color, hair color, or a combination thereof; (ii) physical characteristic data, such as height, body size, body proportions, limb length, hair color, skin color, clothing, possessions, other similar characteristics or a combination thereof; and (iii) behavioral characteristic data, such as body movements, limb position, direction of movement, speed of movement, walking pattern, the way a person / object stands, moves, speaks, changes in physical characteristics due to interactions with other people / objects, other similar characteristics or a combination thereof.

[0099] The object tracking device 603 may be a device such as a LiDAR sensor that provides various data (map data), including object shape data and location data, that may be used by the system to generate a three-dimensional map of surrounding objects and to track the location and path of a person / object. In one embodiment, data obtained from the object tracking device 603 may be stored in a memory 608 of the device 604 or in a database 610 accessible by the device 604. The shape data of an object may include: (i) facial feature data, such as the relative position, size, shape and / or contour of the eyes, nose, cheekbones, jaw and chin, or a combination thereof; (ii) physical characteristic data, such as height, body size, body proportions, limb length, clothing, possessions, other similar features or a combination thereof; and (iii) behavioral characteristic data, such as body movements, limb position, direction of movement, speed of movement, walking pattern, how a person / object stands, moves, speaks, changes in physical features due to interactions with other people / objects, other similar features or a combination thereof.

[0100] In one embodiment, camera data and map data such as position and resolution (e.g., in terms of XY coordinates, pan / tilt angles and focal length) and / or time data including a timestamp at which one or more persons are identified may also be derived from the imaging device 602 and the object tracking device 603. The camera data, track data and / or time data may be stored in a memory 608 of the device 604 or in a database 610 accessible by the device 604, and the processor 606 is configured to identify and retrieve appearance data, images, object shape data, three-dimensional object maps based on the time data. It should be appreciated that the database 610 may be part of the device 604.

[0101] According to the present disclosure, device 604 may be configured to communicate with imaging device 602, object tracking device 603, and database 610. In one example, device 604 may receive as input multiple images relating to the same field of view of the imaging device from imaging device 602 or retrieved from database 610, and after processing by processor 606 within device 604, may generate output such as person / object ID of person / object identified from the input, which may be used by object tracking device 603 to determine paths and potential paths of objects / persons.

[0102] More specifically, the memory 608 and the computer program code stored therein are configured by the processor 606 to cause the device 604 or the imaging device 602 to directly detect the appearance of an object as it enters the detection space, compare the appearance data with data stored in the database 610, and assign an identification (e.g., a person ID) to the object.

[0103] The memory 608 and computer program code stored therein are configured to cause the device 604 or object tracking device 603, using the processor 606, to directly generate a pulsed laser across the detection space, and detect the pulsed laser reflected by the object to detect and determine multiple positions in the detection space where the object is located at different detection times over a detection time period, and form a path for the object based on the multiple positions of the object and their detection times, for example, by concatenating / combining the multiple positions of the object according to a chronological order (detection times) where the object is detected.

[0104] The memory 608 and computer program code stored therein are configured to cause the device 604, using the processor 606, to further associate identification information assigned to the object with the object detected by the object tracking device 603.

[0105] In various embodiments of the present disclosure, the memory 608 and computer program code stored therein are configured to cause the processor 606 to determine whether a first location at which a first object is detected along a first path at a detection time is within a ground distance from a second location at which a second object is detected along a second path of the second object after the detection time, and in response to determining that the first location is within the ground distance from the second location, to cause the device 604 to determine a potential path of the target (first) object including a plurality of locations at which the target object was detected at corresponding a plurality of detection times by combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object.

[0106] In one embodiment, the memory 608 and the computer program code stored therein are configured to cause the device 604, using the processor 606, to further combine a remainder of the second path, a remainder of at least one of the first location and the second location, and a remainder of the first path to form a continuous potential path for the second object. The memory 608 and the computer program code stored therein are configured to cause the device 604, using the processor 606, to further determine whether the second location is within a ground region centered on the first location.

[0107] In another embodiment, the first location is at a distance away from a location of an object tracker configured to track a first object along a first path and a second object along a second path and at an angular distance from a reference line of the object tracker, and the ground region centered on the first location is at an axial distance away proximate to the location of the object tracker and an angular distance away proximate to the reference line of the object tracker. distance), the memory 608 and computer program code stored therein are configured to cause the device 604, using the processor 606, to determine whether the position of the object tracking device is closer to the second position than the first position and whether the height of the second object is greater than the height of the first object, and further responsive to one of (a) determining that the position of the object tracking device is closer to the second position and the height of the second object is greater than the height of the first object, and (b) determining that the position of the object tracking device is closer to the first position and the height of the first object is greater than the height of the second object, combine at least a portion of the first path detected before the detection time, at least one of the first and second positions, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object.

[0108] In yet another embodiment, the memory 608 and computer program code stored therein may be configured to cause the device 604, using the processor 606, to obtain (a) a size of a first object at a first preceding location along a first path, the preceding location being a location where the first object was detected outside the ground area prior to the detection time, the size of the first object at the preceding location being comparable to a size of a second object at a second subsequent location along a second path, the second subsequent location being a location where the second object was detected outside the ground area after the detection time, and (b) a size of a first object at a preceding location along a first path, the size of the first object at a preceding location being comparable to a size of a second object at a second subsequent location along a second path, the second subsequent location being a location where the second object was detected outside the ground area after the detection time. and further configured to compare at least one of the speed of movement of the first object along the first path at one location (comparable to the speed of movement of the second object along the second path at a subsequent second location), and depending on the result of the comparison of at least one of the size and speed of movement of the first object and the second object, combine at least a portion of the first path detected before the detection time, at least one of the first and second positions detected after the detection time, and at least a portion of the second path to form a continuous potential path of the first object. The memory 608 and the computer program code stored therein are further configured to cause the processor 606 to cause the device 604 to apply further different weightings to the comparison of the magnitude and the comparison of the movement speed to calculate a score and determine whether the score is higher than a score calculated based on another comparison of (a) the size of the first object at the subsequent first location to the size of the first object at the previous first location and (b) the movement speed of the first object along the first path at the subsequent first location (wherein the subsequent first location is a location where the first object was detected outside the ground area after the detection time) to the movement speed of the first object along the first path at the previous first location, and if it is determined that the score is higher than the score calculated based on the another comparison, combine at least a portion of the first path detected before the detection time, at least one of the first and second locations, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object.

[0109] Alternatively and additionally, the memory 608 and the computer program code stored therein are configured to cause the device 604, using the processor 606, to further determine whether at least one of the sizes of the first object and / or the second object is within a normal human size range and the movement speed of the first object and / or the second object is lower than a human movement speed limit, and then, in response to further determining at least one of the sizes of the first object and / or the second object is within a normal human size range and the movement speed of the first object and / or the second object is lower than a human movement speed limit, to combine at least a portion of the first path detected before the detection time, at least one of the first position and the second position, and at least a portion of the second path detected after said detection time to form a continuous potential path of the first object.

[0110] Alternatively and additionally, the memory 608 and computer program code stored therein are configured to, using the processor 606, cause the device 604 to further display both the first path and the second path on a graphical interface (not shown) and to receive an input (not shown) in response to a query displayed on the graphical interface to combine at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object; and in response to receiving the input, combine at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time.

[0111] In the present disclosure, it is checked whether the tracking of the object is reliable, and if not, potential paths of the person (e.g., possible nearby tracks) are determined and collected. One way to determine the reliability of the object tracking is to determine the reliability of each detection point of the person along the current path detected by the object tracking device by checking whether other objects also appear within the detection distance or area from the detection point of the person. Such reliability evaluation and determination of potential paths can be performed retroactively after all detection data (location and time data) are obtained and all object paths are generated, or can be performed simultaneously at the time of identifying and tracking the object path and continuously upon detection of every new detection point of the object.

[0112] The following paragraphs present a first embodiment of the present disclosure in which a ground area used to determine the reliability of an object's current tracking and potential path for the object is constructed with reference to a reference line / plane and a reference position / location of the object tracking device.

[0113] FIG. 7 is a schematic diagram illustrating a system 700 for determining a potential path of a person according to a first embodiment of the present disclosure. In this embodiment, the system 700 includes an object tracking subsystem 701 configured to detect all objects moving within its line of sight at a certain time interval, identify paths of each object within its line of sight, and determine potential paths of the objects. As shown in FIG. 7, the object tracking subsystem 701 detects four people (herein shown as persons A-D) at different positions (detection points 702a-702d) at a detection time t. The person monitoring server 706 of the system 700 may store their detection data (e.g., object shape / contour data, location data, and time data) in association with their respective person IDs in a database in communication with the person monitoring server 706. The person monitoring server 706 also checks for the detection point 702a of person A whether other people are also detected (based on the detection point) within a determination area 704 centered on the detection point 702a.

[0114] XY coordinates relative to the object tracking subsystem 701 or a corner of a room may be used to indicate the detection points. For example, the position of the object tracking subsystem 701 has coordinates of (0,0) and the detection point 702a has coordinates of (x, y), indicating that the detection point 702a is an x ​​distance away from the object tracking subsystem 701 along the x axis and a y distance away from the object tracking subsystem 701 on the y axis perpendicular to the x axis. Alternatively, polar coordinates relative to the object tracking subsystem 701 may be used to indicate the detection points. For example, the position of the object tracking subsystem 701 has polar coordinates of (0,0) and the detection point 702a has polar coordinates of (p,r), indicating that the detection point 702a is an angular distance of p radians (or degrees) away from a reference line or plane of the object tracking subsystem 701 and an axial distance of r away from a reference position of the object tracking subsystem 701 at (0,0).

[0115] In this embodiment, as shown in the rectangular block 704 with length s1+s2 and angle 2q and with four edge points (pq,r-s1), (pq,r+s2), (p+q,r-s1), (p+q,r+s2) in polar coordinates in Fig. 7, the decision region 704 extends from the detection point 702a (p,r) to a ground distance closer to the reference line / plane of the object tracking subsystem 701 (in this case, angular distance q), a ground distance away from the reference line / plane of the object tracking subsystem 701 (in this case, angular distance q), a ground distance closer to the reference position of the object tracking subsystem 701 (in this case, axial distance s1), and a ground distance away from the reference position of the object tracking subsystem 701 (in this case, axial distance s2). It is determined that person B at detection point 702b is within the decision region 704, and therefore the tracking of person A may not be reliable (or the confidence level of person A's path is low), and person B is also identified as a potential associate of person A. Potential paths of person A are then collected, for example, by considering subsequent detections and paths of both person A and person B detected after detection time t. Such detection data and their associated IDs are then stored in a database in communication with the people monitoring server 706. The multiple detection points and potential detection points of the object may then be linked / combined according to a chronological order and displayed on the client device or server 710 to determine two potential paths 812, 814 of the object in the room, as shown in FIG. 8A. In one embodiment, the user may interact with the client device or server 810 to select one of the possible paths (e.g., 812′) as a more accurate path of person A, as shown in FIG. 8B. This selection may be stored in the database of the people monitoring server 706. Although different axial distances s1 and s2 are shown, it is understood that s1 and s2 may have the same length s0. Similarly, although the same angular distance q is shown, it will be understood that the ground distance closer to the reference line / plane and the ground distance further from the reference line / plane can be replaced with different angular distances q1 and q2, respectively.

[0116] FIG. 9 shows a schematic diagram 900 illustrating an example decision region angle according to an embodiment of the present disclosure. As mentioned above, if an object 902 is detected at an angle (e.g., p degrees) from the reference line / plane of the object tracking sensor, the decision region angle of the detection point may range from p to an angle or angular distance (e.g., q degrees) closer to the reference line / plane 904 of the object tracking sensor to an angle or angular distance (e.g., q degrees) farther from the reference line / plane 904. The decision region angle in this case is defined as 2q. If another object 906 is detected at an angle p1 that is within the decision region angle (i.e., p-p1 is less than q degrees), it is determined that the two people 902, 906 are too close to each other and tracking of these two people 902, 906 is not reliable because the main part of one person may be blocked by the other person, resulting in swapped track IDs. Such decision region angle (e.g., q degrees) may be configured based on the ability of the object tracking sensor to identify and track the two people. For example, q degrees is set because it is the smallest angle at which the object tracking sensor can distinguish and track two different people / objects.

[0117] FIG. 10 shows a schematic diagram 1000 illustrating an example decision region length according to an embodiment of the present disclosure. As described above, when an object 1002 is detected at an axial distance (in this case, radius r) from the reference position 1004 of the object tracking sensor 1001, the length of the decision region of the detection point may extend from r to an axial distance (e.g., s1) close to the reference position 1004 of the object tracking sensor 1001 and an axial distance (e.g., s2) far from the reference position 1004 of the object tracking sensor 1001. The length of the decision region in this case is defined as s1+s2. If another object 1006, 1008 is detected at an axial distance or radius r1, r2 that falls within the decision region length (r-r1 is smaller than s1 or r-r2 is smaller than s2), it is determined that the tracking of these two people is not reliable because the two people are too close to each other and most of one person may be blocked by the other person, resulting in swapped tracking IDs. The length of such decision regions (e.g., s1 or s2) may be configured based on the ability of the object tracking sensor to distinguish and track two people, and can be related to the tilt angle of the object tracking sensor 1001. For example, s1 and s2 (or the tilt angle, e.g., angle 1010) are set to be the minimum distance at which the object tracking can distinguish and track two different people / objects.

[0118] In one embodiment, the height of the tracked object can be taken into account to accurately determine the reliability of tracking. Figures 11A-11C show three different person tracking scenarios in which two people (target person and person A) with different heights and distances from the tracking sensor are tracked according to an embodiment of the present disclosure. In this embodiment, the reliability of the target person's tracking is evaluated. In Figure 11A, if person A is closer to the sensor and shorter than the target person, it is determined that the target person's tracking is still reliable. In Figure 11B, if person A is closer to the sensor and taller than the target person, it is determined that the target person's tracking is unreliable. In Figure 11C, if the target person is closer to the sensor and taller than person A, it is determined that the target person's tracking is unreliable.

[0119] 7, the object tracking subsystem 701 may further determine the height of person A at the detection point 702a and the height of person B at the detection point 702b. For example, if the height of person A at the detection point 702a is 180 cm and the height of person B at the detection point 702b is 150 cm, even if person B is detected within the decision region 704 of the detection point 702a, in this case, the tracking of person A may be determined to be reliable.

[0120] According to the present disclosure, an object tracking sensor may track an object and determine its path and potential paths based on the size of the object. FIG. 12 shows a schematic diagram 1200 illustrating an object tracking subsystem 1201 for determining a potential path of a person according to various embodiments of the present disclosure. The object tracking subsystem 1201 detects person A with output size of size (x, y, z) and two other people (person B and person C with output size (not shown)) at different positions (illustrated with same person figure blocks) along the path (as shown with arrows 1222, 1224, 1226) over a period of time. In this case, the target person is person A, and the tracking confidence of person A is evaluated at detection point 1202b using a decision region 1220 around detection point 1202b. Then, persons B and C are determined to be detected within the decision area 1220 by the object tracking sensor 1200 at approximately the same time t or shortly thereafter (e.g., t+1s), and persons B and C are determined to be potential associates of person A.

[0121] The object tracking sensor 1201 may then compare the magnitudes of person A and his potential companions (person B and person C) when they are detected outside the decision region 1220, i.e., before they enter the decision region 1220 (e.g., detection points 1202a, 1204a, 1206a where persons A, B, and C are detected at times before detection time t) and / or after they leave the decision region 1220 (e.g., detection points 1202c, 1204c, 1206c where persons A, B, and C are detected after detection time t at times after detection time t). The respective sizes of Person A, Person B and Person C detected before entering the decision area 1220 at detection points 1202a, 1204a and 1206a before time t, and / or the respective sizes of Person A, Person B and Person C detected after exiting the decision area 1220 at detection points 1202c, 1204c and 1206c after time t, are compared to determine their potential paths. In this example, after identifying that tracking of person A may be unreliable based on detection point 1202b detected at detection time t, the object tracking subsystem 1201 may compare the size of each person detected after leaving the decision area 1220, identify a person (e.g., the person at detection point 1202c) that is the same size as person A (e.g., at detection point 1202a) before entering the decision area 1220, and concatenate / combine person A's previous detection point 1202a detected before time t, detection point 1202b detected at time t, and detection point 1202c detected after time t to form a continuous potential path for person A.

[0122] Additionally or alternatively, the object tracking sensor may track an object and determine its path and potential paths based on the object's velocity or moving speed. FIG. 13 shows a diagram 1300 illustrating an object tracking subsystem 1301 for determining a potential path of a person according to various embodiments of the present disclosure. The object tracking subsystem 1301 detects person A with a velocity (e.g., x-axis and y-axis velocity) and two other people (persons B and C with respective velocities) at different positions (illustrated with same person figure blocks) along a path (as shown with arrows 1322, 1324, 1326) over a period of time. In this case, the target person is person A, and the tracking confidence of person A is evaluated at detection point 1302b using a decision region 1320 around detection point 1302b. Then, persons B and C are determined to be detected within the decision area 1320 by the object tracking sensor 1301 at approximately the same time t or shortly thereafter (e.g., t+1s), and persons B and C are determined to be potential companions of person A.

[0123] The object tracking sensor 1301 may then compare the velocities of person A and his potential companions (person B and person C) when they are detected outside the decision region 1320, i.e., before they enter the decision region 1320 (e.g., detection points 1302a, 1304a, 1306a where persons A, B, and C are detected at times before detection time t) and / or after they leave the decision region 1320 (e.g., detection points 1302c, 1304c, 1306c where persons A, B, and C are detected after detection time t at times after detection time t). The velocities of Person A, Person B, and Person C detected before entering the decision area 1320 at detection points 1302a, 1304a, and 1306a before time t, and / or the velocities of Person A, Person B, and Person C detected after exiting the decision area 1320 at detection points 1302c, 1304c, and 1306c after time t, are compared to determine their potential paths. In this example, after identifying that tracking of person A may be unreliable based on detection point 1302b detected at detection time t, the object tracking subsystem 1301 may compare the speed of each person detected after leaving the decision area 1320, identify a person (e.g., the person at detection point 1302c) with the same speed as person A (e.g., at detection point 1302a) before entering the decision area 1320, and concatenate / combine previous detection point 1302a detected before time t, detection point 1302b detected at time t, and detection point 1302c detected after time t to form a continuous potential path for person A.

[0124] In one embodiment, the object tracking sensor 1201, 1301 may detect person A (target person) and determine its potential path based on both its size and speed. For example, after identifying that the tracking of the target person may be unreliable based on the detection points, the object tracking sensor may compare the size and speed of each person after leaving the decision area, generate a confidence score according to the likelihood that the detected person is the target person, identify the person with the highest confidence score as the target person, and concatenate / combine the previous detection points 1202a, 1302a of the target person detected before time t, the detection points 1202b, 1302b of the target person detected at time t, and the detection points associated with the highest confidence score detected after time t to form a continuous potential path of the target person. Furthermore, the size and speed parameters of the object are weighted and affect the confidence score differently. For example, the size parameter may have a larger weight than the speed parameter because a person can change speed and direction. Thus, a person of similar size to the target person will generally have a higher score than a person who only has a similar speed to the target person.

[0125] FIG. 14 illustrates a reliability score calculation table 1400 according to an embodiment of the present disclosure. In this embodiment, five parameters, such as the size x, size y and size z of the person in x, y and z axis and the velocity x, velocity y of the person in x and y axis, are used as shown in the first column of the table to calculate scalar values ​​of four different contributing factors of the reliability score, such as area, height, speed and heading, as shown in the second column of table 1400 in FIG. 14. Then, the difference between the original known track and the possible track is calculated and normalized to a value between 0 and 1, as shown in the third and fourth columns of table 1400 in FIG. 14, respectively. Different weights (W area , Wheight , W speed , W heading ) are applied to different contributing factors and the weighted values ​​are summed to arrive at a reliability score. These reliability scores for each person are compared and ranked to determine which person has the highest reliability score and can therefore be considered a target person.

[0126] According to the present disclosure, potential paths for a target object can be determined from paths of objects that have more than one association level from the target object. In various embodiments below, the association level indicates the degree of association from the target object. For example, a target object may have an association level of 0, while an object or path with an association level of 2 may have a second degree of association (2 degrees) with the target object or the path of the target object, respectively. nd Such an object and a path having a second degree of association may be referred to as a second-level (or level 2) potential associate and a level 2 path, respectively, in the following embodiments.

[0127] 15A illustrates an example track group table 1500 for a target object according to one embodiment of the present disclosure. A target object 1502a with track_ID "1" is tracked. The target object 1502a has an association level 0. If another object 1502b with track_ID "2" is tracked and found to have one detection point of its path located within the ground area 1504 of one detection point of the target object's path at approximately the same detection time, the object 1502b is identified as a first level potential association (association level 1) whose path is at least very close to the path of the target object at one point, directly affecting the tracking reliability of the target object. The path of the first level potential association 1502b is identified as a level 1 path. Similarly, if another object (not shown) with track_ID "3" is tracked and found to have one detection point of its path located within the ground area of ​​one detection point of the target object's path at approximately the same detection time, it is identified as a second level potential association (association level 2) whose path is at least very close to the path of the first level potential association (association level 1) at one point, directly affecting the tracking reliability of the target object. The paths of the second level potential associations 1502b are identified as level 2 paths. The same can be applied to search for further levels of potential associations and paths. Such track ID and association level information is stored in a database for further searching and display of paths. In various embodiments below, objects and their potential tracks having an association level with the target object are arranged in a track group of the target object.

[0128] In one embodiment, the object tracking system stops tracking objects and further levels of tracks when the relevance level reaches a maximum value, for example, 3 relevance levels. This is to balance between tracking depth and computer resources. In particular, increasing depth allows the system to track for a longer period of time. The number of tracks may increase exponentially with each relevance level, and more computer resources are required to process tracks of higher relevance levels. Furthermore, tracked objects of higher relevance levels may be less meaningful in determining the target object of level 0. Therefore, by limiting the relevance level, tracks and computer resources can be limited.

[0129] FIG. 15B is a schematic diagram 1510 illustrating paths with different association levels according to an embodiment of the present disclosure. An original path (track) 1512 identified under person ID “Bob” (target object) is determined. Another path 1514 is found to have at least one point located within the ground region 1513 of the original path (level 0 path), affecting the tracking reliability of the target object, so the path 1514 is identified as a level 1 path. Similarly, two other paths 1516, 1518 are found to have at least one point located within the ground region 1515a, 1515b of the level 1 path, affecting the tracking reliability of the first level of potential associations, so the paths 1516, 1518 are identified as level 2 paths. Yet another path 1520 is found to have at least one point located within the ground region 1517 of the level 2 path 1516, affecting the tracking reliability of the second level of potential associations, so the path 1520 is identified as a level 3 path. According to the present disclosure, all such routes having association levels 1 through 3 are determined to be potential Bob's tracks.

[0130] FIG. 16 shows a flow chart illustrating a process of forming a track group for an object according to an embodiment of the present disclosure. In step 1602, a step of extracting a track in a track group of a person U (a target person) is performed. In step 1604, it is determined whether the track level is less than 3. If the track level is less than 3, step 1606 is performed. If the track level is 3 or more, step 1612 is performed. In step 1606, a step of creating a decision region using the angle p and radius r of each point of the track is performed. In step 1608, it is determined whether there is a point of an existing track in the decision region. If there is a point of an existing track in the decision region, step 1610 is performed. If there is no point of an existing track in the decision region, step 1612 is performed. In step 1610, a step of adding an ID of an existing track in the decision region to the track group of the person U with a higher level than the track (the level of the track + 1). In step 1612, it is checked whether there is another existing track in the track group of the person U. If so, step 1614 is executed to extract another track from person U's track group, and steps 1604-1612 are executed based on the other track. If there are no other existing tracks in person U's track group, the process may end.

[0131] Although the above embodiments have been described using a person or person, it will be appreciated that the object tracking sensor can track the path of any moving object, such as a vehicle.

[0132] In one embodiment, the object tracking sensor may be configured to filter unwanted objects and be more efficient in detecting and determining potential paths for humans. In such a case, the object tracking sensor may be further configured to determine whether the size of the object is within a normal human size range or the size of the object is too small or too large to be a human, and / or whether the moving speed of the object is within normal human moving speed limits or the moving speed is too fast to be a human.

[0133] The following paragraphs present a second embodiment of the present disclosure in which the ground region used to determine the current tracking of an object and the reliability of the object's potential path is configured with reference to the object's location.

[0134] FIG. 17 shows a schematic diagram illustrating a system including an object tracking subsystem 1701 for determining a potential path of a person according to a second embodiment of the present disclosure. In this embodiment, the object tracking subsystem 1701 is configured to detect all objects moving within its line of sight at a certain interval, identify the path of each object within its line of sight, and determine the potential path of the object. As shown in FIG. 17, the object tracking subsystem 1701 tracks a person under track ID "1". However, the path of the person is lost and the last detection point 1702a is detected at detection time t. Then, the object tracking subsystem 1701 is configured to find a continuation pathway by checking whether the detection points of other people are also detected within a decision region 1704 centered on the detection point 1702a. In this example, the decision region 1704 is a circular region with a certain ground distance (radius R2) in all directions from the detection point 1702a. The object tracking sensor 1701 detects the path of track ID "5" starting from detection point 1703a at some later time t+i. The object tracking subsystem 1701 may identify the path of the person with track ID "5" as a possible continuation path of the person with track ID "1" who lost track at time t. Such detection data and associated IDs are then communicated to the people monitoring server 1706 and stored in a database. The multiple detection points and possible detection points of the object may then be linked / combined in chronological order to determine the path 1722 of the object in the room and displayed on the client device or server 1710.

[0135] 18 shows an example track group table 1800 for a target object according to the second embodiment of the present disclosure. In addition to the track ID and its association level, unlike the table in FIG. 15A, the origin of the target person search area is also tabulated, i.e., the location / detection point where the track of the target object was lost or last detected and the location / detection point where the decision area is centered. This is particularly useful for lost track situations and continued track finding, as the user is informed of the last detection point of the person.

[0136] 19A-19C are a flow chart 1900 showing a process of forming a track group for a target person in the second embodiment of the present invention. In this example, as shown in block 1903 of FIG. 19B, the track of the target person is lost at time 25201.3. In step 1902, a track is extracted from the track search area (SA) list of the target person. In step 1904, it is determined whether there is a track starting within radius R2 from the origin (x, y) 1901 of the track SA. As shown in block 1903, if a new track / person 1905 is detected at time 25201.5 after the track of the target object is lost and there is a track starting within radius R2 from the origin (x, y) 1901 of the SA, then step 1906 is performed. Otherwise, step 1908 is performed. In step 1906, as shown in block 1907, an association level of 1 is added to the track ID "5", and the ID of the detected track is added to the target person track group with a higher track level (track level + 1). The track ID and association level are then stored in database 1909. In step 1908, it is checked whether there are other existing tracks in the SA list. If there are, in step 1910, other tracks are extracted from the SA list, and steps 1904 to 1908 are executed based on the extracted tracks. If there are no other existing tracks in the SA list, the process may end.

[0137] 20A-20D show a flowchart 2000 illustrating a process of displaying potential paths of an object under multiple sensor configurations according to an embodiment of the present disclosure. C1-C6 refer to cameras or sensors 1-6. In step 2002, search for track points of the target person up to time t. In step 2004, display the track of the target person up to time t and the video playback at time t. In step 2006, extract a track V in the track group (see FIG. 20B) of the target person. In step 2008, determine whether the point of track V at time t is in the footage area of ​​the surveillance camera. For example, as shown in block 2009, it is determined that the target person is detected within the line of sight 2011b, 2011c of both C2 and C3 at different positions, indicating that there are two different detections of the target person and therefore there may be two potential paths. If it is determined that the point of track V at time t is within the footage area of ​​the surveillance camera, step 2010 is executed. If not, step 2012 is executed. In step 2010, potential paths are displayed and the video playback area of ​​the surveillance camera is highlighted (see FIG. 20D). In step 2012, it is determined whether there are other existing tracks V in the track group of the target person. If there are, step 2014 is performed, where other tracks in the track group are extracted, and steps 2008-2012 are performed based on the extracted tracks. If there are no other existing tracks V in the track group of the target person, the process can end.

[0138] According to the present disclosure, for example, two different ground domain configurations, one configured with reference to the object's location and one configured with reference to the reference line / plane and position of the object tracking sensor, may be used to determine the reliability of the object's current tracking and the object's potential path.

[0139] FIG. 21A illustrates an exemplary target person's track group table 2102 and associated tracks according to an embodiment of the present disclosure. According to the track group table 2102, the target object is detected with high confidence under track 1, and thus is identified as a confirmed track with association level 0. While the object tracking system and the person monitoring server are operating, other track points under different track IDs may be detected, and nearby tracks are checked to determine whether they are relevant tracks of the target person. In this case, two relevant tracks (track 2 and track 5) are detected and added to the track group table with association level 1. In this embodiment, track 2 may be determined from a rectangular decision area of ​​the target person's detection points configured according to what is described in the first embodiment above, and track 5 may be determined from a circular decision area of ​​the target person's detection points (which may be the same or different detection points as the detection points used to determine track 2) configured according to what is described in the second embodiment above. When the user requests to display the target person's activities and paths, both the confirmed track (track 1) and associated tracks 2 and 5 are displayed, as shown in FIG. 21B. Video playback in which the target person may appear is also highlighted.

[0140] 22A and 22B show a schematic diagram 2200 illustrating a system including two object tracking sensors 2202, 2204 for determining potential paths of an object 2206 according to one embodiment of the present disclosure. In this embodiment, two LiDAR sensors 2202, 2204 are used in the system to track the object 2206 at different locations. The fields of view of each of the two LiDAR sensors are indicated by circles 2023 and 2025. The LiDAR sensors 2202, 2204 send their respective detection data to a LiDAR server 2208, which processes and compiles them based on the capabilities of the sensors to determine the path of the object. The LiDAR server 2208 then classifies the track of the detected object based on its attributes and stores the detection data, the classification and the assigned track ID in a database (not shown) in communication with the LiDAR server 2208. A sample of object attributes stored in the database is shown in table 2210, where three tracks and objects are identified under three different track IDs classified as person, vehicle and unknown object based on object attributes. Unknown object is classified when there is no classification criteria matching the object attributes. The three objects have current 3D positions in X, Y, Z, and sizes of height for person, width for vehicle and depth for unknown object, and are moving with X, Y, Z velocities respectively.

[0141] In another embodiment, an object tracking system according to various embodiments of the present disclosure can be utilized to accurately identify potential contacts of subjects (people) by determining nearby tracks, which may be useful for identifying the source or spread of infectious diseases or for identifying allies of criminal activity.

[0142] FIG. 23 shows a schematic diagram illustrating a system 2300 and an object tracking subsystem 2301 for determining nearby tracks and potential contacts of a target person according to an embodiment of the present disclosure. The operation of person identification and object tracking using the person identification subsystem and the object tracking subsystem is the same as that described in FIG. 2. In this embodiment, the person identification subsystem 2302 of the system 2300 identifies a target person who is assigned a track ID "Bob" and enters a room based on the image 2303 of the target person, and the object tracking subsystem 2301 tracks the target person at regular intervals over a period of time and detects the target person at different locations (detection points). The person monitoring server 2306 of the system 2300 may associate the detection data (e.g., object shape / contour data, position data and time data) under the track ID, communicate with the person monitoring server 2306 to store in a database, and link / combine the detection points according to a time order to determine the path 2322 of the target object in the room, as shown in block 2321, and display on the client device or server 2310.

[0143] Furthermore, for each detection point (indicated by a black circle) along the path 2322 (hereinafter referred to as the "main track") of the target person "Bob", it is checked whether other people are also detected within a decision area centered on that detection point. The detection data and associated IDs for such other objects can be retrieved from the database of the people monitoring server 2306. For example, at the detection point 2302f, it is determined that at the same detection time, two other objects are also located 2303, 2304 within the decision area of ​​the detection point 2302f. The people monitoring server 2306 then identifies the other two objects as nearby contacts of the target person "Bob" and their tracks 2324, 2326 as nearby tracks. The track IDs of the potential contacts are stored in the database of the people monitoring server 2306, for example in the track group table of the target person.

[0144] On the other hand, the object with track ID "4" is not detected as a proximity contact because it is not detected within the decision area of ​​any of the detection points of the target person. When the user subsequently requests data regarding Bob's potential contacts, for example via the client device 2310, the target person's path 2322 and two proximity tracks 2324, 2326 are displayed on the client device 2310, as shown in block 2321.

[0145] 24 is a schematic diagram illustrating a system 2400 including an object tracking subsystem 2401 for determining proximity tracks and potential contacts of a target person according to another embodiment of the present disclosure. The object tracking subsystem 2401 detects a target person with track ID "Bob" at different locations (only detection point 2402 is shown) over a period of time.

[0146] The people monitoring server 2406 of the system 2400 may store the detection data (e.g., object shape / outline data, location data and time data) associated with the track ID "Bob" in a database in communication with the people monitoring server 2406, and connect / combine the detection points according to time series to determine the path 2422 of the target person in the room and display it on the client device or server 2410.

[0147] For each of the detection points of the target person "Bob", the people monitoring server 2406 may check whether another person is also detected within a circular decision area 2404 (radius R3) centered on the detection point. The object tracking subsystem 2401 detects two people 2403a, 2403b located within the circular decision area 2404 of the detection point 2402, thereby identifying the other two people 2423a, 2403b as nearby contacts of the target person "Bob" and their tracks 2424, 2426 as nearby tracks. The track IDs of the potential contacts are stored in the database of the people monitoring server 2406, for example in the track group table of the target person. Thereafter, when the user requests data regarding the potential contacts of Bob, the path 2422 of the target person and the nearby tracks 2424, 2426 may be displayed on the client device 2410, as shown in block 2421.

[0148] It should be noted that the neighboring tracks are algorithmic estimates and that these tracks may sometimes be incorrectly associated. Correction and verification of the potential or neighboring tracks may be necessary. Correction and verification may be accomplished by the system correcting the track ID, i.e., by re-identifying the subject at the next face recognition point. Alternatively, correction and verification of the potential or neighboring tracks may be performed by the user correcting and verifying the track ID assigned to the potential or neighboring track.

[0149] FIG. 25 shows a flow chart 2500 illustrating a process of determining potential paths of an object and modifying the potential paths and its data entries in a database (DB) 2501 in communication with a sensor and / or a people monitoring server according to an embodiment of the present disclosure. In step 2502, a (target) person is first identified by face recognition at point 2521, i.e., by comparing the data of the image obtained from the sensor with the person data stored in the database 2501, matching the image data with the person data under a person ID, and assigning a person ID to the person data and the image data. The person is then tracked and the position data detected over a period of time is associated with the person ID and combined into the person's track (track 1) 2522. The detection data and the main track (track 1) are then stored in the database 2501. Potential tracks of the person may also be determined if another track (e.g., track 2) 2523 is found to have at least one point within the ground area of ​​a point of track 1, and track 2 is then identified as a potential track of level 1 (having the first association level or degree with the main track). The information of the potential track (track 2) 2523, its association level, and currently assigned person ID are then stored in the database 2501 under the track group 2503 of the target person.

[0150] In step 2504, two level 2 potential tracks 2524, 2525, i.e. track 3 and track 4, having a second level of association or second degree of association with the main track, are identified by finding track 3 (2524) and track 4 (2525), each of which has at least one point located within the ground area of ​​track 2 (2523) (a level 1 potential track). Then, the information of the two potential tracks (track 3 and track 4) 2524, 2525, their association levels, and currently assigned person IDs are stored under the target person's track group 2505 in the database 2501.

[0151] In one example, track verification may be performed through the system by capturing an image of the object on the potential track at the next facial recognition point 2526. In this case, in step 2506, the object in track 3 (2524) is verified using a sensor as it passes the next facial recognition point 2526. For example, an image of the object in track 3 (2524) is captured and the image data is compared with the person data in the database 2501 to re-identify whether the person or person ID detected at this point 2526 matches that of the person in track 1 (2522) or that of the person detected at the last facial recognition point 2521. In this case, the person in track 3 (2524) is found to match the person in track 1 (2521), so the detection data and person ID of track 4 are removed from the target person's track group 2507.

[0152] Alternatively or additionally, the user can interact with the interface and watch a video playback of the track to perform verification of track 3 (2524) himself, especially if track 3 (2524) does not intersect with the face recognition point 2526. The user can select either "correct" to indicate that the object in track 3 corresponds to the object in track 1, or "wrong" to indicate that the object in track 3 is not the object in track 1. In this case, the user selects "correct". In step 2508, the system can link parts of tracks 1, 2 and 3 to form consecutive potential tracks 2527 of the target person and assign the person ID of the target person to them. Then, the information of the final consecutive potential tracks (part of track 1 + part of track 2 + part of track 3) 2527 of the target person, their association levels, and their newly assigned person IDs are updated in the track group 2509 of the target person in the database 2501. Optionally, the remaining unlinked parts of Track 1, Track 2 and Track 3 are not deleted from the database, and the system may create a new person ID for each of the remaining unlinked parts, or link them to one or more other paths (not shown) to form another potential continuous track of another person (not shown) and assign them to the other person's person ID. The information of these parts and the newly assigned person IDs are updated in the respective track groups in database 2501.

[0153] Figure 26 illustrates a schematic diagram of an exemplary computing device 2600, hereinafter interchangeably referred to as computer system 2600. One or more such computing devices 2600 may be used, or may be suitable for use, to perform the method of Figure 5 and implement the apparatus of Figure 6. The following description of computing device 2600 is provided by way of example only and is not intended to be limiting.

[0154] 26, the exemplary computing device 2600 includes a processor 2604 for executing software routines. Although a single processor is shown for clarity, the computing device 2600 may also include a multi-processor system. The processor 2604 is connected to a communications infrastructure 2606 for communicating with other components of the computing device 2600. The communications infrastructure 2206 may include, for example, a communications bus, crossbar, or network.

[0155] The computing device 2600 further includes a main memory 2608, such as a random access memory (RAM), and a secondary memory 2610. The secondary memory 2610 may include a storage drive 2612, which may be, for example, a hard disk drive, a solid state drive, or a hybrid drive, and / or a removable storage drive 2614, which may include a magnetic tape drive, an optical disk drive, a solid state storage drive (USB flash drive, a flash memory device, a solid state drive, a memory card, etc.), etc. The removable storage drive 2614 reads from and / or writes to a removable storage medium 2618, in a well-known manner. The removable storage medium 2618 may include a magnetic tape, an optical disk, a non-volatile memory storage medium, etc., which is read from and written to by the removable storage drive 2614. As will be appreciated by those skilled in the art, the removable storage medium 2618 includes a computer readable storage medium having computer executable program code instructions and / or data stored thereon.

[0156] In another embodiment, the secondary memory 2610 may additionally or alternatively include other similar means for allowing computer programs or other instructions to be loaded into the computing device 2600. Such means may include, for example, a removable storage unit 2622 and an interface 2620. Examples of removable storage units 2622 and interfaces 2620 include program cartridges and cartridge interfaces (such as those found in gaming consoles), removable memory chips (such as EPROMs or PROMs) and associated sockets, removable solid-state storage drives (such as USB flash drives, flash memory devices, solid-state drives, memory cards, etc.), and other removable storage units 2622 and interfaces 2620 that allow software and data to be transferred from the removable storage unit 2622 to the computer system 2600.

[0157] The computing device 2600 also includes at least one communication interface 2624. The communication interface 2624 allows software and data to be transferred between the computing device 2600 and external devices via a communication path 2626. In various disclosed embodiments, the communication interface 2624 allows data to be transferred between the computing device 2600 and a data communication network, such as a public or private data communication network. The communication interface 2624 can be used to exchange data between different computing devices 2600, such as computing devices 2600 that form part of an interconnected computer network. Examples of the communication interface 2624 can include a modem, a network interface (such as an Ethernet card), a communication port (such as serial, parallel, printer, GPIB, IEEE 1394, RJ45, USB, etc.), an antenna with associated circuitry, etc. The communication interface 2624 can be wired or wireless. The software and data transferred via the communication interface 2624 is in the form of a signal, which can be an electronic signal, an electromagnetic signal, an optical signal, or other signal receivable by the communication interface 2624. These signals are provided to communications interface 2624 via communications path 2626 .

[0158] As shown in FIG. 26, the computing device 2600 further includes a display interface 2602 that performs operations for rendering images on an associated display 2630, and an audio interface 2632 that performs operations for playing audio content via associated speakers 2634.

[0159] As used herein, the term "computer program product" may refer, in part, to removable storage media 2618, removable storage unit 2622, a hard disk attached to storage drive 2612, or a carrier wave that carries the software over communications path 2626 (wireless link or cable) to communications interface 2624. Computer readable storage media refers to any non-transitory, non-volatile tangible storage medium that provides recorded instructions and / or data to computing device 2600 for execution and / or processing. Examples of such storage media include magnetic tape, CD-ROM, DVD, Blu-ray disk, hard disk drive, ROM or integrated circuit, solid-state storage drive (such as USB flash drive, flash memory device, solid-state drive, memory card), hybrid drive, magneto-optical disk, or computer readable card such as PCMCIA card, whether such device is internal or external to computing device 2600. Examples of transitory or non-tangible computer-readable transmission media that may also be involved in providing software, application programs, instructions and / or data to computing device 2600 include wireless or infrared transmission channels, network connections to other computers or networked devices, and the Internet or intranets, including e-mail transmissions and recorded information on websites and the like.

[0160] Computer programs (also referred to as computer program code) are stored in main memory 2608 and / or secondary memory 2610. Computer programs may also be received via communications interface 2624. Such computer programs, when executed, enable computing device 2600 to perform one or more features of the embodiments described herein. In various embodiments, the computer programs, when executed, enable processor 2604 to perform the features of the embodiments. Such computer programs thus represent controllers of computer system 2600.

[0161] The software may be stored in a computer program product and loaded into the computing device 2600 using the removable storage drive 2614, the storage drive 2612, or the interface 2620. The computer program product may be a non-transitory computer readable medium. Alternatively, the computer program product may be downloaded to the computing system 2600 via communications path 2626. The software, when executed by the processor 2604, causes the computing device 2600 to perform the necessary operations to perform the method of FIG. 5 and to implement the apparatus of FIG. 6.

[0162] It should be understood that the embodiment of Figure 26 is presented only as an example to explain the operation and structure of the apparatus. Thus, in some embodiments, one or more features of computing device 2600 may be omitted. Also, in some embodiments, one or more features of computing device 2600 may be combined together. Furthermore, in some embodiments, one or more features of computing device 2600 may be split into one or more components.

[0163] It will be appreciated by those skilled in the art that various changes and / or modifications may be made to the present disclosure as illustrated in the specific embodiments without departing from the spirit or scope of the disclosure as broadly described. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive.

[0164] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. (Appendix 1) 1. A method for determining potential paths for an object, comprising: the potential path of the object includes a plurality of locations where the object is detected at a corresponding plurality of detection times; The method comprises: determining whether a first location at which a first object was detected at a detection time along a first path of the first object is within a ground distance from a second location at which a second object was detected after the detection time along a second path of the second object; in response to determining that the first location is within the ground distance from the second location, combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object; A method comprising: (Appendix 2) and combining a remainder of the second path, a remainder of at least one of the first location and the second location, and a remainder of the first path to form a continuous potential path for the second object. The method described in Appendix 1. (Appendix 3) 2. The method of claim 1, wherein determining whether the first location where the first object was detected at the detection time along the first path of the first object is within the ground distance from the second location where the second object was detected after the detection time along the second path of the second object includes determining whether the second location is within a ground region centered on the first location. (Appendix 4) the first location is at a distance away from a location of an object tracking device configured to track the first object along the first path and track the second object along the second path and at an angular distance from a reference line of the object tracking device; the ground region centered about the first location spans axial distances toward and away from the location of the object tracking device and angular distances toward and away from a reference line of the object tracking device; The method described in Appendix 3. (Appendix 5) determining whether a position of the object tracking device is closer to the second position than to the first position and whether a height of the second object is greater than a height of the first object; connecting at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object is performed in response to either: (a) determining that the location of the object tracking device is close to the second location and that a height of the second object is greater than a height of the first object; and (b) determining that the location of the object tracking device is close to the first location and that a height of the first object is greater than a height of the second object. The method described in Appendix 4. (Appendix 6) The ground region centered on the first location is a circle having a preset radius and centered on the first location. The method described in Appendix 3. (Appendix 7) comparing at least one of: (a) a size of the first object at a previous first location along the first path; and (b) a speed of movement of the first object along the first path at the previous first location; the preceding first location is a location where the first object was detected outside the ground region prior to the detection time; a size of the first object at the preceding first location is comparable to a size of the second object at a subsequent second location along the second path; the subsequent second location is a location where the second object is detected outside the ground region after the detection time; a rate of movement of the first object along the first path at the preceding first location is comparable to a rate of movement of the second object along the second path at the subsequent second location; combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object is performed in response to a comparison of at least one of a size of the first object and the second object and a speed of movement of the first object and the second object. 7. The method according to any one of claims 3 to 6. (Appendix 8) combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object is performed in response to a comparison of both the size of the first object and the second object and the speed of movement of the first object and the second object; The method comprises: calculating a score by applying different weights to the result of the size comparison and the result of the movement speed comparison; determining whether the score is higher than a score calculated based on a result of a separate comparison between (a) a size of the first object at the subsequent first location and a size of the first object at the preceding first location, and (b) a speed of movement of the first object along the first path at the subsequent first location and a speed of movement of the first object along the first path at the preceding first location; Including, the subsequent first location is a location where the first object was detected outside the ground region after the detection time; combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object in response to determining that the score is higher than a score calculated based on a result of the further comparison. The method described in Appendix 7. (Appendix 9) a weight applied to the result of the determination of the size is greater than a weight applied to the result of the determination of the moving speed; The method described in Appendix 8. (Appendix 10) determining whether at least one of a size of the first object and / or the second object is within a range of normal human sizes and whether a speed of movement of the first object and / or the second object is slower than a human speed limit; combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object is performed in response to determining at least one of: a size of the first object and / or the second object is within a range of normal human sizes; and a speed of movement of the first object and / or the second object is slower than a human speed limit. 10. The method according to any one of claims 1 to 9. (Appendix 11) displaying both the first path and the second path on a graphical interface; receiving input in response to queries displayed on the graphical interface and combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object; Including, combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object, is performed in response to receiving the input. 11. The method according to any one of claims 1 to 10. (Appendix 12) an object tracker generating a pulsed laser traversing a detection volume; The object tracking device detects a pulsed laser reflected by each of the first object and the second object to detect each of the first object and the second object at a plurality of positions within the detection space and detection period; Including, each of a plurality of positions in the detection space at which the first object and the second object are detected within each detection period defines the first path of the first object and the second path of the second object; The method described in Appendix 1. (Appendix 13) the object tracking device detecting one of the first object and the second object when it enters the detection space, and an image capture device detecting an appearance of the one of the first object and the second object when it enters the detection space; assigning an identity to the one of the first object and the second object based on the appearance data; and Associating the identification information with a corresponding one of the first path of the first object and the second path of the second object; 13. The method of claim 12, comprising: (Appendix 14) 1. An apparatus for determining potential paths of an object, comprising: the path includes a plurality of locations at which the object is detected at a corresponding plurality of detection times; The apparatus comprises: At least one processor; at least one memory containing computer program code; Equipped with The at least one memory and the computer program code are adapted to cause the device, using the at least one processor, to at least: determining whether a first location at which a first object was detected at a detection time along a first path of the first object is within a ground distance from a second location at which a second object was detected after the detection time along a second path of the second object; in response to determining that the first location is within the ground distance from the second location, combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object; Execute the Device. (Appendix 15) The at least one memory and the computer program code, using the at least one processor, cause the apparatus to at least further combine a remainder of the second path, a remainder of at least one of the first location and the second location, and a remainder of the first path to form a continuous potential path for the second object. 15. The apparatus of claim 14. (Appendix 16) The at least one memory and the computer program code, using the at least one processor, cause the device to at least determine whether the second location is within a ground region centered on the first location. 15. The apparatus of claim 14. (Appendix 17) the first location is at a distance away from a location of an object tracking device configured to track the first object along the first path and track the second object along the second path and at an angular distance from a reference line of the object tracking device; the ground region centered about the first location spans axial distances toward and away from the location of the object tracking device and angular distances toward and away from a reference line of the object tracking device; 17. The apparatus of claim 16. (Appendix 18) The at least one memory and the computer program code, using the at least one processor, cause the device to at least determine whether a position of the object tracking device is closer to the second position than to the first position and whether a height of the second object is greater than a height of the first object; The at least one memory and the computer program code, using the at least one processor, cause the device to, in response to either (a) a determination that the position of the object tracking device is near the second position and a height of the second object is greater than a height of the first object, and (b) a determination that the position of the object tracking device is near the first position and a height of the first object is greater than a height of the second object, connect at least a portion of the first path detected before the detection time, at least one of the first position and the second position, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object. 18. The apparatus of claim 17. (Appendix 19) The ground region centered on the first location is a circle having a preset radius and centered on the first location. 17. The apparatus of claim 16. (Appendix 20) The at least one memory and the computer program code, using the at least one processor, cause the apparatus to at least further compare at least one of: (a) a size of the first object at a previous first location along the first path; and (b) a speed of movement of the first object along the first path at the previous first location; the preceding first location is a location where the first object was detected outside the ground region prior to the detection time; a size of the first object at the preceding first location is comparable to a size of the second object at a subsequent second location along the second path; the subsequent second location is a location where the second object is detected outside the ground region after the detection time; a rate of movement of the first object along the first path at the preceding first location is comparable to a rate of movement of the second object along the second path at the subsequent second location; The at least one memory and the computer program code, using the at least one processor, cause the device to, in response to a comparison of at least one of the sizes of the first object and the second object and the moving speeds of the first object and the second object, connect at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object. 20. The apparatus of any one of claims 16 to 19. (Appendix 21) The at least one memory and the computer program code are adapted to cause the device, using the at least one processor, to further include at least: calculating a score by applying different weights to the result of the size comparison and the result of the movement speed comparison; determining whether the score is higher than a score calculated based on a result of a separate comparison between (a) a size of the first object at the subsequent first location and a size of the first object at the preceding first location, and (b) a speed of movement of the first object along the first path at the subsequent first location and a speed of movement of the first object along the first path at the preceding first location; Run the command, the subsequent first location is a location where the first object was detected outside the ground region after the detection time; and wherein the at least one memory and the computer program code, using the at least one processor, cause the apparatus to, in response to at least determining that the score is higher than a score calculated based on the results of the further comparison, combine at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object. 21. The apparatus of claim 20. (Appendix 22) and wherein the at least one memory and the computer program code, using the at least one processor, at least further cause the apparatus to apply a higher weighting to the magnitude determination than to the movement speed determination. 22. The apparatus of claim 21. (Appendix 23) The at least one memory and the computer program code, using the at least one processor, cause the device to at least further determine whether at least one of a size of the first object and / or the second object is within a normal human size range and whether a speed of movement of the first object and / or the second object is slower than a human speed limit; The at least one memory and the computer program code, using the at least one processor, cause the device to, in response to at least one of determining that a size of the first object and / or the second object is within a range of normal human sizes and that a moving speed of the first object and / or the second object is slower than a human moving speed limit, combine at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object. 23. The apparatus of any one of claims 14 to 22. (Appendix 24) The at least one memory and the computer program code are adapted to cause the device, using the at least one processor, to further include at least: displaying both the first path and the second path on a graphical interface; receiving input in response to queries displayed on the graphical interface and combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object; Run the command, The at least one memory and the computer program code, using the at least one processor, cause the device to, in response to receiving the input, combine at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object. 24. The apparatus of any one of claims 14 to 23. (Appendix 25) The at least one memory and the computer program code are adapted to cause the device, using the at least one processor, to further include at least: generating a pulsed laser across a detection volume by an object tracker; Detecting a pulsed laser reflected by each of the first object and the second object by the object tracking device to detect each of the first object and the second object at a plurality of positions within the detection space and detection period; Run the command, each of a plurality of positions in the detection space at which the first object and the second object are detected within each detection period defines the first path of the first object and the second path of the second object; 15. The apparatus of claim 14. (Appendix 26) The at least one memory and the computer program code are adapted to cause the device, using the at least one processor, to further include at least: detecting one of the first object and the second object by the object tracking device when it enters the detection space, and detecting an appearance of the one of the first object and the second object by an image capture device when it enters the detection space; assigning an identity to the one of the first object and the second object based on the appearance data; and Associating the identification information with a corresponding one of the first path of the first object and the second path of the second object; Execute the 26. The apparatus of claim 25. (Appendix 27) 27. A system for determining potential paths of an object, comprising: a device according to any one of claims 14 to 26, an object tracking device, and an imaging device.

[0165] This application claims priority to Singapore Patent Application No. 10202203271V, filed on March 30, 2022, the disclosure of which is incorporated herein in its entirety.

Claims

1. 1. A method for determining potential paths for an object, comprising: the potential path of the object includes a plurality of locations where the object is detected at a corresponding plurality of detection times; The method comprises: determining whether a first location at which a first object was detected at a detection time along a first path of the first object is within a ground distance from a second location at which a second object was detected after the detection time along a second path of the second object; in response to determining that the first location is within the ground distance from the second location, combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object; A method comprising:

2. and combining a remainder of the second path, a remainder of at least one of the first location and the second location, and a remainder of the first path to form a continuous potential path for the second object. The method of claim 1.

3. 2. The method of claim 1, wherein determining whether the first location where the first object was detected at the detection time along the first path of the first object is within the ground distance from the second location where the second object was detected after the detection time along the second path of the second object includes determining whether the second location is within a ground region centered on the first location.

4. the first location is at a distance away from a location of an object tracking device configured to track the first object along the first path and track the second object along the second path and at an angular distance from a reference line of the object tracking device; the ground region centered about the first location spans axial distances toward and away from the location of the object tracking device and angular distances toward and away from a reference line of the object tracking device; The method of claim 3.

5. determining whether a position of the object tracking device is closer to the second position than to the first position and whether a height of the second object is greater than a height of the first object; connecting at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object is performed in response to either: (a) determining that the location of the object tracking device is close to the second location and that a height of the second object is greater than a height of the first object; and (b) determining that the location of the object tracking device is close to the first location and that a height of the first object is greater than a height of the second object. The method of claim 4.

6. The ground region centered on the first location is a circle having a preset radius and centered on the first location. The method of claim 3.

7. comparing at least one of: (a) a size of the first object at a previous first location along the first path; and (b) a speed of movement of the first object along the first path at the previous first location; the preceding first location is a location where the first object was detected outside the ground region prior to the detection time; a size of the first object at the preceding first location is comparable to a size of the second object at a subsequent second location along the second path; the subsequent second location is a location where the second object is detected outside the ground region after the detection time; a rate of movement of the first object along the first path at the preceding first location is comparable to a rate of movement of the second object along the second path at the subsequent second location; combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object is performed in response to a comparison of at least one of a size of the first object and the second object and a speed of movement of the first object and the second object.

7. The method according to any one of claims 3 to 6.

8. combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object is performed in response to a comparison of both the size of the first object and the second object and the speed of movement of the first object and the second object; The method comprises: calculating a score by applying different weights to the result of the size comparison and the result of the movement speed comparison; determining whether the score is higher than a score calculated based on a result of a separate comparison between (a) a size of the first object at the subsequent first location and a size of the first object at the preceding first location, and (b) a moving speed of the first object along the first path at the subsequent first location and a moving speed of the first object along the first path at the preceding first location; Including, the subsequent first location is a location where the first object was detected outside the ground region after the detection time; combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object in response to determining that the score is higher than a score calculated based on a result of the further comparison. The method of claim 7.

9. a weight applied to the result of the determination of the size is greater than a weight applied to the result of the determination of the moving speed; The method of claim 8.

10. determining whether at least one of a size of the first object and / or the second object is within a range of normal human sizes and whether a speed of movement of the first object and / or the second object is slower than a human speed limit; combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object is performed in response to determining at least one of: a size of the first object and / or the second object is within a range of normal human sizes; and a speed of movement of the first object and / or the second object is slower than a human speed limit.

10. The method according to any one of claims 1 to 9.

11. displaying both the first path and the second path on a graphical interface; receiving input in response to queries displayed on the graphical interface and combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object; Including, combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object, is performed in response to receiving the input.

11. The method according to any one of claims 1 to 10.

12. an object tracker generating a pulsed laser traversing a detection volume; The object tracking device detects a pulsed laser reflected by each of the first object and the second object to detect each of the first object and the second object at a plurality of positions within the detection space and detection period; Including, each of a plurality of positions in the detection space at which the first object and the second object are detected within each detection period defines the first path of the first object and the second path of the second object; The method of claim 1.

13. the object tracking device detecting one of the first object and the second object when it enters the detection space, and an image capture device detecting an appearance of the one of the first object and the second object when it enters the detection space; assigning an identity to the one of the first object and the second object based on the appearance data; and Associating the identification information with a corresponding one of the first path of the first object and the second path of the second object; 13. The method of claim 12, comprising:

14. 1. An apparatus for determining potential paths of an object, comprising: the path includes a plurality of locations at which the object is detected at a corresponding plurality of detection times; The apparatus comprises: At least one processor; at least one memory containing computer program code; Equipped with The at least one memory and the computer program code are adapted to cause the device, using the at least one processor, to at least: determining whether a first location at which a first object was detected at a detection time along a first path of the first object is within a ground distance from a second location at which a second object was detected after the detection time along a second path of the second object; in response to determining that the first location is within the ground distance from the second location, combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object; Execute the Device.

15. The at least one memory and the computer program code, using the at least one processor, cause the apparatus to at least further combine a remainder of the second path, a remainder of at least one of the first location and the second location, and a remainder of the first path to form a continuous potential path for the second object.

15. The apparatus of claim 14.

16. The at least one memory and the computer program code, using the at least one processor, cause the device to at least determine whether the second location is within a ground region centered on the first location.

15. The apparatus of claim 14.

17. the first location is at a distance away from a location of an object tracking device configured to track the first object along the first path and track the second object along the second path and at an angular distance from a reference line of the object tracking device; the ground region centered about the first location spans axial distances toward and away from the location of the object tracking device and angular distances toward and away from a reference line of the object tracking device; 17. The apparatus of claim 16.

18. The at least one memory and the computer program code, using the at least one processor, cause the device to at least determine whether a position of the object tracking device is closer to the second position than to the first position and whether a height of the second object is greater than a height of the first object; The at least one memory and the computer program code, using the at least one processor, cause the device to, in response to either (a) a determination that the position of the object tracking device is near the second location and a height of the second object is greater than a height of the first object, and (b) a determination that the position of the object tracking device is near the first location and a height of the first object is greater than a height of the second object, connect at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object.

18. The apparatus of claim 17.

19. The ground region centered on the first location is a circle having a preset radius and centered on the first location.

17. The apparatus of claim 16.

20. The at least one memory and the computer program code, using the at least one processor, cause the apparatus to at least further compare at least one of: (a) a size of the first object at a previous first location along the first path; and (b) a speed of movement of the first object along the first path at the previous first location; the preceding first location is a location where the first object was detected outside the ground region prior to the detection time; a size of the first object at the preceding first location is comparable to a size of the second object at a subsequent second location along the second path; the subsequent second location is a location where the second object is detected outside the ground region after the detection time; a rate of movement of the first object along the first path at the preceding first location is comparable to a rate of movement of the second object along the second path at the subsequent second location; The at least one memory and the computer program code, using the at least one processor, cause the device to, in response to a comparison of at least one of the sizes of the first object and the second object and the moving speeds of the first object and the second object, connect at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object.

20. Apparatus according to any one of claims 16 to 19.

21. The at least one memory and the computer program code are adapted to cause the device, using the at least one processor, to further include at least: calculating a score by applying different weights to the result of the size comparison and the result of the movement speed comparison; determining whether the score is higher than a score calculated based on a result of a separate comparison between (a) a size of the first object at the subsequent first location and a size of the first object at the preceding first location, and (b) a moving speed of the first object along the first path at the subsequent first location and a moving speed of the first object along the first path at the preceding first location; Run the command, the subsequent first location is a location where the first object was detected outside the ground region after the detection time; and wherein the at least one memory and the computer program code, using the at least one processor, cause the apparatus to, in response to at least determining that the score is higher than a score calculated based on the results of the further comparison, combine at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object.

21. The apparatus of claim 20.

22. and wherein the at least one memory and the computer program code, using the at least one processor, at least further cause the apparatus to apply a higher weighting to the magnitude determination than to the movement speed determination.

22. The apparatus of claim 21.

23. The at least one memory and the computer program code, using the at least one processor, cause the device to at least further determine whether at least one of a size of the first object and / or the second object is within a normal human size range and whether a speed of movement of the first object and / or the second object is slower than a human speed limit; The at least one memory and the computer program code, using the at least one processor, cause the device to, in response to at least one of determining that a size of the first object and / or the second object is within a range of normal human sizes and that a moving speed of the first object and / or the second object is slower than a human moving speed limit, combine at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path of the first object.

23. Apparatus according to any one of claims 14 to 22.

24. The at least one memory and the computer program code are adapted to cause the device, using the at least one processor, to further include at least: displaying both the first path and the second path on a graphical interface; receiving input in response to queries displayed on the graphical interface and combining at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object; Run the command, The at least one memory and the computer program code, using the at least one processor, cause the device to, in response to receiving the input, combine at least a portion of the first path detected before the detection time, at least one of the first location and the second location, and at least a portion of the second path detected after the detection time to form a continuous potential path for the first object.

24. Apparatus according to any one of claims 14 to 23.

25. The at least one memory and the computer program code are adapted to cause the device, using the at least one processor, to further include at least: generating a pulsed laser across a detection volume by an object tracker; Detecting a pulsed laser reflected by each of the first object and the second object by the object tracking device to detect each of the first object and the second object at a plurality of positions within the detection space and detection period; Run the command, each of a plurality of positions in the detection space at which the first object and the second object are detected within each detection period defines the first path of the first object and the second path of the second object; 15. The apparatus of claim 14.

26. The at least one memory and the computer program code are adapted to cause the device, using the at least one processor, to further include at least: detecting one of the first object and the second object by the object tracking device when it enters the detection space, and detecting an appearance of the one of the first object and the second object by an image capture device when it enters the detection space; assigning an identity to the one of the first object and the second object based on the appearance data; and Associating the identification information with a corresponding one of the first path of the first object and the second path of the second object; Execute the 26. The apparatus of claim 25.

27. 27. A system for determining potential paths of an object, comprising an apparatus according to any one of claims 14 to 26, an object tracking apparatus, and an imaging apparatus.

Images