COMPUTER-IMPLEMENTED OBJECT TRACKING METHOD, OBJECT TRACKING DEVICE, AND OBJECT TRACKING SYSTEM

Abstract

According to various embodiments, a computer-implemented method for tracking an object is provided, the method including receiving real-time streaming video, the method further including, for each frame of the streaming video, determining a presence probability of an object in the frame, determining whether the presence probability of the object in the frame is below a probability threshold, determining whether the object satisfies a set of proximity criteria based on a determination that the presence probability is below the probability threshold, and generating an adjusted presence probability based on a determination of whether the object satisfies the set of proximity criteria.

Description

[Technical field]

[0001] Various embodiments relate to computer-implemented object tracking methods, apparatus, and systems. [Background technology]

[0002] It is useful for a vehicle to track a target, such as a vehicle ahead, to avoid collisions. A vehicle typically detects and tracks a target by applying image recognition techniques to images captured by a camera mounted on the vehicle. However, a vehicle may lose sight of a target when the target approaches the camera. When a target approaches the camera, the image of the target may be blurred or only a partial image of the target may be captured. This makes it impossible for conventional image recognition techniques to recognize the target, thus adversely affecting the vehicle's ability to track the target. Summary of the Invention [Problem to be solved by the invention]

[0003] In view of the above, a solution is needed to overcome the difficulties of tracking targets at close range. [Means for solving the problem]

[0004] According to various embodiments, a computer-implemented method for tracking an object is provided, the method including receiving real-time streaming video, the method further including, for each frame of the streaming video, determining a presence probability of an object in the frame, determining whether the presence probability of the object in the frame is below a probability threshold, determining whether the object satisfies a set of proximity criteria based on a determination that the presence probability is below the probability threshold, and generating an adjusted presence probability based on a determination of whether the object satisfies the set of proximity criteria.

[0005] According to various embodiments, a non-transitory computer-readable storage medium is provided that includes instructions that, when executed by a processor, perform the computer-implemented methods described above.

[0006] According to various embodiments, an object tracking apparatus is provided that includes the non-transitory computer-readable storage medium described above and at least one processor, which may be coupled to the non-transitory computer-readable storage medium and configured to perform the computer-implemented method described above.

[0007] According to various embodiments, an object tracking system is provided that includes the device described above and a camera configured to capture streaming video.

[0008] Further features of advantageous embodiments are set forth in the dependent claims.

[0009] In the drawings, like reference numbers generally refer to the same elements across different views. The drawings are not necessarily to scale, with emphasis generally being placed upon illustrating the principles of the invention. In the following description, various embodiments are described with reference to the following drawings: [Brief description of the drawings]

[0010] [Figure 1] 1 illustrates a conceptual diagram of a tracking device according to various embodiments. [Diagram 2] 1 illustrates a conceptual diagram of an object tracking system in accordance with various embodiments. [Diagram 3] 1 illustrates a conceptual diagram of a storage medium according to various embodiments. [Figure 4A] 4 illustrates examples of images processed for target tracking according to various embodiments. [Figure 4B] 4 illustrates examples of images processed for target tracking according to various embodiments. [Diagram 5] 1 illustrates a flow diagram of a computer-implemented object tracking method according to various embodiments. [Figure 6A] 1 summarizes a flow diagram of a method for tracking objects according to various embodiments. [Figure 6B] 1 summarizes a flow diagram of a method for tracking objects according to various embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] It will be understood that the embodiments described below from the apparatus point of view are equally valid for the respective methods and vice versa. Furthermore, it will be understood that the embodiments described below may be combined, e.g., parts of one embodiment may be combined with parts of another embodiment.

[0012] It will be understood that characteristics described herein for a particular apparatus may also apply to any of the apparatus described herein. It will be understood that characteristics described herein for a particular method may also apply to any of the methods described herein. Furthermore, it will be understood that for any apparatus or method described herein, not all of the described elements or steps may necessarily be included in the apparatus or method, but only some (but not all) of the elements or steps may be included.

[0013] It will be understood that the term "coupled" (or "connected") in this specification may be understood to mean electrically coupled or mechanically coupled, for example, attached or fixed, or in contact without being fixed, and may be both directly coupled or indirectly coupled (in other words, coupled without direct contact).

[0014] In the above context, the devices described herein may include a memory, for example, for use in processes performed within the device. The memory used in the embodiments may be a volatile memory, for example, a DRAM (Dynamic Random Access Memory), or a non-volatile memory, for example, a PROM (Programmable Read Only Memory), an EPROM (Erasable PROM), an EEPROM (Electrically Erasable PROM), or a flash memory, for example, a floating gate memory, a charge trap memory, an MRAM (Magnetoresistive Random Access Memory), or a PCRAM (Phase Change Random Access Memory). The memory may include a non-transitory storage medium.

[0015] The term "object" to be tracked may be interchangeably referred to herein as a "target."

[0016] In order that the invention may be easily understood and put into practice, various embodiments will now be described, by way of non-limiting example, with reference to the drawings in which:

[0017] FIG. 1 illustrates a conceptual diagram of a tracking device 100 according to various embodiments. The tracking device 100 may also be referred to herein as an object tracking device. The tracking device 100 may include a storage medium 102 and a processor 104. The tracking device 100 may include an additional processor 104. The storage medium 102 may be a non-transitory computer-readable storage medium. The storage medium 102 may include at least one memory. The storage medium 102 and the processor 104 may be electrically and / or communicatively coupled to each other by coupling wires 110. The processor 104 may be configured to perform the object tracking method described with respect to FIG. 5 and FIGS. 6A-6B.

[0018] 2 illustrates a conceptual diagram of an object tracking system 200 in accordance with various embodiments. System 200 may include device 100. System 200 may further include a camera 306 configured to capture images. Camera 306 may be configured to capture video streaming including a sequence of images. Device 100 and camera 306 may be electrically and / or communicatively coupled to each other by coupling wires 130, for example.

[0019] According to various embodiments, the apparatus 100 may be located in a vehicle, referred to herein as the ego-vehicle. In an alternative embodiment, the apparatus 100 may be located separately from the ego-vehicle, for example in a remote server or cloud server.

[0020] According to various embodiments, the system 200 may be located at least partially in the host vehicle.

[0021] 3 illustrates a conceptual diagram of the storage medium 102 according to various embodiments. The storage medium 102 may include a detection module 112, a tracking extension module 114, a probability of presence (PoE) module 116, and a near-field module 118. At least one of the detection module 112, the tracking extension module 114, the PoE module 116, and the near-field module 118 may be a computer algorithm stored on the storage medium 102 and executable by the processor 104. In an alternative embodiment, at least one of the detection module 112, the tracking extension module 114, the PoE module 116, and the near-field module 118 may be stored on a separate storage medium 102. The detection module 112 may be configured to detect and classify targets in an image captured by a camera, e.g., camera 306, based on image processing and recognition techniques. The PoE module 116 may be configured to determine the PoE of the target based on an output from the detection module 112. The PoE module 116 may also determine the PoE based further on other information, such as the age of the target, and input from other sensors of the host vehicle. The PoE indicates a measure of certainty, i.e., probability, of the target's presence. The near-range module 118 may be configured to determine whether the target is within a predetermined distance from the host vehicle. The tracking extension module 114 may be configured to modify the PoE determined by the PoE module 116 based at least in part on the output of the near-range module 118. The tracking extension module 114 may extend the period of time that the target is tracked by the tracking device 100 based at least in part on determining that the target is within a short distance from the camera. The detection module 112, the tracking extension module 114, the PoE module 116, and the near-range module 118 may be electrically and / or communicatively coupled to each other by coupling wires 140.

[0022] 4A and 4B show examples of images being processed to track a target 402, according to various embodiments. These images may be taken by a camera, such as the camera 306 mounted on the vehicle. The tracking device 100 can receive these images from the camera 306 and detect the target 402 based on the received images. For example, the detection module 112 of the tracking device 100 can identify features of the target 402 in the images and recognize the target 402 based on the identified features. These features may include, for example, an outline of the target 402, characteristic shapes and colors of parts of the target 402.

[0023] 4A, when the target 402 is far away from the host vehicle, the camera 306 can capture an image 400A including the entire outline of the target 402. In this example, the target 402 is more than 10 meters away from the host vehicle. The detection module 112 can generate a bounding box 404 surrounding the target 402 in the image 400A by recognizing the overall shape of the target 402. The PoE module 116 can output a high PoE based on the detection result because the detection module 112 can indicate a high confidence of detection of the target 402.

[0024] Referring to FIG. 4B, when the target 402 is close to the host vehicle, a portion of the target 402 may be outside the field of view of the camera 306. In this example, the target 402 is within 10 meters of the host vehicle. If the host vehicle and the target 402 move relatively closer to each other, the features of the target 402 are enlarged in the camera image as shown in image 400B. The overall shape of the target 402 is not visible in image 400B. As a result, the detection module 112 may not be able to detect and / or recognize the target 402. Therefore, the detection module 112 may indicate a low confidence that the target 402 was detected, and the PoE module 116 may output a low PoE based on the detection result. As the target 402 gets closer to the host vehicle, the PoE module 116 may further reduce the determined PoE because it becomes more difficult for the detection module 112 to detect the target 402. If the PoE module 116 continues to reduce its determined PoE until it reaches a minimum threshold, the tracking device 100 may stop tracking the target 402. This may result in a dangerous situation where the target 402 is not only present but is in close range to the host vehicle, which may result in a collision with the host vehicle. To overcome this problem, the tracking device 100 may include a close range module 118 and a tracking extension module 114. The close range module 118 may determine that the target 402 is within a predetermined close range of the host vehicle based on a number of close range conditions. These close range conditions are described with respect to 610 in FIG. 6A. The tracking extension module 114 may modify or override the output PoE of the detection module 112 to continue tracking the target 402 if the close range module 118 determines that the target 402 is within a close range. The tracking extension module 114 allows the tracking device 100 to extend the time to track the target 402 by adjusting the target's PoE value even if it is not detected for multiple frames of the streamed video.

[0025] 5 illustrates a flow diagram of a computer-implemented object tracking method 500 according to various embodiments. The object being tracked is also referred to herein as a target. The method 500 can address missed detections at close ranges by further tracking the target even if the detection module 112 fails to detect or recognize the target. The method 500 can include determining whether the target is within a close range from the host vehicle, and then complementing the missed detection of the detection module if the target is determined to be within a close range from the host vehicle within the expected extended time limit.

[0026] The method 500 may include, at step 502, receiving the streamed video in real time. The method 500 may further, at step 504, determine a PoE of an object in each frame of the streamed video. The PoE indicates a confidence level that an object is present and may be determined by the detection module 112. The detection module 112 may receive the streamed video as an input and may generate a detection result as an output. The PoE module 118 may receive the detection from the detection module 112 and generate a PoE as an output. The method 500 may, at step 506, determine whether the PoE of the object in the frame is greater than a probability threshold (P T ) in step 508. The method 500 may further include determining whether the PoE is less than P T Based on the determination that the PoE is lower than the set of near-distance criteria, the method 500 may further generate an adjusted PoE based on the determination that the PoE is lower than the set of near-distance criteria at step 510. Each of steps 504, 506, 508, and 510 may be performed for each frame of the streamed video and may be performed by the tracking extension module 114.

[0027] Method 500 may continuously track an object based on the current scenario (captured by the current frame of the streamed video) instead of relying solely on a predicted future position. This improves the accuracy of tracking. Also, the algorithm performs simple arithmetic operations, which is computationally inexpensive in terms of resources and time and may be useful for tracking objects at very close ranges. Method 500 may be able to track targets even if they are 3-6 meters away from the ego-vehicle or even closer. Method 500 may not rely solely on specific features of the target and thus is not limited to tracking a specific type of object. Method 500 may further determine the probability of object presence based on at least one of vehicle dynamics, camera, environmental parameters, etc. Method 500 addresses challenges faced in close range tracking, which may help prevent collisions.

[0028] According to an embodiment, which may be combined with any of the above-mentioned embodiments or further embodiments described below, the generation of the adjusted PoE may be further based on the difference between the adjusted probability of the object in the previous frame and the PoE of the object in the frame. In other words, the adjusted PoE may be generated at least in part based on the change in PoE from the previous time instance. For example, a sudden drop in PoE may indicate a detection error that may require intervention from the tracking extension module 114.

[0029] According to an embodiment, which may be combined with any of the above-mentioned embodiments or further embodiments described below, the method 500 may further include determining the probability decrement value based on whether the object satisfies a set of close distance criteria, thereby allowing the PoE to be appropriately adjusted depending on whether the object is within a close distance from the host vehicle.

[0030] According to one embodiment, which may be combined with any of the above-mentioned embodiments or further embodiments described below, determining the probability decrement value includes assigning a first value to the probability decrement value based on a determination that the object satisfies the set of near distance criteria, and further includes assigning a second value to the probability decrement value based on a determination that the object does not satisfy the set of near distance criteria, the first value being less than the second value. In other words, the tracking extension module 114 may reduce the PoE to a smaller value if the near distance module 118 determines that the object is within the near distance.

[0031] According to one embodiment, which may be combined with any of the above-mentioned embodiments or further embodiments described below, method 500 may further include a step of determining whether the difference between the adjusted PoE of the object in the previous frame and the PoE of the object in the frame exceeds a probability decrement value.

[0032] According to one embodiment, which may be combined with any of the above-mentioned embodiments or further embodiments described below, generating the adjusted PoE includes subtracting a probability decrement value from the adjusted probability of the previous frame based on a determination that the difference exceeds the probability decrement value.

[0033] According to one embodiment, which may be combined with any of the above-mentioned embodiments or further embodiments described below, generating the adjusted PoE includes setting the PoE of the object in the frame as the adjusted PoE based on a determination that the difference does not exceed a probability decrement value.

[0034] According to an embodiment, which may be combined with any of the above-mentioned embodiments or further embodiments described below, the method 500 may further include turning on a predicted extension flag and incrementing a predicted extension time limit based on determining that the object satisfies the set of near distance criteria. In other words, if the near distance module 118 determines that the object is in the near distance, the tracking extension module 114 may switch the tracking process to a predicted extension mode, and the tracking extension module 114 may overwrite the PoE calculated by the PoE module 116. The tracking extension module 114 may also extend the time window in which to operate in the predicted extension mode.

[0035] According to one embodiment, which may be combined with any of the above-described embodiments or further embodiments described below, method 500 may further include a step of determining whether the age of the object in the distributed video exceeds the predicted extension time limit, and a step of resetting the predicted extension time limit based on a determination that the age of the object in the distributed video exceeds the predicted extension time limit.

[0036] According to one embodiment, which may be combined with any of the above-described embodiments or further embodiments described below, the method 500 may further include setting a near distance extension counter based on the time until the object collides with the vehicle based on a determination that the object satisfies the set of near distance criteria.

[0037] According to one embodiment, which may be combined with any of the above-described embodiments or further embodiments described below, the method 500 may further include decrementing a near-distance extension counter based on determining that the PoE is lower than a probability threshold.

[0038] According to one embodiment, which may be combined with any of the above-mentioned embodiments or further embodiments described below, the method 500 may further include a step of determining whether the object satisfies a set of near distance criteria further based on determining that the near distance extension counter of the previous frame is greater than zero.

[0039] According to one embodiment, which may be combined with any of the above-mentioned embodiments or further embodiments described below, the method 500 may further include determining a time until a collision between the vehicle and the object based on the relative speed between the object and the vehicle and the distance between the object and the vehicle.

[0040] 6A and 6B collectively illustrate a flow diagram of an object tracking method 600 according to various embodiments. The method 600 illustrated in FIGS. 6A and 6B may include the method 500 or may be a part of the method 500.

[0041] Referring to FIG. 6A, the method 600 may begin at step 602. The method 600 may be performed by a processor 104 executing at least one of the detection module 112, the tracking extension module 114, the near-field module 118, and the PoE module 116 stored in the storage medium 102. The method 600 may determine, at step 604, (i) a PoE of a previous frame, denoted as "PoE_old", (ii) a PoE of a current frame, denoted as "PoE_curr", calculated by the PoE module 116, and (iii) a difference value between PoE_old and PoE_current, denoted as "PoE_delta". PoE_old may be updated with the value of the last adjusted PoE of the previous frame. PoE_curr may be calculated by the PoE module 116 based at least in part on the output of the detection module 112. PoE_curr may be calculated based on relevant measurements generated by the detection module 112, i.e., whether the detection module 112 can detect an object from the current frame. PoE_curr may further be calculated based on the life cycle of the object detection, i.e., the length of time the object continues to be detected from the first frame of the streamed video. The life cycle of the object detection may be the duration of intermittent detection or the duration of continuous detection.

[0042] The method 600 determines in step 606 whether the target's near field extension counter is greater than zero and whether PoE_curr is greater than or equal to P TIt can be determined whether the probability threshold value is less than the probability threshold value denoted as: The close range extension counter is a counter that tracks the number of frames for which the PoE needs to be adjusted when the target is in close range to the host vehicle.

[0043] The short distance extension counter is zero and / or PoE_curr is P T If the above is not satisfied, the method proceeds to output the final PoE of the current frame in step 638 (shown in FIG. 6B). In other words, the PoE generated by the PoE module 116 is not adjusted and the predicted extension mode may be terminated.

[0044] If the near extension counter is greater than zero and PoE_curr is less than PT, then in step 608 the near extension counter may be decremented, for example by one count.

[0045] After decrementing the near distance extension counter in step 608, a near distance check is performed in step 610. The near distance module 114 can determine whether the tracked object is within a predetermined near distance from the host vehicle. Checking whether the object is within the predetermined near distance may include determining whether the following conditions are met: (a) the speed of the object in a first direction is less than a first speed limit; (b) the speed of the object in a second direction is less than a second speed limit. (c) The relative speed between the vehicle and the object is less than a third speed limit. (d) The decrease in PoE of the PoE module 116 between successive frames is greater than a decrease threshold. (e) The size of the object in the image is large. (f) The distance from the vehicle to the object is less than a predetermined short distance. (g) The classification of the object is a known identified class. (h) The speed of the host vehicle in the first direction is less than a fourth limit value. (i) The speed of the host vehicle is above a minimum threshold that guarantees the host vehicle's movement.

[0046] In cases (a) and (b), the first direction may be orthogonal to the second direction. In (D), the decrease threshold may be, for example, about 0.19. A significant drop in the PoE generated by the PoE module 116 between successive frames may indicate a loss of detection from the detection module 112. In case (e), the object size may be determined by the number of pixels the object occupies in the image. For at least one of (a), (b), (c), and (f), the speed and / or distance may be obtained from another sensor, for example, a radar sensor or a lidar sensor.

[0047] If at least a predetermined number of the above conditions are met, or if all of the above conditions are met, the near-distance module 118 can determine that the object is in the near distance. In step 610, the near-distance module 118 may continue to monitor the output of the detection module 112. If the detection module 112 can detect the object, the near-distance module 118 may set the near-distance condition to false and reset the near-distance extension counter to zero. Otherwise, the near-distance module 118 sets the near-distance condition to true, and the tracking extension module 114 calculates the near-distance extension counter in step 612. The tracking extension module 114 may calculate the near-distance extension counter based on an estimated time point of collision between the host vehicle and the target. The tracking extension module 114 may estimate the collision time between the host vehicle and the target based on the distance between the host vehicle and the target determined in (f) and the relative speed between the host vehicle and the target determined in (c).

[0048] The near-distance extension counter can determine the number of frames to extend to track the object at a near distance. As an example, if the relative speed between the host vehicle and the object is less than the speed threshold, the near-distance extension counter can be set to a predetermined number, such as 30. If the near-distance extension counter exceeds the predetermined number, the tracking extension module 114 can reduce the number of frames to track to the predetermined number, assuming that the number cannot exceed the predetermined number.

[0049] If the relative speed between the host vehicle and the object exceeds the speed threshold, the near distance extension counter can be calculated based on the object age measured in frames and the distance between the object and the host vehicle, based on the relative speed between the host vehicle and the object, and based on the frame rate. For example, the near distance extension counter can be calculated as follows:

number

[0050] Next, in step 64, the tracking extension module 114 may set the Extend_prediction flag to ON and decrease the parameter PoE_decrease to a value lower than the default value, for example 0.01. PoE_decrease is also referred to herein as the "probability decrement value." The Extend_prediction flag set to ON indicates that the prediction extension mode is enabled, and thus the object may continue to be tracked up to the prediction extension time limit even if it is not detected by the detection module. PoE_decrease may be set to a small value so that the PoE does not decrease too rapidly, allowing the tracking device 100 to continue tracking the object for a longer time.

[0051] If, in step 610, the near field module 118 determines that no object is present in the near field, the method may proceed to 616. In step 616, the Extend_prediction flag may be set to OFF, while the PoE_decrease value may be set to a default value, for example, 0.2.

[0052] The method then proceeds to node A 620 which leads to step 622 of FIG. 6B.

[0053] 6B, in step 622, the tracking extension module 114 may determine whether the PoE_delta determined in step 604 exceeds the PoE_decrease calculated in step 614 or 616. If PoE_delta is greater than PoE_decrease, in step 624, a new PoE value, denoted as PoE_new, is set as the difference between PoE_old and PoE_decrease. The new PoE value may also be referred to herein as "adjusted PoE." In other words, PoE_new=PoE_old-PoE_decrease. Otherwise, if PoE_delta is less than or equal to PoE_decrease, the tracking extension module 114 may determine in step 626 that PoE_new is equal to PoE_curr.

[0054] Next, in step 628, the tracking extension module 114 may determine whether the Extend_prediction flag is ON. If the Extend_prediction flag is ON, the tracking extension module 114 may increment the predicted extension max age in step 632. The predicted extension max age is an upper limit set on the maximum age of the object in the predicted extension mode. This is a safe interval in which the predicted extension is valid. The predicted extension max age may be initialized based on the age of the object, i.e., the duration the object continues to be detected, and added to the calculated number of frames. The calculated number of frames may be the number of frames the object has been tracked in the predicted extension mode multiplied by a factor. The factor may be, for example, 0.5. The calculated number of frames may be limited to a range of, for example, 3 to 10. If the factor multiplied by the number of frames the object has been tracked in the predicted extension mode falls below the lower limit of the range, for example, 2, the calculated number of frames may be adjusted to the lower limit, for example, 3. Also, if the factor multiplied by the number of frames an object has been tracked in under the prediction extension mode exceeds the upper limit of the range, e.g., 11, then the calculated number of frames may be adjusted to an upper limit value, e.g., 10.

[0055] The tracking extension module 114 may then proceed to output PoE_new as the final PoE of the current frame in step 638. In other words, the adjusted PoE of the current frame = PoE_new. The adjusted PoE of the current frame may be an output of the methods 500 and 600.

[0056] Otherwise, if the Extend_prediction flag is OFF, the tracking extension module 114 may determine whether the target age exceeds the predicted extension maximum age in step 634. If the target age exceeds the predicted extension maximum age, the tracking extension module 114 may reset the PoE extension information in step 636 before proceeding to output the final PoE of the current frame in step 638. In other words, if the target age exceeds the predicted extension maximum age, the tracking extension module 114 may exit the prediction extension mode. The target age exceeds the predicted extension age when, for example, an object is very close to the host vehicle, causing the close range module 118 to fail to verify that the close range condition is met, and thus the ExtendPrediction flag is OFF and the predicted extension maximum age is not further incremented. In such a scenario, the target age increases until it exceeds the predicted extension maximum age. The tracking extension module 114 may then exit the prediction extension mode by rapidly reducing the PoE, for example by subtracting 0.2 from the PoE value.

[0057] The PoE extension information reset in step 636 may include data used in processing the PoE extension, including (a) the number of frames the prediction was extended by, (b) the predicted extension maximum age, (c) achievement of the near distance criterion, and (d) the near distance extension counter.

[0058] The above-described methods 500 and 600 may provide several advantages.

[0059] Training a classification neural network for near-field detection is typically a costly and resource intensive process. By improving the accuracy of PoE determination, methods 500 and 600 can reduce the cost of training required for a classification neural network for near-field detection.

[0060] Additionally, the method 500 or 600 can prevent a close-range collision by tracking a target at a close range. If either method 500, 600 determines that a collision between the host vehicle and the tracked target is imminent, the host vehicle may initiate an emergency brake assist function to prevent the collision.

[0061] Additionally, the method 500 or 600 can continue to track the target at close range, which can assist in determining whether the target is moving toward or away from the host vehicle, which can then be useful information in determining whether to continue tracking the target.

[0062] The methods 500 and 600 can be applied to any camera-based tracking application. These methods 500, 600 can further assist in detecting road conditions based on changes in the motion characteristics of the tracked object. These methods 500, 600 can be used to improve tracking continuity when the target is temporarily lost to the ego-vehicle's sensors because the target entered a tunnel.

[0063] The following examples illustrate further technical aspects of the above-described devices, systems and methods and should not be construed as claims.

[0064] The following examples may additionally be combined with any of the above-mentioned apparatus, systems and methods, or with any of the claims of the parent application. EXAMPLES

[0065] Example 1 is a computer-implemented object tracking method that includes the steps of receiving real-time streaming video; for each frame of the streaming video, determining a PoE of an object in the frame; determining whether the PoE of the object in the frame is below a probability threshold; based on a determination that the PoE is below the probability threshold, determining whether the object satisfies a set of near-distance criteria; and generating an adjusted PoE based on a determination of whether the object satisfies the set of near-distance criteria.

[0066] In Example 2, the subject matter of Example 1 may optionally include, wherein the step of generating an adjusted PoE is further based on a difference between the adjusted probability of a previous frame and the PoE of the object in the frame.

[0067] In Example 3, the subject matter of any one of Examples 1-2 may optionally include determining the probability decrement value based on determining whether the object satisfies a set of near distance criteria.

[0068] In Example 4, the subject matter of Example 3 optionally includes the step of determining the probability decrement value including a step of assigning a first value to the probability decrement value based on a determination that the object satisfies the set of near distance criteria, and further including a step of assigning a second value to the probability decrement value based on a determination that the object does not satisfy the set of near distance criteria, the first value being less than the second value.

[0069] In Example 5, the subject matter of any one of Examples 3 to 4 may optionally include a step of determining whether the difference between the adjusted probability of the previous frame and the PoE of the object in the frame exceeds a probability decrement value.

[0070] In Example 6, the subject matter of Example 5 may optionally include the step of generating an adjusted PoE including subtracting a decrement value from the adjusted probability of the preceding frame based on a determination that the difference exceeds a probability decrement value.

[0071] In Example 7, any one of the subject matters of Examples 5 to 6 may optionally include a step of generating an adjusted PoE including setting the PoE of the object in the frame as the adjusted PoE based on a determination that the difference does not exceed a probability decrement value.

[0072] In Example 8, the subject matter of any one of Examples 1 to 7 may optionally include turning on a predicted extension flag based on a determination that the object satisfies a set of near distance criteria, and incrementing the predicted extension time limit.

[0073] In Example 9, the subject matter of Example 8 may optionally include a step of determining whether the age of the object in the distributed video exceeds the predicted extension time limit, and a step of resetting the predicted extension time limit based on a determination that the age of the object in the distributed video exceeds the predicted extension time limit.

[0074] In Example 10, the subject matter of any one of Examples 1-9 may optionally include a step of setting a near-distance extension counter based on a time until collision between the object and the vehicle based on a determination that the object satisfies a set of near-distance criteria.

[0075] In Example 11, the subject matter of Example 10 may optionally include decrementing a near field extension counter based on determining that the PoE is lower than a probability threshold.

[0076] In Example 12, the subject matter of any one of Examples 10 to 11 may optionally include a step of determining whether the object satisfies a set of near-distance criteria further based on determining that the near-distance extension counter of the previous frame is greater than zero.

[0077] Example 13 is a non-transitory computer-readable storage medium comprising instructions that, when executed by a processor, perform the computer-implemented method of any one of Examples 1 to 12.

[0078] Example 14 is an object tracking device, the device including a non-transitory computer-readable storage medium of Example 13 and at least one processor coupled to the non-transitory computer-readable storage medium, the at least one processor configured to execute any one of the computer-implemented methods of Examples 1 to 12.

[0079] Example 15 is an object tracking system, which includes the device of Example 14 and a camera configured to capture streaming video.

[0080] In Example 16, the subject matter of any one of Examples 1-12 may optionally include the steps of recognizing an object in the frame and determining a confidence of the recognition.

[0081] In Example 17, the subject matter of Example 16 may optionally include a step of determining the PoE of an object in a frame based on the confidence of recognition and further based on the number of frames in which the object is detected.

[0082] In Example 18, the subject matter of any one of Examples 1 to 12 and 16 may optionally include a step of setting the determined PoE of the object in the frame as the adjusted PoE based on a determination that the determined PoE of the object in the frame is not lower than a probability threshold.

[0083] In Example 19, the subject matter of Example 10 may optionally include determining a time to collision based on a relative speed between the object and the vehicle and a distance between the object and the vehicle.

[0084] Although several embodiments of the present invention have been specifically shown and described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. The scope of the present invention is therefore indicated by the appended claims, and all changes that come within the meaning and range of equivalency of the claims are therefore intended to be embraced. It will be understood that common numerals used in associated drawings refer to elements serving similar or identical purposes.

[0085] Those skilled in the art will appreciate that the terminology used herein is for the purpose of describing various embodiments only and is not intended to limit the invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that the terms "comprise" and / or "comprising" as used herein specify the presence of stated features, integers, steps, operations, elements, and / or elements, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, elements, and / or groups thereof.

[0086] It is understood that the particular order or hierarchy of blocks in the disclosed process / flow diagrams is an illustration of an example approach. It is understood that the particular order or hierarchy of blocks in the process / flow diagrams may be rearranged based on design preferences. Further, some blocks may be combined or omitted. The accompanying method claims present elements present in the various blocks in a sample order, and are not intended to be limited to the particular order or hierarchy presented.

[0087] The above description is presented to enable any person skilled in the art to practice the various embodiments described herein. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein apply to other embodiments. Thus, the claims are not intended to be limited to the embodiments shown herein, but are to be given the full scope consistent with the language of the claims, and the use of an element in the singular is intended to mean "one or more" rather than "only one" unless specifically stated otherwise. The term "exemplary" is used herein to mean "serving as an example, illustration, or illustration." Any embodiment described herein as "exemplary" should not necessarily be construed as preferred or advantageous over other embodiments. Unless specifically stated otherwise, the term "some" refers to one or more. Combinations such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof" include any combination of A, B, and / or C, and may include multiple As, multiple Bs, or multiple Cs. Specifically, combinations such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," "A, B, C, or any combination thereof" may be A only, B only, C only, A and B, A and C, B and C, or A and B and C, and any such combination may include one or more individuals or populations of A, B, or C. All structural and functional equivalents to the elements of the various embodiments described throughout this disclosure that are known or that later become known to those of skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims.

Claims

1. A computer-implemented object tracking method (500), comprising: Receiving real-time streaming video (502); For each frame of the distribution video, determining (504) a probability of presence of said object in said frame; determining (506) whether the probability of the presence of the object in the frame is below a probability threshold; determining (508) whether the object satisfies a set of proximity criteria based on a determination that the presence probability is less than the probability threshold; The method (500) includes generating (510) an adjusted presence probability based on a determination of whether the object satisfies the set of proximity criteria.

2. 2. The computer-implemented method of claim 1, wherein generating the adjusted presence probability is further based on a difference between an adjusted presence probability of the object in a previous frame and a presence probability of the object in the frame.

3. determining a probability decrement value based on whether the object satisfies the set of near distance criteria; 3. A computer-implemented method (500) according to claim 1 or 2.

4. 4. The computer-implemented method (500) of claim 3, wherein determining the probability decrement value comprises: assigning a first value to the probability decrement value based on a determination that the object satisfies the set of near distance criteria; and further comprising: assigning a second value to the probability decrement value based on a determination that the object does not satisfy the set of near distance criteria, the first value being less than the second value.

5. determining whether a difference between the adjusted presence probability of the previous frame and the presence probability of the object in the frame exceeds the probability decrement value; A computer implemented method (500) according to claim 3 or 4.

6. 6. The computer-implemented method (500) of claim 5, wherein generating the adjusted presence probability comprises subtracting the decrement value from the adjusted probability of the previous frame based on a determination that the difference exceeds the probability decrement value.

7. 7. The computer-implemented method (500) of claim 5 or 6, wherein generating the adjusted presence probability comprises setting the presence probability of the object in the frame as the adjusted presence probability based on a determination that the difference does not exceed the probability decrement value.

8. setting a prediction extension flag ON based on a determination that the object satisfies the set of near distance criteria; further comprising the step of incrementing the predicted extension time limit. A computer implemented method (500) according to any one of claims 1 to 7.

9. determining whether the age of the object in the distribution video exceeds the predicted extension time limit; The method further includes a step of resetting the predicted extension time limit based on a determination that the age of the object in the distribution video has exceeded the predicted extension time limit.

9. The computer-implemented method (500) of claim 8.

10. based on a determination that the object satisfies the set of proximity criteria, setting a proximity extension counter based on a time to collision of the object with the vehicle. A computer implemented method (500) according to any one of the preceding claims.

11. and decrementing the near distance extension counter based on a determination that the presence probability is less than the probability threshold.

11. The computer-implemented method (500) of claim 10.

12. determining whether the object satisfies the set of near distance criteria further based on determining that a near distance extension counter of a previous frame is greater than zero; 12. A computer implemented method (500) according to claim 10 or 11.

13. A non-transitory computer-readable storage medium (102) comprising instructions which, when executed by a processor, perform a computer-implemented method (500) according to any one of claims 1 to 12.

14. An apparatus (100) for tracking an object, comprising: A non-transitory computer readable storage medium (102) according to claim 13; An apparatus (100) comprising at least one processor (104) coupled to the non-transitory computer-readable storage medium (302), the at least one processor (104) configured to execute a computer-implemented method (500) according to any one of claims 1 to 12.

15. An apparatus (100) according to claim 14, a camera (306) configured to capture video for distribution; An object tracking system (200).

Images