Computer-implemented method for tracking object, device for tracking object and system for tracking object
By receiving real-time video frames and adjusting the probability of existence, the failure problem when tracking targets is solved, improving tracking accuracy and collision avoidance capabilities.
Patent Information
- Application Number
- CN202380030769.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-02
- Filing Date
- 2023-02-22
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-02-22
AI Technical Summary
The prior art fails when tracking targets at close range, resulting in the vehicle potentially losing track of the targets, affecting the ability to avoid collisions.
By receiving real-time video feeds, it is determined whether the probability of the object's existence in the video frame is below the probability threshold, and the adjusted probability of existence is generated based on the close-range criteria, extending the duration of the tracking target.
Improves the accuracy and duration of tracking targets, and can effectively track objects at very close distances to help prevent collisions.
Smart Images

Figure CN119013696A_ABST
Abstract
Description
Technical Field
[0001] Various embodiments relate to a computer-implemented method for tracking an object, an apparatus for tracking an object, and a system for tracking an object. Background Art
[0002] It is useful for a vehicle to track a target, such as a vehicle ahead, to avoid a collision. The vehicle typically detects the target and tracks the target by applying image recognition technology to images captured by a camera mounted on the vehicle. However, when the target gets close to the camera, the vehicle may lose track of the target. When the target gets close to the camera, the image of the target may become out of focus, or the target may be only partially captured in the image. This makes conventional image recognition technology ineffective in identifying the target, and thus affects the vehicle's ability to track the target.
[0003] In view of the above, a solution is needed to address the challenge of tracking targets at close range. Summary of the invention
[0004] According to various embodiments, a computer-implemented method for tracking an object is provided. The method includes: receiving a real-time video feed. The method further includes: for each frame of the video feed, determining a probability of presence of the object in the frame; determining whether the probability of presence of the object in the frame is below a probability threshold; determining whether the object meets a set of close distance criteria based on determining that the probability of presence is below the probability threshold; and generating an adjusted probability of presence based on the determination of whether the object meets the set of close distance criteria.
[0005] According to various embodiments, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium comprising instructions that, when executed by a processor, perform the above-described computer-implemented method.
[0006] According to various embodiments, a device for tracking an object is provided. The device includes the above-mentioned non-transitory computer-readable storage medium and at least one processor. The at least one processor can be coupled to the non-transitory computer-readable storage medium and can be configured to perform the above-mentioned computer-implemented method.
[0007] According to various embodiments, a system for tracking an object is provided. The system includes the above device and a camera configured to capture the video feed.
[0008] Additional features of advantageous embodiments are provided in the dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] In the drawings, like reference numerals generally refer to like parts throughout the different views. The drawings are not necessarily drawn to scale, but emphasis is generally placed on illustrating the principles of the invention. In the following description, various embodiments are described with reference to the following drawings, in which:
[0010] Figure 1 A conceptual diagram of a tracking device according to various embodiments is shown.
[0011] Figure 2 A conceptual diagram of a system for tracking an object in accordance with various embodiments is shown.
[0012] Figure 3 A conceptual diagram of a storage medium according to various embodiments is shown.
[0013] Figure 4A and Figure 4B An example of an image being processed to track a target in accordance with various embodiments is shown.
[0014] Figure 5 A flow chart of a computer-implemented method for tracking an object is shown in accordance with various embodiments.
[0015] Fig. 6A and Figure 6B Together, a flow chart of a method for tracking an object according to various embodiments is shown. DETAILED DESCRIPTION
[0016] The embodiments described below in the context of the apparatus are similarly valid for the corresponding methods, and vice versa.In addition, it should be understood that the embodiments described below may be combined, for example, part of one embodiment may be combined with part of another embodiment.
[0017] It should be understood that any property described herein for a specific device may also apply to any device described herein. It should be understood that any property described herein for a specific method may also apply to any method described herein. In addition, it should be understood that for any device or method described herein, not all of the described components or steps must be included in the device or method, but only some (but not all) components or steps may be included.
[0018] The term "coupling" (or "connection") in this document may be understood as electrical coupling or mechanical coupling (such as attachment or fixation), or merely contact without any fixation, and it should be understood that both direct coupling and indirect coupling (in other words: coupling without direct contact) may be provided.
[0019] In this context, a device as described in this specification may include a memory, which is used, for example, in the processing performed in the device. The memory used in the embodiments may be a volatile memory, such as a DRAM (dynamic random access memory), or a non-volatile memory, such as a PROM (programmable read-only memory), an EPROM (erasable PROM), an EEPROM (electrically erasable PROM) or a flash memory, such as a floating gate memory, a charge trap memory, an MRAM (magnetoresistive random access memory) or a PCRAM (phase change random access memory). The memory may include a non-transitory storage medium.
[0020] The term tracked "object" may also be interchangeably referred to herein as a "target."
[0021] In order that the invention may be readily understood and put into practice, various embodiments will now be described by way of example and not limitation and with reference to the accompanying drawings.
[0022] Figure 1 A conceptual diagram of a tracking device 100 according to various embodiments is shown. The tracking device 100 may also be referred to herein as a device for tracking an object. The tracking device 100 may include a storage medium 102 and a processor 104. The tracking device 100 may include an additional processor 104. The storage medium 102 may be a non-transitory computer-readable storage medium. The storage medium 102 may include at least one memory. The storage medium 102 and the processor 104 may be coupled to each other via a coupling line 110, such as an electrical coupling and / or a communication coupling. The processor 104 may be configured to execute a command regarding a tracking object. Figure 5 as well as Fig. 6A and Figure 6B Describes a method for tracking objects.
[0023] Figure 2 A conceptual diagram of a system 200 for tracking an object according to various embodiments is shown. The system 200 may include the device 100. The system 200 may further include a camera 306 configured to capture images. The camera 306 may be configured to capture a video feed including a sequence of images. The device 100 and the camera 306 may be coupled to each other via a coupling line 130, such as an electrical coupling and / or a communication coupling.
[0024] According to various embodiments, the device 100 may be provided on a vehicle (referred to herein as the host vehicle). In alternative embodiments, the device 100 may be provided separately from the host vehicle, such as on a remote server or a cloud server.
[0025] According to various embodiments, the system 200 may be at least partially disposed onboard a host vehicle.
[0026] Figure 3 A conceptual diagram of a storage medium 102 according to various embodiments is shown. The storage medium 102 may include a detection module 112, a tracking extension module 114, a probability of existence (PoE) module 116, and a close distance module 118. At least one of the detection module 112, the tracking extension module 114, the PoE module 116, and the close distance module 118 may be a computer algorithm stored on the storage medium 102 and executable by the processor 104. In an alternative embodiment, at least one of the detection module 112, the tracking extension module 114, the PoE module 116, and the close distance module 118 may be stored on a different storage medium 102. The detection module 112 may be configured to detect and classify a target in an image captured by a camera (e.g., camera 306) based on image processing and recognition techniques. The PoE module 116 may be configured to determine the PoE of the target based on output from the detection module 112. The PoE module 116 may also further determine the PoE based on other information such as the age of the target and input from other sensors on the host vehicle. The PoE indicates a measure of certainty (i.e., probability) that a target is present. The proximity module 118 may be configured to determine whether a target is within a predefined distance from the host vehicle. The tracking extension module 114 may be configured to modify the PoE determined by the PoE module 116 based at least in part on the output of the proximity module 118. The tracking extension module 114 may extend the duration that the tracking device 100 tracks the target based at least in part on determining that the target is within a close range of the camera. The detection module 112, the tracking extension module 114, the PoE module 116, and the proximity module 118 may be coupled to each other via a coupling line 140, such as an electrical coupling and / or a communication coupling.
[0027] Figure 4A and Figure 4B An example of an image processed to track a target 402 according to various embodiments is shown. The image may be captured by a camera (such as camera 306) onboard the host vehicle. Tracking device 100 may receive the image from camera 306 and may detect target 402 based on the received image. For example, detection module 112 of tracking device 100 may discern features of target 402 in the image, thereby identifying target 402 based on the discerned features. These features may include, for example, the outline of target 402, characteristic shapes and colors of portions of target 402.
[0028] refer to Figure 4A, when the target 402 is far from the host vehicle, the camera 306 can capture an image 400A including the entire outline of the target 402. In this example, the target 402 is more than 10 meters away from the host vehicle. The detection module 112 can identify the overall shape of the target 402, thereby generating a bounding box 404 around the target 402 in the image 400A. The PoE module 116 may output a high PoE based on the detection result because the detection module 112 indicates that the confidence level of detecting the target 402 is high.
[0029] refer to Figure 4B , when the target 402 approaches the host vehicle, a portion of the target 402 will be outside the field of view of the camera 306. In this example, the target 402 is less than 10 meters away from the host vehicle. When the host vehicle and the target 402 move relative to each other so that the distance between them is shortened, the features of the target 402 are magnified in the camera image (as shown in image 400B). The overall shape of the target 402 cannot be seen in image 400B. Therefore, the detection module 112 may not be able to detect and / or identify the target 402. Therefore, the PoE module 116 may output a low PoE based on the detection results because the detection module 112 may indicate that the confidence level of the detection of the target 402 is low. When the target 402 moves even closer to the host vehicle, it may become increasingly difficult for the detection module 112 to detect the target 402, and therefore, the PoE module 116 may further reduce its determined PoE. If the PoE module 116 continues to reduce its determined PoE until the PoE reaches the minimum threshold, the tracking device 100 may stop tracking the target 402. This may be a dangerous outcome, since target 402 is not only present, but also within close range of the host vehicle and therefore poses a collision threat to the host vehicle. To overcome this problem, tracking device 100 may include close range module 118 and tracking extension module 114. Close range module 118 may determine whether target 402 is within a predefined close range of the host vehicle based on a plurality of close range conditions. Fig. 6A These close range conditions are described in 610. When the close range module 118 determines that the target 402 is within a close range, the tracking extension module 114 can modify or override the output PoE of the detection module 112 in order to continue tracking the target 402. The tracking extension module 114 can assist the tracking device 100 in tracking the target 402 for a longer duration by adjusting the PoE value of the target, even when no detection results are available for multiple frames of the video feed.
[0030] Figure 5A flow chart of a computer-implemented method 500 for tracking an object is shown in accordance with various embodiments. The tracked object may also be referred to herein as a target. The method 500 may address a lack of detection results at close range by further tracking the detection results even when the detection module 112 fails to detect or identify the target. The method 500 may include determining whether the target is within close range of the host vehicle, and then overriding the lack of detection results of the detection module when it is determined that the target is within close range of the host vehicle within an extended prediction time limit.
[0031] The method 500 may include receiving a real-time video feed at 502. The method 500 may further include determining a PoE of an object in each frame of the video feed at 504. The PoE indicates a confidence level that the object is present and may be determined by the detection module 112. The detection module 112 may receive the video feed as an input and may generate a detection result as an output. The PoE module 118 may receive the detection result from the detection module 112 and may generate a PoE as an output. The method 500 may further include determining whether the PoE of the object in the frame is below a probability threshold (P T ). The method 500 may further include: at 508, based on determining that the PoE is lower than the P T The method 500 may further include generating an adjusted PoE based on the determination of whether the object satisfies the set of close distance criteria at 510. Each of 504, 506, 508, 510 may be performed for each frame of the video feed and may be performed by the tracking extension module 114.
[0032] Method 500 can continuously track an object based on the current scene (as captured by the current frame of the video feed) rather than relying solely on predicted future positions. This improves the accuracy of tracking. Moreover, the computational cost in terms of resources and time is less because the algorithm involves simple arithmetic operations and can effectively track objects at very close distances. The method 500 may be able to track a target even at 3 to 6 meters or even closer to the vehicle. The method 500 may not only depend on the specific characteristics of the target, and therefore may not be limited to tracking a specific type of object. The method 500 may further determine the probability of the existence of an object based on at least one of vehicle dynamics, cameras, environmental parameters, etc. The method 500 solves the challenges faced when tracking at close range, thereby helping to prevent collisions.
[0033] According to embodiments that may be combined with the above embodiments or with any of the embodiments further described below, generating the adjusted PoE may be further based on a difference between the adjusted probability of the object in the previous frame and the PoE of the object in the frame. In other words, the adjusted PoE may be generated based at least in part on a change in PoE since the previous time instance. For example, a sharp decrease in PoE may indicate a detection error that may require intervention from the tracking extension module 114.
[0034] According to embodiments which may be combined with the above embodiments or with any of the embodiments further described below, method 500 may further include: determining a probability decrement value based on a determination of whether the object meets the set of close distance criteria. This allows the PoE to be appropriately adjusted depending on whether the object is within close distance of the host vehicle.
[0035] According to an embodiment that can be combined with the above embodiments or with any of the embodiments further described below, determining the probability decrement value includes assigning a first value to the probability decrement value based on determining that the object meets the set of close distance criteria, and further includes assigning a second value to the probability decrement value based on determining that the object does not meet the set of close distance criteria, wherein the first value is less than the second value. In other words, if the close distance module 118 determines that the object is within the close distance, the tracking extension module 114 can reduce the PoE to a lesser extent.
[0036] According to embodiments which may be combined with the above embodiments or with any embodiments further described below, method 500 may further include determining whether a difference between an adjusted PoE of the object in a previous frame and the PoE of the object in the frame exceeds a probability decrement value.
[0037] According to embodiments which may be combined with the above embodiments or with any embodiments described further below, generating the adjusted PoE comprises subtracting the probability decrement value from the adjusted probability of the previous frame based on determining that the difference exceeds the probability decrement value.
[0038] According to an embodiment which may be combined with the above embodiments or with any embodiment described further below, generating the adjusted PoE comprises setting the PoE of the object in the frame to the adjusted PoE based on determining that the difference does not exceed the probability decreasing value.
[0039] According to an embodiment that can be combined with the above embodiments or with any of the embodiments further described below, the method 500 can further include: turning on an extended prediction flag based on determining that the object meets the set of close distance criteria; and incrementing the extended prediction time limit. In other words, when the close distance module 118 determines that the object is within the close distance, the tracking extension module 114 can switch the tracking process to an extended prediction mode, in which the tracking extension module 114 can overwrite the PoE calculated by the PoE module 116. The tracking extension module 114 can also extend the time window of operation in the extended prediction mode.
[0040] According to an embodiment that can be combined with the above embodiments or with any embodiment further described below, method 500 may further include: determining whether the duration of the object in the video feed exceeds the extended prediction time limit; and resetting the extended prediction time limit based on determining that the duration of the object in the video feed exceeds the extended prediction time limit.
[0041] According to embodiments which may be combined with the above embodiments or with any embodiments further described below, method 500 may further include: based on determining that the object meets the set of close distance criteria, setting a close distance extension counter based on a collision time between the object and the vehicle.
[0042] According to embodiments which may be combined with the above embodiments or with any embodiments further described below, the method 500 may further include decrementing a close extension counter based on determining that the PoE is below a probability threshold.
[0043] According to embodiments which may be combined with the above embodiments or with any embodiments further described below, method 500 may further include determining whether the object meets the set of close distance criteria further based on determining that a close distance extension counter of a previous frame is greater than zero.
[0044] According to embodiments which may be combined with the above embodiments or with any embodiments further described below, method 500 may further include determining a collision time between the vehicle and the object based on a relative velocity between the object and the vehicle and a distance between the object and the vehicle.
[0045] Fig. 6A and Figure 6B Together, a flow chart of a method 600 for tracking an object is shown in accordance with various embodiments. Fig. 6A and Figure 6B The method 600 shown in FIG. 6 may include the method 500 or may be a part of the method.
[0046] refer to Fig. 6A, the method 600 may include starting the method in 602. The method 600 may be performed by the processor 104, which runs at least one of the detection module 112, the tracking extension module 114, the close distance module 118, and the PoE module 116 stored on the storage medium 102. The method 600 may include defining in 604 the values of: (i) the PoE of the previous frame, denoted as "PoE_old", (ii) the PoE of the current frame as calculated by the PoE module 116, denoted as "PoE_curr", and (iii) the difference between PoE_old and PoE_current, denoted as "PoE_delta". PoE_old may be updated with the value of the final adjusted PoE of the immediately previous frame. PoE_curr may be calculated by the PoE module 116 based at least in part on the output of the detection module 112. PoE_curr may be calculated based on the associated measurement results generated by the detection module 112, in other words, based on whether the detection module 112 is able to detect the object from the current frame. PoE_curr may further be based on the lifetime of the object detection, in other words, based on the time the object has been detected since the first frame of the video feed. The lifetime of the object detection may be the duration of intermittent detection, or may be the duration of continuous detection.
[0047] The method 600 may further include determining at 606 whether the target's close range extension counter is greater than zero and whether PoE_curr is less than a value denoted as P T The close range extension counter is a counter of the number of frames that need to adjust its PoE to keep tracking when the target is within close range of the host vehicle.
[0048] If the proximity extension counter is zero, and / or PoE_curr is equal to or greater than P T , the method continues to output the final PoE ( Figure 6B In other words, the PoE generated by the PoE module 116 is not adjusted and the extended prediction mode may be terminated.
[0049] If the close extension counter is greater than zero and PoE_curr is less than P T , the close distance extension counter can be decremented in 608, for example, by one count.
[0050] After decrementing the close distance extension counter in 608, a close distance check is performed in 610. The close distance module 114 may determine whether the tracked object is within a predefined close distance from the host vehicle. Checking whether the object is within a predefined close distance may involve determining whether the following conditions are met: (a) the velocity of the object in the first direction is less than a first speed limit; (b) the velocity of the object in the second direction is less than a second speed limit; (c) the relative speed between the host vehicle and the object is less than the third speed limit; (d) the reduction of PoE by the PoE module 116 between consecutive frames is greater than a reduction threshold; (e) The size of the objects in the image is large; (f) The distance between the object and the vehicle is less than the predefined close distance; (g) Classification of objects is to determine known categories; (h) the speed of the vehicle in the first direction is less than a fourth limit; and (i) The speed of the host vehicle is greater than a minimum threshold for ensuring that the host vehicle is moving.
[0051] For (a) and (b), the first direction may be orthogonal to the second direction. In (d), the reduction threshold may be, for example, about 0.19. A substantial drop in the PoE generated by the PoE module 116 between consecutive frames may indicate a missing detection result by the detection module 112. For (e), the object size may be determined by the number of pixels occupied by the object in the image. For at least one of (a), (b), (c), and (f), the velocity and / or distance may be obtained from another sensor (e.g., a radar sensor or a lidar sensor).
[0052] If at least a predefined number of the above conditions are met, or if all of the above conditions are met, the close distance module 118 may determine that the object is within close range. In 610, the close distance module 118 may also continue to monitor the output of the detection module 112. If the detection module 112 is able to detect the object, the close distance module 118 may set the close distance condition to false and reset the close distance extension counter to zero. Otherwise, the close distance module 118 sets the close distance condition to true, and the tracking extension module 114 calculates the close distance extension counter in 612. The tracking extension module 114 may calculate the close distance extension counter based on the estimated collision time between the host vehicle and the target. The tracking extension module 114 may estimate the collision time between the host vehicle and the target based on the distance between the host vehicle and the target (as determined in (f)) and their relative velocity (as determined in (c)).
[0053] The close distance extension counter may define the number of frames to be extended to track an object in close distance. As an example, if the relative velocity between the host vehicle and the object is less than a velocity threshold, the close distance extension counter may be set to a predefined number, such as 30. If the close distance extension counter exceeds the predefined number, the tracking extension module 114 may assume that the number of frames to be tracked cannot be greater than the predefined number and reduce it to the predefined number.
[0054] If the relative speed between the host vehicle and the object exceeds the speed threshold, a close distance extension counter may be calculated based on the object duration measured in frames and the distance between the object and the host vehicle, and further based on the relative speed between the host vehicle and the object, and further based on the frame rate. For example, the close distance extension counter may be calculated as follows: In other words, the close distance extension counter may be calculated based on the number of frames that may pass in one second, taking into account that both the host vehicle and the object are moving. For example, the velocity threshold may be 0.001 m / s.
[0055] Next, in 64, the tracking extension module 114 can set the Extend_prediction flag to on and can reduce the parameter PoE_decrease to a value below the default value, for example, 0.01. PoE_decrease is also referred to herein as a "probability decrement value." Setting Extend_prediction to on indicates that the extended prediction mode is turned on, and therefore the object can continue to be tracked within the extended prediction time limit, even if the detection module does not detect the object. PoE_decrease can be set to a smaller value so that the PoE does not decrease abruptly, thereby allowing the tracking device 100 to continue to track the object for a longer time.
[0056] If at 610 the close range module 118 determines that the object is not within close range, the method may proceed to 616. At 616, the Extend_prediction flag may be set to off, and the PoE_decrease value may be set to a default value, such as 0.2.
[0057] Next, the method proceeds to Figure 6B 622 of "A" 620.
[0058] refer to Figure 6B, in 622, the tracking extension module 114 may determine whether PoE_delta, as defined in 604, exceeds PoE_decrease calculated in 614 or 616. If PoE_delta is greater than PoE_decrease, then in 624, a new PoE value, denoted as PoE_new, is set to the difference between PoE_old and PoE_decrease. The new PoE value may also be referred to herein as "adjusted PoE." In other words, PoE_new = PoE_old - PoE_decrease. Otherwise, if PoE_delta is equal to or less than PoE_decrease, then in 626, the tracking extension module 114 may define PoE_new to be equal to PoE_curr.
[0059] Next, in 628, the tracking extension module 114 can determine whether the Extend_prediction flag is on. If the Extend_prediction flag is on, then in 632, the tracking extension module 114 can increment the prediction extension maximum duration. For the extended prediction mode, the prediction extension maximum duration is an upper limit imposed on the maximum duration of the object. It is a safety interval during which the prediction extension is activated. The prediction extension maximum duration can be initialized based on the duration of the object (i.e., the duration for which the object is detected) plus the calculated number of frames. The calculated number of frames can be the result of a factor multiplied by the number of frames in which the object is tracked in the extended prediction mode. The factor can be, for example, 0.5. The calculated number of frames can be limited to, for example, a range of 3 to 10. When the result of the factor multiplied by the number of frames in which the object is tracked in the extended prediction mode is lower than the lower limit of the range (e.g., the result is 2), the calculated number of frames can be adjusted to the lower limit value (e.g., 3). Also, when a result of multiplying the factor by the number of frames in which the object is tracked in the extended prediction mode exceeds an upper limit of the range (eg, the result is 11), the calculated number of frames may be adjusted to the upper limit value (eg, 10).
[0060] Next, in 638 , the tracking extension module 114 may continue to output PoE_new as the final PoE of the current frame. In other words, the adjusted PoE of the current frame=PoE_new. The adjusted PoE of the current frame may be the output of the methods 500 and 600 .
[0061] Otherwise, if the Extend_prediction flag is off, then in 634, the tracking extension module 114 can determine whether the target duration is greater than the predicted extension maximum duration. If the target duration is greater than the predicted extension maximum duration, the tracking extension module 114 can reset the PoE extension information in 636 and then proceed to output the final PoE of the current frame in 638. In other words, if the target duration is greater than the predicted extension maximum duration, the tracking extension module 114 can terminate the extended prediction mode. For example, when the object is too close to the host vehicle, so that the close distance module 118 fails to confirm that the close distance condition is met, and therefore the ExtendPrediction flag is off and the predicted extension maximum duration is no longer increasing, the target duration can become greater than the predicted extension maximum duration. In this scenario, the target duration increases until the target duration is greater than the predicted extension maximum duration. Then, the tracking extension module 114 can quickly reduce the PoE, for example, by subtracting 0.2 from the PoE value, thereby terminating the extended prediction mode.
[0062] The PoE extension information reset in 636 may include data for handling PoE extension, including: (a) the predicted number of frames that have been extended, (b) the predicted maximum duration of the extension, (c) the satisfaction of the proximity criteria, and (d) the proximity extension counter.
[0063] The above-described methods 500 and 600 may provide several advantages.
[0064] The training process of a classification neural network for close range is typically expensive and resource intensive. By improving the accuracy of PoE determination, methods 500 and 600 can reduce the training cost required for a classification neural network for close range detection.
[0065] Furthermore, the method 500 or 600 can prevent a near collision by closely tracking the target. When any of the methods 500, 600 determines that a collision between the host vehicle and the tracked target is imminent, the host vehicle can activate its emergency brake assist to prevent the collision.
[0066] Further, the method 500 or 600 can assist in determining whether the target is moving toward or away from the host vehicle, as the target can continue to be tracked at close range. This in turn can be useful information for determining whether the target should continue to be tracked.
[0067] Methods 500 and 600 can be applied to any camera-based tracking application. These methods 500, 600 can further assist in detecting road conditions based on kinematic changes of the tracked object. These methods 500, 600 can be used to improve the continuity of tracking when the target enters a tunnel so that the sensors on the host vehicle cannot detect the target temporarily.
[0068] The following examples describe further technical aspects of the above-described devices, systems, and methods and should not be construed as claims.
[0069] The following examples may additionally be combined with any of the devices, systems, and methods as described above or with any of the originally filed claims.
[0070] Example 1 is a computer-implemented method for tracking an object, the method comprising: receiving a real-time video feed; for each frame of the video feed: determining a PoE of the object in the frame; determining whether the PoE of the object in the frame is below a probability threshold; determining whether the object meets a set of close distance criteria based on determining that the PoE is below the probability threshold; and generating an adjusted PoE based on the determination of whether the object meets the set of close distance criteria.
[0071] In Example 2, the subject matter of Example 1 can optionally include generating the adjusted PoE further based on a difference between the adjusted probability of a previous frame and the PoE of the object in the frame.
[0072] In Example 3, the subject matter of any one of Examples 1 to 2 may optionally include determining a probability decrement value based on the determination of whether the object satisfies the set of close distance criteria.
[0073] In Example 4, the subject matter as described in Example 3 may optionally include: determining the probability decrement value includes assigning a first value to the probability decrement value based on determining that the object meets the set of close distance criteria, and further includes assigning a second value to the probability decrement value based on determining that the object does not meet the set of close distance criteria, wherein the first value is less than the second value.
[0074] In Example 5, the subject matter of any one of Examples 3 to 4 may optionally include determining whether a difference between the adjusted probability of a previous frame and the PoE of the object in the frame exceeds the probability decrement value.
[0075] In Example 6, the subject matter of Example 5 can optionally include generating the adjusted PoE comprising subtracting the probability decrement value from the adjusted probability of the previous frame based on determining that the difference exceeds the probability decrement value.
[0076] In Example 7, the subject matter of any one of Examples 5 to 6 may optionally include generating the adjusted PoE comprising setting the PoE of the object in the frame to the adjusted PoE based on determining that the difference does not exceed the probability decrement value.
[0077] In Example 8, the subject matter of any one of Examples 1 to 7 may optionally include: turning on an extended prediction flag based on determining that the object meets the set of close distance criteria; and incrementing an extended prediction time limit.
[0078] In Example 9, the subject matter as described in Example 8 may optionally include: determining whether the duration of the object in the video feed exceeds the extended prediction time limit; and resetting the extended prediction time limit based on determining that the duration of the object in the video feed exceeds the extended prediction time limit.
[0079] In Example 10, the subject matter of any one of Examples 1 to 9 may optionally include: based on determining that the object meets the set of close range criteria, setting a close range extension counter based on a collision time between the object and the vehicle.
[0080] In Example 11, the subject matter of Example 10 can optionally include decrementing the close extension counter based on determining that the PoE is below the probability threshold.
[0081] In Example 12, the subject matter of any one of Examples 10 to 11 may optionally include determining whether the object satisfies the set of close distance criteria further based on determining that the close distance extension counter of the previous frame is greater than zero.
[0082] Example 13 is a non-transitory computer-readable storage medium comprising instructions that, when executed by a processor, perform the computer-implemented method of any one of Examples 1 to 12.
[0083] Example 14 is a device for tracking an object, the device comprising: a non-transitory computer-readable storage medium as described in Example 13; and at least one processor coupled to the non-transitory computer-readable storage medium, wherein the at least one processor is configured to execute a computer-implemented method as described in any one of Examples 1 to 12.
[0084] Example 15 is a system for tracking an object, the system comprising: the device as described in Example 14; and a camera configured to capture the video feed.
[0085] In Example 16, the subject matter of any one of Examples 1 to 12 may optionally include: identifying the object in the frame; and determining a confidence level of the identification.
[0086] In Example 17, the subject matter of Example 16 can optionally include determining the PoE of the object in the frame based on the confidence of the recognition and further based on the number of frames in which the object is detected.
[0087] In Example 18, the subject matter of any one of Examples 1 to 12 and 16 may optionally include: based on determining that the determined PoE of the object in the frame is not below the probability threshold, setting the determined PoE of the object in the frame to the adjusted PoE.
[0088] In Example 19, the subject matter of Example 10 can optionally include determining the collision time based on a relative velocity between the object and the vehicle and a distance between the object and the vehicle.
[0089] Although embodiments of the present invention have been specifically shown and described with reference to particular embodiments, it will be appreciated by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention as defined in the appended claims. Therefore, the scope of the present invention is indicated by the appended claims, and it is therefore intended to cover all changes that fall within the meaning and scope of the equivalents of the claims. It should be understood that common numerals used in the relevant drawings refer to components used for similar or identical purposes.
[0090] It will be understood by those skilled in the art that the terms used herein are only for the purpose of various embodiments and are not intended to limit the present invention. As used herein, unless the context clearly indicates otherwise, the singular forms "a / an" and "the" are intended to also include plural forms. It will be further understood that the terms "include" and / or "comprise" used in this specification specify the presence of stated features, integrations, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integrations, steps, operations, elements, parts and / or groups thereof.
[0091] It should be understood that the specific order or hierarchy of the boxes in the disclosed process / flowchart is a display of an exemplary method. It should be understood that the specific order or hierarchy of the boxes in the process / flowchart can be rearranged based on design preferences. Further, some boxes can be combined or omitted. The attached method requires the elements of each box to be presented in a sample order, and is not intended to be limited to the specific order or hierarchy presented.
[0092] Provide the previous description so that those skilled in the art can practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applicable to other aspects. Therefore, the claims are not intended to be limited to the various aspects shown herein, but are intended to be given the full scope consistent with the language claims, wherein the reference to the singular element is not intended to mean "there is and only one" (unless specifically stated), but refers to "one or more". The word "exemplary" is used herein to mean "serving as an example, instance or display". Any aspect described as "exemplary" herein is not necessarily interpreted as being preferred or advantageous over other aspects. Unless otherwise specifically stated, the term "some" refers to one or more. Combinations such as "at least one of A, B or C", "one or more of A, B or C", "at least one of A, B and C", "one or more of A, B and C", and "A, B, C or any combination thereof" include any combination of A, B and / or C, and may include multiple A, multiple B or multiple C. In particular, combinations such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof" can be A only, B only, C only, A and B, A and C, B and C, or A and B and C, where any such combination can include one or more members of A, B, or C. All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims.
Claims
1. A computer-implemented method (500) for tracking an object, the method (500) comprising: Receive a real-time video feed (502); For each frame of the video feed: Determine the existence probability of the object in the frame (504); Determining whether the probability of the object existing in the frame is below a probability threshold (506); determining whether the object meets a set of close distance criteria based on determining that the probability of presence is below the probability threshold (508); and An adjusted presence probability is generated based on the determination of whether the object satisfies the set of close distance criteria (510).
2. The computer-implemented method (500) of claim 1, wherein: Generating the adjusted presence probability (510) is further based on a difference between the adjusted presence probability of the object in a previous frame and the presence probability of the object in the current frame.
3. The computer-implemented method (500) of claim 1, further comprising: A probability decrement value is determined based on the determination of whether the object satisfies the set of close distance criteria.
4. The computer-implemented method (500) of claim 3, wherein: Determining the probability decrement value includes assigning a first value to the probability decrement value based on determining that the object meets the set of close distance criteria, and further includes assigning a second value to the probability decrement value based on determining that the object does not meet the set of close distance criteria, wherein the first value is less than the second value.
5. The computer-implemented method (500) of claim 3, further comprising: A determination is made as to whether a difference between the adjusted existence probability of the previous frame and the existence probability of the object in the frame exceeds the probability decrement value.
6. The computer-implemented method (500) of claim 5, wherein: Generating the adjusted presence probability includes subtracting the probability decrement value from the adjusted probability of the previous frame based on determining that the difference exceeds the probability decrement value.
7. The computer-implemented method (500) of claim 5, wherein: Generating the adjusted presence probability includes setting the presence probability of the object in the frame to the adjusted presence probability based on determining that the difference does not exceed the probability decrement value.
8. The computer-implemented method (500) of claim 1, further comprising: turning on an extended prediction flag based on determining that the subject meets the set of close distance criteria; as well as Increase the extended forecast time limit incrementally.
9. The computer-implemented method (500) of claim 8, further comprising: determining whether a duration of the object in the video feed exceeds the extended prediction time limit; as well as The extended prediction time limit is reset based on a determination that the duration of the object in the video feed exceeds the extended prediction time limit.
10. The computer-implemented method (500) of claim 1, further comprising: Based on determining that the object satisfies the set of close range criteria, a close range extension counter is set based on a time of collision between the object and the vehicle.
11. The computer-implemented method (500) of claim 10, further comprising: The close distance extension counter is decremented based on determining that the presence probability is below the probability threshold.
12. The computer-implemented method (500) of claim 10, further comprising: A determination is further made based on determining that the close distance extension counter for the previous frame is greater than zero as to whether the object satisfies the set of close distance criteria.
13. A non-transitory computer-readable storage medium (102) comprising instructions that, when executed by a processor, perform the computer-implemented method (500) of any one of claims 1 to 12.
14. A device (100) for tracking an object, the device (100) include: The non-transitory computer readable storage medium (102) of claim 13; and At least one processor (104) coupled to the non-transitory computer-readable storage medium (302), wherein the at least one processor (104) is configured to perform the computer-implemented method (500) of any one of claims 1 to 12.
15. A system (200) for tracking an object, the system (200) comprising: The device (100) as claimed in claim 14; as well as A camera (306) is configured to capture the video feed.
Citation Information
Patent Citations
A method and a system for detecting an object in a video
CN108885684A
Automatic error repair method in video tracking
CN112767448A
Object detection in videos
US9830503B1
