Process noise adjustment in target tracker
By iteratively adjusting process noise and grouping evaluation, the process noise of the camera target tracker is optimized, solving the tracking performance problem under nonlinear motion and improving the accuracy and stability of the tracker.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- AXIS
- Filing Date
- 2025-11-24
- Publication Date
- 2026-05-29
Smart Images
Figure CN122120618A_ABST
Abstract
Description
Technical Field
[0001] This invention relates generally to the field of camera surveillance, and more specifically, to adjusting the process noise of a target tracker in a camera. Background Technology
[0002] Target tracking in camera surveillance is typically performed based on the principle of tracking detection. A detector model provides target detection predictions, which usually include bounding boxes, target categories, and classification confidence scores. These predictions are sent to a tracker, which determines which target trajectory should be updated and which target detections should update which trajectory. Once a detection is associated with a trajectory, the trajectory state is updated accordingly.
[0003] The trajectory state can be modeled using a statistical motion filter that assumes linear motion. Since real-world targets may move non-linearly, the model must handle this situation in some way. One way to do this is to set the process noise of the filter to an appropriate value. The process noise determines how much the filter is allowed to deviate from its assumption (in this case, linear motion).
[0004] However, setting process noise is an important task, especially since linear motion in the real world can transform into nonlinear motion in the camera's image plane. Therefore, there is a need for improvements in target tracking within the camera. Summary of the Invention
[0005] In view of the above and other disadvantages of the prior art, the object of the present invention is to provide an improvement on the process noise of adjusting a target tracker in a camera.
[0006] According to a first aspect of the invention, a computer-implemented method is provided for adjusting process noise in a target tracker in a camera configured to monitor a scene including a moving target, the target tracker using a motion model associated with the process noise, the method comprising: using the target tracker to determine a target trajectory of a moving target traveling along a path in the scene over a time period using a process noise level; evaluating at least one spatial parameter of the determined target trajectory relative to a predetermined criterion; repeating the steps of determining the target trajectory and evaluating at least one spatial parameter of the target trajectory using an increasing process noise in each iteration during a subsequent time period until the predetermined criterion is met or a maximum process noise limit is reached; and storing data indicating the final process noise.
[0007] This invention is based on the implementation of adjusting process noise until one of predetermined criteria associated with spatial parameters is met or maximum process noise is reached. The adjustment of process noise is performed iteratively to improve target tracking in each iteration, wherein improved tracking may involve further extending the trajectory with each increase in process noise and avoiding so-called ID switching. The method begins at relatively low process noise, thereby evaluating at least one spatial parameter of the determined target trajectory based on predetermined criteria. If the criterion is met, the method completes; otherwise, the process noise is increased, and the evaluation is repeated over subsequent time periods. Preferably, the process noise is increased until the predetermined criteria are met. However, in some cases, maximum process noise may be reached before the predetermined criteria are met.
[0008] The increase in process noise can be performed as a real-time calibration of the process noise of the target tracker in the field camera. However, it is also envisioned that for local portions of the image, such as at a trajectory or a set of trajectories, the process noise can be increased to improve tracking performance even after the initial calibration.
[0009] The increase in process noise is performed incrementally. This increase can be performed in predetermined steps where the process noise is increased by the same amount in each step. Alternatively, the increase can be performed by varying the amount of increase in process noise based on improvements in target tracking in the previous iteration.
[0010] A time period allows for the detection of multiple target trajectories, enabling the identification and evaluation of an average trajectory. A time period can be set such that a certain number of target trajectories have already been identified. The number of target trajectories can be set to ensure a sufficient statistical basis for subsequent evaluation steps. As another example, the time period can be a fixed time period set based on the maximum expected time period.
[0011] In each iteration of this method, a set of target trajectories is determined, and an evaluation is performed on the entire set or a subset of the target trajectories. In this paper, a subset of the target trajectories will be discussed and referred to as a group of target trajectories.
[0012] A camera includes an image sensor configured to capture image data that can be encoded into image data frames. A video stream is typically a series of consecutive encoded image data frames captured over time.
[0013] It should be understood that the target can refer to a physical object, a person, or an animal; in other words, it can refer to any type of target that can be captured in a video stream and tracked. A physical object can be, for example, a vehicle, which is particularly advantageous to this invention.
[0014] Object detection algorithms are known and can be selected from a range of algorithms including transformers, convolutional neural networks (CNNs), recurrent neural networks, and decision tree classifiers such as random forest classifiers, which are also effective for classification. Additionally, classifiers such as support vector machines and logistic regression classifiers are also conceivable. An object detection algorithm can be a target tracker using a convolutional neural network trained to detect and track targets based on its training. Furthermore, object / instance segmentation algorithms can also be applied to object detection.
[0015] In an embodiment, the predetermined standard may be a length standard, wherein the evaluation step may include determining the length parameter of the determined target trajectory as a spatial parameter and comparing the determined length parameter with the length standard. The length parameter may be the length of the determined trajectory. For example, the length parameter may be a statistical length parameter derived from a group of target trajectories. Using length as a parameter and standard advantageously allows for increased process noise until the length of the determined trajectory is satisfactory, i.e., a sufficiently long trajectory length that can be tracked (preferably across the entire field of view of the camera) and is useful for practical applications.
[0016] In this embodiment, the length parameter can be the average length of a defined target trajectory, and the length criterion can be that the change in the average length of the defined target trajectory between successive iterations is close to zero or below a predetermined threshold. In other words, if the average length of the target trajectory no longer increases despite the increase in process noise, then the length criterion is met, and the final process noise is reached. Using the average length as an evaluation parameter provides an easily measurable and reliable parameter for testing and discovering the final process noise suitable for the current application at hand.
[0017] In an embodiment, the method may include: calculating the change in process noise between two iterations; and determining, based on the calculated change in process noise, an expected change in the average length of a determined trajectory between the two iterations, wherein the determined expected change is used as a predetermined threshold for evaluation in the next iteration. Advantageously, this can allow the final process noise to be found in a shorter time. For example, if a large change in process noise is determined, a large increase in the average length is expected. If this expectation is realized, the noise increase may again be large in the next iteration. If the increase in the average length is less than expected, the increase in process noise can be reduced for the next iteration. In this way, the process noise increase is dynamically set based on previous iterations.
[0018] In this embodiment, during the current iteration, process noise can be increased along the path of the determined target trajectory. That is, process noise is increased along the detected target trajectory, or at least along the average target trajectory in the target tracker's motion model. In this way, the probability of detecting an improved or longer trajectory in the next iteration increases.
[0019] Preferably, process noise is increased at locations along the trajectory where the acceleration of the size or position of the target in the detected image plane is measured.
[0020] The increase in process noise along a path defined as a target trajectory can be based on the size of the target detected along that path. The increase in process noise is greater at locations or points with larger targets compared to locations or points along the path with smaller targets.
[0021] In some embodiments, process noise can be increased at the endpoints of a defined target trajectory. For example, process noise can be increased only at the endpoints of a defined target trajectory to allow the length of the target trajectory to increase in the next iteration.
[0022] In an embodiment, the step of determining the target trajectory may further include: determining which group of multiple target trajectories the determined target trajectory belongs to based on grouping criteria, wherein the steps of evaluating and increasing process noise are performed separately for different groups of target trajectories. In other words, the method can be performed individually for each of the sets of target trajectory groups. After an observation period, the trajectories are grouped or clustered based on grouping criteria. Then, for the average trajectory of each group, the process noise is increased separately. Grouping the trajectories and increasing the process noise of each group separately advantageously provides the benefit of increasing process noise where it is most needed. Furthermore, it allows the increase to be tailored to different parts of the image in different ways, such that the increase does not affect parts of the image where it is not needed and where the increase in process noise might actually impair tracking performance.
[0023] In an embodiment, the grouping criterion can be a spatial criterion, wherein the method includes: determining the start and / or end positions of the determined target trajectories; and grouping the target trajectories based on the start and / or end positions of the target trajectories and a spatial criterion indicating the average start and / or average end positions of the group. In this way, process noise is adjusted for spatially separated groups of trajectories, meaning that process noise will be increased differently for different parts of the observed scene. This results in process noise being increased much more where it is needed than where it is not needed, thereby improving the overall target tracking performance in the scene.
[0024] In this embodiment, the grouping criterion can be one or more of spatial criteria, target size, target trajectory variability, and target velocity. That is, grouping can be based on different characteristics of the trajectory, leading to more general grouping possibilities. Preferably, a single spatial criterion is used for grouping.
[0025] In an embodiment, the evaluation step may include: determining the start and / or end positions of determined target trajectories within a group of target trajectories; comparing at least one of the start and / or end positions with corresponding average start and / or average end positions as defined by a predetermined standard, wherein process noise is increased for the group of target trajectories whenever the deviation between at least one of the start and / or end positions and the corresponding average start and / or average end positions exceeds a position threshold. The average start and average end positions may be based on the average target trajectories of the corresponding group or may be based on, for example, a target trajectory that best represents the group, determined by Mahalanobis distance. Under the assumption that trajectories in each group should start and end at similar positions, such as in the case of vehicles traveling on a road, process noise is increased whenever the start and / or end positions deviate too much from the corresponding average start and / or average end positions, which is an indication of lost or poor tracking performance. This is an advantageous method for evaluating whether process noise should be increased, requiring only the evaluation of the start and / or end positions. If the new trajectory ends at a position that is either longer than or after the average end position—meaning the trajectory is longer than the average path—it indicates that the previously observed trajectories were shorter than they should have been, resulting in lost tracking and requiring an increase in process noise. Similarly, if the new trajectory begins at a position that deviates from the average start position, the complete trajectory may not have been captured, necessitating an increase in processing noise.
[0026] In an embodiment, the evaluation steps may include: determining the start position of a determined target trajectory and the average end position of the determined target trajectories in a group of target trajectories; calculating the number of start positions in the group of target trajectories that fall within a distance threshold from the average end position, the distance threshold being a predetermined standard, wherein the process noise of the group of target trajectories is increased whenever the number of start positions falling within the distance threshold from the average end position exceeds a counter threshold. It is possible that a trajectory may be lost and then reappear as a new trajectory in a new detection. Although this new detection is the result of lost and reappeared trajectories, it will have a new start position that is likely relatively close to the average end position. If this occurs too frequently, the process noise is increased so that target trajectories are not lost as reappearances of new trajectories.
[0027] In an embodiment, the evaluation step may include: determining the size variation of the target along a defined target trajectory, wherein only the group of target trajectories with size variations equal to or exceeding a size threshold is considered, increasing process noise. Groups with unchanged size may not be moving in the depth direction of the image. However, targets with large acceleration or deceleration in the image plane but actually moving at a constant speed, either large or small, may be moving towards or away from the camera; this is a case that is preferably addressed herein, and linear motion may appear nonlinear.
[0028] In one embodiment, the camera may be a static camera arranged to monitor the road on which vehicles travel as moving targets.
[0029] In this embodiment, the target tracker may use a linear motion model. For example, the linear motion model may be a statistical motion filter such as a Kalman filter. It is assumed that the motion is linear in the image plane.
[0030] This method can be executed locally on the camera. However, it is conceivable that this method can be executed on a remote server connected to the camera.
[0031] According to the second aspect, a control unit is provided for performing a method according to any of the embodiments disclosed herein.
[0032] Other embodiments of this second aspect of the invention and the effects obtained through this second aspect of the invention are largely similar to the embodiments and effects described above in the first and second aspects of the invention.
[0033] According to a third aspect of the invention, a camera is provided that includes a control unit according to the second aspect.
[0034] In one embodiment, the control unit may be connected to the output of the image sensor to receive image data directly from the image sensor.
[0035] Other embodiments of this third aspect of the invention and the effects obtained through this third aspect of the invention are largely similar to the embodiments and effects described above in the first and second aspects of the invention.
[0036] According to a fourth aspect of the invention, a computer program product including program code is provided for performing any of the methods in the embodiments discussed herein when executed by a control unit.
[0037] Other embodiments of this fourth aspect of the invention and the effects obtained through this fourth aspect of the invention are largely similar to the embodiments and effects described above in other aspects of the invention.
[0038] Further, a computer program product is provided that includes a computer-readable storage medium for storing a computer program. The computer-readable storage medium may be, for example, non-transitory and may be provided as, for example, a hard disk drive (HDD), a solid-state drive (SDD), a USB flash drive, an SD card, a CD / DVD, and / or any other storage medium capable of non-transitory storage of data.
[0039] Other features and advantages of the invention will become apparent as the claims are presented and the following description is given. Those skilled in the art will recognize that different features of the invention can be combined to create embodiments other than those described below, without departing from the scope of the invention. Attached Figure Description
[0040] The various aspects of the invention, including its particular features and advantages, will be readily understood from the following detailed description and accompanying drawings, in which:
[0041] Figure 1 The image capture apparatus according to an embodiment of the present invention is illustrated schematically.
[0042] Figure 2 The illustration shows a vehicle traveling on a road monitored by a camera according to an embodiment of the present invention;
[0043] Figure 3 This is a flowchart of the method steps according to an embodiment of the present invention;
[0044] Figure 4 This is a flowchart of the method steps according to an embodiment of the present invention;
[0045] Figure 5 The illustration shows a vehicle traveling on a road monitored by a camera according to an embodiment of the present invention;
[0046] Figure 6 This is a flowchart of the method steps according to an embodiment of the present invention;
[0047] Figure 7 It is a set of graphs illustrating the average length of different groups of target trajectories according to embodiments of the present invention;
[0048] Figure 8 This is a flowchart of the method steps according to an embodiment of the present invention;
[0049] Figure 9 This is a flowchart of the method steps according to an embodiment of the present invention; and
[0050] Figure 10 This is a block diagram of a camera according to an embodiment of the present invention. Detailed Implementation
[0051] The invention will be described more fully below with reference to the accompanying drawings, in which presently preferred embodiments of the invention are illustrated. However, the invention may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided for thoroughness and completeness and to fully convey the scope of the invention to those skilled in the art. The same reference numerals refer to the same elements throughout.
[0052] Now turn to the attached diagram and in particular to... Figure 1 The image capture device 100, such as a camera (e.g., a surveillance camera), is shown as scene 1. In scene 1, there is a group of targets 102a to 102b, which are illustrated here as vehicles 102a to 102b present in scene 1.
[0053] Camera 100 continuously monitors scene 1 by capturing a video stream of images of scene 1 and targets 102a to 102b therein. Camera 100 and control unit 106 are part of system 110, wherein control unit 106 may be a separate, independent control unit or part of camera 100. It is also envisioned that control unit 106 is set up remotely (e.g., on a server) and thus operates as a cloud-based service.
[0054] Depending on the specific application at hand, camera 100 can be mounted on a building, a utility pole, or any other suitable location. In the case of monitoring vehicles 102a to 102b, the camera is mounted as a roadside camera. That is, camera 100 is a fixed or static camera that does not move relative to scene 1 or within scene 1. Furthermore, camera 100 can be a visible light camera, an infrared (IR) sensitive camera, or a thermal (long-wavelength infrared (LWIR)) camera. Additionally, image acquisition devices employing lidar and radar functions are also conceivable. It is also conceivable that camera 100 is a combination of the aforementioned camera technologies.
[0055] The camera 100 further includes an image capture module 112, an image processing pipeline 114, an encoder 116, a data storage 130, and optional input and output interfaces 120 configured to communicate between the camera 100 and the network 122 via a radio link 124.
[0056] The image capture module 112 includes various components such as a lens and an image sensor 103 comprising multiple pixels, wherein the lens is adapted to project an image onto the pixels of the image sensor 103. Image data from the image sensor is forwarded to the image processing pipeline 114 and the encoder 116.
[0057] Image processing pipeline 114 is configured to perform a range of operations on image frames received from image sensor 103. These operations may include filtering, desacrifice, color correction, noise filtering for eliminating spatial and / or temporal noise, distortion correction for eliminating effects such as barrel distortion, global and / or local tone mapping (e.g., to achieve imaging of scenes with a wide range of brightness), transformation (e.g., rotation), flattening correction (e.g., to eliminate vignetting), and applying overlays (e.g., privacy masking, explanatory text, etc.). However, it should be noted that some of these operations, such as transformation operations (e.g., correction of barrel distortion, rotation, etc.), may be performed by one or more modules, components, or circuits disposed outside image processing pipeline 114 (e.g., one or more units between image processing pipeline 114 and encoder 116).
[0058] Following image processing pipeline 114, image frames are forwarded to encoder 116, where they are encoded and signed according to an encoding protocol, and then forwarded via network 122 using input / output interface 120 to a receiver, such as client 126 and / or server 128. It should be noted that... Figure 1 The camera 100 illustrated in the figure also includes many other components (such as processors, memory, etc.) that are common in conventional camera systems and whose purpose and operation are well known to those skilled in the art. For clarity, [the following has been omitted]. Figure 1 This component is omitted from the illustrations and descriptions.
[0059] The camera 100 may also include a data storage 130 for optionally storing data related to the capture of the video stream. Therefore, the data storage can store the captured video stream. The data storage can be a non-volatile memory such as an SD card.
[0060] There are many traditional video coding formats. Some common video coding formats that work with the various embodiments of the present invention include: JPEG, Motion JPEG (MJPEG), High Efficiency Video Coding (HEVC), also known as H.265 and MPEG-H Part 2; Advanced Video Coding (AVC), also known as H.264 and MPEG-4 Part 10; Universal Video Coding (VVC), also known as H.266, MPEG-I Part 3 and Future Video Coding (FVC); VP9, VP10 and AOMedia Video 1 (AV1), to name just a few.
[0061] Control unit 106 employs target tracker 107 for tracking moving targets 102a to 102b. Target tracker 107 uses a motion model associated with process noise for detecting and tracking the targets. The motion model is typically a statistical motion filter such as a Kalman filter, where the motion is assumed to be linear. However, this is not always the case, and the process noise of the filter is set to an appropriate value to handle deviations from linear motion.
[0062] Typically, process noise determines the amount or uncertainty of the deviation between the target's true motion and the chosen motion model. That is, if process noise is set too low, the tracker will struggle to track targets with nonlinear motion. Conversely, if process noise is set too high, a decrease in tracking performance can occur, for example, due to an increased number of identity switching events. This can happen to targets with linear motion or even static targets due to spatial noise and jitter in target detection.
[0063] Furthermore, even if (for example, in the case of a vehicle traveling on a highway) the motion of a target is actually linear, the motion may become nonlinear in the image plane.
[0064] Embodiments of this disclosure address the shortcomings of the prior art by proposing a method that iteratively increases process noise until target tracking becomes more complete. While other moving targets are also envisioned for embodiments of the invention, the invention is particularly advantageous for tracking targets such as vehicles on roads that move along relatively well-defined paths.
[0065] Figure 2 The illustration shows vehicle 102a traveling on road 200, and Figure 3 This is a flowchart of the method steps according to an embodiment of the present invention. First, let's go to... Figure 2 A static camera 100 monitors vehicle 102a and other vehicles on road 200. The camera 100 uses a target tracker that employs a motion model to track the target (here, an example of vehicles including vehicle 102a (only one is shown) traveling on road 200).
[0066] exist Figure 3 In step S100, using the target tracker 107 of camera 100 and the process noise level, the target trajectories 202a to 202c (dashed trajectories) of a moving target traveling along a path in the scene within a time period are determined. The target tracker uses a motion model to track vehicles entering and leaving the field of view of camera 100 during the time period. One trajectory 202a to 202c represents the tracking of a single vehicle. That is, the example three trajectories 202a to 202c shown here correspond to three detected and tracked vehicles.
[0067] In step S105, at least one spatial parameter of the target trajectories 202a to 202c is evaluated and determined by the control unit 106 relative to a predetermined standard. That is, the control unit 106 determines, for example, the spatial parameters of the target trajectories 202a to 202c that are related to the start point, end point, length, or motion pattern of the determined target trajectories 202a to 202c.
[0068] If a predetermined standard is met or the maximum process noise is reached, in step S110, data indicating the current process noise is stored in memory 130 as the final process noise for use by the target tracker in camera 100.
[0069] However, in the negative case, if the predetermined criteria are not met, the process noise is increased in step S111, and the method restarts at step S100.
[0070] In the next iteration, a new set of target trajectories 204a to 204b (solid line trajectories) is determined using the increased process noise from step S100. In step S105, the new set of target trajectories is evaluated against predetermined criteria. If the predetermined criteria are met or the maximum process noise is reached, then in step S110, data indicating the current process noise is stored in memory 130 as the final process noise for use by the target tracker in camera 100. If the new set of target trajectories does not meet the predetermined criteria, the process noise is increased S111, and the method restarts again at step S100.
[0071] During the subsequent time period, the steps of determining the target trajectory S100 and evaluating at least one spatial parameter of the target trajectory S105 are repeated with increasing process noise in each iteration until a predetermined criterion is met or the maximum process noise limit is reached.
[0072] In one embodiment, the predetermined standard is a length standard. In this case, the evaluation step S105 includes determining the length parameter of the target trajectory determined in S105a as a spatial parameter and comparing the determined length parameter with the length standard in S105b. The length parameters are related to how long the observed trajectory is; the longer they are, the better the tracking performance, and therefore the target tracker will not lose the trajectory or suffer ID switching along the path in the scene. Figure 2In the case shown, the vehicle represented by vehicle 102a is traveling along a road in a space that is not explicitly defined, and it can be assumed that the vehicle will travel within the boundary 206 of road 200. Therefore, the possible length of the observed trajectory can be assumed to be from one end 208a to the other end 208b or within the field of view 209 of camera 100 of road 200. In other words, length criteria and length parameters considering the length of the trajectory can be advantageously used to evaluate the performance of the target tracker using a motion model such as a linear motion model. Such a linear motion model can be a Kalman filter that assumes linear motion in the image plane of the camera.
[0073] In one example, the length parameter is the average length of the defined target trajectories 202a to 202c and 204a to 204b. In this case, the length criterion is that the change in the average length of the defined target trajectories between consecutive iterations approaches zero or falls below a predetermined threshold. In other words, the final process noise is reached when an increase in process noise no longer increases the average length of the target trajectories. As an example, in... Figure 2 In this process, the first average length is the average length of the target trajectories 202a to 202c captured and tracked during the first time period. The second average length is the average length of the target trajectories 204a to 204b captured and tracked during the second consecutive time period. If the second average length is greater than the first average length, the method repeats steps S100 to S105 with increased process noise. If the average length does not increase, or does not increase significantly with a certain margin, the final process noise is reached, and no further increase is required.
[0074] Process noise can be amplified at different locations along the trajectory. For example, in the current iteration, process noise is amplified along the path of the defined target trajectory. That is, in a motion model such as a Kalman filter, process noise is amplified at locations along the path of the target trajectory. In one example, process noise is amplified at the endpoints of the defined target trajectory. However, it is preferable to amplify process noise over the entire trajectory length. The amplification of process noise can be scaled along the trajectory or the average trajectory based on the size of the detected target along the trajectory; that is, the amount of amplification of process noise is greater at points where the detected target is relatively large than at points where the detected target is relatively small.
[0075] Preferably, process noise is increased at locations where the target's position or size is accelerated within the image plane of camera 100. These locations are where linear motion models are more likely to lose their trajectory.
[0076] Go to Figure 4The flowchart illustrates this. In some embodiments, the change in process noise between two iterations (the first iteration and the second iteration) can be used to estimate how much the change in average length should be in the second iteration relative to the first iteration. That is, a first process noise is used in the first iteration, and that process noise is increased to a second process noise. The difference between the first process noise and the second process noise is then used to estimate how much the change in the average length of the trajectory should be after the increase in process noise, thus using the second process noise.
[0077] Step 106 includes calculating the change in process noise between two iterations, and step 108 includes determining, based on the calculated change in process noise, the expected change in the average length of the determined target trajectory between the two iterations. The determined expected change is used as a predetermined threshold for evaluating the change in average length in step S105b of the next iteration. The expected increase or change in average length can be determined by evaluating the historical increase in process noise and what change in the average length of the determined trajectory it causes.
[0078] Figure 5 The illustration shows vehicles 102a to 102c traveling on road 200. In some embodiments, detected target trajectories are clustered into groups. Figure 5 This is illustrated by three groups, 302, 304, and 306. Grouping is implemented by individually increasing the process noise for each group of trajectories, and thus tailoring the process noise for different parts of the image without affecting other parts of the image that do not need it.
[0079] Figure 6 This is a flowchart of the method steps according to an embodiment of the present invention. In addition to the steps described above, the method includes step S102, which step determines which group of multiple target trajectories the determined target trajectory belongs to based on grouping criteria. For different groups 302, 304, and 306 of the target trajectory, step S105 is performed to evaluate and subsequently increase the process noise. In other words, after the time period of detecting target trajectory 301, the grouping of target trajectory 301 is performed using a clustering algorithm. The method evaluates and adjusts the process noise for each in the group separately. Separate final process noise data is stored for each in the group. That is, groups can have different final process noise.
[0080] Trajectory clustering can be performed in several ways, such as using a thresholding method, where positions or points along one trajectory are compared with positions or points along another trajectory, and if a certain number of points are less than a threshold distance from each other, they are determined to belong to the same group. Another approach is to compare the start and end positions of the trajectories, which should not deviate from each other by more than a preset distance.
[0081] Grouping criteria can be spatial criteria, target size, target trajectory variability, and target velocity.
[0082] In embodiments where the grouping criterion is a spatial criterion, the method may include step S101 of determining the start positions 310a, 312a, 314a and / or end positions 310b, 312b, 314b of the determined target trajectories 301. Step S102 of grouping the target trajectories 301 is based on their start and / or end positions and a spatial criterion indicating the average start and / or average end position of the group.
[0083] In other words, in this embodiment, the clustering algorithm groups the trajectories based on the determined start positions 310a, 312a, and 314a of the target trajectory. In other embodiments, the clustering algorithm may group the trajectories based on the determined end positions 310b, 312b, and 314b of the target trajectory. In yet another embodiment, the clustering algorithm may group the trajectories based on the determined start and end positions 310a, 312a, 314a and 310b, 312b, and 314b of the target trajectory.
[0084] Figure 7 The diagram illustrates the average lengths of groups 302, 304, and 306 of the target trajectory as a function of the number of iterations, or equivalently as a function of time. The graphs are for illustrative purposes only and to make the invention clear. In practice, the number of iterations may differ from the examples shown herein.
[0085] As described in step S100, the target trajectory 301 is determined over a duration. In step S101, the determined target trajectory 301 is grouped according to a grouping criterion, such as the determined start positions 310a, 312a, 314a and / or end positions 310b, 312b, 314b. For the current iteration (1), the average length of the trajectory in each group 302, 304, 306 is determined in step S105a. In this paper, process noise can be increased from an initial low level.
[0086] The same process is repeated for a new set of grouped and evaluated target trajectories, but with increased process noise, until a predetermined criterion is met, such as the average length no longer increasing. For group 302, the average length is m1 and no longer increases in iteration 4; for group 304, the average length is m2 and no longer increases in iteration 3; and for group 306, the average length is m3 and no longer increases in iteration 5. The final process noise reached when the average length no longer increases for the corresponding group is output and stored as the final process noise.
[0087] Go to Figure 8The flowchart is shown in the figure. In some embodiments, the evaluation in step S105 includes determining the start position 310a, 312a, 314a and / or end position 310b, 312b, 314b of the target trajectory 301 determined in groups 302, 304, 306 of step S105c.
[0088] In step S105d, at least one of the start position and / or end position is compared with the corresponding average start position and / or average end position. That is, for each group 302, 304, 306, when a new set of target trajectories has been determined, the start position and / or end position are compared with the average start position and / or average end position of groups 302, 304, 306. In one example, before the comparison, the start position and / or end position of the new trajectory contribute to the average start position and / or average end position of the corresponding groups 302, 304, 306. In another example, the average start position and / or average end position of the new set of target trajectories is compared with the average start position and / or average end position of the corresponding groups 302, 304, 306. In this case, if the deviation between at least one of the new start position and / or the new end position, or their average, and the corresponding average start position and / or average end position exceeds a position threshold, process noise is increased for groups 302, 304, 306 of the target trajectories. In other words, it is desirable that the start and / or end positions of the new trajectory are relatively close to the average start and / or average end positions. If this is not the case, process noise is increased, especially at start positions 310a, 312a, 314a and / or end positions 310b, 312b, 314b.
[0089] In some embodiments, the evaluation steps may include step S105c of determining the start position of a determined target trajectory and the average end position of the determined target trajectories in a group of target trajectories. Following this step, the control unit 106 may calculate the number of start positions 310a falling within a distance threshold from the average end position in the group 302 of target trajectories. The distance threshold serves as a predetermined standard. In this case, whenever the number of start positions falling within the distance threshold from the average end position exceeds a counter threshold, the process noise of the group of target trajectories is increased. New trajectory detection will have a new start position. If the new start position is relatively close to the average end position, it may be the result of a lost and reappearing trajectory. If this occurs too frequently, the process noise is increased at the position of the new start position and at positions prior to the new start position, thereby improving target tracking performance.
[0090] In some embodiments, here Figure 9As listed, the evaluation step S105 further includes a step S105e for determining the size change of the target along the determined target trajectory. Here, the predetermined criterion is that the size change is equal to or exceeds a size threshold; in other words, only the group of target trajectories with size changes equal to or exceeding the size threshold is subject to increased process noise. Groups with unchanged size may not be moving in the depth direction of the image. However, a target with large acceleration or deceleration in the image plane but actually moving at a constant speed, large or small, may be moving towards or away from the camera. In this case, linear motion may appear nonlinear, and for this situation, increased process noise may be appropriate.
[0091] This method can be executed by the control unit 106.
[0092] Preferably, the described method can be executed locally in camera 100. However, it is also envisioned that the method be executed on a remote server connected to camera 100.
[0093] Figure 10 This is a block diagram of a camera 100 including a control unit 106, which is configured to adjust process noise 132 in a target tracker 107 configured to monitor a scene including a moving target. The target tracker 107 is stored in a memory 130 accessible to the control unit 106 and uses a motion model 134 associated with the process noise 132. The control unit 106 is configured to use the target tracker 107, using the level of the process noise 132, to determine a target trajectory 136 of a moving target traveling along a path in the scene over a time period. The control unit 106 is further configured to evaluate at least one spatial parameter 138 of the determined target trajectory relative to a predetermined criterion 140.
[0094] During subsequent time periods, the control unit 106 repeatedly determines the target trajectory and evaluates at least one spatial parameter 138 of the target trajectory using the increasing process noise 132 of each iteration until a predetermined criterion 140 is met or the maximum process noise limit 142 is reached. Once one of the conditions is met, the control unit 106 stores data 144 indicating the final process noise in the memory 130.
[0095] Further, a computer program including instructions is provided, which, when executed by a computer, cause the computer to perform any of the methods in the embodiments disclosed herein. The computer program may be stored or distributed on a data carrier. As used herein, "data carrier" can be a temporary data carrier such as modulated electromagnetic waves or light waves, or a non-temporary data carrier. Non-temporary data carriers include volatile and non-volatile memories such as permanent and non-permanent storage media of magnetic, optical, or solid-state types. Still within the scope of "data carrier," such memory may be fixedly mounted or portable.
[0096] The computer program includes code for: using a target tracker, determining a target trajectory of a moving target traveling along a path in a scene over a time period using a process noise level; evaluating at least one spatial parameter of the determined target trajectory relative to a predetermined criterion; repeating the steps of determining the target trajectory and evaluating at least one spatial parameter of the target trajectory using an increasing process noise level in each iteration during subsequent time periods until the predetermined criterion is met or a maximum process noise limit is reached; and storing data indicating the final process noise.
[0097] The control unit includes a microprocessor, a microcontroller, a programmable digital signal processor, or other programmable device. The control unit may also, or alternatively, include an application-specific integrated circuit (ASIC), a programmable gate array (FPGA) or programmable array logic, a programmable logic device, or a digital signal processor. Where the control unit includes a programmable device such as a microprocessor, microcontroller, or programmable digital signal processor as described above, the processor may further include computer-executable code that controls the operation of the programmable device.
[0098] The control functions of this disclosure can be implemented using existing computer processors or dedicated computer processors of suitable systems introduced for this or another purpose, or by hardwired systems. Embodiments within the scope of this disclosure include program products comprising machine-readable media for carrying or having machine-executable instructions or data structures stored thereon. Such machine-readable media can be any available medium accessible by a general-purpose or special-purpose computer or other machine with a processor. For example, such machine-readable media may include RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of machine-executable instructions or data structures and accessible by a general-purpose or special-purpose computer or other machine with a processor. When information is transmitted or provided to a machine via a network or another communication connection (hardwired, wireless, or a combination of hardwired and wireless), the machine appropriately considers that connection as a machine-readable medium. Therefore, any such connection is appropriately referred to as a machine-readable medium. Combinations of the above are also included within the scope of machine-readable media. Machine-executable instructions include, for example, instructions and data that cause a general-purpose computer, special-purpose computer, or special-purpose processor to perform a particular function or set of functions.
[0099] Although the accompanying drawings may show a sequence, the order of the steps may differ from what is depicted. Furthermore, two or more steps may be performed concurrently or partially concurrently. This variation will depend on the chosen software and hardware system and the designer's choices. All these variations are within the scope of this disclosure. Similarly, standard programming techniques with rule-based logic and other logic can be used to accomplish the various connection steps, processing steps, comparison steps, and decision steps. Additionally, although the invention has been described with reference to specific exemplary embodiments, many different changes and modifications will become apparent to those skilled in the art.
[0100] Furthermore, by studying the accompanying drawings, this disclosure, and the appended claims, those skilled in the art can understand and implement variations of the disclosed embodiments when practicing the claimed invention. Additionally, in the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plurality.
Claims
1. A computer-implemented method for adjusting process noise in a target tracker in a camera (100), the camera (100) being configured to monitor a scene (200) including moving targets (102a to 102c), the target tracker (107) using a motion model (134) associated with process noise (132), the method comprising: Using the target tracker, the target trajectory (136, 202a to 202c, 204a to 204b, 301) of a moving target traveling along a path in the scene within a time period is determined (S100) using the process noise level. Evaluate (S105) at least one spatial parameter of the determined target trajectory relative to the predetermined standard (140); During the subsequent time period, the steps of determining (S100) the target trajectory and evaluating (S105) at least one spatial parameter of the target trajectory are repeated with increasing process noise in each iteration until the predetermined criterion is met or the maximum process noise limit is reached. as well as Store (S110) data (144) indicating the final process noise.
2. The method according to claim 1, wherein the predetermined standard is a length standard, wherein, The evaluation steps include: Determine (S105a) the length parameter of the determined target trajectory as the spatial parameter, and The determined length parameter is compared with the length standard (S105b).
3. The method according to claim 2, wherein, The length parameter is the average length (M) of the determined target trajectory, and the length criterion is that the change in the average length of the determined target trajectory between consecutive iterations is close to zero or below a predetermined threshold.
4. The method according to claim 3, comprising: Calculate the change in process noise between the two iterations (S106); Based on the change in the process noise of the calculation, (S108) the expected change in the average length of the determined trajectory between the two iterations is determined, wherein the determined expected change is used as the predetermined threshold for the evaluation in the next iteration.
5. The method according to claim 1, wherein, In the current iteration, the process noise is amplified along the path of the determined target trajectory.
6. The method according to claim 5, wherein, The process noise is amplified at the endpoints of the defined target trajectory.
7. The method according to claim 1, wherein the step of determining the target trajectory further comprises: Based on the grouping criteria, (S102) it is determined which group of multiple target trajectories the determined target trajectory belongs to, wherein the evaluation and the step of increasing the process noise are performed on different groups of target trajectories respectively.
8. The method according to claim 7, wherein the grouping criterion is a spatial criterion, wherein, The method includes: Determine (S101) the start position (210a, 310a, 312a, 314a) and / or end position (210b, 310b, 312b, 314b) of the determined target trajectory. The target trajectories are grouped based on the start and / or end positions of the target trajectories and the spatial criteria indicating the average start and / or average end positions of the group (S102).
9. The method according to claim 7, wherein, The grouping criteria are one or more of the following: spatial criteria, target size, target trajectory variability, and target velocity.
10. The method according to claim 7, wherein, The evaluation steps include: Determine (S105c) the start position and / or end position of the determined target trajectory in the group of target trajectories; At least one of the start position and / or the end position is compared with the corresponding average start position and / or average end position for the predetermined standard (S105d). Wherein, if the deviation between at least one of the start position and / or the end position and the corresponding average start position and / or average end position exceeds a position threshold, the process noise of the target trajectory group is increased.
11. The method according to claim 1, wherein, The evaluation steps include: Determine (S105e) the size change of the target along the determined target trajectory. Specifically, only the group of target trajectories whose size changes are equal to or exceed the size threshold increases the process noise.
12. The method of claim 1, wherein the camera is a static camera arranged to monitor a road on which a vehicle travels as a moving target, and wherein, The target tracker uses a linear motion model.
13. A control unit (106) for performing the method according to any one of claims 1 to 12.
14. A camera (100) comprising a control unit (106) according to claim 13.
15. A non-transitory computer-readable storage medium comprising program code for performing the method according to any one of claims 1 to 12 when executed by a control unit.