Target tracking device and target tracking method

Abstract

A target tracking device is provided with: a state prediction unit (1) that predicts, on the basis of respective state amounts of a plurality of tracking targets at a previous observation time, the state of each of the tracking targets at a current observation time to output a predicted value of the state of each of the tracking targets; a shielding occurrence probability calculation unit (3) that calculates, on the basis of the predicted value output from the state prediction unit (1), a shielding occurrence probability such that any two or more of the tracking targets from among the plurality of tracking targets overlap with each other; and a shielding target setting unit (4) that sets a shielding target including two or more tracking targets if the shielding occurrence probability calculated by the shielding occurrence probability calculation unit (3) is greater than or equal to a threshold value. Additionally, the target tracking device is provided with a state estimation unit (5) that calculates, on the basis of a predicted value output from the state prediction unit (1), the predicted value of the state of the shielding target set by the shielding target setting unit (4) at the current observation time and that estimates, by using the predicted value of the state of the shielding target at the current observation time, the state amount of each tracking target included in the shielding target at the current observation time.

Description

Target tracking device and target tracking method

[0001] This disclosure relates to a target tracking device and a target tracking method.

[0002] There are target tracking devices that track each of multiple tracking targets. As an example of such a target tracking device, Patent Document 1 discloses a target tracking device in which, when parts of multiple tracking targets overlap each other, a portion of each tracking target that is not hidden by other tracking targets is detected, and the detected portion is designated as the target of tracking.

[0003] International Publication No. 2021-171498

[0004] The target tracking device disclosed in Patent Document 1 has a problem in that, when multiple tracking targets overlap each other, if it cannot detect a portion of each tracking target that is not hidden by other tracking targets, it cannot track each target.

[0005] This disclosure was made to solve the above-mentioned problems, and aims to provide a target tracking device that can track each target regardless of whether or not there are any parts of each target that are not obscured by other targets, even when multiple tracking targets overlap each other.

[0006] The target tracking device according to this disclosure includes: a state prediction unit that predicts the state of each tracking target at the current observation time based on the state quantities of each tracking target at the previous observation time and outputs predicted values ​​for the state of each tracking target; an occlusion occurrence probability calculation unit that calculates the probability that any two or more tracking targets among the multiple tracking targets overlap each other based on the predicted values ​​output from the state prediction unit; and an occlusion target setting unit that sets an occlusion target containing two or more tracking targets if the occlusion occurrence probability calculated by the occlusion occurrence probability calculation unit is equal to or greater than a threshold. Furthermore, the target tracking device includes a state estimation unit that calculates predicted values ​​for the state of the occlusion target set by the occlusion target setting unit at the current observation time based on the predicted values ​​output from the state prediction unit, and estimates the state quantities of each tracking target included in the occlusion target at the current observation time using the predicted values ​​for the state of the occlusion target at the current observation time.

[0007] According to this disclosure, when multiple tracking targets overlap each other, each tracking target can be tracked regardless of whether or not there are any parts of each target that are not obscured by other tracking targets.

[0008] This is a configuration diagram showing a target tracking device according to Embodiment 1. This is a hardware configuration diagram showing the hardware of the target tracking device according to Embodiment 1. This is a hardware configuration diagram of a computer when the target tracking device is implemented by software or firmware, etc. This is a flowchart showing the target tracking method, which is the processing procedure of the target tracking device. This is an explanatory diagram showing an example of multiple tracking targets.

[0009] To provide a more detailed explanation of this disclosure, the forms for implementing this disclosure will be described below with reference to the attached drawings.

[0010] Embodiment 1. Figure 1 is a configuration diagram showing a target tracking device according to Embodiment 1. Figure 2 is a hardware configuration diagram showing the hardware of the target tracking device according to Embodiment 1. The target tracking device shown in Figure 1 comprises a state prediction unit 1, a correlation target identification unit 2, an occlusion occurrence probability calculation unit 3, an occlusion target setting unit 4, a state estimation unit 5, and an occlusion target deletion unit 6.

[0011] The state prediction unit 1 is implemented, for example, by the state prediction circuit 11 shown in Figure 2. The state prediction unit 1 obtains the state quantities of each of the multiple tracking targets at the previous observation time from the state estimation unit 5. Based on the state quantities of each tracking target at the previous observation time, the state prediction unit 1 predicts the state of each tracking target at the current observation time. The state prediction unit 1 outputs the predicted value of the state of each tracking target at the current observation time to the correlation target identification unit 2, the occlusion occurrence probability calculation unit 3, and the state estimation unit 5, respectively.

[0012] The correlation target identification unit 2 is implemented, for example, by the correlation target identification circuit 12 shown in Figure 2. The correlation target identification unit 2 obtains predicted values ​​of the state of each tracking target at the current observation time from the state prediction unit 1. The correlation target identification unit 2 obtains observed values ​​of the state of each tracking target at the current observation time from the radar device shown in Figure 5. Based on the predicted values ​​of the state of each tracking target at the current observation time and the observed values ​​of the state of each tracking target at the current observation time, the correlation target identification unit 2 identifies tracking targets among the multiple tracking targets for which there is no correlation between the predicted values ​​and the observed values. The correlation target identification unit 2 outputs information indicating tracking targets for which there is no correlation between the predicted values ​​and the observed values ​​to the occlusion occurrence probability calculation unit 3 and the state estimation unit 5, respectively.

[0013] The occlusion probability calculation unit 3 is implemented, for example, by the occlusion probability calculation circuit 13 shown in Figure 2. The occlusion probability calculation unit 3 obtains predicted values ​​of the state of each tracking target at the current observation time from the state prediction unit 1. The occlusion probability calculation unit 3 obtains information from the correlation target identification unit 2 indicating tracking targets for which there is no correlation between the predicted value and the observed value. Based on the predicted values ​​of the state of each tracking target at the current observation time, the occlusion probability calculation unit 3 calculates the occlusion probability, which is the probability that two or more tracking targets among the multiple tracking targets overlap each other. Specifically, if there are two or more tracking targets identified by the correlation target identification unit 2, the occlusion probability calculation unit 3 calculates the occlusion probability, which is the probability that two or more tracking targets overlap each other, based on the predicted values ​​of two or more tracking targets for which there is no correlation between the predicted value and the observed value. The occlusion probability calculation unit 3 outputs the occlusion probability to the occlusion target setting unit 4.

[0014] The occlusion target setting unit 4 is implemented, for example, by the occlusion target setting circuit 14 shown in Figure 2. The occlusion target setting unit 4 obtains the occlusion occurrence probability from the occlusion occurrence probability calculation unit 3. If the occlusion occurrence probability calculated by the occlusion occurrence probability calculation unit 3 is greater than or equal to a threshold, the occlusion target setting unit 4 sets an occlusion target that includes two or more tracking targets with no correlation between the predicted value and the observed value. The threshold may be stored, for example, in the internal memory of the occlusion target setting unit 4, or it may be provided from outside the target tracking device shown in Figure 1.

[0015] The state estimation unit 5 is implemented, for example, by the state estimation circuit 15 shown in Figure 2. The state estimation unit 5 includes a correlation determination unit 5a and a state estimation processing unit 5b. The state estimation unit 5 obtains predicted values ​​of the state of each tracking target at the current observation time from the state prediction unit 1. Based on the predicted values ​​of the state of each tracking target at the current observation time, the state estimation unit 5 calculates predicted values ​​of the state of the occlusion target set by the occlusion target setting unit 4 at the current observation time. Using the predicted values ​​of the state of the occlusion target at the current observation time, the state estimation unit 5 estimates the state quantities of each tracking target included in the occlusion target at the current observation time. The state estimation unit 5 outputs the state quantities of each tracking target included in the occlusion target at the current observation time to the state prediction unit 1 as state quantities of each tracking target included in the occlusion target at the previous observation time. The state quantities of each tracking target at the current observation time estimated by the state estimation unit 5 are displayed, for example, on a display device (not shown).

[0016] The correlation determination unit 5a obtains predicted values ​​of the current observation time state of each tracking target from the state prediction unit 1. The correlation determination unit 5a obtains observed values ​​of the current observation time state of each tracking target from, for example, the radar device shown in Figure 5. Based on the predicted values ​​of the current observation time state of each tracking target, the correlation determination unit 5a predicts the current observation time state of the occlusion target set by the occlusion target setting unit 4. Based on the predicted values ​​of the current observation time state of the occlusion target and the observed values ​​of the current observation time state of each tracking target included in the occlusion target, the correlation determination unit 5a determines whether or not there is a correlation between each tracking target included in the occlusion target and the occlusion target. The correlation determination unit 5a outputs the predicted values ​​of the current observation time state of the occlusion target and the determination result indicating whether or not there is a correlation to the state estimation processing unit 5b. The correlation determination unit 5a also outputs the determination result to the occlusion target deletion unit 6.

[0017] The state estimation processing unit 5b obtains from the correlation determination unit 5a the predicted value of the state of the occluded target at the current observation time and a determination result indicating whether or not there is a correlation. Based on the predicted value of the state of the occluded target at the current observation time, the state estimation processing unit 5b estimates the state quantity at the current observation time of the tracking targets among the multiple tracking targets that the correlation determination unit 5a has determined to be correlated. The state estimation processing unit 5b outputs the state quantity at the current observation time of each tracking target to the state prediction unit 1 as the state quantity at the previous observation time of each tracking target.

[0018] The shielding target deletion unit 6 is implemented, for example, by the shielding target deletion circuit 16 shown in Figure 2. The shielding target deletion unit 6 obtains a determination result from the correlation determination unit 5a indicating whether or not there is a correlation. If there are no tracking targets among the two or more tracking targets included in the shielding target that the correlation determination unit 5a has determined to be correlated, the shielding target deletion unit 6 deletes the shielding target set by the shielding target setting unit 4.

[0019] In Figure 1, it is assumed that the components of the target tracking device—the state prediction unit 1, the correlated target identification unit 2, the occlusion occurrence probability calculation unit 3, the occlusion target setting unit 4, the state estimation unit 5, and the occlusion target deletion unit 6—are each implemented by dedicated hardware as shown in Figure 2. Specifically, the target tracking device is assumed to be implemented by a state prediction circuit 11, a correlated target identification circuit 12, an occlusion occurrence probability calculation circuit 13, an occlusion target setting circuit 14, a state estimation circuit 15, and an occlusion target deletion circuit 16. Each of the state prediction circuit 11, correlation target identification circuit 12, shielding occurrence probability calculation circuit 13, shielding target setting circuit 14, state estimation circuit 15, and shielding target deletion circuit 16 can be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0020] The components of a target tracking device are not limited to those implemented by dedicated hardware; the target tracking device may also be implemented by software, firmware, or a combination of software and firmware. The software or firmware is stored as a program in the computer's memory. A computer refers to the hardware that executes the program, and includes, for example, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor).

[0021] FIG. 3 is a hardware configuration diagram of a computer when the target tracking device is implemented by software, firmware, or the like. When the target tracking device is implemented by software, firmware, or the like, a program for causing a computer to execute respective processing procedures in the state prediction unit 1, the correlation target identification unit 2, the occlusion occurrence probability calculation unit 3, the occlusion target setting unit 4, the state estimation unit 5, and the occlusion target deletion unit 6 is stored in the memory 21. Then, the processor 22 of the computer executes the program stored in the memory 21.

[0022] Further, FIG. 2 shows an example in which each component of the target tracking device is realized by dedicated hardware, and FIG. 3 shows an example in which the target tracking device is realized by software, firmware, or the like. However, this is merely an example, and some components in the target tracking device may be realized by dedicated hardware and the remaining components may be realized by software, firmware, or the like.

[0023] Next, the operation of the target tracking device shown in FIG. 1 will be described. The target tracking device shown in FIG. 1 performs tracking of a tracking target using a state estimation algorithm such as a Kalman filter, an extended Kalman filter, an unscented Kalman filter, or a particle filter. FIG. 4 is a flowchart showing a target tracking method which is a processing procedure of the target tracking device.

[0024] FIG. 5 is an explanatory diagram showing an example of a plurality of tracking targets. In FIG. 5, Tg １ to Tg ５ are tracking targets. In the example of FIG. 5, the tracking target Tg ３ and the tracking target Tg ４ are overlapped with each other, and a part of the tracking target Tg ４ is hidden by the tracking target Tg ３ . In FIG. 5, a part of the tracking target Tg ４ is shown to be hidden by the tracking target Tg ３ . However, this is merely an example, and all of the tracking target Tg ４ may be hidden by the tracking target Tg ３It may also be hidden by [something]. In the example in Figure 5, the tracking target Tg ３ and tracking target Tg ４ The two targets are overlapping each other. However, this is just one example, and there may be three or more tracking targets Tg that are overlapping each other. In the example in Figure 5, there are five tracking targets Tg. However, there may be two or more tracking targets Tg, and the number of tracking targets Tg is not limited to five.

[0025] The state prediction unit 1 receives the tracking target Tg from the state estimation unit 5. ｎ The state variable x-hat at the previous observation time t for (n=1, ..., N) ｎ，ｔ｜ｔ Obtain the value. N is an integer greater than or equal to 2. Due to the requirements of electronic filing, the symbol "^" cannot be placed above the character x in the text of the specification, therefore the state variable x is represented as a hat. ｎ，ｔ｜ｔ It is written as follows: State quantity x hat ｎ，ｔ｜ｔ For example, tracking target Tg ｎ Position, tracking target Tg ｎ The speed, or tracking target Tg ｎ These are the image features. The state prediction unit 1 tracks the target Tg as shown in equation (1) below. ｎ The state variable x-hat at the previous observation time t. ｎ，ｔ｜ｔ Based on this, tracking target Tg ｎ The state of the current observation time t+1 is predicted (step ST1 in Figure 4). The state prediction unit 1 predicts the state of the tracking target Tg ｎ Predicted value x hat for the current observed state at time t+1 ｎ，ｔ＋１｜ｔ This is output to the correlation target identification unit 2, the shielding occurrence probability calculation unit 3, and the state estimation unit 5, respectively. In the text of the specification, due to the requirements of electronic filing, the symbol "^" cannot be placed above the character x, so the predicted value x hat is used. ｎ，ｔ＋１｜ｔ It is written as follows.

[0026]

[0027] The state prediction unit 1 tracks the target Tg as shown in equation (2) below. ｎ The prediction error covariance matrix P at the current observation time t+1. ｎ，ｔ＋１｜ｔ Calculate Q in equation (2). ｔThis is the drive noise covariance matrix, which is a Jacobian matrix Φ expressed as shown in equation (3) below. The state prediction unit 1 tracks the target Tg ｎ The prediction error covariance matrix P at the current observation time t+1. ｎ，ｔ＋１｜ｔ This is output to the correlation target identification unit 2.

[0028]

[0029] The correlation target identification unit 2 receives the tracking target Tg from the state prediction unit 1. ｎ The predicted value x hat of the current observed state at time t+1 for (n=1, ..., N) ｎ，ｔ＋１｜ｔ And, tracking target Tg ｎ The prediction error covariance matrix P at the current observation time t+1. ｎ，ｔ＋１｜ｔ The correlation target identification unit 2 obtains, for example, the tracking target Tg from the radar device shown in Figure 5. ｎ Observed value y of the current state at observation time t+1 ｎ，ｔ＋１ Obtain the observed value y of the state. ｎ，ｔ＋１ For example, tracking target Tg ｎ Position, tracking target Tg ｎ The speed, or tracking target Tg ｎ These are the image features. The correlation target identification unit 2 calculates the predicted value x hat as shown in equation (4) below. ｎ，ｔ＋１｜ｔ Using this, track target Tg ｎ Observed state vector y hat ｎ，ｔ＋１｜ｔ The correlation target identification unit 2 calculates the prediction error covariance matrix P as shown in equation (5) below. ｎ，ｔ＋１｜ｔ The observation error covariance matrix S is calculated using this method.

[0030]

[0031] The correlation target identification unit 2 identifies the tracking target Tg ｎ Predicted value x hat for the current observed state at time t+1 ｎ，ｔ＋１｜ｔ And, tracking target Tg ｎ Observed value y of the current state at observation time t+1 ｎ，ｔ＋１ Based on this, N tracking targets Tg １ ~Tg Ｎ Within that, predicted value x hat ｎ，ｔ＋１｜ｔ and the observable value y ｎ，ｔ＋１The tracking target Tg that has no correlation with the target is identified (step ST2 in Figure 4). Specifically, the correlation target identification unit 2 identifies the tracking target Tg ｎ Observed state vector y hat ｎ，ｔ＋１｜ｔ And, tracking target Tg ｎ Observed value y of the current state at observation time t+1 ｎ，ｔ＋１ Based on this, N tracking targets Tg １ ~Tg Ｎ Among these, for tracking targets Tg where the following equation (6) does not hold true, the predicted value x hat ｎ，ｔ＋１｜ｔ and the observable value y ｎ，ｔ＋１ The tracking target Tg is identified as having no correlation with the predicted value. The correlation target identification unit 2 outputs information indicating the tracking target Tg, which has no correlation with the predicted value and the observed value, to the occlusion occurrence probability calculation unit 3 and the state estimation unit 5, respectively.

[0032] In equation (6), S －１ is the inverse of the observation error covariance matrix S. d is a pre-set parameter.

[0033] In the target tracking device shown in Figure 1, the correlation target identification unit 2 calculates the predicted value x hat based on equation (6). ｎ，ｔ＋１｜ｔ and the observable value y ｎ，ｔ＋１ It identifies tracking targets Tg that have no correlation with the target. However, this is just one example, and the correlation target identification unit 2 can, for example, execute a correlation algorithm such as GNN (Global Nearest Neighbor) or MHT (Multiple Hyperthesis Tracking) to predict the x-hat value. ｎ，ｔ＋１｜ｔ and the observable value y ｎ，ｔ＋１ You may also choose to identify a tracking target Tg that has no correlation with the other parameters.

[0034] The state estimation unit 5 receives the tracking target Tg from the state prediction unit 1. ｎ The predicted value x hat of the current observed state at time t+1 for (n=1, ..., N) ｎ，ｔ＋１｜ｔ The state estimation unit 5 obtains, for example, the tracking target Tg from the radar device shown in Figure 5. ｎ Observed value y of the current state at observation time t+1 ｎ，ｔ＋１ The state estimation unit 5 obtains the predicted value x hat from the correlation target identification unit 2. ｎ，ｔ＋１｜ｔ and the observable value yｎ，ｔ＋１ Obtain information indicating a tracking target Tg that has no correlation with [it].

[0035] The state estimation processing unit 5b of the state estimation unit 5 has a predicted value x hat ｎ，ｔ＋１｜ｔ and an observed value y ｎ，ｔ＋１ Based on the information indicating the tracking target Tg that has no correlation with [them], for N tracking targets Tg １ to Tg Ｎ among them, count the number M of tracking targets Tg for which there is no correlation between the predicted value x hat ｎ，ｔ＋１｜ｔ and the observed value y ｎ，ｔ＋１ The state estimation processing unit 5b counts the number M of tracking targets Tg for which there is no correlation between the predicted value x hat ｎ，ｔ＋１｜ｔ and the observed value y ｎ，ｔ＋１ If the number M of tracking targets Tg for which there is no correlation between the predicted value x hat １ and the observed value y Ｎ is less than 2 (in the case of YES in step ST3 of FIG. 4), for the N tracking targets Tg ｎ，ｔ＋１｜ｔ to Tg ｎ，ｔ＋１ among them, estimate the state quantity x hat ｎ at the current observation time t + 1 of the tracking target Tg ｎ，ｔ＋１｜ｔ＋１ that has a correlation between the predicted value x hat ｎ and the observed value y ｎ，ｔ＋１｜ｔ (step ST4 of FIG. 4). That is, the state estimation processing unit 5b, as shown in the following formulas (7) to (9), uses the predicted value x hat ｎ of the state at the current observation time t + 1 of the tracking target Tg ｎ，ｔ＋１ (n = 1,..., N) and the observed value y ｎ of the state at the current observation time t + 1 of the tracking target Tg ｎ，ｔ＋１｜ｔ＋１ to estimate the state quantity x hat

[0036]

[0037] The state estimation processing unit 5b １ to Tg Ｎ among the N tracking targets Tg, for the state quantity x hat ｎ，ｔ＋１｜ｔ of the tracking target Tg ｎ，ｔ＋１ for which there is no correlation between the predicted value x hat ｎ and the observed value y ｎ，ｔ＋１｜ｔ＋１ estimate as follows according to the following formula (10). The state estimation processing unit 5b has the predicted value x hat ｎ，ｔ＋１｜ｔ and the observed value y ｎ，ｔ＋１Tracking target Tg with no correlation between it and the target Tg ｎ The prediction error covariance matrix P at the current observation time t+1. ｎ，ｔ＋１｜ｔ＋１ This is estimated as shown in equation (11) below. The state estimation processing unit 5b estimates the tracking target Tg ｎ As a state variable at the time of the previous observation, the tracking target Tg ｎ The state variable x-hat at the current observation time t+1 ｎ，ｔ＋１｜ｔ＋１ This is output to the state prediction unit 1.

[0038]

[0039] The occlusion probability calculation unit 3 calculates the tracking target Tg from the state prediction unit 1. ｎ The predicted value x hat of the current observed state at time t+1 for (n=1, ..., N) ｎ，ｔ＋１｜ｔ The shielding probability calculation unit 3 obtains the predicted value x hat from the correlation target identification unit 2. ｎ，ｔ＋１｜ｔ and the observable value y ｎ，ｔ＋１ Information is obtained indicating the tracking target Tg, which has no correlation with the occlusion probability calculation unit 3. ｎ，ｔ＋１｜ｔ and the observable value y ｎ，ｔ＋１ Based on information indicating that there is no correlation between the tracking target Tg and the N tracking target Tg １ ~Tg Ｎ Within that, predicted value x hat ｎ，ｔ＋１｜ｔ and the observable value y ｎ，ｔ＋１ Count the number M of tracking targets Tg that have no correlation with the target.

[0040] The occlusion probability calculation unit 3 calculates the occlusion probability p, which is the probability that two or more tracking targets Tg overlap each other, based on the predicted values ​​of two or more uncorrelated tracking targets Tg (step ST5 in Figure 4), if there are two or more uncorrelated tracking targets Tg (step ST5 in Figure 4). In the example in Figure 5, tracking targets Tg ３ and tracking target Tg ４ Because they overlap each other, tracking target Tg ３ However, the predicted value x hat ３，ｔ＋１｜ｔ and the observable value y ３，ｔ＋１ It was identified as a tracking target Tg with no correlation between it and the target Tg. ４ However, the predicted value x hat ４，ｔ＋１｜ｔ and the observable value y ４，ｔ＋１A tracking target Tg may be identified as having no correlation with the following. In this case, the occlusion probability calculation unit 3 calculates the tracking target Tg as shown in the following equation (12). ３ and tracking target Tg ４ The shielding probability p, which is the probability that the two overlap, is calculated. The shielding probability calculation unit 3 outputs the shielding probability p to the shielding target setting unit 4.

[0041]

[0042] The shielding target setting unit 4 obtains the shielding occurrence probability p from the shielding occurrence probability calculation unit 3. The shielding target setting unit 4 determines that the shielding occurrence probability p is at threshold p ｔｈｒｅｓｈ If the above conditions are met (step ST6 in Figure 4: YES), the predicted value x hat ｎ，ｔ＋１｜ｔ and the observable value y ｎ，ｔ＋１ Two or more tracking targets Tg that are not correlated with each other ｎ A shielding target PTg is set that includes (step ST7 in Figure 4). In the example in Figure 5, the shielding target setting unit 4 sets the shielding occurrence probability p to threshold p ｔｈｒｅｓｈ If the above conditions are met, then tracking target Tg ３ and tracking target Tg ４ Set one shielding target PTg that includes and . Threshold p ｔｈｒｅｓｈ This is a pre-set value. The shielding target setting unit 4 outputs the setting result of the shielding target PTg to the state estimation unit 5 and the shielding target deletion unit 6, respectively. The shielding target setting unit 4 determines that the shielding occurrence probability p is equal to the threshold p ｔｈｒｅｓｈ If the value is less than (step ST6 in Figure 4: NO), proceed to step ST9 without setting the shielding target PTg.

[0043] The correlation determination unit 5a of the state estimation unit 5 determines the tracking target Tg from the state prediction unit 1. ｎ The predicted value x hat of the current observed state at time t+1 for (n=1, ..., N) ｎ，ｔ＋１｜ｔ The correlation determination unit 5a obtains, for example, the tracking target Tg from the radar device shown in Figure 5. ｎ Observed value y of the current state at observation time t+1 ｎ，ｔ＋１ The correlation determination unit 5a obtains the setting result of the shielding target PTg from the shielding target setting unit 4. The correlation determination unit 5a determines that the shielding target PTg is, for example, the tracking target Tg３ and tracking target Tg ４ If the occlusion target includes the following, then the tracking target Tg will be tracked as shown in equation (13) below. ３ Predicted value x hat for the current observed state at time t+1 ３，ｔ＋１｜ｔ And, tracking target Tg ４ Predicted value x hat for the current observed state at time t+1 ４，ｔ＋１｜ｔ From this, the predicted value xHat OCC1,t+1|t of the current observation time t+1 state of the occlusion target PTg is calculated. If the occlusion target PTg is, for example, the tracking target Tg ３ and tracking target Tg ４ If the occlusion target includes , then the predicted value of the occlusion target PTg x Hat OCC1, t+1|t is equal to the tracking target Tg ３ Predicted value x hat ３，ｔ＋１｜ｔ And, tracking target Tg ４ Predicted value x hat ４，ｔ＋１｜ｔ Information indicating that it was calculated based on the above is added to the predicted value xhat OCC1,t+1|t of the shielding target PTg. The correlation determination unit 5a calculates the observed state vector yhat OCC1,t+1|t of the shielding target PTg using the predicted value xhat OCC1,t+1|t, as shown in the following equation (14).

[0044]

[0045] The correlation determination unit 5a determines the prediction error covariance matrix P of the current observation time t+1 of the shielding target PTg, as shown in the following equation (15). ＯＣＣ１，ｔ＋１｜ｔ The correlation determination unit 5a calculates the prediction error covariance matrix P as shown in the following equation (16). ＯＣＣ１，ｔ＋１｜ｔ The observation error covariance matrix S is calculated using this method.

[0046]

[0047] The correlation determination unit 5a determines the state of the occlusion target PTg at the current observation time t+1 by comparing the predicted value xHat OCC1,t+1|t with the tracking target Tg ｎ Observed value y of the current state at observation time t+1 for (n=1, ..., N) ｎ，ｔ＋１ Based on this, tracking target Tg ｎThe system determines whether there is a correlation between the tracking target Tg and the shielding target PTg (step ST8 in Figure 4). Specifically, the correlation determination unit 5a determines whether there is a correlation between the tracking target Tg if the following equation (17) is true. ｎ If a correlation is determined between and the shielding target PTg, and equation (17) does not hold, then the tracking target Tg ｎ It is determined that there is no correlation between the occlusion target PTg and the tracking target Tg. ３ and tracking target Tg ４ If the occlusion target includes the following, the correlation determination unit 5a determines the predicted value x Hat OCC1, t+1|t and the tracking target Tg ３ Observed value y of the current state at observation time t+1 ３，ｔ＋１ Based on this, tracking target Tg ３ The system determines whether there is a correlation between the predicted value xHat OCC1,t+1|t and the tracking target Tg. ４ Observed value y of the current state at observation time t+1 ４，ｔ＋１ Based on this, tracking target Tg ４ Determine whether or not there is a correlation between the occlusion target PTg.

[0048]

[0049] The correlation determination unit 5a outputs to the state estimation processing unit 5b a determination result indicating whether or not there is a correlation between the predicted value xhat OCC1,t+1|t of the state of the occlusion target PTg at the current observation time t+1. The correlation determination unit 5a also outputs the determination result to the occlusion target deletion unit 6.

[0050] The state estimation processing unit 5b obtains from the correlation determination unit 5a the predicted value xhat OCC1,t+1|t of the state of the occlusion target PTg at the current observation time t+1, and a determination result indicating whether or not there is a correlation. As shown in the following equations (18) to (20), the state estimation processing unit 5b determines the N tracking targets Tg based on the predicted value xhat OCC1,t+1|t. １ ~Tg Ｎ Of these, the tracking target Tg that the correlation determination unit 5a determined to have a correlation with ｎ The state variable x-hat at the current observation time t+1 ｎ，ｔ＋１｜ｔ＋１ This is estimated (step ST9 in Figure 4). If the occlusion target PTg is, for example, the tracking target Tg３ and tracking target Tg ４ When the target is an obstruction target including the target, the tracking target Tg ３ If a correlation is determined between the tracking target Tg, then the state estimation processing unit 5b will determine the tracking target Tg as shown in equation (21) below. ３ The state variable x-hat at the current observation time t+1 ３，ｔ＋１｜ｔ＋１ To estimate the tracking target Tg. ４ If a correlation is determined between the tracking target Tg, then the state estimation processing unit 5b will determine the tracking target Tg as shown in equation (22) below. ４ The state variable x-hat at the current observation time t+1 ４，ｔ＋１｜ｔ＋１ The state estimation processing unit 5b estimates the tracking target Tg. ｎ As a state variable at the time of the previous observation, the tracking target Tg ｎ The state variable at the current observation time t+1 ｎ，ｔ＋１｜ｔ＋１ This is output to the state prediction unit 1.

[0051]

[0052] The occlusion target removal unit 6 obtains a determination result from the correlation determination unit 5a indicating whether or not there is a correlation. The occlusion target removal unit 6 removes two or more tracking targets Tg included in the occlusion target PTg. ｎ If there are no tracking targets determined to be correlated by the correlation determination unit 5a, the occlusion target PTg is deleted. Thereafter, the processing of steps ST1 to ST9 is repeated. In the following steps ST1 to ST8, the occlusion target PTg is deleted from the tracking target Tg. ｎ It is treated as a similar goal.

[0053] In the above embodiment 1, the system includes: a state prediction unit 1 that predicts the state of each tracking target at the current observation time based on the state quantities of each tracking target at the previous observation time and outputs a predicted value for the state of each tracking target; an occlusion occurrence probability calculation unit 3 that calculates the probability that any two or more tracking targets among the multiple tracking targets overlap each other based on the predicted value output from the state prediction unit 1; and an occlusion target setting unit 4 that sets an occlusion target containing two or more tracking targets if the occlusion occurrence probability calculated by the occlusion occurrence probability calculation unit 3 is equal to or greater than a threshold. Furthermore, the target tracking device includes a state estimation unit 5 that calculates a predicted value for the state of the occlusion target at the current observation time set by the occlusion target setting unit 4 based on the predicted value output from the state prediction unit 1, and estimates the state quantities of each tracking target included in the occlusion target at the current observation time using the predicted value for the state of the occlusion target at the current observation time. Therefore, when multiple tracking targets overlap, the target tracking device can track each target regardless of whether or not there are any parts of each target that are not obscured by other targets.

[0054] In the target tracking device shown in Figure 1, the occluded target PTg is, for example, the tracking target Tg ３ and tracking target Tg ４ If the occlusion target includes the above, the correlation determination unit 5a calculates the predicted value x Hat OCC1, t+1|t of the state of the occlusion target PTg at the current observation time t+1, as shown in equation (13). However, this is only one example, and the correlation determination unit 5a may also calculate the predicted value x Hat OCC1, t+1|t of the state of the occlusion target PTg at the current observation time t+1 using the weight ρ, as shown in equation (23) below.

[0055]

[0056] In the target tracking device shown in Figure 1, the occluded target PTg is, for example, the tracking target Tg ３ and tracking target Tg ４ If the occlusion target includes the tracking target Tg, the correlation determination unit 5a calculates the predicted value xhat OCC1,t+1|t of the state of the occlusion target PTg at the current observation time t+1, as shown in equation (13).３ If the occlusion target includes an already set occlusion target, the correlation determination unit 5a determines the tracking target Tg as shown in the following equation (24). ３ Predicted value x hat ３，ｔ＋１｜ｔ Alternatively, the predicted value xhat OCC1,t+1|t of the current observation time t+1 of the shielding target PTg may be calculated based on the predicted value xhat OCC2,t+1|t of the pre-set shielding target.

[0057]

[0058] In the target tracking device shown in Figure 1, the occlusion probability calculation unit 3 calculates the occlusion probability p according to equation (12). However, this is just one example, and the occlusion probability calculation unit 3 may also calculate the occlusion probability p as the absolute value of the simple difference between two predicted values ​​x-hat, as shown in equation (25) below.

[0059]

[0060] Embodiment 2. In Embodiment 2, the tracking target Tg ｎ When the target is a linear object, a target tracking device that tracks a linear object using an optical camera will be described. Examples of linear objects include power lines. The configuration of the target tracking device according to Embodiment 2 is the same as that of the target tracking device according to Embodiment 1, and Figure 1 is a diagram showing the configuration of the target tracking device according to Embodiment 2.

[0061] First, the tracking target Tg ｎ The state vector x-hat is defined as shown in equation (26) below. The x-hat contains the tracking target Tg ｎ An ID (ID identification) is assigned to identify it.

[0062] In equation (26), x ｃｅｎｔｅｒ The tracking target Tg ｎ The centroid position in the image, x ｓｉｚｅ The tracking target Tg ｎ The size of the image, x ｓｌｏｐｅ This is the tracking target Tg, assuming the horizontal direction of the image is 0 degrees. ｎ This is the angle.

[0063] The drive noise error covariance matrix Q is defined as shown in equation (27) below.

[0064] In equations (27) and (28), σ is the pre-set drive noise, and diag is a symbol indicating diagonalization. occ indicates that the tracking target is an obstruction target.

[0065] Tracking target Tg ｎ The observed value y-hat is expressed as shown in equation (29) below, and the observed noise covariance matrix R is expressed as shown in equation (30) below.

[0066]

[0067] The state prediction unit 1 tracks the target Tg as shown in equation (1). ｎ The state variable x-hat at the previous observation time t. ｎ，ｔ｜ｔ Based on this, tracking target Tg ｎ The state of the current observation time t+1 is predicted. Furthermore, the state prediction unit 1 predicts the tracking target Tg as shown in equation (2). ｎ The prediction error covariance matrix P at the current observation time t+1. ｎ，ｔ＋１｜ｔ The result is calculated. In this case, in Embodiment 2, f(x hat) ｎ，ｔ｜ｔ ) is expressed as shown in equation (31) below, and Φ is expressed as shown in equation (32) below.

[0068] In equations (31) to (32), V is the movement speed of the imaging device, C ｆ The focal length of the camera is [mm], C Ｓ is the vertical size [mm] of the camera sensor. S is the actual tracking target Tg. ｎ Size [m], I Ｈ is the vertical size of the image [pixels], d ｇ This represents the time change [deg] of the camera gimbal's pitch angle.

[0069] The correlation target identification unit 2 performs correlation processing using a GNN correlation algorithm to identify N tracking targets Tg １ ~Tg Ｎ Within that, predicted value x hat ｎ，ｔ＋１｜ｔ and the observable value y ｎ，ｔ＋１ Tracking target Tg with no correlation between it and the target Tgｎ When identifying the target Tg for which equation (6) does not hold, ｎ The observed value y ｎ，ｔ＋１ The assignment is to be avoided. The correlation target identification unit 2 identifies the tracking target Tg included in the shielding target PTg. ｎ Also, the observed value y ｎ，ｔ＋１ Avoid assigning it.

[0070] The occlusion probability calculation unit 3 calculates the occlusion probability p as shown in the following equations (33) and (34) if there are two or more uncorrelated tracking targets Tg. Equations (33) and (34) show the case where there are two uncorrelated tracking targets Tg, but they are also applicable to cases where there are three or more.

[0071]

[0072] The shielding target setting unit 4, as shown in the following equation (35), sets the probability of shielding occurrence p to a threshold p ｔｈｒｅｓｈ If the above conditions are met, it is determined that the M tracking targets Tg overlap each other, and an occlusion target PTg containing the M tracking targets Tg is set.

[0073]

[0074] The correlation determination unit 5a calculates the predicted value x Hat OCC1, t+1|t of the state of the occlusion target PTg set by the occlusion target setting unit 4 at the current observation time t+1, as shown in the following equation (36). If the occlusion target PTg is, for example, the tracking target Tg ３ and tracking target Tg ４ If the occlusion target includes the following, the predicted value x Hat OCC1, t+1|t will be the tracking target Tg ３ An ID to identify the target, and the tracking target Tg ４ An identifying ID is added.

[0075]

[0076] Subsequently, the observed values ​​that were not assigned in the initial correlation process are subjected to correlation processing again, and if a correlation is found, the subsequent processing is carried out in the same manner as in Embodiment 1. This makes it possible to maintain tracking even when occlusion occurs between two or more tracking targets captured by the optical camera. In addition, even when the position of an object causing occlusion, such as an artificial object, is known, occlusion can be determined by treating the position or size of the known object as a tracking target, so it can be applied to objects other than the target. Furthermore, the setting of the occlusion target is as shown in equation (13) or equation (36), tracking target Tg ｎ It may also be modified by the state vector.

[0077] Furthermore, this disclosure allows for free combination of each embodiment, modification of any component in each embodiment, or omission of any component in each embodiment.

[0078] This disclosure enables tracking of each tracking target even when multiple tracking targets overlap, regardless of whether or not there are any parts of each tracking target that are not obscured by other tracking targets, and can be used in target tracking devices and target tracking methods.

[0079] 1 State prediction unit, 2 Correlation target identification unit, 3 Shielding occurrence probability calculation unit, 4 Shielding target setting unit, 5 State estimation unit, 5a Correlation determination unit, 5b State estimation processing unit, 6 Shielding target deletion unit, 11 State prediction circuit, 12 Correlation target identification circuit, 13 Shielding occurrence probability calculation circuit, 14 Shielding target setting circuit, 15 State estimation circuit, 16 Shielding target deletion circuit, 21 Memory, 22 Processor.

Claims

1. A target tracking device comprising: a state prediction unit that predicts the state of each tracking target at the current observation time based on the state quantities of each tracking target at the previous observation time, and outputs a predicted value for the state of each tracking target; an occlusion occurrence probability calculation unit that calculates the probability that any two or more tracking targets among the multiple tracking targets overlap each other, based on the predicted value output from the state prediction unit; an occlusion target setting unit that sets an occlusion target including the two or more tracking targets if the occlusion occurrence probability calculated by the occlusion occurrence probability calculation unit is equal to or greater than a threshold; and a state estimation unit that calculates a predicted value for the state of the occlusion target set by the occlusion target setting unit at the current observation time based on the predicted value output from the state prediction unit, and estimates the state quantities of each tracking target included in the occlusion target at the current observation time using the predicted value for the state of the occlusion target at the current observation time.

2. The target tracking device according to claim 1, characterized in that the state estimation unit includes: predicting the state of the occlusion target set by the occlusion target setting unit at the current observation time based on the predicted value output from the state prediction unit; a correlation determination unit that determines whether or not there is a correlation between each tracking target included in the occlusion target and the occlusion target based on the predicted value of the state of the occlusion target at the current observation time and the observed value of the state of each tracking target included in the occlusion target at the current observation time; and a state estimation processing unit that estimates the state quantity of the tracking target at the current observation time that the correlation determination unit has determined to be correlated, based on the predicted value of the state of the occlusion target at the current observation time.

3. The target tracking device according to claim 2, further comprising an occlusion target deletion unit that deletes an occlusion target set by the occlusion target setting unit if there is no tracking target among the two or more tracking targets included in the occlusion target that has been determined to be correlated by the correlation determination unit.

4. A target tracking device according to any one of claims 1 to 3, comprising a correlation target identification unit that identifies a tracking target among the plurality of tracking targets in which there is no correlation between the predicted value and the observed value, based on the predicted value of the state of each tracking target output from the state prediction unit and the observed value of the state of each of the plurality of tracking targets at the current observation time, wherein the occlusion occurrence probability calculation unit calculates an occlusion occurrence probability, which is the probability that two or more tracking targets identified by the correlation target identification unit overlap each other, based on the predicted value output from the state prediction unit.

5. The target tracking device according to any one of claims 1 to 4, characterized in that each of the plurality of tracking targets is a linear object.

6. A target tracking method comprising: a state prediction unit predicts the state of each tracking target at the current observation time based on the state quantities of each tracking target at the previous observation time, and outputs a predicted value for the state of each tracking target; an occlusion occurrence probability calculation unit calculates an occlusion occurrence probability, which is the probability that any two or more tracking targets among the multiple tracking targets overlap each other, based on the predicted value output from the state prediction unit; an occlusion target setting unit sets an occlusion target including the two or more tracking targets if the occlusion occurrence probability calculated by the occlusion occurrence probability calculation unit is equal to or greater than a threshold; a state estimation unit calculates a predicted value for the state of the occlusion target set by the occlusion target setting unit at the current observation time based on the predicted value output from the state prediction unit, and estimates the state quantities of each tracking target included in the occlusion target at the current observation time using the predicted value for the state of the occlusion target at the current observation time.

Images