Object Tracking Device and Object Tracking Method

Abstract

To integrate information acquired from a plurality of external sensors.SOLUTION: An object tracking device 100 comprises: a first acquisition unit 131 for acquiring first information including first position information representing a position of a first object OB1 which is detected by a first external sensor 20 that monitors a front area R1, and first identification information for identifying the first object; a second acquisition unit 132 for acquiring second information including second position information representing a position of a second object OB2 which is detected by a second external sensor 30 that monitors a front side area R2, and second identification information for identifying the second object; and an integration unit 112 that integrates the first information with the second information by recording the second identification information of the second information in the first information and deleting the second information in a case where it is determined that the first object and the second object are the same object, which are detected at the same time in an overlap area where the first region and the second region overlap each other.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an object tracking device and an object tracking method.

Background Art

[0002] Patent Document 1 discloses an in-vehicle radar device including a millimeter-wave radar that detects a target existing in a predetermined region in front of a vehicle, and a camera that images a region wider than the predetermined region and including the predetermined region. In this radar device, using information from the millimeter-wave radar, the distance between the vehicle and the target, the relative speed of the target with respect to the vehicle, and the azimuth of the target with respect to the vehicle are calculated, and using information from the camera, the azimuth of the target with respect to the vehicle is corrected.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, when monitoring the surroundings of a vehicle using an external sensor such as the millimeter-wave radar or camera in the above-mentioned document, the area that can be monitored is limited if only one external sensor is provided. Therefore, the inventors of the present application considered expanding the area that can be monitored by providing a plurality of external sensors, and further considered overlapping the ends of adjacent areas so that there are no dead zones between the areas monitored by each sensor. Then, when the ends of adjacent areas are overlapped with each other, even if there is only one object in the overlapping part of the ends of the two areas, the object may be detected simultaneously by the external sensor monitoring one area and the external sensor monitoring the other area, so it was found that there is a possibility of misrecognizing that there are two objects in the said part. Furthermore, when tracking an object moving from one of the adjacent areas to the other, even though the object tracked by the external sensor monitoring one area and the object being tracked by the external sensor monitoring the other area are the same object, it was found that there is a possibility that the object may be misrecognized as a different object and the tracking of the object may be interrupted.

Means for Solving the Problems

[0005] The present disclosure can be realized in the following forms.

[0006] According to one aspect of the present disclosure, an object tracking device (100) is provided. The object tracking device includes a first acquisition unit (131) that acquires first information including first position information representing the position of a first object (OB1) detected by a first external sensor (20) that monitors a first region (R1) located in a first direction with respect to a vehicle (10), and first identification information for identifying the first object; a second acquisition unit (132) that acquires second information including second position information representing the position of a second object (OB2) detected by a second external sensor (30) that monitors a second region (R2) located in a second direction different from the first direction with respect to the vehicle, and second identification information for identifying the second object; and an integration unit (112) that integrates the first information and the second information. The first region and the second region have an overlapping region (RD) in which their respective ends overlap each other. The integration unit determines whether the first object and the second object detected simultaneously in the overlapping region are the same object by using the first information and the second information. When it is determined that the first object and the second object detected simultaneously in the overlapping region are the same object, the integration unit integrates the first information and the second information by recording the second identification information in the first information and deleting the second information among the second information.

[0007] According to the object tracking device of this form, in the overlapping area monitored by the first external sensor and the second external sensor, when the first object is detected by the first external sensor and at the same time the second object is detected by the second external sensor, the integration unit determines whether the first object and the second object are the same object. If it is determined that the first object and the second object are the same object, the integration unit records the second identification information for identifying the second object among the second information related to the second object in the first information related to the first object and deletes the second information, thereby integrating the first information and the second information. Therefore, when one object existing in the overlapping area is simultaneously detected by the first external sensor and the second external sensor, it is possible to suppress the misrecognition that two objects exist in the overlapping area. In addition, since the integration unit integrates the first information and the second information by recording the second identification information in the first information and deleting the second information, the first object detected by the first external sensor can be recognized as the same object as the second object detected by the second external sensor. Therefore, it is possible to suppress the tracking of the object from being interrupted due to the misrecognition that the first object and the second object are different objects.

Brief Description of Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Best Mode for Carrying Out the Invention

[0009] A. First Embodiment: As shown in FIG. 1, the object tracking device 100 in the first embodiment is mounted on the vehicle 10 and tracks an object existing around the vehicle 10. In this embodiment, the vehicle 10 is an automobile. The object means, for example, a pedestrian, a bicycle, another vehicle, or an obstacle. In the following description, in order to distinguish the vehicle 10 from other vehicles, the vehicle 10 is referred to as the host vehicle 10, and other vehicles are referred to as other vehicles. Note that the host vehicle 10 may not be an automobile but, for example, a motorcycle.

[0010] The host vehicle 10 is equipped with an automatic driving control system 15. In this embodiment, the automatic driving control system 15 includes a first external sensor 20, a second external sensor 30, an internal sensor 40, a brake device 50, and the above-described object tracking device 100.

[0011] The first external sensor 20 monitors a front region R1 located in front of the host vehicle 10 and detects an object existing in the front region R1. In this embodiment, the first external sensor 20 includes a front millimeter-wave radar 21 and a front camera 22.

[0012] The front millimeter-wave radar 21 is a millimeter-wave radar that irradiates millimeter waves on the front region R1 to detect an object existing in the front region R1. The front millimeter-wave radar 21 measures the position and relative speed of an object with respect to the host vehicle 10. In the present embodiment, the front millimeter-wave radar 21 generates first position information representing the position of an object with respect to the host vehicle 10, first speed information representing the relative speed of the object with respect to the host vehicle 10, first detection count information representing the number of times the detected object has been detected, and first identification information for identifying the detected object, and transmits the first position information, the first speed information, the first detection count information, and the first identification information to the object tracking device 100. The first identification information is represented by, for example, a combination of a symbol and a number. When the front millimeter-wave radar 21 detects the same object as the object detected at the previous detection timing, the front millimeter-wave radar 21 generates first identification information having the same content as the first identification information of the object detected at the previous detection timing. When the front millimeter-wave radar 21 detects a plurality of objects, the front millimeter-wave radar 21 generates first position information, first speed information, first detection count information, and first identification information for each object, and transmits them to the object tracking device 100.

[0013] The front camera 22 is a camera that images the front region R1. By analyzing the image obtained by imaging the front region R1, the front camera 22 detects an object existing in the front region R1 and specifies the type of the detected object. The type of the object is, for example, a pedestrian, a bicycle, another vehicle, or an obstacle. In the present embodiment, the front camera 22 generates classification information representing the type of the object and transmits the classification information to the object tracking device 100. When the front camera 22 detects a plurality of objects, the front camera 22 generates classification information for each object and transmits it to the object tracking device 100.

[0014] In the following description, the object detected by the first external sensor 20 may be referred to as the first object. The front may be referred to as the first direction, and the front region R1 may be referred to as the first region R1. The information generated by the first external sensor 20 may be referred to as the first information. In the present embodiment, the first information includes first position information, first speed information, first detection count information, first identification information, and classification information. When a plurality of objects are simultaneously detected by the first external sensor 20, a plurality of first information is generated.

[0015] The second external sensor 30 monitors the front-side region R2 located on the left front side of the host vehicle 10 and the front-side region R2 located on the right front side of the host vehicle 10, and detects an object existing in each front-side region R2. The left front side means the left diagonally front, and the right front side means the right diagonally front. The front region R1 and the front-side region R2 have an overlapping region RD in which the respective ends overlap each other. In the present embodiment, the second external sensor 30 includes a left front-side millimeter-wave radar 31L and a right front-side millimeter-wave radar 31R.

[0016] The left front millimeter-wave radar 31L is a millimeter-wave radar that irradiates millimeter waves on the front-side region R2 on the left front side to detect an object existing in the front-side region R2 on the left front side. The right front millimeter-wave radar 31R is a millimeter-wave radar that irradiates millimeter waves on the front-side region R2 on the right front side to detect an object existing in the front-side region R2 on the right front side. Each of the front-side millimeter-wave radars 31L and 31R measures the position and relative speed of an object with respect to the host vehicle 10. In the present embodiment, each of the front-side millimeter-wave radars 31L and 31R generates second position information representing the position of the object with respect to the host vehicle 10, second speed information representing the relative speed of the object with respect to the host vehicle 10, second detection count information representing the number of detections of the detected object, and second identification information for identifying the detected object, and transmits the second position information, the second speed information, the second detection count information, and the second identification information to the object tracking device 100. The second identification information is represented by, for example, a combination of a symbol and a number. When each of the front-side millimeter-wave radars 31L and 31R detects the same object as the object detected at the previous detection timing, it generates second identification information having the same content as the second identification information of the object detected at the previous detection timing. When each of the front-side millimeter-wave radars 31L and 31R detects a plurality of objects, it generates the second position information, the second speed information, the second detection count information, and the second identification information for each object, and transmits them to the object tracking device 100.

[0017] In the following description, when the left front millimeter-wave radar 31L and the right front millimeter-wave radar 31R are not particularly distinguished, they are simply referred to as the front-side millimeter-wave radar 31. An object detected by the second external sensor 30 may be referred to as a second object. The front side may be referred to as the second direction, and the front-side region R2 may be referred to as the second region R2. Information generated by the second external sensor 30 may be referred to as second information. In the present embodiment, the second information includes second position information, second speed information, second detection count information, and second identification information. When a plurality of objects are simultaneously detected by the second external sensor 30, a plurality of pieces of second information are generated.

[0018] The in-vehicle sensor 40 detects the motion state of the host vehicle 10. In the present embodiment, the in-vehicle sensor 40 includes a steering angle sensor 41. The steering angle sensor 41 detects the steering angle of the host vehicle 10. Information regarding the steering angle detected by the steering angle sensor 41 is transmitted to the object tracking device 100. Note that, in addition to the steering angle sensor 41, the in-vehicle sensor 40 may include, for example, a vehicle speed sensor and a yaw rate sensor.

[0019] The braking device 50 decelerates or stops the host vehicle 10 by generating a braking force on the host vehicle 10. In the present embodiment, when a collision between the object being tracked by the object tracking device 100 and the host vehicle 10 is predicted, the braking device 50 operates without receiving a braking operation by the occupant, thereby avoiding a collision between the host vehicle 10 and the object or mitigating the impact caused by the collision between the host vehicle 10 and the object. When a collision between the host vehicle 10 and the first object is predicted, the braking device 50 generates a stronger braking force than when a collision between the host vehicle 10 and the second object is predicted. When the braking device 50 operates due to a predicted collision between the host vehicle 10 and the first object, it transmits the first identification information of the first object to the object tracking device 100, and when it operates due to a predicted collision between the host vehicle 10 and the second object, it transmits the second identification information of the second object to the object tracking device 100.

[0020] The object tracking device 100 is configured as a computer including a CPU 110, a memory 120, an input / output interface 130 for inputting and outputting signals to and from the outside, and an internal bus 140. The input / output interface 130 includes a first interface 131, a second interface 132, a third interface 133, and a fourth interface 134. A front millimeter-wave radar 21 and a front camera 22 of the first external sensor 20 are connected to the first interface 131. A left front-side millimeter-wave radar 31L and a right front-side millimeter-wave radar 31R of the second external sensor 30 are connected to the second interface 132. A steering angle sensor 41 of the internal sensor 40 is connected to the third interface 133. A brake device 50 is connected to the fourth interface 134. The CPU 110, the memory 120, and the interfaces 131 to 134 are connected to each other via the internal bus 140. Note that the first interface 131 may be referred to as the first acquisition unit 131, the second interface 132 may be referred to as the second acquisition unit 132, the third interface 133 may be referred to as the third acquisition unit 133, and the fourth interface 134 may be referred to as the fourth acquisition unit 134.

[0021] By executing a program stored in the memory 120, the CPU 110 functions as a first integration unit 111, a second integration unit 112, and a tracking execution unit 113. The first integration unit 111 acquires first position information, first speed information, first detection count information, and first identification information input from the front millimeter-wave radar 21 to the first interface 131, and acquires classification information input from the front camera 22 to the first interface 131. The first integration unit 111 integrates the first position information, the first speed information, the first detection count information, and the first identification information, which are part of the first information, with the classification information, which is the remaining part of the first information. In the present embodiment, the first integration unit 111 integrates the first position information, the first speed information, the first detection count information, the first identification information, and the classification information by adding the classification information to the first position information, the first speed information, the first detection count information, and the first identification information. The integrated first information by the first integration unit 111 is transmitted to the second integration unit 112.

[0022] The second integration unit 112 acquires the integrated first information by the first integration unit 111 and the second information input to the second interface 132. The second integration unit 112 determines whether the first object and the second object simultaneously detected in the overlapping region RD are the same object. In the present embodiment, the second integration unit 112 determines whether the first object and the second object simultaneously detected in the overlapping region RD are the same object by executing the same-object determination process described later. When the second integration unit 112 determines that the first object and the second object simultaneously detected in the overlapping region RD are the same object, the second integration unit 112 generates integrated information including the first position information of the first object determined to be the same object and the second identification information of the second object, and transmits the integrated information to the tracking execution unit 113. When the second integration unit 112 does not determine that the first object and the second object simultaneously detected in the overlapping region RD are the same object, the second integration unit 112 transmits the first information and the second information to the tracking execution unit 113. Note that the second integration unit 112 may be simply referred to as the integration unit 112.

[0023] The tracking execution unit 113 tracks the objects existing around the host vehicle 10 using the first information, the second information, and the integrated information. In the present embodiment, when the tracking execution unit 113 acquires the first information from the second integration unit 112, the tracking execution unit 113 recognizes the position of the object using the first position information included in the first information. When the tracking execution unit 113 acquires the second information from the second integration unit 112, the tracking execution unit 113 recognizes the position of the object using the second position information included in the second information. When the tracking execution unit 113 acquires the integrated information from the second integration unit 112, the tracking execution unit 113 recognizes the position of the object using the first position information included in the integrated information. In the present embodiment, the tracking execution unit 113 predicts whether the object being tracked will collide with the host vehicle 10. The tracking execution unit 113 can predict whether the object being tracked will collide with the host vehicle 10 using the first information, the second information, or the integrated information. The tracking execution unit 113 transmits the prediction result to the brake device 50. The prediction result includes the first identification information of the first object for which a collision with the host vehicle 10 is predicted and the second identification information of the second object for which a collision with the host vehicle 10 is predicted. Note that the process executed by the object tracking device 100 including the same-object determination process may be referred to as an object tracking method.

[0024] Figure 2 shows a flowchart indicating the content of the identical object determination process executed by the second integration unit 112. This process is executed by the second integration unit 112 each time the first information and the second information are input to the second integration unit 112.

[0025] First, in step S110, the second integration unit 112 acquires the first information of the first object detected in the overlapping region RD by the first external sensor 20 and the second information of the second object detected in the overlapping region RD by the second external sensor 30. Next, in step S120, the second integration unit 112 acquires information regarding the motion state of the host vehicle 10. In the present embodiment, the second integration unit 112 acquires, as information regarding the motion state of the host vehicle 10, information regarding the steering angle detected by the steering angle sensor 41 and information regarding the operating state of the brake device 50.

[0026] In step S130, the second integration unit 112 selects one of the pairs of the first object and the second object that are simultaneously detected in the overlapping region RD. As shown in Figure 3, for example, when three first objects OB1 and two second objects OB2 are detected in the overlapping region RD, there are a total of six pairs as indicated by the dashed lines. In this case, the second integration unit 112 selects one of the six pairs.

[0027] In step S140 of FIG. 2, the second integration unit 112 determines whether or not to calculate a score for the selected pair. As will be described later, this score is used to determine whether the first object OB1 and the second object OB2 constituting the pair are the same object. In the present embodiment, the second integration unit 112 does not calculate a score for the selected pair when at least one of the following conditions A to E is satisfied. Condition A is a condition that it has already been determined that the first object OB1 and the second object OB2 constituting the pair are the same object. Condition B is a condition that the distance between the host vehicle 10 and the first object OB1 or the distance between the host vehicle 10 and the second object OB2 is greater than or equal to a predetermined distance. Condition C is a condition that the detection count of the first object OB1 or the detection count of the second object OB2 is less than or equal to a predetermined count. Condition D is a condition that the host vehicle 10 is turning at a steering angle greater than or equal to a predetermined steering angle. The predetermined steering angle can be, for example, the steering angle when the host vehicle 10 makes a U-turn. Condition E is a condition that the type of the first object OB1 included in the pair is another vehicle and the braking device 50 is not operating to avoid a collision between the second object OB2 included in the pair and the host vehicle 10. By providing the above-described conditions, pairs that have little need to be determined whether they are the same object can be excluded from the score calculation target, and the calculation load can be reduced.

[0028] When it is determined in step S140 that the second integration unit 112 is to calculate the score for the selected pair, the second integration unit 112 calculates, in step S150, a score for determining whether the first object OB1 and the second object OB2 constituting the pair are the same object. In the present embodiment, the second integration unit 112 calculates a position score based on the similarity between the position of the first object OB1 with respect to the host vehicle 10 and the position of the second object OB2 with respect to the host vehicle 10, and a speed score based on the similarity between the relative speed of the first object OB1 with respect to the host vehicle 10 and the relative speed of the second object OB2 with respect to the host vehicle 10. The second integration unit 112 calculates a score for determining whether the first object OB1 and the second object OB2 constituting the pair are the same object by adding the position score and the speed score. The second integration unit 112 calculates the position score using the first position information among the first information and the second position information among the second information. The higher the similarity between the position of the first object OB1 with respect to the host vehicle 10 and the position of the second object OB2 with respect to the host vehicle 10, the higher the position score. The second integration unit 112 calculates the speed score using the first speed information among the first information and the second speed information among the second information. The higher the similarity between the relative speed of the first object OB1 with respect to the host vehicle 10 and the relative speed of the second object OB2 with respect to the host vehicle 10, the higher the speed score. The weight of the position score is larger than that of the speed score.

[0029] In other embodiments, in addition to the position score and the speed score, the second integration unit 112 calculates a detection count score based on the number of times the first object OB1 is detected and the number of times the second object OB2 is detected, and uses the score obtained by adding the position score, the speed score, and the detection count score as the score for determining whether the first object OB1 and the second object OB2 constituting the pair are the same object. The second integration unit 112 can calculate the detection count score using the first detection count information in the first information and the second detection count information in the second information. The higher the number of times the first object OB1 is detected or the number of times the second object OB2 is detected, the higher the detection count score. In this case, it is possible to suppress the determination that the first object OB1 erroneously detected by the first external sensor 20 is the same as the second object OB2, or the determination that the second object OB2 erroneously detected by the second external sensor 30 is the same as the first object OB1.

[0030] If it is determined in step S140 that the second integration unit 112 does not target the selected pair for score calculation, in step S155, without calculating the position score and the speed score, it determines the score for determining whether the first object OB1 and the second object OB2 constituting the pair are the same object as "0". As will be described later, a pair with a score of "0" is not determined to be the same object.

[0031] In step S160, the second integration unit 112 determines whether the scores of all pairs have been determined. If it is determined in step S160 that the scores of all pairs have been determined, the process proceeds to step S170. If it is not determined in step S160 that the scores of all pairs have been determined, the processes from step S130 to step S160 are repeated until the scores of all pairs are determined.

[0032] In step S170, the second integration unit 112 determines whether the first object OB1 and the second object OB2 that are simultaneously detected in the overlapping region RD are the same object. In the present embodiment, the second integration unit 112 determines whether the first object OB1 and the second object OB2 that constitute each pair are the same object based on the scores of each pair. Specifically, the second integration unit 112 first assigns, to each second object OB2, the first object OB1 for which the score is maximized when paired with the second object OB2. When different first objects OB1 are assigned to each second object OB2, the second integration unit 112 determines that the second object OB2 and the first object OB1 assigned to the second object are the same object. When different first objects OB1 are not assigned to each second object OB2, in other words, when one first object OB1 is repeatedly assigned to a plurality of second objects OB2, the second integration unit 112 uses a combinatorial optimization algorithm to optimize the combination of the first object OB1 and the second object OB2 so that different first objects OB1 are assigned to each second object OB2 and the sum of the scores of each pair is maximized. As the combinatorial optimization algorithm, for example, the GNN (Global Nearest Neighbor) method or the Greedy method can be used. FIG. 4 shows an example in which the combination of the first object OB1 and the second object OB2 is optimized. In this example, it is determined that "S1" and "F3" are the same object, "S2" and "F4" are the same object, and "S4" and "F1" are the same object. There is no first object OB1 that is the same object for "S3". When the number of first objects OB1 simultaneously detected in the overlapping region RD is one and the number of second objects OB2 is one, the second integration unit 112 determines that the first object OB1 and the second object OB2 are the same object when the score of the pair constituted by the first object OB1 and the second object OB2 is greater than "0", and does not determine that the first object OB1 and the second object OB2 are the same object when the score is "0".

[0033] In step S180 of FIG. 2, the second integration unit 112 generates a determination result as to whether the first object OB1 and the second object OB2 that are simultaneously detected in the overlapping region RD are the same object, and stores the determination result in the memory 120. The determination result represents the first identification information and the second identification information regarding the first object OB1 and the second object OB2 determined to be the same object. Thereafter, the second integration unit 112 ends this process. As described above, when new first information and second information are input to the object tracking device 100, the second integration unit 112 starts this process again. Note that the part of step S110 that acquires the first information may be referred to as the first acquisition step, and the part of step S110 that acquires the second information may be referred to as the second acquisition step. Information representing the motion state of the host vehicle 10 may be referred to as third information, and the part of step S120 that acquires information representing the motion state of the host vehicle 10 may be referred to as the third acquisition step. Information representing the operating state of the brake device 50 may be referred to as fourth information, and the part of step S120 that acquires information representing the operating state of the brake device 50 may be referred to as the fourth acquisition step. The part from step S130 to step S180 may be referred to as the determination step.

[0034] As shown in FIG. 5, when the second integration unit 112 determines that the first object OB1 and the second object OB2 are the same object in the same object determination process, the second integration unit 112 integrates the first information and the second information regarding the first object OB1 and the second object OB2 determined to be the same object, and generates integrated information. In the present embodiment, the second integration unit 112 generates integrated information by overwriting and recording the first identification information included in the first information of the first object OB1 with the second identification information of the second object OB2 determined to be the same object as the first object OB1. That is, in the present embodiment, the integrated information includes the first position information, the first speed information, the first detection frequency information, the classification information, and the second identification information. The second integration unit 112 transmits the integrated information, in other words, the first information in which the first identification information is overwritten with the second identification information, to the tracking execution unit 113, and deletes the second information. When the second integration unit 112 does not determine that the first object OB1 and the second object OB2 are the same object in the same object determination process, the second integration unit 112 transmits the first information and the second information to the tracking execution unit 113 without generating integrated information.

[0035] Thereafter, when the second integration unit 112 newly acquires first information and second information regarding the first object OB1 and the second object OB2 determined to be the same, the second integration unit 112 integrates the newly acquired first information and second information to generate integrated information, deletes the second information, and transmits the integrated information to the tracking execution unit 113. When the second integration unit 112 newly acquires the first information regarding the first object OB1 but does not newly acquire the second information regarding the second object OB2 determined to be the same as the first object OB1, the second integration unit 112 overwrites the first identification information included in the newly acquired first information with the second identification information of the second object OB2 determined to be the same as the first object OB1 to generate integrated information, and transmits the integrated information to the tracking execution unit 113. The second integration unit 112 can acquire the second identification information of the second object OB2 determined to be the same as the first object OB1 by referring to the determination result of the same-object determination process stored in the memory 120. Therefore, for the first object OB1 and the second object OB2 determined to be the same in the same-object determination process, it is not necessary to determine again whether they are the same in the subsequent same-object determination process, so the computational load of the same-object determination process can be reduced. When the second integration unit 112 newly acquires the second information regarding the second object OB2 but does not newly acquire the first information regarding the first object OB1 determined to be the same as the second object OB2, the second integration unit 112 transmits the second information to the tracking execution unit 113 without generating integrated information. Note that the process of generating integrated information may be referred to as an integration process.

[0036] FIG. 6 schematically shows a state in which an object moving from the front-side region R2 outside the overlapping region RD to the front region R1 outside the overlapping region RD is tracked by the tracking execution unit 113. An object that enters the front-side region R2 outside the overlapping region RD is detected as the second object OB2 by the second external sensor 30. In FIG. 6, the second object OB2 is given second identification information "S1" by the second external sensor 30. Second information including second position information representing the position of the second object OB2 and the second identification information is transmitted to the tracking execution unit 113.

[0037] When an object enters the overlapping region RD from the front-side region R2 outside the overlapping region RD, the object that has entered the overlapping region RD is detected not only as the second object OB2 by the second external sensor 30, but also as the first object OB1 by the first external sensor 20. In FIG. 6, the first object OB1 is given first identification information "F5" by the first external sensor 20. When it is not determined by the same-object determination process that the first object OB1 and the second object OB2 are the same object, the tracking execution unit 113 is transmitted with first information including first position information representing the position of the first object OB1 and the first identification information, and second information including second position information representing the position of the second object OB2 and the second identification information. At this timing, since the first information and the second information are transmitted to the tracking execution unit 113, although there is actually only one object in the overlapping region RD, the tracking execution unit 113 recognizes that there are two objects in the overlapping region RD. After that, when it is determined by the same-object determination process that the first object OB1 and the second object OB2 moving within the overlapping region RD are the same object, integrated information including the first position information and the second identification information is transmitted to the tracking execution unit 113, and the second information is no longer transmitted. Therefore, the tracking execution unit 113 comes to recognize that there is one object in the overlapping region RD. Since the integrated information includes the second identification information "S1", the tracking execution unit 113 recognizes that the first object OB1 detected in the overlapping region RD is the same object as the second object OB2.

[0038] When an object enters from the overlapping region RD into the front region R1 outside the overlapping region RD, the object that has entered the front region R1 outside the overlapping region RD is detected as the first object OB1 by the first external sensor 20, but is not detected by the second external sensor 30. Since the first object OB1 has already been determined to be the same object as the second object OB2 by the same object determination process, integrated information is transmitted to the tracking execution unit 113. As described above, since the integrated information includes the second identification information "S1", the tracking execution unit 113 recognizes that the first object OB1 detected in the front region R1 outside the overlapping region RD is the same object as the second object OB2. Therefore, the tracking execution unit 113 can continuously track an object that moves from the front side region R2 outside the overlapping region RD, through the overlapping region RD, and into the front region R1 outside the overlapping region RD without interruption.

[0039] FIG. 7 shows a situation in which when the host vehicle 10 attempts to go straight through an intersection, another vehicle VH enters the intersection from a road intersecting with the road on which the host vehicle 10 is traveling. When the other vehicle VH enters the front-side region R2, the other vehicle VH is detected as a second object by the second external sensor 30. When a collision between the host vehicle 10 and the second object is predicted, the automatic driving control system 15 generates a braking force for the host vehicle 10 by the braking device 50. Thereafter, when the other vehicle VH enters the front region R1, the other vehicle VH is detected as a first object by the first external sensor 20. When a collision between the host vehicle 10 and the first object is predicted, the automatic driving control system 15 increases the braking force by the braking device 50. Therefore, the possibility of a collision between the host vehicle 10 and the other vehicle VH can be reduced, and when a collision between the host vehicle 10 and the other vehicle VH is inevitable, the impact at the time of the collision between the host vehicle 10 and the other vehicle VH can be mitigated. Further, in the present embodiment, since the first external sensor 20 and the second external sensor 30 are provided on the host vehicle 10, the monitorable area is expanded as compared with a configuration in which only the first external sensor 20 is provided without providing the second external sensor 30. Therefore, in the situation described with reference to FIG. 7, an approaching other vehicle can be detected early for the host vehicle 10, and braking of the host vehicle 10 can be started early. Therefore, the possibility of a collision between the host vehicle 10 and the other vehicle can be reduced. Further, in the present embodiment, since the automatic driving control system 15 increases the braking force step by step, it is possible to suppress a large impact being applied to the passengers due to the sudden generation of a strong braking force. Furthermore, in the present embodiment, an object moving from the front-side region R2 outside the overlapping region RD through the overlapping region RD to the front region R1 outside the overlapping region RD can be tracked without interruption. Therefore, it is possible to suppress the tracking of the object from being interrupted and the operation of the braking device 50 from being interrupted.

[0040] FIG. 8 shows a pedestrian WK crossing a crosswalk in the same traveling direction as that of the host vehicle 10 before a right turn when the host vehicle 10 attempts to cross the crosswalk after making a right turn at an intersection. When the pedestrian WK enters the front side region R2, the second external sensor 30 detects the pedestrian WK as a second object. When a collision between the host vehicle 10 and the second object is predicted, the automatic driving control system 15 generates a braking force for the host vehicle 10 by the brake device 50. Thereafter, when the pedestrian WK enters the front region R1, the first external sensor 20 detects the pedestrian WK as a first object. When a collision between the host vehicle 10 and the first object is predicted, the automatic driving control system 15 increases the braking force by the brake device 50. Therefore, the possibility of a collision between the host vehicle 10 and the pedestrian WK can be reduced, and when a collision between the host vehicle 10 and the pedestrian WK is inevitable, the impact at the time of the collision between the host vehicle 10 and the pedestrian WK can be mitigated. Further, in the present embodiment, since the first external sensor 20 and the second external sensor 30 are provided on the host vehicle 10, the monitorable area is expanded as compared with a configuration in which only the first external sensor 20 is provided without providing the second external sensor 30. Therefore, the pedestrian WK can be easily detected in the situation described with reference to FIG. 8. Furthermore, in the present embodiment, an object moving from the front side region R2 outside the overlapping region RD through the overlapping region RD to the front region R1 outside the overlapping region RD can be tracked without interruption. Therefore, it is possible to suppress the tracking of the object from being interrupted and the operation of the brake device 50 from being interrupted.

[0041] According to the object tracking device 100 of the present embodiment described above, in the overlapping region RD monitored by the first external sensor 20 and the second external sensor 30, when the first object OB1 is detected by the first external sensor 20 and at the same time the second object OB2 is detected by the second external sensor 30, and the first information regarding the first object OB1 and the second information regarding the second object OB2 are acquired, the second integration unit 112 determines whether the first object OB1 and the second object OB2 are the same object by the same object determination process. When the second integration unit 112 determines in the same object determination process that the first object OB1 and the second object OB2 are the same object, the second integration unit 112 generates integrated information by overwriting the first identification information of the first information with the second identification information of the second information, transmits the integrated information to the tracking execution unit 113, and deletes the second information. The tracking execution unit 113 recognizes the position of the object existing in the overlapping region RD using the first position information included in the integrated information. Therefore, when one object existing in the overlapping region RD is simultaneously detected by the first external sensor 20 and the second external sensor 30, it is possible to suppress the tracking execution unit 113 from erroneously recognizing that two objects exist in the overlapping region RD. Further, in the present embodiment, even when the second integration unit 112 newly acquires the first information after determining that the first object OB1 and the second object OB2 are the same object by the same object determination process, the second integration unit 112 generates integrated information by overwriting the first identification information of the newly acquired first information with the second identification information and transmits the integrated information to the tracking execution unit 113. Therefore, the tracking execution unit 113 can recognize that the second object OB2 that has moved from the front side region R2 outside the overlapping region RD to the overlapping region RD and the first object OB1 that has moved from the overlapping region RD to the front region R1 outside the overlapping region RD are the same object. Therefore, it is possible to suppress the tracking execution unit 113 from erroneously recognizing that the second object OB2 that has moved from the front side region R2 outside the overlapping region RD to the overlapping region RD and the first object OB1 that has moved from the overlapping region RD to the front region R1 outside the overlapping region RD are different objects and the tracking of the object is interrupted.

[0042] Also, in the present embodiment, in the same object determination process, the second integration unit 112 calculates a position score according to the degree of similarity between the position of the first object OB1 with respect to the host vehicle 10 represented by the first position information and the position of the second object OB2 with respect to the host vehicle 10 represented by the second position information, and a speed score according to the degree of similarity between the relative speed of the first object OB1 with respect to the host vehicle 10 represented by the first speed information and the relative speed of the second object OB2 with respect to the host vehicle 10 represented by the second speed information, and determines whether the first object OB1 and the second object OB2 are the same object based on the score obtained by adding the position score and the speed score. Therefore, it is possible to suppress the misjudgment that the first object OB1 and the second object OB2 are the same object even though they are different objects.

[0043] Also, in the present embodiment, in the same object determination process, when the second integration unit 112 determines that the host vehicle 10 is turning with a steering angle equal to or greater than a predetermined value, it omits the calculation of the score for determining whether the first object OB1 and the second object OB2 are the same object, and determines the score for determining whether the first object OB1 and the second object OB2 are the same object as "0". For example, during a turn with a large steering angle such as a U-turn, the situations described with reference to FIG. 7 and FIG. 8 are hardly likely to occur. Therefore, when the host vehicle 10 is turning with a steering angle equal to or greater than a predetermined value, the calculation load can be reduced by omitting the calculation of the score for determining whether the first object OB1 and the second object OB2 are the same object.

[0044] Also, in the present embodiment, in the same object determination process, when the type of the first object OB1 detected in the overlapping region RD is another vehicle, the second integration unit 112 calculates the score for determining whether the first object OB1 and the second object OB2 are the same object only when the host vehicle 10 is in the process of braking for a collision with the second object OB2. Therefore, as described with reference to FIG. 7, it is possible to suppress a large impact on the passengers of the host vehicle 10 due to the sudden generation of a strong braking force.

[0045] Further, in the present embodiment, when a plurality of first objects OB1 and a plurality of second objects OB2 are simultaneously detected in the overlapping region RD, the second integration unit 112 calculates a score for determining whether each pair of the first object OB1 and the second object OB2 is the same object in the same object determination process, and determines whether the first object OB1 and the second object OB2 are the same object based on the scores of each pair. In particular, in the present embodiment, when determining whether the first object OB1 and the second object OB2 are the same object based on the scores of each pair, if one first object OB1 is a candidate for the same object as a plurality of second objects OB2, the second integration unit 112 optimizes the combination of the first object OB1 and the second object OB2 by a combinatorial optimization algorithm so that one first object OB1 is not a candidate for the same object as a plurality of second objects OB2. Therefore, even when a plurality of first objects OB1 and a plurality of second objects OB2 are simultaneously detected in the overlapping region RD, it is possible to determine whether the first object OB1 and the second object OB2 are the same object.

[0046] B. Other Embodiments: (B-1) In the first embodiment described above, the first external sensor 20 monitors the front region R1 located in front of the host vehicle 10, and the second external sensor 30 monitors the front side region R2 located obliquely in front of the host vehicle 10. In contrast, the first external sensor 20 may monitor, for example, a rear region located behind the host vehicle 10, and the second external sensor 30 may monitor, for example, a rear side region located obliquely behind the host vehicle 10. In this case, the rear may be referred to as the first direction, the rear region may be referred to as the first region, the obliquely rear may be referred to as the second direction, and the rear side region may be referred to as the second region.

[0047] (B-2) In the above-described first embodiment, the first external sensor 20 includes the front millimeter-wave radar 21 and the front camera 22, and the second external sensor 30 includes the front side millimeter-wave radar 31. In contrast, the first external sensor 20 may include, for example, LiDAR (Light Detection and Ranging) or an ultrasonic radar instead of the front millimeter-wave radar 21. The first external sensor 20 may not include the front camera 22. The second external sensor 30 may include, for example, LiDAR or an ultrasonic radar instead of the front side millimeter-wave radar 31.

[0048] (B-3) In the object tracking device 100 of the above-described first embodiment, the second integration unit 112 omits the calculation of the score for determining whether the first object OB1 and the second object OB2 are the same object in the same object determination process when the motion state of the host vehicle 10 is turning with a steering angle equal to or greater than a predetermined value. In contrast, the second integration unit 112 may not execute the same object determination process during the period when the motion state of the host vehicle 10 is turning with a steering angle equal to or greater than a predetermined value.

[0049] (B-4) In the object tracking device 100 of the above-described first embodiment, when the type of the first object OB1 detected in the overlapping region RD is an other vehicle, the second integration unit 112 calculates the score for determining whether the first object OB1 and the second object OB2 are the same object only when the motion state of the host vehicle 10 is in braking for a collision with the second object OB2. In contrast, when the type of the first object OB1 detected in the overlapping region RD is an other vehicle, the second integration unit 112 may calculate the score for determining whether the first object OB1 and the second object OB2 are the same object even when the motion state of the host vehicle 10 is not in braking for a collision with the second object OB2.

[0050] (B-5) In the object tracking device 100 of the first embodiment described above, the second integration unit 112 calculates a position score and a velocity score, and uses the score obtained by adding the position score and the velocity score as a score for determining whether the first object OB1 and the second object OB2 are the same object. In contrast, the second integration unit 112 may use the position score as a score for determining whether the first object OB1 and the second object OB2 are the same object. In this case, the second integration unit 112 does not necessarily need to calculate the velocity score.

[0051] (B-6) In the object tracking device 100 of the first embodiment described above, when the second integration unit 112 determines whether the first object OB1 and the second object OB2 are the same object based on the scores of each pair in the same object determination process, if one first object OB1 is a candidate for the same object as a plurality of second objects OB2, the second integration unit 112 optimizes the combination of the first object OB1 and the second object OB2 by a combinatorial optimization algorithm so that one first object OB1 is not a candidate for the same object as a plurality of second objects OB2. In contrast, the second integration unit 112 does not necessarily need to optimize the combination of the first object OB1 and the second object OB2 by a combinatorial optimization algorithm in the same object determination process. For example, the second integration unit 112 first selects the first object OB1 with the maximum score when paired with the first second object OB2, determines that the selected first object OB1 and the second object OB2 are the same object, and then, for the second second object OB2, excludes the first object OB1 determined to be the same object as the first second object OB2, selects the first object OB1 with the maximum score when paired, and determines that the selected first object OB1 and the second object OB2 are the same object. The second integration unit 112 may sequentially determine the first object OB1 that is the same object as the second object OB2 in this way.

[0052] The present disclosure is not limited to the above-described embodiments, and can be implemented in various configurations without departing from the gist thereof. For example, the technical features in the embodiments can be appropriately replaced or combined in order to solve some or all of the above-described problems, or to achieve some or all of the above-described effects. Further, if the technical feature is not described as essential in this specification, it can be appropriately deleted.

Description of Reference Numerals

[0053] 10…Vehicle, 15…Autonomous driving control system, 20…First external sensor, 21…Millimeter wave radar, 22…Camera, 30…Second external sensor, 31…Millimeter wave radar, 40…Internal sensor, 41…Steering angle sensor, 50…Brake device, 100…Object tracking device, 110…CPU, 111…First integration unit, 112…Second integration unit, 113…Tracking execution unit, 120…Memory, 130…Input / output interface, 131…First interface, 132…Second interface, 133…Third interface, 134…Fourth interface, 140…Internal bus, R1…First region, R2…Second region, RD…Overlapping region

Claims

1. An object tracking device (100), a first acquisition unit (131) that acquires first information including first position information representing the position of a first object (OB1) detected by a first external sensor (20) that monitors a first region (R1) located in a first direction with respect to a vehicle (10), and first identification information for identifying the first object; a second acquisition unit (132) that acquires second information including second position information representing the position of a second object (OB2) detected by a second external sensor (30) that monitors a second region (R2) located in a second direction different from the first direction with respect to the vehicle, and second identification information for identifying the second object; an integration unit (112) that integrates the first information and the second information; comprising: the first region and the second region each have an overlapping region (RD) where their respective ends overlap each other; the integration unit: uses the first information and the second information to determine whether the first object and the second object simultaneously detected in the overlapping region are the same object; when it is determined that the first object and the second object simultaneously detected in the overlapping region are the same object, integrates the first information and the second information by recording the second identification information in the first information and deleting the second information; an object tracking device.

2. The object tracking device according to claim 1, comprising a third acquisition unit (133) that acquires third information representing the motion state of the vehicle; when the motion state represented by the third information is a turning motion at a steering angle equal to or greater than a predetermined steering angle, the integration unit does not determine whether the first object and the second object are the same object even if the first object and the second object are simultaneously detected in the overlapping region. An object tracking device.

3. The object tracking device according to claim 1 or claim 2, the vehicle is equipped with a brake device (50) that generates braking force without receiving an operation by a passenger when a collision with the first object and the second object is predicted; when a collision with the first object is predicted, the brake device generates a stronger braking force than when a collision with the second object is predicted; comprising a fourth acquisition unit (134) that acquires fourth information representing the operating state of the brake device; the first information includes classification information representing the type of the first object; When the type of the first object is a vehicle, the integration unit integrates the first information and the second information only when the braking device is in operation due to a predicted collision with the second object. An object tracking device.

4. The object tracking device according to any one of Claims 1 to 3, The first information includes first velocity information representing the relative velocity of the first object with respect to the vehicle. The second information includes second velocity information representing the relative velocity of the second object with respect to the vehicle. The integration unit calculates a score according to the degree of similarity between the position represented by the first position information and the position represented by the second position information, and the degree of similarity between the relative velocity represented by the first velocity information and the relative velocity represented by the second velocity information, and determines whether the first object and the second object are the same based on the score. An object tracking device.

5. The object tracking device according to Claim 4, When a plurality of the first objects and a plurality of the second objects are simultaneously detected in the overlapping region, the integration unit calculates the score for each pair of the first object and the second object, and determines whether the first object and the second object are the same based on the score. An object tracking device.

6. The object tracking device according to Claim 5, When a plurality of the second objects are candidates for being the same object with respect to one of the first objects, the integration unit optimizes the combination so that one of the second objects is a candidate for being the same object with respect to one of the first objects by a predetermined combinatorial optimization algorithm. An object tracking device.

7. An object tracking method executed by a computer, A first acquisition step of acquiring first information including first position information representing the position of a first object (OB1) detected by a first external sensor (20) that monitors a first region (R1) located in a first direction with respect to a vehicle (10), and first identification information for identifying the first object; A second acquisition step of acquiring second information including second position information representing the position of a second object (OB2) detected by a second external sensor (30) that monitors a second region (R2) located in a second direction different from the first direction with respect to the vehicle, and second identification information for identifying the second object; An integration step of integrating the first information and the second information; having The first region and the second region each have an overlapping region (RD) where their respective ends overlap each other. In the integration step, using the first information and the second information, it is determined whether the first object and the second object simultaneously detected in the overlapping region are the same object. When it is determined that the first object and the second object simultaneously detected in the overlapping region are the same object, the second identification information in the second information is recorded in the first information and the second information is deleted, thereby integrating the first information and the second information. Object tracking method.

Images