Target detection device

Abstract

The reliability of a target is determined appropriately. Provided is a target detection device that detects a target near a vehicle on the basis of sensor information measured by a plurality of sensors. The device comprises: a first sensor; a second sensor; a fusion processing unit that periodically fuses target detection information from the first sensor and target detection information from the second sensor; a fusion history management unit that manages the results of the fusion by the fusion processing unit in a time series as a fusion history; and a reliability determination unit that determines the reliability of the class of the target on the basis of the fusion history and the target detection information from the first and second sensors. The target detection information from the first sensor includes class information about the target.

Description

[Technical field]

[0001] The present invention relates to a target detection device. [Background technology]

[0002] Driving assistance systems and autonomous driving systems have been developed to achieve various goals, including reducing traffic accidents, easing the burden on drivers, improving fuel efficiency to reduce the burden on the global environment, and providing transportation for vulnerable road users to realize a sustainable society. In these driving assistance systems and autonomous driving systems, multiple sensors (cameras, radar, etc.) are installed on the vehicle to monitor the surroundings of the vehicle instead of the driver. In addition, systems have been developed that use the recognition results of multiple sensors installed in the vehicle to automatically brake for specific targets such as pedestrians and vehicles.

[0003] For example, Patent Document 1 discloses an object recognition device including a moving target discrimination unit that discriminates whether an object detected by a radar device is a moving target, and an object extraction unit that extracts a specific object from an image captured by a camera unit. The specific object discrimination unit discriminates whether an object detected by the radar device is a specific object based on the discrimination result of the moving target discrimination unit and the extraction result of the object extraction unit.

[0004] However, in the device of Patent Document 1, since the reliability of a type is determined by counting the number of detections by each sensor or by both sensors for targets simultaneously detected by both sensors of the radar device and the camera unit, if an object moves out of the camera detection range after simultaneous detection by both sensors, the reliability of the target cannot be determined appropriately. If the reliability of the target cannot be determined appropriately, there is a risk that the decision to perform automatic braking will not be made appropriately. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2014-21524 A Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention has been made in consideration of the above problems, and provides a target detection device that can appropriately determine the reliability of a target. [Means for solving the problem]

[0007] A target detection device according to the present invention is a target detection device that detects targets around a vehicle based on sensor information measured by a plurality of sensors, and includes a first sensor, a second sensor, an integration processing unit that periodically integrates the target detection information of the first sensor and the target detection information of the second sensor, an integration history management unit that chronologically manages the integration results of the integration processing unit as an integration history, and a reliability determination unit that determines reliability of a target type based on the integration history and the target detection information of the first sensor and the second sensor. The target detection information of the first sensor includes information on the type of the target. Effect of the Invention

[0008] According to the present invention, it is possible to provide a target detection device capable of appropriately determining the reliability of a target. [Brief description of the drawings]

[0009] [Figure 1] 1 is a functional block diagram of a target detection device (sensor fusion device 1) according to a first embodiment. [Diagram 2] 11 is a flowchart illustrating a procedure for determining the reliability and assigning detailed type information in a reliability determining unit 50 and a detailed type assigning unit 60. [Diagram 3] FIG. 11 is a schematic diagram illustrating the reliability determination and the assignment of detailed type information according to one scenario. [Figure 4] FIG. 11 is a schematic diagram illustrating the reliability determination and the assignment of detailed type information according to one scenario. [Diagram 5] FIG. 11 is a schematic diagram illustrating the reliability determination and the assignment of detailed type information according to one scenario. [Figure 6] FIG. 11 is a schematic diagram illustrating the reliability determination and the assignment of detailed type information according to one scenario. [Figure 7] FIG. 11 is a functional block diagram of a target detection device (sensor fusion device 1) according to a second embodiment. [Figure 8] FIG. 11 is a schematic diagram illustrating the reliability determination and the assignment of detailed type information in the second embodiment according to one scenario. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the present embodiment will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be indicated by the same numbers. Note that the accompanying drawings show embodiments according to the principles of the present disclosure, but these are for understanding the present disclosure and are not to be used to interpret the present disclosure in a limiting manner. The description in this specification is merely a typical example and does not limit the scope or application of the present disclosure in any sense.

[0011] In the present embodiment, the description is given in sufficient detail for a person skilled in the art to implement the present disclosure, but it should be understood that other implementations and forms are possible, and that changes in configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the present disclosure. Therefore, the following description should not be interpreted as being limited thereto.

[0012] [First embodiment] Fig. 1 is a functional block diagram of a target detection device (sensor fusion device 1) according to a first embodiment. As shown in Fig. 1, the sensor fusion device 1 according to the first embodiment includes an observation information processing unit 20, an integration processing unit 30, an integration history management unit 40, a reliability determination unit 50, and a detailed classification assignment unit 60. In addition, output signals from a first sensor 10a, a second sensor 10b, and a host vehicle behavior detection sensor 10c are input to the sensor fusion device 1.

[0013] The first sensor 10a and the second sensor 10b are sensors that detect targets around the vehicle. As an example, the first sensor 10a is a sensor that can determine the type of a target based on image data of the captured target, such as a camera (visible light camera, near-infrared camera, mid-infrared camera, far-infrared camera). The second sensor 10b is a sensor that can determine the presence and position of a target but does not have a function of determining the type of the target, such as a millimeter wave radar, LiDAR (Light Detection and Ranging), sonar, TOF (Time of Flight) sensor, or a sensor that combines them. The vehicle behavior detection sensor 10c is a group of sensors that detect the speed, steering angle, yaw rate, etc. of the vehicle. As an example, the vehicle behavior detection sensor 10c includes a wheel speed sensor, a steering angle sensor, an acceleration sensor, a gyro sensor, etc.

[0014] The detection information of the first sensor 10a includes at least the position, speed, type, and target ID of the detected target, and the detection information of the second sensor 10b includes at least the position, speed, and target ID of the detected target. The sensor fusion device 1 (electronic control device) and various sensors (such as the first sensor 10a and the second sensor 10b) of this embodiment include a computer (microcomputer) including an arithmetic unit, a memory, and an input / output device.

[0015] The arithmetic device includes a processor and executes a program stored in a memory. A part of the processing performed by the arithmetic device executing the program may be executed by another arithmetic device (for example, hardware such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC)).

[0016] The memory includes ROM and RAM, which are non-volatile storage elements. ROM stores unchanging programs (e.g., BIOS), etc. RAM is a high-speed and volatile storage element such as DRAM (Dynamic Random Access Memory) and a non-volatile storage element such as SRAM (Static Random Access Memory), and stores programs executed by the arithmetic device and data used when the programs are executed.

[0017] The input / output device is an interface that transmits the contents of processing by the electronic control device or the sensor to the outside and receives data from the outside according to a predetermined protocol. The programs executed by the arithmetic unit are stored in a non-volatile memory that is a non-transitory storage medium of the electronic control device or the sensor.

[0018] The observation information processor 20 has a function of receiving detection information from the first sensor 10a and the second sensor 10b, as well as the speed, steering angle, yaw rate, etc. of the vehicle output from the vehicle behavior detection sensor 10c, and converting the information into a standard format. The code given as the target ID is the same when tracking the same target in the time direction. The above format includes at least time and coordinate information, where the time indicates, for example, the timing of fusion execution, and the coordinate indicates, for example, a coordinate system with the center of the vehicle as the origin, the front of the vehicle as the x-axis, and the left of the vehicle as the y-axis.

[0019] The integration processing unit 30 receives the detection information of the target output from the observation information processing unit 20 and performs integration processing, which will be described later. The integration processing unit 30 includes a grouping unit 30a that groups a plurality of pieces of detection information, an integration unit 30b that integrates the detection information of the grouped targets to generate or update a fusion target, and a prediction update unit 30c that integrates the detection information and performs prediction update to track the same target in a time series.

[0020] The grouping unit 30a uses at least position information among the detection information of the target to determine whether or not the multiple detection information are detection information of the same target, and groups the multiple detection information determined to be the same target into one set of information. The multiple detection information grouped by the grouping unit 30a is integrated in the integration unit 30b and converted into a fusion target, and is assigned the same target ID. The grouping in the grouping unit 30a and the integration process in the integration unit 30b are executed at predetermined intervals.

[0021] In addition, when the prediction update unit 30c obtains the estimated information of the fusion target in the previous cycle, the grouping unit 30a performs an identity determination between the estimated information of the fusion target obtained in the previous cycle and the multiple pieces of detection information obtained in the current cycle, prior to the identity determination between the multiple pieces of detection information for grouping. The identity determination for the estimated information of the fusion target obtained in the previous cycle is performed using at least the position information of the estimated information of the fusion target obtained in the previous cycle and the position information included in the multiple pieces of detection information obtained in the current cycle. When it is determined that the estimated information of the fusion target obtained in the previous cycle is identical to the detection information obtained in the current cycle, the target ID assigned to the multiple pieces of detection information is set to be the same as the target ID of the fusion target in the previous cycle that is determined to be identical.

[0022] The plurality of pieces of detection information grouped by the grouping unit 30a are output to the integrating unit 30b. The integrating unit 30b integrates the plurality of pieces of grouped detection information to generate or update a fusion target. In the integration process, for example, the error characteristics of the detections of the first sensor 10a and the second sensor 10b are given in advance as parameters to the integrating unit 30b, and integration can be performed based on the position and speed corrected according to the parameters of the error characteristics. Alternatively, the error characteristics may be defined by a covariance matrix, and integration may be performed by adopting the position and speed calculated by the probability average.

[0023] Moreover, the error characteristics of each sensor may be estimated during the integration process in the integration unit 30b instead of being provided to the integration unit 30b in advance. Moreover, data on such error characteristics may be included in the detection information of the first sensor 10a and the second sensor 10b. Moreover, even if the detection information of a sensor included in the fusion target is one, the integration unit 30b outputs the detection result as the fusion target. In the following description, even if the detection information of one sensor is grouped, it is expressed as "integrated" as a fusion target. The position and speed information in the fusion target may be the same as the detection information of the sensor, or may indicate the position and speed internally estimated in time series according to the detection information of the sensor.

[0024] The information of the fusion target, which is a result of the integration process of the detection information in the integration unit 30b, is output to the prediction update unit 30c. In addition, the fusion target and the detection information of the first sensor 10a and the second sensor 10b included in the fusion target are output to the integration history management unit 40. The integration history management unit 40 has a function of managing the generated or updated information of the fusion target in chronological order.

[0025] The prediction update unit 30c calculates an estimated value of the fusion target in the next integration process cycle using the execution cycle of the integration process, the position and speed information included in the detection information of the fusion target, and the speed, steering angle, yaw rate, etc. of the host vehicle output from the host vehicle behavior detection sensor 10c. For example, the prediction update unit 30c calculates the time change of the target position using at least the speed of the fusion target and the execution cycle of the integration process, and estimates the position of the fusion target based on the calculated time change of the position, thereby obtaining estimated information of the fusion target. Note that the position and speed of the target may be estimated by taking into account the turning behavior of the host vehicle using the speed, steering angle, yaw rate, etc. of the host vehicle. The estimated information of the fusion target obtained by the prediction update unit 30c is output to the grouping unit 30a as described above.

[0026] The integrated history management unit 40 receives the fusion target output from the integration processing unit 30 and the detection information of the first sensor 10a and the second sensor 10b included in the fusion target, and stores and manages the fusion target in time series for each target ID. The information managed by the integrated history management unit 40 is output to a reliability determination unit 50 and a detailed classification assignment unit 60.

[0027] The reliability determination unit 50 receives the integrated history of the fusion target from the integrated history management unit 40, determines the reliability of the fusion target based on the integrated history, and assigns the reliability information, which is the determination result, to the fusion target. In addition, the detailed type assignment unit 60 assigns information on the type of target indicated by the fusion target and its reliability as detailed type information to the fusion target. The information generated by the detailed type assignment unit 60 is output to the driving control device 2, and the control of the automobile, including automatic driving and driving assistance, is executed.

[0028] The reliability determination unit 50 of the first embodiment determines the reliability of the fusion target with respect to the type of target, such as a pedestrian or a vehicle, from a combination of the fusion target of the previous cycle obtained by the integrated history management unit 40 and the fusion target of the current cycle. Specifically, the reliability determination unit 50 determines the reliability of a specific type (pedestrian, car, bicycle, etc.) indicated by each fusion target according to the contents of the detection information of the fusion target obtained in the previous cycle and the contents of the detection information of the target of the fusion target of the current cycle. In this embodiment, as an example, the reliability is divided into at least three types, "confirmed", "past", and "split", in descending order of reliability. Note that one of the basic criteria for reliability determination is that the reliability is determined to be high when grouping is performed based on the detection information of the first sensor 10a. Another criterion is that the grouping of the current cycle is more reliable than the grouping of the previous cycle.

[0029] The "Definite" type, which has the highest reliability among the judgment results of the specific type, is judged when the detection information of the first sensor 10a is integrated as a fusion target in the current cycle. The reason why the "Definite" type reliability is the highest is because the detection information of the first sensor 10a, which has high reliability, is obtained in the current cycle and is the most reliable of the possible combinations of integration.

[0030] “Past”, which has the second highest reliability after “Definite”, is assigned when the detection information of the first sensor 10a is not grouped in the current cycle, but was grouped in the previous cycle.

[0031] The reason why the reliability of "Past" is the second highest after "Determined" is that while the consistency of grouping between the previous cycle and the current cycle is maintained, the detection information of the first sensor 10a is not grouped in the current cycle, unlike "Determined". For example, if the detection information of the first sensor 10a that was grouped in the previous cycle does not exist in the current cycle and the detection information of the first sensor 10a is not grouped, this means that the first sensor 10a no longer detects the target due to the target moving outside the detection range (detection range) of the first sensor 10a, and the detection information is no longer input to the sensor fusion device 1. In other words, if the detection information of the first sensor 10a that was grouped in the previous cycle does not exist in the current cycle, it is inevitable that the detection information will not be grouped in the current cycle, so it can be said that the consistency of grouping is maintained between multiple cycles. Therefore, "Past" is determined to have the second highest reliability after "Determined".

[0032] Of the three reliability categories, "split" is the lowest reliability. Specifically, for one fusion target, when the detection information of the first sensor 10a is not grouped in the current cycle, while the detection information of the first sensor 10a is grouped in the previous cycle, and further, when the detection information of the first sensor 10a exists elsewhere in the current cycle, the reliability is determined to be "split." The reason why "split" is lower in reliability than "past" is that, unlike "past," the consistency of grouping between multiple cycles is not maintained.

[0033] If the detection information of the first sensor of a certain fusion target that was grouped in the previous cycle exists as the detection information of the first sensor of another fusion target in the current cycle, based on the policy that the grouping in the current cycle is more correct, it is correct for the certain fusion target not to integrate its detection information. In other words, the grouping of the detection information of the first sensor in the previous cycle was actually incorrect, and the consistency of the grouping between multiple cycles is not maintained. Therefore, the reliability of "split" is judged to be lower than that of "past".

[0034] Next, a procedure for determining the reliability and assigning the detailed type information in the reliability determining unit 50 and the detailed type assigning unit 60 will be described with reference to the flowchart in Fig. 2. The procedure in this flowchart is executed for each target ID in each cycle of the integration process.

[0035] The integration history management unit 40 receives the grouping and integration results of the detection information from the integration processing unit 30 for each period, and manages the integration history for each period. The reliability determination unit 50 determines the reliability of the specific type of the fusion target obtained for each period. The detailed type assignment unit 60 assigns a detailed type, which is the result of the reliability determination, to the fusion target.

[0036] First, the reliability determination unit 50 receives the detection information obtained in the current cycle and determines whether the integration processing unit 30 has grouped the detection information with the fusion target associated with the target ID in the current cycle and assigned specific type information (step S100). If the determination result is positive (YES), the process proceeds to step S130, where the reliability information is assigned as "confirmed" for the grouped fusion target to which the specific type information has been assigned. On the other hand, if the determination in step S100 is negative (NO), the process proceeds to step S110.

[0037] In step S110, the reliability determination unit 50 receives the detection information obtained in the previous period from the integrated history management unit 40, and determines whether or not the detection information was grouped with the corresponding fusion target and specific type information was added in the previous period (step S110). If YES, the process proceeds to step S120. If NO, the reliability is not determined and the process ends (END).

[0038] If the determination result in step S110 is YES, then in step S120, it is determined whether or not the detection information relating to the specific type grouped and assigned in the previous cycle exists elsewhere in the current cycle (as a fusion target with a different target ID). If the determination result in step S120 is YES, then in step S140, the reliability of the specific type of the fusion target is determined to be "split", and detailed type information indicating that is assigned. On the other hand, if the determination result in step S130 is NO, then in step S150, the reliability of the specific type of the fusion target is determined to be "past", and detailed type information indicating that is assigned. In this manner, the procedure of determining the reliability and assigning the detailed type information is completed.

[0039] Next, with reference to FIGS. 3 to 6, it will be described how the determination procedure in FIG. 2 is executed assuming various scenarios.

[0040] Fig. 3 is a schematic diagram for explaining the procedure for determining the reliability and detailed type when a pedestrian 200 crosses in front of the host vehicle 210. The following explanation will be given assuming that time progresses in the order of (B) → (C) → (D) in Fig. 3.

[0041] In Fig. 3, the detection range CFOV of the first sensor (camera) 10a is shown as a dashed sector, and the detection range RFOV of the second sensor (radar) 10b is shown as a solid sector. The detection information C1 and C2 of the first sensor 10a are shown as circles, and the detection information R1 of the second sensor 10b is shown as a rectangle. The fusion target F1, which is generated by integrating multiple detection information and has detailed type information of "confirmed" for a specific type, is shown as a solid oval. The fusion target F2, which has been determined to be "split", is shown as a dotted oval.

[0042] 3B to 3D show the detection results of the first sensor 10a and the second sensor 10b at each moment (t=t1, t2, t3) and the integration results of the integration unit 30b. However, in order to facilitate the explanation of the reliability determination, Figs. 3B to 3D show only some of the integration results, and do not necessarily show a continuous period.

[0043] As shown in FIG. 3(A), when a pedestrian 200 crosses in front of the vehicle 210, in the situation of (B), the pedestrian 200 exists in an overlapping portion of the detection range CFOV of the first sensor 10a and the detection range RFOV of the second sensor 10b, and is detected by both sensors 10a and 10b at the same time. The first sensor 10a detects the pedestrian 200 as a pedestrian target C1 with target ID=1, and the second sensor 10b detects the pedestrian 200 as a target R1 with target ID=1 of unknown type. The grouping unit 30a groups the detection information of the pedestrian target C1 with target ID=1 of the first sensor 10a and the detection information of the target R1 with target ID=1 of unknown type of the second sensor 10b by assigning the target ID of target ID=1 based on the coincidence of the position information of the detection information. The integration unit 30b integrates the grouped detection information to generate a fusion target F1.

[0044] The integrated history management unit 40 holds the fusion target F1, to which the target ID=1 has been assigned, as the integration result for the current cycle (t=t1). The reliability determination unit 50 determines whether or not the detection information has been grouped and specific type information has been assigned to the fusion target F1 in the current cycle, as the operation of step S100 in Fig. 2. In this situation (B), a YES determination is made in step S100, and detailed type information, specific type=pedestrian, reliability="confirmed", is assigned to the fusion target F1.

[0045] After FIG. 3(B), when the pedestrian 200 moves further to the right and the integration processing unit 30 moves to the next period (t=t2), the situation moves to that shown in FIG. 3(C). In FIG. 3(C), the pedestrian 200 is still within the detection range CFOV of the first sensor 10a and within the detection range RFOV of the second sensor 10b. Therefore, the detection information of the first sensor 10a and the detection information of the second sensor 10b are grouped based on the coincidence of their position information and integrated into a fusion target F1 with a target ID=1. In step S100, a YES determination is made, a determination is made that the specific type is a pedestrian and the reliability is "confirmed", and the detailed type information is assigned to the fusion target F1.

[0046] After Fig. 3(C), when the pedestrian 200 crosses the host vehicle 210 further to the right and the integrated processing unit 30 moves to the next period (t=t3), the situation transitions to that shown in Fig. 3(D). In Fig. 3(D), the pedestrian 200 is still within the detection range CFOV of the first sensor 10a, but has moved out of the detection range RFOV of the second sensor 10b.

[0047] Although the pedestrian 200 has already moved outside the detection range RFOV of the second sensor 10b, the second sensor 10b detects the pedestrian 200 as an unknown type target R1 with target ID = 1 due to the internal interpolation function of the second sensor 10b. The internal interpolation is a function in which the sensor behaves as if it continues to detect the target by interpolating the target that moves outside the detection range at the detection end in order to prevent the control target from going undetected.

[0048] On the other hand, since the first sensor 10a includes the pedestrian 200 within the detection range CFOV, the first sensor 10a detects the pedestrian 200 and determines that the specific type is a pedestrian. However, since the position information of the detection information of the second sensor 10b does not match the position information of the detection information of the first sensor 10a, the detection information of the first sensor 10a is not grouped with the same target ID as the detection information of the second sensor 10b. Since the detection information of the first sensor 10a with target ID=1 is not grouped, it is grouped with another fusion target F2 with target ID=2, and is determined to be specific type=pedestrian and reliability="confirmed". Therefore, in the situation of FIG. 3(D), in step S100, a NO determination is made for the fusion target F1' with target ID=1, and the process proceeds to step S110. On the other hand, a YES determination is made for the fusion target F2 with target ID=2, and detailed type information is assigned with specific type=pedestrian and reliability="confirmed".

[0049] For the fusion target F1' with target ID=1, in the next step S110, it is determined whether or not the detection information was grouped and the specific type information was added to the fusion target F1 with target ID=1 in the previous period. In the case of FIG. 3(D), in FIG. 3(C) (t=t2), the specific type information of specific type=pedestrian was added to the fusion target F1 with target ID=1, so the determination is YES in step S110 and the process proceeds to step S120. Then, in step S120, since there is another fusion target F2 determined to be specific type=pedestrian, the determination is YES, and detailed type information of specific type=pedestrian and reliability="split" is added to the fusion target F1' with target ID=1. As described above, in the case of FIG. 3, the reliability is determined to be "split".

[0050] Next, another case will be described with reference to FIG. 4. FIG. 4(A) shows a case where the vehicle 210, which is turning left (turning left), approaches the pedestrian 200 walking parallel to the left side of the vehicle 210. FIG. 4(B) shows a situation before the vehicle 210 starts turning left, and the pedestrian 200 is present in an overlapping portion of the detection range CFOV of the first sensor 10a and the detection range RFOV of the second sensor 10b. The detection information of both sensors 10a and 10b shows the same position information. The pedestrian 200 is recognized as a target C1 with a target ID=1 and a specific type=pedestrian by the detection information of the first sensor 10a, and is recognized as a target R1 with a target ID=1 and a specific type=unknown by the detection information of the second sensor 10b. Therefore, the detection information of the first sensor 10a and the detection information of the second sensor 10b are grouped into the same target ID=1 based on the coincidence of the position information, and are integrated to generate a fusion target F1.

[0051] The integrated history management unit 40 stores and manages the fusion object F1 with the object ID=1 as data of the integrated history of the current period. In addition, the reliability determination unit 50 assigns detailed type information of specific type=pedestrian and reliability=“confirmed” to the fusion object F1 by judging YES in step S100.

[0052] Next, as shown in FIG. 4C, when the integration processing unit 30 moves to the next cycle (t=t2), the host vehicle 210 starts to turn left, and the pedestrian 200 approaches the left side of the host vehicle 210. Here, the pedestrian 200 is present in the detection range RFOV of the second sensor 10b, but is in a blind spot relative to the detection range CFOV of the first sensor 10a, and the pedestrian 200 is not present within the detection range CFOV. Therefore, in the cycle t=t2, only the detection information of the second sensor 10b is obtained, and the detection information is integrated as the target R1 of the specific type=unknown in the fusion target F1′ of the target ID=1.

[0053] The reliability determination unit 50 executes the determination of step S100 for the fusion target F1' with target ID=1, and determines NO. Next, in step 110, it is determined whether or not the detection information was grouped with the fusion target F1 in the previous cycle and specific type information was given. In this example, the determination is YES, and the process proceeds to step S120. Next, in step S120, it is determined whether or not there is other detection information related to the specific type that was grouped with the fusion target F1 in the previous cycle and determined to be the specific type in the current cycle (step S120). Here, as described in FIG. 4(C), since the type of the fusion target F1' is determined to be unknown, the determination is NO. As described above, in the case of FIG. 4, the reliability of the fusion target F1' with target ID=1 is determined to be "past".

[0054] 5 shows a procedure for determining whether or not a preceding vehicle 230 is traveling ahead of the vehicle 210 and passes a manhole 220 on the road surface during the journey. As shown in FIG. 5(A), the preceding vehicle 230 is traveling ahead of the vehicle 210 at approximately the same speed, but if there is a manhole 220 ahead of the preceding vehicle 230, the first sensor 10a does not detect the manhole 220 because it is blocked by the preceding vehicle 230. Similarly, the second sensor 10b does not detect the manhole 220 and only detects the preceding vehicle 230 as a target.

[0055] In this case, the detection information of the first sensor 10a and the detection information of the second sensor 10b have substantially the same position information, and the first sensor 10a detects the preceding vehicle 230 as a target C2c of a vehicle with a target ID=1, and the second sensor 10b detects it as a target R1c of an unknown type with a target ID=1. The grouping unit 30a groups the detection information of the target C2c of a vehicle with a target ID=1 of the first sensor 10a and the detection information of the target R2c of an unknown type with a target ID=1 of the second sensor 10b by assigning the target ID of the target ID=1 based on the coincidence of the position information of the detection information. The integration unit 30b integrates the grouped detection information to generate a fusion target F1.

[0056] The integrated history management unit 40 holds the fusion target F1, to which the target ID=1 has been assigned, as the integration result for the current cycle (t=t1). The reliability determination unit 50 determines whether or not the detection information has been grouped and specific type information has been assigned to the fusion target F1 in the current cycle, as the operation of step S100 in Fig. 2. In this situation (B), a YES determination is made in step S100, and detailed type information, specific type=vehicle, reliability="confirmed", is assigned to the fusion target F1.

[0057] After FIG. 5(B), when the preceding vehicle 230 further travels on the road surface and passes through the manhole 220, and the integration processing unit 30 moves to the next cycle (t=t2), the situation moves to that shown in FIG. 5(C). Here, FIG. 5(C) is assumed to be immediately after the preceding vehicle 230 passes through the manhole 220. In this case, the preceding vehicle 230 may be detected by the first sensor 10a and detected as a specific type=vehicle with a target ID=1, while the manhole 220 may be detected by the second sensor 10b and detected as a specific type=unknown with a target ID=1. In this case, the detection information (target C2c) of the first sensor 10a and the detection information (target R2m) of the second sensor 10b should not be grouped and integrated, but since the position information is approximately the same, they may be erroneously integrated as a fusion target F1' with a target ID=1.

[0058] 5(D) shows a case where the preceding vehicle 230 continues traveling and travels ahead, away from the manhole 220. The integration processing unit 30 proceeds to the next period (t=t3). In this case, even if the first sensor 10a detects the preceding vehicle 230 and obtains detection information indicating that the specific type is a vehicle, and the second sensor 10b detects the manhole 220 and obtains detection information indicating that the specific type is unknown, the two pieces of detection information have different position information, and are therefore integrated into different fusion targets (F2 (ID=2), F1″ (ID=1)).

[0059] For the fusion object F2, it is determined that the specific type is a vehicle, so a YES determination is made in step S100, and detailed type information of specific type=vehicle and reliability="confirmed" is assigned to the fusion object F2. On the other hand, for the fusion object F1'', it is determined that the specific type is unknown, so a NO determination is made in step S100, followed by a YES determination in step S110 and a YES determination in step S120, and detailed type information of specific type=vehicle and reliability="split" is assigned.

[0060] As described above, in the case of Fig. 5, at the time point (C), the detection information of the preceding vehicle 230 by the first sensor 10a and the detection information of the manhole 220 by the second sensor 10b are erroneously integrated, and detailed type information of specific type = vehicle and reliability = "confirmed" is erroneously given, but at the next time point (D), this is corrected to "split." Thus, according to the determination procedure of Fig. 2, even if an erroneous reliability determination is made once in one cycle, this is corrected in the next cycle, and the accuracy of the reliability determination can be guaranteed.

[0061] Yet another case will be described with reference to Fig. 6. Fig. 6 shows a case in which a pedestrian 200 is walking parallel to the vehicle 210 in front of the vehicle, but there is an obstacle 240 (such as a wall) in the way, and at a certain point in time the pedestrian 200 is blocked by the obstacle 240 and becomes invisible to the vehicle 210.

[0062] As shown in Fig. 6(A), when the vehicle 210 is traveling forward, but the pedestrian 200 is still located to the left front of the vehicle 210 and is not blocked by an obstacle 240, various detection information is as shown in Fig. 6(B). The first sensor 10a detects the pedestrian 200 and recognizes it as a target ID=1, specific type=pedestrian, while the second sensor 10b recognizes it as a target ID=1, specific type=unknown. The detection information of the first sensor 10a (target C1) and the detection information of the second sensor 10b (target R2) are integrated into a fusion target F1 (target ID=1) in the integration processing unit 30 because their position information matches.

[0063] Next, when the host vehicle 210 and the pedestrian 200 both move forward and an obstacle 240 blocks the space between the first sensor 10a and the pedestrian 200, a situation as shown in FIG. 6C is obtained as an example. The first sensor 10a cannot capture an image of the pedestrian 200, and no detection information of the first sensor 10a is obtained. On the other hand, the second sensor 10b continues to obtain detection information based on the above-mentioned internal interpolation function, etc., and the pedestrian is detected as a target R1 and integrated into the fusion target F1. In the situation of FIG. 6C, NO is determined in step S100, YES is determined in step S110, and NO is determined in step S120, and the specific type is determined to be a pedestrian and the reliability is determined to be "past".

[0064] As described above with various concrete examples such as Fig. 3 to Fig. 6, the sensor fusion device 1 of the first embodiment can determine the specific type of the fusion target with high reliability, thereby improving the accuracy of automatic driving and driving assistance. Specifically, the integration history management unit 40 manages the integration result obtained by integrating the detection information of each sensor obtained in each cycle, and compares the integration result obtained in the previous cycle with the integration result obtained in the current cycle, thereby making it possible to determine the specific type with high reliability.

[0065] [Second embodiment] Next, a target detection device according to a second embodiment will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is a functional block diagram of a target detection device (sensor fusion device 1) according to the second embodiment. In Fig. 7, the same components as those in the first embodiment (Fig. 1) are given the same reference numerals, and overlapping descriptions will be omitted below.

[0066] The sensor fusion device 1 of the second embodiment is different from the first embodiment in that the integration processing unit 30 has a movement prediction unit 30d in addition to the same components as the sensor fusion device 1 of the first embodiment. The movement prediction unit 30d predicts movement prediction information such as the moving speed and moving direction of the target according to the detection information of the first sensor 10a and the second sensor 10b and the detection information of the vehicle behavior detection sensor 10c. According to these prediction results of the movement prediction unit 30d, the grouping unit 30a and the integration unit 30b of the integration processing unit 30 execute grouping and integration. The movement prediction information predicted by the movement prediction unit 30d includes the moving speed and direction of the target, as well as acceleration, continuous deviation of position, and the direction of the target obtained from image recognition (for example, the direction of the face of a pedestrian, line of sight), and is not limited to a specific combination.

[0067] Next, a procedure for determining the reliability and assigning the detailed type information in the reliability determination unit 50 and the detailed type assignment unit 60 of the second embodiment will be described. The procedure itself is similar to that in the first embodiment (FIG. 2), but as described above, in the grouping and integration processing in the integration processing unit 30, movement prediction information is referred to.

[0068] A description will be given by applying it to a specific scenario with reference to Fig. 8. Here, as in Fig. 3, a case where a pedestrian 200 crosses in front of the host vehicle 210 will be described as an example. Substantially the same description is also applicable to the cases where the actions as shown in Figs. 4 to 6 are performed.

[0069] The situation in Fig. 8(B) is the same as the situation in Fig. 3(B). The grouping unit 30a adds the detection information of the pedestrian target C1 with target ID = 1 of the first sensor 10a and the detection information of the target R1 of unknown type with target ID = 1 of the second sensor 10b to the position information of the detection information, and groups them by giving the target ID of target ID = 1 based on the agreement of the movement prediction information. The integration unit 30b integrates this grouped detection information to generate a fusion target F1.

[0070] The situation in Fig. 8(C) is similar to that in Fig. 3(C). By grouping using not only the position information of the target but also the movement prediction information, the fusion target F1 with target ID=1, which had grouped the detection information of the pedestrian of the first sensor 10a up until the previous cycle (t=t1) in Fig. 8(C), can also integrate the detection information of the first sensor 10a in the current cycle (t=t2).

[0071] After FIG. 8(C), when the pedestrian 200 crosses the vehicle 210 further to the right and the integrated processing unit 30 moves to the next period (t=t3), the situation moves to that shown in FIG. 8(D). In FIG. 8(D), the pedestrian 200 is still within the detection range CFOV of the first sensor 10a, but has moved outside the detection range RFOV of the second sensor 10b. However, due to the internal interpolation function of the second sensor 10b, detection information of a specific type=unknown (target object R2) is obtained. On the other hand, the first sensor 10a detects the pedestrian 200 and obtains detection information of a specific type=pedestrian (target object C2) since the pedestrian 200 is included within the detection range CFOV.

[0072] Because the position information of the detection information of the second sensor 10b does not match the position information of the detection information of the first sensor 10a, the detection information of the first sensor 10a (target C2) is not grouped with the detection information of the second sensor 10b (target R2) under the same target ID. Target C2 is integrated into fusion target F1', and target R2 is integrated into another fusion target F2.

[0073] At this time, the grouping unit 30a and the integration unit 30b assign the same target ID = 1 as the fusion target F1 of the previous cycle to the fusion target F1' into which the detection information (target C2) of the first sensor 10a is integrated based on the degree of agreement with the previous cycle regarding the movement prediction information. A different target ID = 2 is assigned to the fusion target F2. For the fusion target F1', detailed type information of specific type = pedestrian and reliability = "confirmed" is given based on the target identification result by the first sensor 10a. For the fusion target F2, identification information of specific type = unknown is given because only the detection information of the second sensor 10b is obtained.

[0074] As described above, according to the second embodiment, in addition to obtaining the same effects as the first embodiment, by referring to the movement prediction information, it becomes possible to perform a more reliable determination of the specific type of the fusion target.

[0075] [others] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. Furthermore, a part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, other configurations may be added, deleted, or replaced with respect to a part of the configuration of each embodiment.

[0076] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits, or may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as programs, tables, files, etc. that realize each function can be stored in storage devices such as memories, hard disks, and SSDs (Solid State Drives), or in recording media such as IC cards, SD cards, DVDs, and BDs.

[0077] In addition, the control lines and information lines shown in the attached drawings are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines that are necessary for implementation. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0078] 1. Sensor fusion device 2... Operation control device 10a…First sensor 10b…Second sensor 10c…Vehicle behavior detection sensor 20... Observation information processing section 30... Integrated processing section 30a…Grouping section 30b…Integration Department 30c…Prediction Update Section 40…Integrated History Management Department 50…Reliability judgment section 200…pedestrians 210…Personal vehicle 220…Manhole 230… Leading vehicle 240…Obstacle

Claims

1. A target detection device that detects targets around a vehicle based on sensor information measured by a plurality of sensors, A first sensor; A second sensor; an integration processing unit that integrates detection information of the target by the first sensor and detection information of the target by the second sensor every period; an integration history management unit that manages the integration results of the integration processing unit in chronological order as an integration history; a reliability determination unit that determines reliability of a type of the target based on the integrated history and detection information of the target by the first sensor and the second sensor; Equipped with The detection information of the target by the first sensor includes type information of the target, The reliability determination unit is a first step of determining whether or not detection information has been grouped and specific type information has been assigned to one fusion target in the current period; a second step of determining whether or not the detection information was grouped and specific type information was assigned to the corresponding fusion target in the previous period; a third step of determining whether or not there is other detection information related to the specific type grouped and assigned in the previous period in the current period; configured to execute A target detection device comprising:

2. the first sensor and the second sensor are configured to output position information of the target as detection information; The target detection device according to claim 1 , wherein the integration processing unit performs the integration in accordance with the position information.

3. The target detection device according to claim 1 , wherein the integration history management unit manages, as the integration result, a target ID and a target type of the fusion target in chronological order.

4. 2. The target detection device according to claim 1, wherein the reliability determining unit determines a reliability of the determination of the specific type information in accordance with the results of the determinations in the first to third steps.

5. the first sensor and the second sensor are configured to output position information of the target as detection information; The integration processing unit further includes a movement prediction unit that predicts a movement of the target, The target detection device according to claim 1 , wherein the integration processing unit executes the integration in accordance with the position information and a prediction of a movement of the target by the movement prediction unit.

6. The target detection device according to claim 1 , further comprising a host vehicle behavior detection sensor that detects a behavior of the host vehicle.

Images