Object identification device and object identification program

Abstract

To provide a technology that enables better object recognition than conventional methods. [Solution] An object identification device (500) mounted on a vehicle includes a detection information acquisition unit (522) that acquires object detection information based on captured image information acquired by a camera mounted on the vehicle, and a ghost determination unit (505) that determines whether the detection information is actual object detection information corresponding to the detection of an actual object or ghost detection information corresponding to the detection of a non-existent ghost. The ghost determination unit determines that if the object is cut off from the captured image or the distance to the object is less than or equal to a threshold distance when the object is first detected, the detection information corresponding to the object is ghost detection information.

Description

Technical Field

[0001] The present invention relates to an object recognition device mounted on a vehicle and an object recognition program executed by such an object recognition device.

Background Art

[0002] Patent Document 1 discloses an image processing device that can correctly identify whether or not a person is a pedestrian even when the entire body of a pedestrian existing in the surroundings cannot be detected. Specifically, such an image processing device includes a detection unit, an identification area correction amount calculation unit, an identification area correction unit, and an identification processing unit.

[0003] The detection unit detects image information of the external environment. The identification area correction amount calculation unit calculates a correction amount for an identification area that identifies a three-dimensional object from the image information detected by the detection unit. The identification area correction unit corrects the identification area of the image information for identifying a three-dimensional object based on the correction amount calculated by the identification area correction amount calculation unit. The identification processing unit performs an identification process on a three-dimensional object within the identification area corrected by the identification area correction unit. Here, the detection unit detects the distance from the imaging unit to the three-dimensional object, and when the distance detected by the detection unit is equal to or less than a distance set value and it is determined that the upper part of the three-dimensional object does not enter the identification area, the identification area correction unit performs correction to expand the upper area of the identification area.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, Patent Document 1 does not address cases where object identification cannot be performed correctly even when the identification area is widened (for example, unnecessary operation due to misrecognition). The present invention has been made in view of the circumstances exemplified above. That is, the present invention provides a technology that enables object identification to be performed better than in the conventional method. [Means for solving the problem]

[0006] The object identification device (500) mounted on a vehicle according to claim 1 is A detection information acquisition unit (522) acquires object detection information based on image information captured by a camera (2) mounted on the vehicle, A ghost determination unit (505) determines whether the detection information corresponds to the detection of an actual object or to the detection of a ghost that is not an actual object. It has, When the ghost detection unit first detects an object, if the object is cut off from the captured image or the distance to the object is less than or equal to a threshold distance, it determines that the detection information corresponding to the object is the ghost detection information. The object identification program executed by the object identification device according to claim 8 is: The process performed by the object identification device is as follows: A detection information acquisition process that acquires object detection information based on image information captured by a camera (2) mounted on the vehicle, A ghost determination process that determines whether the detection information corresponds to the detection of the object as a physical entity or to the detection of a ghost that is not a physical entity, Includes, In the ghost detection process described above, if the object is initially detected and is partially obscured from the captured image or the distance to the object is less than or equal to a threshold distance, the detection information corresponding to the object is determined to be the ghost detection information.

[0007] In addition, each element in the application documents may be given a reference numeral in parentheses. However, such reference numerals merely indicate one example of the correspondence between the element and the specific means described in the embodiments described later. Therefore, the present invention is not limited in any way by the notation of the above reference numerals. [Brief explanation of the drawing]

[0008] [Figure 1] This block diagram shows a schematic configuration of an in-vehicle system to which one embodiment of the present invention is applied. [Figure 2] This is a block diagram showing a schematic functional configuration of an object identification device according to one embodiment of the present invention. [Figure 3] This is a schematic diagram illustrating an example of a situation where a pedestrian is cut off from the captured image. [Figure 4] Figure 2 is a schematic diagram showing an example of the operation of the object identification device. [Figure 5] Figure 2 is a schematic diagram showing an example of the operation of the object identification device. [Figure 6] Figure 2 is a schematic diagram showing an example of the operation of the object identification device. [Figure 7] Figure 2 is a schematic diagram showing an example of the operation of the object identification device. [Figure 8] Figure 2 is a schematic diagram showing an example of the operation of the object identification device. [Figure 9] Figure 2 is a flowchart illustrating an example of the operation of the object identification device shown. [Modes for carrying out the invention]

[0009] (Embodiment) Hereinafter, exemplary embodiments or specific examples of the present invention will be described with reference to the drawings as appropriate. It should be noted that the following embodiments, their modifications, and the corresponding drawings are schematic or simplified for the purpose of concisely explaining the content of the present invention, and do not in any way limit the content of the present invention. Therefore, it goes without saying that the descriptions in the drawings do not necessarily correspond to the specific device configurations actually manufactured and sold. In other words, unless explicitly limited by the applicant in the application history, the present invention should not be interpreted restrictively by the descriptions in the drawings and the corresponding descriptions of the device configurations, functions, or operations described below.

[0010] (In-vehicle system configuration) First, referring to Figure 1, the in-vehicle system 1 is configured to perform various operations in a vehicle when mounted on it. Hereinafter, the vehicle on which the in-vehicle system 1 is mounted will be referred to as "the vehicle itself." In this embodiment, the vehicle itself is a so-called ordinary automobile capable of traveling on roads, and has a box-shaped body.

[0011] The in-vehicle system 1 is configured to perform various operations, including motion control operations and related notification operations, based on the detection results of objects around the vehicle using the camera 2 and radar sensor 3 mounted on the vehicle. Specifically, in this embodiment, the in-vehicle system 1 has the configuration of a so-called driving automation system. "Driving automation" is a concept that includes driver assistance and autonomous driving.

[0012] That is, the in-vehicle system 1 is configured to be capable of realizing a driving automation level corresponding to at least one of levels 1 to 5 defined in the standard "SAE J3016" published by SAE International. SAE is the abbreviation of Society Of Automotive Engineers. Level X in "SAE J3016" is hereinafter simply referred to as "SAE level X". X is any one of 0 to 5. SAE level 0 is called manual driving. SAE level 1 is called driving assistance. SAE level 2 is called advanced driving assistance. SAE level 3 is called conditional automated driving. SAE level 4 is called highly automated driving. SAE level 5 is called full automated driving.

[0013] In the present embodiment, the in-vehicle system 1 is configured to be capable of continuously executing at least one of the vertical vehicle motion control subtasks and the lateral vehicle motion control subtasks included in the dynamic driving task. The vertical vehicle motion control subtasks are starting, acceleration / deceleration, and stopping. The lateral vehicle motion control subtask is steering. Specifically, the in-vehicle system 1 is configured to be capable of realizing, for example, AEB or AES. AEB is the abbreviation of Autonomous Emergency Braking and is also called collision damage mitigation braking. AES is the abbreviation of Automatic Emergency Steering or Autonomous Emergency Steering and is also called automatic steering avoidance.

[0014] The camera 2 includes an image sensor such as a CCD or a CMOS and is mounted at a predetermined position in the host vehicle to photograph the surroundings of the host vehicle. CCD is the abbreviation of Charge Coupled Device. CMOS is the abbreviation of Complementary Metal Oxide Semiconductor. In the present embodiment, at least a front camera is mounted on the host vehicle as the camera 2. The front camera is provided so as to be able to photograph the road surface of the road ahead of the host vehicle, an object located above such a road surface, and the foreground in front of and on the front side of the host vehicle.

[0015] The radar sensor 3 is configured to detect an object around the host vehicle based on the transmission and reception results of radar waves, which are radio waves in the millimeter-wave band. In the present embodiment, at least a front radar sensor for detecting an object in front of the host vehicle is mounted on the host vehicle as the radar sensor 3. The front radar sensor is attached to the front end portion of the vehicle body of the host vehicle.

[0016] In addition to the camera 2 and the radar sensor 3, the in-vehicle system 1 includes a vehicle state sensor 4, a vehicle control ECU 5, a motion control device 6, a notification device 7, and a communication device 8. ECU is an abbreviation for Electronic Control Unit. The camera 2, the radar sensor 3, the vehicle state sensor 4, the vehicle control ECU 5, the motion control device 6, the notification device 7, and the communication device 8 are connected via an in-vehicle network so as to be able to exchange information or signals.

[0017] The vehicle state sensor 4 is provided to detect various quantities related to the driving state of the host vehicle. The "driving state" includes the driving operation state, the driving behavior state, and the driving environment state of the host vehicle. The "driving operation state" is the state of the driving operation input of the host vehicle by the driver or the vehicle control ECU 5 of the host vehicle, and includes, for example, the accelerator opening, the brake operation amount, the steering amount, the shift range, and the like. The "driving behavior state" is the state related to the movement or behavior of the host vehicle, and includes, for example, the vehicle speed, the acceleration, the yaw rate, and the like. The "driving environment state" is different from the object presence state that is the detection target by the camera 2 or the radar sensor 3 among the environments around the host vehicle, and includes, for example, the illuminance around the host vehicle, the weather, and the like. That is, the vehicle state sensor 4 is a general term for well-known in-vehicle sensors such as an accelerator position sensor, a vehicle speed sensor, an outside air temperature sensor, and the like.

[0018] The vehicle control ECU 5 is configured to control the operation of various parts of the vehicle based on the object detection results using the camera 2 and radar sensor 3, and the detection results of the vehicle's driving state using the vehicle state sensor 4. Specifically, the vehicle control ECU 5 is a driving automation ECU, also known as a driving assistance ECU or autonomous driving ECU, and controls the above-mentioned detection results and driving assistance operations or autonomous driving operations and various accompanying notification operations based on those results.

[0019] In this embodiment, the vehicle control ECU 5 has a configuration as an in-vehicle microcomputer comprising at least a processor 51 and a memory 52. ​​The processor 51 comprises at least one arithmetic unit having a configuration as a CPU or MPU, and peripheral circuits (e.g., a timer circuit). The memory 52 includes at least RAM and ROM and / or non-volatile rewritable memory from among various non-transitional physical storage media such as ROM, RAM, and non-volatile rewritable memory. Non-volatile rewritable memory is a storage device that allows information to be rewritten when the power is on, but keeps information unrewritable when the power is off, and is, for example, flash memory.

[0020] In this embodiment, the vehicle control ECU 5 is configured such that the processor 51 reads a computer program from the memory 52 and executes it to realize a predetermined function for recognizing objects around the vehicle. The term "object" is a concept that includes not only three-dimensional "objects" but also two-dimensional detection targets such as road markings and road lane markings. The memory 52 stores the above-mentioned computer program along with various data necessary for its execution, such as initial values, maps, and lookup tables.

[0021] The motion control device 6 is configured as an on-board microcomputer that controls the operation of the vehicle's drive force generation mechanism, drive force transmission mechanism, braking mechanism, steering mechanism, etc., based on control signals generated and output by the vehicle control ECU 5. In other words, the motion control device 6 is provided to perform longitudinal and / or lateral motion control of the vehicle. More specifically, the motion control device 6 is configured to perform at least a portion of the motion control of the vehicle, such as starting, acceleration / deceleration, braking, stopping, and steering.

[0022] The notification device 7 is equipped with display devices, audio output devices, etc., for providing various information and warnings to the occupants of the vehicle. Display devices may include meters, meter displays, center information displays, head-up displays, electronic mirrors, etc.

[0023] The communication device 8 is an in-vehicle communication module, also known as a DCM, and is configured to communicate information with an external server Z via base stations in the vicinity of the vehicle using wireless communication compliant with communication standards such as LTE or 5G. DCM stands for Data Communication Module. LTE stands for Long Term Evolution. 5G stands for 5th Generation. The communication device 8 is configured to acquire various information such as road traffic information including congestion information and the latest map information from the external server Z and output it to the vehicle control ECU 5 and the notification device 7.

[0024] (Object identification device) Figure 2 shows an example of the functional block configuration of the object identification device 500, which is realized by the processor 51 in the vehicle control ECU 5 shown in Figure 1 executing a computer program. That is, the object identification device 500 shown in Figure 2 is realized by the processor 51 reading and executing the object identification program according to the present invention from the memory 52. ​​This object identification device 500 identifies objects included in the captured image based on the image captured by the camera 2 mounted on the vehicle ahead of the vehicle's movement.

[0025] As shown in Figure 2, the object identification device 500, as a functional configuration realized by executing a computer program, includes a sensor information acquisition unit 501, an image target information acquisition unit 502, a radar target information acquisition unit 503, a fusion unit 504, a ghost detection unit 505, and an identification information output unit 506. The details of these will be described below.

[0026] The sensor information acquisition unit 501 acquires detection information from the vehicle condition sensor 4. Specifically, the sensor information acquisition unit 501 receives signals or information from the vehicle condition sensor 4 that correspond to the detection results of the vehicle's driving operation state, driving behavior state, and driving environment state, and stores them for a predetermined capacity.

[0027] The image target information acquisition unit 502 is configured to acquire image target information, that is, information related to the results of detection and recognition of image targets. An "image target" is a target detected or recognized based on the captured image information acquired by the camera 2. In this embodiment, the image target information acquisition unit 502 acquires, or generates, image target information by performing image processing and image recognition processing on the captured image information received from the camera 2.

[0028] More specifically, the image target information acquisition unit 502 includes an image information acquisition unit 521 and a detection information acquisition unit 522. The image information acquisition unit 521 acquires captured image information from the camera 2, that is, it receives captured image information from the camera 2 and stores it for a predetermined amount of time. The detection information acquisition unit 522 acquires object detection information based on the captured image information. That is, the detection information acquisition unit 522 performs image processing and image recognition processing on the captured image information. The image recognition processing includes identification of the type of target and the tracking status of the detected object. The tracking status includes a tracking established state and a lost state. The tracking established state is a state in which the same object is continuously detected. The lost state is when the tracking established state is interrupted.

[0029] The radar target information acquisition unit 503 is configured to acquire radar target information, that is, detection information of radar targets. A "radar target" is a target detected based on the transmission and reception results of radar waves by the radar sensor 3. In this embodiment, the radar target information acquisition unit 503 acquires, or generates, radar target information by receiving the transmission and reception information of radar waves from the radar sensor 3 and performing signal processing.

[0030] The fusion unit 504 acquires object recognition information using so-called sensor fusion technology, which integrates image target information and radar target information. A target that is both an image target and a radar target, and in which the integration of image target information and radar target information is achieved, is referred to as a "fusion target." Since the image target recognition technology, the radar target detection or recognition technology, and the sensor fusion technology are already well-known technologies at the time of filing this application, further detailed explanations are omitted.

[0031] The ghost detection unit 505 determines whether the object detection information is real object detection information or ghost detection information. "Detection information" includes image target information, radar target information, and fusion information as a result of integrating these. "Real object detection information" is detection information corresponding to the detection of an object as a physical entity. In contrast, "ghost detection information" is detection information corresponding to the detection of a ghost that is not a physical entity. The determination that the detection information corresponds to ghost detection will be referred to as "ghost determination" below. Details of the ghost detection unit 505 and ghost determination will be described later.

[0032] Here, "ghost" refers to the phenomenon in which detection information appears even though an object does not actually exist, and can also be called "false detection" or "misrecognition." The definition of such "ghost" is also described in the "FY2022 Demonstration and Support Project for CASE Response such as Unmanned Autonomous Driving (Advanced Mobility Service Research and Development and Social Demonstration Project for Autonomous Driving Level 4 etc. (Theme 1: Initiatives to realize autonomous driving services with remote monitoring only (Lv4) in limited areas and vehicles in FY2022))" published by the National Institute of Advanced Industrial Science and Technology (AIST) in June 2023.

[0033] The identification information output unit 506 generates object identification information based on the processing results from the image target information acquisition unit 502, the radar target information acquisition unit 503, the fusion unit 504, and the ghost detection unit 505. The identification information output unit 506 also outputs the generated object identification information to applications for automated driving operations in the vehicle control ECU 5, as well as to the motion control device 6 and the notification device 7.

[0034] (Ghost detection unit) In this embodiment, the ghost detection unit 505 performs ghost detection based on the manner of initial detection of an object, and appropriately cancels the ghost detection according to the driving scene of the vehicle. "Initial detection" typically refers to the first detection of an object during the current trip, but also includes cases where the object is detected again after being lost. "Trip" refers to the period from when the vehicle's ignition switch is turned on until it is turned off. Specifically, the ghost detection unit 505 has a ghost affirmation detection unit 551 and a ghost denial detection unit 552.

[0035] The ghost detection unit 551 performs ghost detection based on the initial detection characteristics of the object. Specifically, the ghost detection unit 551 includes a first affirmation condition determination unit 5511, a second affirmation condition determination unit 5512, and a third affirmation condition determination unit 5513.

[0036] The first affirmation condition determination unit 5511 determines the first affirmation condition in ghost detection, that is, whether or not the object is cut off from the captured image during initial detection. Such cut-off determination can be performed, for example, by determining whether or not the lower edge of the detection rectangle or bounding box surrounding the object overlaps with the outer frame of the captured image. The second affirmation condition determination unit 5512 determines the second affirmation condition in ghost detection, that is, whether or not the distance to the object during initial detection is less than or equal to a threshold distance (e.g., 8m). The third affirmation condition determination unit 5513 determines whether or not the logical OR of the first affirmation condition and the second affirmation condition, which are used as ghost detection conditions, is successful, and sets a close-range initial detection ghost flag according to the result of this determination. In other words, the third affirmation condition determination unit 5513 sets the close-range initial detection ghost flag if either the first or second affirmation condition is met.

[0037] The ghost denial determination unit 552 is configured to cancel the ghost detection according to the driving scene of the vehicle. Specifically, the ghost denial determination unit 552 has a first denial condition determination unit 5521, a second denial condition determination unit 5522, a third denial condition determination unit 5523, a fourth denial condition determination unit 5524, a fifth denial condition determination unit 5525, and a sixth denial condition determination unit 5526. The first to fifth denial condition determination units 5521 to 5525 each determine the first to fifth denial conditions, which are the conditions for canceling the ghost detection, i.e., ghost denial. The sixth denial condition determination unit 5526 determines whether the logical OR of the first to fifth denial conditions, which are the conditions for canceling the ghost detection, is successful, and resets the short-range initial detection ghost flag according to the result of this determination. In other words, the sixth negation condition determination unit 5526 resets the short-range initial detection ghost flag based on the determination result of the first negation condition, etc.

[0038] The first negation condition determination unit 5521 determines whether the detected object has remained in view of the captured image for a predetermined period of time. The second negation condition determination unit 5522 determines whether the movement speed of the detected object exceeds a threshold speed (e.g., 0 km / h). The third negation condition determination unit 5523 determines whether the detected object is a fusion target and whether the movement speed of the object detected by the radar sensor 3 exceeds a threshold speed (e.g., 0 km / h). The threshold speed in the second negation condition determination unit 5522 and the threshold speed in the third negation condition determination unit 5523 may be the same or different.

[0039] The fourth negation condition determination unit 5524 determines whether the turning radius of the vehicle is below a threshold. The fifth negation condition determination unit 5525 determines whether the detected object is a fusion target and whether a threshold time has been maintained since sensor fusion was established. The sixth negation condition determination unit 5526 resets the short-range initial detection ghost flag if any one of the first to fifth negation conditions is met.

[0040] (Operation overview) The following describes the operation of the device configuration according to this embodiment, along with the effects achieved by the device configuration and the methods and programs executed thereunder. In the following description, the device configuration according to this embodiment, and the methods and programs executed thereunder, may be collectively referred to as "this embodiment."

[0041] The vehicle control ECU 5 recognizes image targets based on sensor information acquired from the vehicle state sensor 4 and image information acquired from the camera 2. The vehicle control ECU 5 also detects or recognizes radar targets based on sensor information acquired from the vehicle state sensor 4 and radar wave transmission / reception information acquired from the radar sensor 3. The vehicle control ECU 5 then performs sensor fusion to integrate the image target information and the radar target information. In this way, objects in the vehicle's path are detected or recognized.

[0042] An object that is detected or recognized may be selected as a control target in collision avoidance support operations, etc. Note that the term "control target" used herein refers to the target of collision avoidance, etc., in collision avoidance support operations, etc., and differs in usage from the general technical term "control target," such as the fuel injection amount in air-fuel ratio control. However, this is the normal usage in the technical field to which the present invention belongs.

[0043] In target recognition using camera 2, the near-range region is an area with a high risk of misrecognition, such as when the target is cut off from view. In other words, for image targets, if the lower edge is cut off or the detection is at a very close distance during the initial detection, there is a high possibility that it is a ghost. Therefore, the ghost detection unit 505 determines that if the object is cut off from view in the captured image or the distance to the object is below a threshold distance during the initial detection when the object is first detected, the detection information corresponding to that object is ghost detection information. This makes it possible to effectively suppress unnecessary operations such as collision avoidance support due to misrecognition.

[0044] However, as shown in Figure 3, pedestrians Pd may be initially detected in an image region G corresponding to the field of view of camera 2, but partially obscured. Such initial detection occurs, for example, when a pedestrian Pd suddenly appears from the side of the vehicle ahead of its path, or when the vehicle turns in a manner that brings it close to the pedestrian Pd, causing the pedestrian Pd to enter the field of view of camera 2 from the side. Such pedestrians Pd should be identified as targets for control, not as ghosts.

[0045] Figures 4 to 8 illustrate driving scenarios of the vehicle in which such initial detection states with a low probability of misrecognition are likely to occur. In these figures, the vehicle is indicated by the symbol Vs. Furthermore, within the imaging area R of the front camera, camera 2, the area in front of the short-range determination line L, i.e., the side close to the vehicle, is defined as the short-range range Rn, and the area behind the short-range determination line L, i.e., the side further away from the vehicle, is defined as the non-short-range range Rf.

[0046] Figure 4 shows a scenario in which another vehicle Vt enters the short-range area Rn from the side while the vehicle is moving forward. In such a scenario, the object may also be a pedestrian Pd. Figure 5 shows a scenario in which another vehicle Vt cuts in front of the vehicle from an adjacent lane, for example, by changing lanes, while the vehicle is moving forward. In these scenarios, objects such as the other vehicle Vt approach from outside the field of view of camera 2 and are initially detected in the vicinity of the field of view.

[0047] Figure 6 shows a scenario where, while the vehicle is moving forward, another vehicle Vt, which was detected in the non-short-range Rf, is temporarily lost due to obstruction by another vehicle, etc., and then re-detected in the short-range Rn. Figure 7 shows a scenario where another vehicle Vt is initially detected in the non-short-range Rf immediately after the vehicle's ignition switch is turned on. Such a situation occurs, for example, when another vehicle Vt enters the non-short-range Rf while the vehicle is parked.

[0048] Figure 8 shows a scene in which a pedestrian Pd is initially detected in the vicinity of the field of view due to the vehicle's turning. Such a scene could also occur if the object is another vehicle Vt.

[0049] In these scenes, the possibility of misrecognition is low, so it is necessary to recognize or identify initial detected objects such as other vehicles Vt and pedestrians Pd as targets for control. Therefore, the ghost detection unit 505 The state in which the object is not cut off from the captured image continues for a predetermined period of time. The object's velocity exceeds the threshold velocity. Sensor fusion is achieved, and the movement speed of the object detected by radar sensor 3 exceeds the threshold speed. The turning radius of your vehicle is below the threshold. The threshold time continues after sensor fusion is established. If any of the following conditions are met, the detection information is determined to be entity detection information.

[0050] For example, if a predetermined time has elapsed since the bottom edge cutoff flag corresponding to the occurrence of bottom edge cutoff was turned off, the object can be recognized with sufficient feature quantity, and the possibility of misrecognition is low. Therefore, the ghost detection unit 505 resets the short-range initial detection ghost flag when a predetermined time has elapsed since the bottom edge cutoff flag was turned off. This makes it possible to accurately identify the object to be controlled in the driving scenes corresponding to Figures 4 to 8.

[0051] Furthermore, the objects that are misrecognized often have features such as stationary objects like manholes or backgrounds. Therefore, the ghost detection unit 505 resets the short-range initial detection ghost flag when the detected object is a moving target, that is, when the speed of the image target as the detected object, or the speed of the radar target coupled with the image target, exceeds a threshold speed. This makes it possible to accurately identify the object to be controlled in the driving scenes corresponding to Figures 4, 5, and 8.

[0052] Furthermore, during vehicle turns, the background changes frequently, making it less likely for misrecognition to persist. Therefore, the ghost detection unit 505 resets the short-range initial detection ghost flag when the vehicle's turning radius is below a threshold. This makes it possible to accurately identify the object to be controlled in the driving scene corresponding to Figure 8.

[0053] Furthermore, objects that have been sufficiently detected, such as those for which sensor fusion has been established for a certain period of time, are less likely to be misrecognized. Therefore, the ghost detection unit 505 resets the short-range initial detection ghost flag when a threshold time has elapsed since sensor fusion was established. This makes it possible to accurately identify the object to be controlled in the driving scenes corresponding to Figures 4 to 8.

[0054] (Example of operation) Figure 9 shows a specific example of the operation for identifying whether an object detected as an image target is a controlled object. In the flowchart shown in Figure 9, "S" is an abbreviation for "step". The processor 51 executes the object identification method according to this embodiment by reading and starting the computer program according to this embodiment from the memory 52. ​​Once the program is started, the processor 51 executes the processing from step 101 onwards.

[0055] First, in step 101, the processor 51 acquires sensor information, which is detection information from the vehicle condition sensor 4. In step 102, the processor 51 acquires captured image information from the camera 2. In step 103, the processor 51 determines whether the setting conditions for the short-range initial detection ghost flag FG are met. The setting conditions for the short-range initial detection ghost flag FG are that the ghost determination conditions described above, namely the first affirmative condition or the second affirmative condition, are met.

[0056] If the setting condition for the short-range initial detection ghost flag FG is met (i.e., step 103 = YES), the processor 51 proceeds to step 104. In step 104, the processor 51 sets the short-range initial detection ghost flag FG (i.e., FG = ON or "1"). Conversely, if the setting condition for the short-range initial detection ghost flag FG is not met (i.e., step 103 = NO), the processor 51 skips the processing in step 104 and proceeds to step 105. In this case, the short-range initial detection ghost flag FG is reset (i.e., FG = OFF or "0").

[0057] In step 105, the processor 51 determines whether the reset condition for the short-range initial detection ghost flag FG is met. The reset condition for the short-range initial detection ghost flag FG is that one of the ghost detection release conditions described above, i.e., the first to fifth negation conditions, is met.

[0058] If the reset condition for the short-range initial detection ghost flag FG is met (i.e., step 105 = YES), the processor 51 proceeds to step 106. In step 106, the processor 51 resets the short-range initial detection ghost flag FG (i.e., FG = OFF or "0"). Conversely, if the reset condition for the short-range initial detection ghost flag FG is not met (i.e., step 105 = NO), the processor 51 skips the processing in step 106 and proceeds to step 107. In this case, the short-range initial detection ghost flag FG is set (i.e., FG = ON or "1").

[0059] In step 107, the processor 51 determines whether the near-range initial detection ghost flag FG is set. If the near-range initial detection ghost flag FG is set (i.e., step 107 = YES), the processor 51 proceeds to step 108. On the other hand, if the near-range initial detection ghost flag FG is reset (i.e., step 107 = NO), the processor 51 proceeds to step 109.

[0060] In step 108, the processor 51 suppresses the operation of collision avoidance support and other functions that target objects initially detected in the short-range Rn. On the other hand, in step 109, the processor 51 permits the operation of collision avoidance support and other functions that target objects initially detected in the short-range Rn.

[0061] Thus, according to this embodiment, unnecessary operations such as collision avoidance support in situations where misrecognition is possible can be effectively suppressed, while the identification of the controlled object in situations where the possibility of misrecognition is low can be effectively achieved. As a result, it is possible to suppress the permanent misrecognition of objects initially detected in the vicinity of the vehicle, and to improve the performance of collision avoidance support and other functions while suppressing misrecognition even in near-field scenes where the entire object cannot be seen by camera 2.

[0062] (modified version) The present invention is not limited to the embodiments and specific examples described above. Therefore, the embodiments and the like can be modified as appropriate. Representative modifications are described below. In the description of the modifications below, the differences from the embodiments and the like will be mainly described. Also, parts that are the same or equivalent in the embodiments and the modifications below are denoted by the same reference numerals. Therefore, in the description of the modifications below, with respect to components that have the same reference numerals as in the embodiments and the like, the descriptions in the embodiments and the like can be appropriately referenced unless there is a technical contradiction or additional explanation to be given.

[0063] The present invention is not limited to the specific uses or device configurations shown in the embodiments described above. That is, for example, the vehicle may be a so-called ordinary automobile or a large automobile. In other words, there are no particular limitations on the shape or size of the vehicle body.

[0064] Object detection sensors other than camera 2 are not limited to radar sensor 2. That is, for example, a laser radar sensor or the like can be used instead of or in conjunction with radar sensor 2. A laser radar sensor may also be called a LiDAR sensor. LiDAR is an abbreviation for Light Detection and Ranging or Laser Imaging Detection and Ranging.

[0065] The computer program according to the present invention, which enables the execution of various operations, procedures, or processes described in the above embodiment, can be downloaded or upgraded via V2X communication using the communication device 8. V2X is an abbreviation for Vehicle to X. Alternatively, such a computer program can be downloaded or upgraded via terminal equipment installed at the vehicle's manufacturing plant, maintenance factory, dealership, etc. The storage location for such a computer program may be a memory card, optical disk, magnetic disk, etc.

[0066] The vehicle control ECU 5 may be configured with all or part of a digital circuit, such as an ASIC or FPGA, that is capable of realizing the functions or operations described above. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field Programmable Gate Array. In other words, the vehicle control ECU 5 can have both an on-board microcomputer and a digital circuit.

[0067] Specifically, for example, the ghost affirmation determination unit 551 may be composed of a determination step corresponding to the first affirmation condition determination unit 5511 and a determination step corresponding to the second affirmation condition determination unit 5512, as in the embodiment described above. In this case, the third affirmation condition determination unit 5513 may be realized by a flowchart structure in which the determination step corresponding to the first affirmation condition determination unit 5511 and the determination step corresponding to the second affirmation condition determination unit 5512 are connected in series. Alternatively, for example, the ghost affirmation determination unit 551 may be composed of a determination logic corresponding to the first affirmation condition determination unit 5511, a determination logic corresponding to the second affirmation condition determination unit 5512, and a third affirmation condition determination unit 5513 as an OR gate that takes the logical OR of these. The same applies to the ghost negation determination unit 552.

[0068] Thus, each of the above functional configurations and processes may be realized by a dedicated computer provided by configuring a processor 51 and memory 52 programmed to execute one or more functions embodied by a computer program. Alternatively, each of the above functional configurations and processes may be realized by a dedicated computer provided by configuring a processor 51 with one or more dedicated hardware logic circuits. Alternatively, each of the above functional configurations and processes may be realized by one or more dedicated computers configured by a combination of one or more processors 51 programmed to execute one or more functions, one or more memories 52, and one or more other processors 51 configured with one or more hardware logic circuits. Furthermore, the computer program may be stored in a computer-readable non-transitional substantial storage medium as instructions to be executed by the computer. That is, each of the above functional configurations and processes can also be represented as a computer program including procedures for realizing them, or as a non-transitional substantial storage medium storing said computer program.

[0069] The detection or recognition of image targets may be performed by the ECU installed in camera 2, or by the vehicle control ECU 5. That is, the image target information acquisition unit 502 may receive image target information from the ECU installed in camera 2, or it may generate it itself. The same applies to the radar target information acquisition unit 503.

[0070] It goes without saying that the elements constituting the above embodiments are not necessarily essential unless explicitly stated to be particularly essential or considered to be fundamentally essential. Furthermore, when numerical values ​​such as the number of components, numerical values, quantities, or ranges are mentioned, the present invention is not limited to those specific numbers unless explicitly stated to be particularly essential or considered to be fundamentally limited to those specific numbers. Similarly, when the shape, direction, positional relationship, etc., of components are mentioned, the present invention is not limited to those shapes, directions, positional relationships, etc., unless explicitly stated to be particularly essential or considered to be fundamentally limited to those specific shapes, directions, positional relationships, etc.

[0071] Similar expressions such as "acquisition," "calculation," "estimation," "detection," and "detection" can be appropriately substituted for each other within the limits of what is technically consistent. Similarly, "exceeding the threshold" and "above the threshold" can be appropriately substituted for each other within the limits of what is technically consistent. The same applies to "below the threshold" and "below the threshold."

[0072] Modifications are not limited to the examples given above. For example, all or part of one of the modifications may be combined with all or part of another, provided that it does not conflict with the technical specifications. Furthermore, all or part of the specific examples may be combined with all or part of the modifications, provided that it does not conflict with the technical specifications. [Explanation of Symbols]

[0073] 2 cameras 500 Object Identification Devices 501 Sensor Information Acquisition Unit 502 Image Target Information Acquisition Unit 522 Detection Information Acquisition Unit 503 Radar Target Information Acquisition Unit 504 Fusion Unit 505 Ghost detection unit 551 Ghost Affirmation Unit 552 Ghost Denial Judgment Unit

Claims

1. An object identification device (500) mounted on a vehicle, A detection information acquisition unit (522) acquires object detection information based on image information captured by a camera (2) mounted on the vehicle, A ghost determination unit (505) determines whether the detection information corresponds to the detection of an actual object or to the detection of a ghost that is not an actual object. It has, The ghost detection unit, upon first detecting the object, determines that if the object is cut off from the captured image or the distance to the object is less than or equal to a threshold distance, the detection information corresponding to the object is the ghost detection information. Object identification device.

2. The ghost detection unit determines that if the object is partially obscured from the captured image when it is first detected, the detection information corresponding to the object is ghost detection information. Conversely, if the object remains visible in the captured image for a predetermined period of time thereafter, the detection information corresponding to the object is entity detection information. The object identification device according to claim 1.

3. The ghost detection unit determines that the detection information corresponding to the object is ghost detection information when the distance to the object is less than or equal to the threshold distance when the object is first detected, while determining that the detection information corresponding to the object is entity detection information when the movement speed of the object exceeds the threshold speed. The object identification device according to claim 1.

4. The ghost detection unit determines that if sensor fusion is achieved, which is the detection of the object by both the camera and the object detection sensor (3) other than the camera, and the movement speed of the object detected by the object detection sensor exceeds a threshold speed, then the detection information corresponding to the object is the entity detection information. The object identification device according to claim 1.

5. The ghost detection unit determines that the detected information is the actual entity detection information if the turning radius of the vehicle is below a threshold. The object identification device according to claim 1.

6. The ghost detection unit determines that if sensor fusion, which is the detection of the object by both the camera and the object detection sensor (3) other than the camera, continues for a threshold time, the detection information corresponding to the object is the entity detection information. The object identification device according to claim 1.

7. The ghost detection unit, The state in which the object is not cut off from the captured image continues for a predetermined period of time. The movement speed of the aforementioned object exceeds the threshold speed. Sensor fusion is achieved, in which the object is detected by both the camera and the object detection sensor (3) other than the camera, and the movement speed of the object detected by the object detection sensor exceeds the threshold speed. The turning radius of the aforementioned vehicle is below the threshold. The threshold time continues after the aforementioned sensor fusion is established. If any of the following conditions are met, it is determined that the detection information is the entity detection information. The object identification device according to claim 1.

8. An object identification program executed by an object identification device (500) mounted on a vehicle, The processing performed by the object identification device is: A detection information acquisition process that acquires object detection information based on image information captured by a camera (2) mounted on the vehicle, A ghost determination process that determines whether the detection information corresponds to the detection of the object as a physical entity or to the detection of a ghost that is not a physical entity, Includes, In the ghost detection process described above, if the object is initially detected and is partially obscured from the captured image or the distance to the object is less than or equal to a threshold distance, the detection information corresponding to the object is determined to be the ghost detection information. Object identification program.

Images