Object determination device, non-transitory storage medium, and object determination method

Abstract

The present disclosure relates to an object determination device, a non-transitory storage medium, and an object determination method. Provided is an object determination device capable of determining the appropriateness of a detection result at a second detection time according to a condition of a terrain when the same object is not detected at the first detection time and at the second detection time. The object determination device determines whether an object exists within a prescribed range from a vehicle based on object detection information output from a sensor provided to the vehicle, determines whether the same object is detected by first object detection information detected at a first detection time and second object detection information detected at a second detection time later than the first detection time, determines whether a prescribed region including a vehicle position at the second detection time satisfies a prescribed condition of a terrain when it is determined that the same object is not detected, and determines that the object exists within the prescribed range from the vehicle when it is determined that the same object is detected or when it is determined that the prescribed region including the vehicle position satisfies the prescribed condition of the terrain, and notifies of the determination result.

Description

Technical Field

[0001] This disclosure relates to an object determination device, a storage medium, and an object determination method. Background Technology

[0002] The automatic control system installed in the vehicle generates a driving route based on the vehicle's current location, destination location, and map information, and controls the vehicle to travel along that route.

[0003] The automatic control system controls the vehicle in a manner that maintains a distance greater than a predetermined distance between the vehicle and other vehicles in the vicinity. When the vehicle moves from its current lane to an adjacent lane, the automatic control system controls the vehicle in a manner that maintains a distance greater than a predetermined distance between the vehicle and other vehicles traveling in the current lane and other vehicles traveling in the adjacent lane.

[0004] The automatic control system uses object detection information output from sensors such as cameras configured on the vehicle to detect other vehicles around the vehicle at each specified time interval (for example, see Patent Document 1).

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2015-161968 Summary of the Invention

[0008] When a vehicle is moving between lanes, while using sensors to detect other vehicles, the automatic control system may miss detecting other vehicles from the previous detection. In such cases, if the automatic control system continues moving between lanes, other vehicles may get too close to the vehicle.

[0009] Consider the following scenarios as reasons for missing other vehicles: there is a reason in the use of sensors to detect other vehicles; other vehicles have moved out of the range that the sensors can detect.

[0010] Therefore, the purpose of this disclosure is to provide an object determination device that determines whether the same object exists based on the terrain conditions at the second detection time when the same object is not detected in the first detection and the second detection.

[0011] According to one embodiment, an object determination device is provided. This object determination device determines whether an object exists within a predetermined range from the vehicle based on object detection information output from sensors disposed on a vehicle. The object determination device is characterized by comprising: a first determination unit that determines whether first object detection information detected during a first detection and second object detection information detected during a second detection later than the first detection detect the same object; a second determination unit that, if determined by the first determination unit that the same object was not detected, determines whether a predetermined area including the vehicle's position during the second detection satisfies predetermined terrain conditions; a third determination unit that, if determined by the first determination unit that the same object was detected, or if determined by the second determination unit that the predetermined area including the vehicle's position satisfies predetermined terrain conditions, determines that the object exists within a predetermined range from the vehicle; and a notification unit that notifies the third determination unit of the determination result.

[0012] Furthermore, in this object detection device, the prescribed terrain conditions are different, preferably in the case where the object detected in the first detection is not detected in the second detection and in the case where the object not detected in the first detection is detected in the second detection.

[0013] Furthermore, in this object determination device, preferably when no object detected in the first detection is detected in the second detection, the second determination unit determines that the area containing the position of the vehicle in the second detection does not contain a lane or road outside the range specified in the second detection if the object detected in the first detection can move there.

[0014] Furthermore, in this object determination device, preferably when an object not detected in the first detection is detected in the second detection, the second determination unit determines that the area containing the position of the vehicle in the second detection does not contain a branch road branching off from the driving road the vehicle is traveling in the defined area containing the position of the vehicle in the second detection, and the defined area contains the branch road branching off from the driving road the vehicle is traveling in the defined area.

[0015] Furthermore, the object detection device preferably includes: a third determination unit that, when the first determination unit determines that no identical object has been detected and the object detected in the first detection is not detected in the second detection, determines whether a second object different from the object detected in the first detection is detected in the second detection within a second range that is narrower than a predetermined range; and a fourth determination unit that, when the second object is detected, determines that the first object detection information and the second object detection information have detected the same object.

[0016] According to another embodiment, a non-temporary storage medium storing a computer program for object determination is provided. The object determination computer program determines whether an object exists within a predetermined range from the vehicle based on object detection information output from sensors configured on the vehicle, and causes a processor to perform processing including: determining whether the first object detection information detected during a first detection and the second object detection information detected during a second detection later than the first detection detect the same object; if it is determined that no identical object was detected, determining whether a predetermined area including the vehicle's position during the second detection satisfies predetermined terrain conditions; if it is determined that the same object was detected, or if it is determined that the predetermined area including the vehicle's position satisfies the predetermined terrain conditions, determining that the object exists within the predetermined range from the vehicle, and notifying the determination result.

[0017] Furthermore, according to another embodiment, an object determination method is provided. In this object determination method, an object is determined to exist within a predetermined range from the vehicle based on object detection information output from sensors disposed on the vehicle. The method is characterized by being performed by an object determination device: determining whether the first object detection information detected during a first detection and the second object detection information detected during a second detection later than the first detection detect the same object; if it is determined that the same object is not detected, determining whether a predetermined area including the vehicle's position during the second detection satisfies predetermined terrain conditions; if it is determined that the same object is detected, or if it is determined that the predetermined area including the vehicle's position satisfies the predetermined terrain conditions, determining that the object exists within a predetermined range from the vehicle, and notifying the determination result.

[0018] The object determination device disclosed herein can determine whether the same object exists based on the terrain conditions during the second detection if the same object is not detected during the first detection and the second detection. Attached Figure Description

[0019] Figure 1 This is a diagram illustrating the operation overview of the vehicle control system including the object detection device of this embodiment.

[0020] Figure 2 This is a schematic structural diagram of a vehicle equipped with the object detection device of this embodiment.

[0021] Figure 3 This is an example of an action flowchart related to the object determination process of the object determination device in this embodiment.

[0022] Figure 4 This is a diagram (Figure 1) illustrating an example of the action of the object determination process of the object determination device.

[0023] Figure 5 This is a diagram (Figure 2) illustrating an example of the action of the object determination process of the object determination device.

[0024] Figure 6 This is a diagram (3) illustrating an example of the action of the object determination process of the object determination device.

[0025] Figure 7 This is a diagram (4) illustrating an example of the action of the object determination process of the object determination device.

[0026] Figure 8 This is a diagram (5) illustrating an example of the action of the object determination process of the object determination device.

[0027] Figure 9 This is a diagram (Figure 6) illustrating an example of the action of the object determination process of the object determination device.

[0028] Figure 10 This is an example of an operation flowchart related to the object determination process of a modified example of the object determination device of this embodiment.

[0029] Figure 11 This is a diagram illustrating an example of the action of object determination processing in a modified example of the object determination device. Detailed Implementation

[0030] Figure 1 This is a diagram illustrating the general operation of the vehicle control system 1, including the object detection device 13 of this embodiment. Hereinafter, refer to... Figure 1 This section provides an overview of the operations related to the object determination process of the object determination device 13 disclosed in this specification.

[0031] Vehicle 10 is traveling on road 50. Road 50 has three lanes 51, 52, and 53. Lanes 51 and 52 are separated by lane dividing line 54, and lanes 52 and 53 are separated by lane dividing line 55. Vehicle 10 is traveling in lane 52.

[0032] The object detection device 13 detects other objects within a detection range L from the vehicle 10 at predetermined time intervals, based on sensor information (an example of object detection information) output from sensors such as the camera 2 disposed on the vehicle 10. Figure 1 In the example shown, the object detection device 13 determines that vehicle 30 exists within the detection range L starting from vehicle 10. The object detection device 13 notifies the driving lane planning device 14, which generates the driving lane plan for vehicle 10, of object detection information indicating the detected position of vehicle 30 and its driving lane. The object detection device 13 uses object identification information (object identification ID) to identify other objects to track vehicle 30.

[0033] The object identification device 13 determines whether the sensor information detected in the previous detection and the sensor information detected in the current detection detect the same vehicle 30. If the object identification device 13 does not detect the vehicle 30 detected in the previous detection (identified by a predetermined identification ID) in the current detection, it determines that the same object has not been detected.

[0034] exist Figure 1 In the example shown, vehicle 30, which was detected in the previous detection, was not detected in this detection. Since the object detection device 13 did not detect the same object, it determines whether the region M containing the location of vehicle 10 in this detection meets the prescribed terrain conditions. Region M preferably includes, for example, the detection range L.

[0035] exist Figure 1 In the example shown, the area M containing the location of vehicle 10 at the time of this detection does not include lanes or roads outside the detection range L of vehicle 30 detected in the previous detection. Therefore, as a reason why vehicle 30 was not detected in this detection, the probability that vehicle 30 moved outside the detection range L of sensor 2, etc., is low. The object determination device 13 determines that the area M containing the location of vehicle 10 meets the prescribed terrain conditions.

[0036] The area M containing the location of vehicle 10 meets the specified terrain conditions, so the object detection device 13 determines that vehicle 10 exists within the detection range L starting from vehicle 10.

[0037] The object detection device 13 notifies the driving lane planning device 14 of object detection information indicating that the vehicle 30 exists within the detection range L from the vehicle 10. This object detection information does not include information indicating the position of the vehicle 30 or the driving lane.

[0038] The driving lane planning device 14, which is notified only of object detection information existing within the detection range L from vehicle 10, generates a new driving lane plan for vehicle 10 in a manner that delays the start of the lane change operation by a predetermined time, for example, if the current driving lane plan includes vehicle 10 moving from lane 52 to lane 53. This is because if vehicle 10 immediately performs the inter-lane movement, vehicle 10 and vehicle 30 may get too close.

[0039] In addition, if the object detection device 13 also detects the vehicle 30 detected in the previous detection during the current detection, it will notify the driving lane planning device 14 of the object detection information, which indicates the position of the detected vehicle 30 and the driving lane, etc.

[0040] On the other hand, if the area M containing the location of vehicle 10 does not meet the specified terrain conditions, it is assumed that vehicle 30 has moved outside the detection range of sensor 2, etc., so vehicle 30 was not detected in this detection.

[0041] If the object detection device 13 does not detect the same object in the previous detection and the current detection, it determines whether the same object exists based on the terrain conditions at the time of the current detection, so that the accurate object detection results can be reflected in the vehicle control.

[0042] Figure 2 This is a schematic structural diagram of a vehicle 10 equipped with a vehicle control system 1. The vehicle control system 1 includes cameras 2a and 2b, LiDAR sensors 3a to 3d, a positioning information receiver 4, a navigation device 5, a user interface (UI) 6, a map information storage device 11, a location calculation device 12, an object detection device 13, a driving lane planning device 14, a driving planning device 15, and a vehicle control device 16. Furthermore, the vehicle control system 1 may also include other ranging sensors (not shown), such as radar sensors, for measuring the distance to objects around the vehicle 10.

[0043] Cameras 2a and 2b, LiDAR sensors 3a to 3d, positioning information receiver 4, navigation device 5, UI 6, map information storage device 11, location calculation device 12, object identification device 13, driving lane planning device 14, driving planning device 15, and vehicle control device 16 can be communicatively connected via an in-vehicle network 18 that follows a standard controller area network.

[0044] Cameras 2a and 2b are examples of camera units installed in vehicle 10. Camera 2a is mounted in front of vehicle 10. Camera 2b is mounted in front of vehicle 10. Cameras 2a and 2b capture camera images of the environment in a predetermined area in front of or behind vehicle 10 at camera image capture times with a predetermined period, for example. The camera images are examples of object detection information. The camera images can show road features, such as lane markings, within the predetermined area in front of or behind vehicle 10. In the camera image captured by camera 2a, other vehicles located to the left front, front, and right front of vehicle 10 can be shown. In the camera image captured by camera 2b, other vehicles located to the left rear, rear, and right rear of vehicle 10 can be shown. Cameras 2a and 2b have a 2D detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or C-MOS, and an imaging optical system that images the area to be photographed onto the 2D detector.

[0045] Each time cameras 2a and 2b capture an image, they output the image and the time of capture via the in-vehicle network 18 to the position estimation device 12 and the object detection device 13, etc. The camera images are used in the position estimation device 12 to calculate the position of the vehicle 10. Furthermore, the camera images are used in the object detection device 13 to detect other objects around the vehicle 10.

[0046] LiDAR sensors 3a, 3b, 3c, and 3d are mounted on the outer surface of vehicle 10, for example, facing the front, rear, left, and right sides of vehicle 10, respectively. At a predetermined periodic time for acquiring reflected wave information, LiDAR sensors 3a to 3d synchronously emit pulsed lasers towards the front, rear, left, and right sides of vehicle 10, respectively, and receive reflected waves reflected by objects. The time required for the reflected waves to return includes distance information between vehicle 10 and other objects located in the direction of laser illumination. LiDAR sensors 3a, 3b, 3c, and 3d output reflected wave information, including the direction of laser illumination and the time required for the reflected waves to return, along with the time of acquiring the reflected wave information, via in-vehicle network 18 to object detection device 13. The reflected wave information is an example of object detection information. The reflected wave information is used in object detection device 13 to detect other objects around vehicle 10. The time for acquiring reflected wave information is preferably, for example, consistent with the time of camera capture.

[0047] The positioning information receiver 4 outputs positioning information indicating the current location of the vehicle 10. For example, the positioning information receiver 4 can be configured as a GNSS receiver. Each time the positioning information receiver 4 acquires positioning information at a predetermined reception period, it outputs the positioning information and the time of acquisition to the navigation device 5 and the map information storage device 11, etc.

[0048] The navigation device 5 generates a navigation route for the vehicle 10 from its current location to its destination location based on navigation map information, the destination location of the vehicle 10 input from the UI 6, and the positioning information indicating the current location of the vehicle 10 input from the positioning information receiver 4. The navigation device 5 generates a new navigation route for the vehicle 10 when a new destination location is set or when the current location of the vehicle 10 deviates from the navigation route. Each time the navigation device 5 generates a navigation route, it outputs the navigation route via the in-vehicle network 18 to the location estimation device 12 and the driving lane planning device 14, etc.

[0049] UI6 is an example of a notification unit. Controlled by navigation device 5, etc., UI6 notifies the driver of driving information such as the vehicle 10's driving status. Furthermore, UI6 generates operation signals corresponding to the driver's actions on the vehicle 10. The vehicle 10's driving information includes the vehicle's current location, navigation route, and other information related to the vehicle's current and future paths. To display driving information, UI6 has a display device 6a such as an LCD or touch panel. Additionally, UI6 may also have an audio output device (not shown) for notifying the driver of driving information. Furthermore, UI6 may have, for example, a touch panel or operation buttons as input devices for inputting operation information from the driver to the vehicle 10. Operation information may include, for example, the destination location, route, vehicle speed, and other control information of the vehicle 10. UI6 outputs the input operation information to navigation device 5 and vehicle control device 16, etc., via in-vehicle network 18.

[0050] The map information storage device 11 stores wide-area map information covering a relatively broad area (e.g., a range of 10 to 30 km square) of the current location of the vehicle 10. This map information includes 3D road surface information, information representing road features and structures such as lane markings, and high-precision map information including legal speed limits. Based on the current location of the vehicle 10, the map information storage device 11 receives the wide-area map information from an external server via a base station through wireless communication via a wireless communication device (not shown) mounted on the vehicle 10 and stores it in the storage device. Whenever positioning information is input from the positioning information receiver 4, the map information storage device 11, referring to the stored wide-area map information, outputs map information including a relatively narrow area (e.g., a range of 100 m to 10 km square) of the current location represented by the positioning information via the in-vehicle network 18 to the location calculation device 12, object detection device 13, driving lane planning device 14, driving planning device 15, and vehicle control device 16, etc.

[0051] The position estimation device 12 estimates the position of the vehicle 10 at the time the camera image was captured, based on the road features surrounding the vehicle 10 as shown in the camera image captured by camera 2a. For example, the position estimation device 12 compares the lane markings identified in the camera image with the lane markings shown in the map information input from the map information storage device 11 to determine the estimated position and estimated azimuth of the vehicle 10 at the time the camera image was captured. Furthermore, the position estimation device 12 estimates the driving lane of the road where the vehicle 10 is located based on the lane markings shown in the map information, the estimated position of the vehicle 10, and the estimated azimuth. Each time the position estimation device 12 calculates the estimated position, estimated azimuth, and driving lane of the vehicle 10 at the time the camera image was captured, it outputs this information to the object determination device 13, the driving lane planning device 14, the driving planning device 15, and the vehicle control device 16, etc. Alternatively, the position estimation device 12 can also estimate the position of the vehicle 10 based on two camera images captured by cameras 2a and 2b.

[0052] The object detection device 13 performs object detection processing, judgment processing, and notification processing. Therefore, the object detection device 13 has a communication interface (IF) 21, a memory 22, and a processor 23. The communication interface 21, memory 22, and processor 23 are connected via signal lines 24. The communication interface 21 has interface circuitry for connecting the object detection device 13 to the in-vehicle network 18.

[0053] All or part of the functions of the object detection device 13 are implemented by functional modules, for example, by a computer program that operates on the processor 23. The processor 23 has an object detection unit 230, a determination unit 231, and a notification unit 232. Alternatively, the functional modules of the processor 23 may be dedicated arithmetic circuits provided on the processor 23. The processor 23 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 23 may also have other arithmetic circuits such as logic operation units, numerical operation units, or graphics processing units. The memory 22 is an example of a storage unit, such as a volatile semiconductor memory or a non-volatile semiconductor memory. Moreover, the memory 22 stores the computer programs used in the information processing executed by the processor 23 and various data.

[0054] The object detection unit 230 performs object detection processing to detect other objects around the vehicle 10 at object detection times with a predetermined period. The period of the object detection time is determined based on the period of the camera capture time and the period of the reflection wave information acquisition time. The object detection unit 230 detects other objects and their types in front of, to the left, front, and front right of the vehicle 10 based on camera images captured by camera 2a. Furthermore, the object detection unit 230 detects other objects and their types in rear left, rear, and rear right of the vehicle 10 based on camera images captured by camera 2b. Other vehicles traveling around the vehicle 10 are included among these other objects. The object detection unit 230 has a recognizer that detects objects represented in an image, for example, by inputting a camera image. As the recognizer, a deep neural network (DNN) can be used, for example, which is pre-learned to detect objects represented in an input image. The object detection unit 230 may also use a recognizer other than a DNN. For example, the object detection unit 230 may also use a support vector machine (SVM) as the recognizer. This SVM is pre-learned to take as input feature quantities (e.g., Histograms of Oriented Gradients, HOG) calculated from a window set on the camera image, and outputs a confidence level that the window represents an object to be detected. Alternatively, the object detection unit 230 may also detect object regions by performing template matching between a template representing an object to be detected and the image.

[0055] Furthermore, the object detection unit 230 detects other objects in front of, to the left, and to the right of the vehicle 10 based on the reflected wave information output by the LiDAR sensor 3a, and detects other objects behind, to the left, and to the right of the vehicle 10 based on the reflected wave information output by the LiDAR sensor 3b. Additionally, the object detection unit 230 detects other objects on the left side of the vehicle 10 based on the reflected wave information output by the LiDAR sensor 3c, and detects other objects on the right side of the vehicle 10 based on the reflected wave information output by the LiDAR sensor 3d.

[0056] The object detection unit 230 determines the orientation of other objects relative to the vehicle 10 based on their positions within the camera image. Based on this orientation and the reflected wave information output by the LiDAR sensors 3a-3d, it calculates the distance between the other objects and the vehicle 10. The object detection unit 230 calculates the positions of other objects, for example, in a world coordinate system, based on the current position of the vehicle 10 and the distances and orientations of other objects relative to the vehicle 10. Furthermore, the object detection unit 230 performs optical flow-based tracking processing, matching other objects detected from the latest camera image with objects detected from past images, thereby tracking other objects detected from the latest image. Moreover, the object detection unit 230 calculates the trajectory of other objects during the tracking process based on their positions in the latest image obtained from past images, expressed in a world coordinate system. The object detection unit 230 calculates the velocity of the object relative to the vehicle 10 based on changes in the position of other objects over time. Furthermore, the object detection unit 230 can calculate the acceleration of other objects based on changes in their velocities over time. Furthermore, the object detection unit 230 determines the driving lane of another object based on the lane markings and the positions of other objects shown in the map information. For example, the object detection unit 230 determines that another object is driving in a driving lane defined by two adjacent lane markings, wherein the two adjacent lanes are located at the center of the horizontal direction sandwiching the other object.

[0057] When the object detection unit 230 detects a moving object, including other vehicles, as another object, it attaches object identification information to identify the other object and tracks it. If the object detection unit 230 does not detect another object with attached object identification information within a specified period, it determines that the other object has been missed. On the other hand, if the object detection unit 230 detects a structure such as a guardrail or side wall as another object, it does not track it.

[0058] The object detection unit 230 notifies the determination unit 231 and the notification unit 232 of object determination information, which includes information indicating the type of other detected objects, their object identification information, information indicating their position, and information indicating their speed, acceleration, and driving lane. Furthermore, the object detection unit 230 outputs the object determination information to the driving lane planning device 14, the driving planning device 15, and the vehicle control device 16, etc. The object detection unit 230 is capable of detecting the detection range of other objects (e.g., ...). Figure 1 The detection range L is determined based on the setup conditions and resolution of cameras 2a, 2b and LiDAR sensors 3a-3d. Details of other operations in the object detection device 13 will be described later.

[0059] At a predetermined interval for generating a driving lane plan, the driving lane planning device 14 selects a lane within the nearest driving range (e.g., 10 km) selected from the navigation route, based on map information, the navigation route, surrounding environment information, and the current position of the vehicle 10, and generates a driving lane plan indicating the predetermined driving lane that the vehicle 10 will travel in. The driving lane planning device 14 generates the driving lane plan, for example, by having the vehicle 10 travel in a lane other than the overtaking lane. Each time the driving lane planning device 14 generates a driving lane plan, it outputs the driving lane plan to the driving planning device 15.

[0060] Furthermore, the driving lane planning device 14, within the nearest driving section selected from the navigation route, determines whether a lane change is necessary based on the driving lane plan, map information, navigation route, and the current position of the vehicle 10, and generates a lane change plan based on the determination result. The lane change plan includes a predetermined lane change section where the vehicle 10 is expected to move to an adjacent lane from its current driving lane. Specifically, the driving lane planning device 14 determines whether a lane change is necessary to move to a lane towards the vehicle 10's destination position based on the navigation route and the vehicle 10's current position. The driving lane planning device 14 determines whether the vehicle 10 is merging from its current driving lane into another road merging to its destination (merging) or exiting from its driving lane into another road branching off to its destination (branching). During merging and branching, the vehicle moves from the driving lane of the driving lane to the driving lane of another road, thus requiring a lane change. The driving lane planning device 14 may also further use surrounding environmental information or vehicle status information to determine whether a lane change is necessary. Surrounding environment information includes the positions and speeds of other vehicles traveling around vehicle 10. Vehicle status information includes the current position, speed, acceleration, and direction of travel of vehicle 10. Furthermore, the driving lane planning device 14 generates a lane change plan based on the driver's request. Information indicating the vehicle speed and acceleration of vehicle 10 is obtained using sensors (not shown) mounted on vehicle 10.

[0061] The driving plan device 15 performs driving plan processing at predetermined intervals, based on the driving lane plan, map information, the current position of vehicle 10, surrounding environment information, and vehicle status information, to generate a driving plan representing a predetermined driving trajectory of vehicle 10 up to a predetermined time (e.g., 5 seconds). The driving plan is represented as a set of target positions of vehicle 10 and target vehicle speeds at each time point from the current time until the predetermined time. The period for generating the driving plan is preferably shorter than the period for generating the driving lane plan. The driving plan device 15 generates the driving plan at intervals that maintain a predetermined distance or more between vehicle 10 and other vehicles. Each time a driving plan is generated, the driving plan device 15 outputs the driving plan to the vehicle control device 16.

[0062] The vehicle control unit 16 controls various parts of the vehicle 10 based on the vehicle 10's current position, speed, yaw rate, and the driving plan generated by the driving plan unit 15. For example, the vehicle control unit 16 calculates the vehicle 10's steering angle, acceleration, and angular acceleration according to the driving plan, speed, and yaw rate, and sets the steering amount, accelerator opening, or braking amount in a manner corresponding to these steering angle, acceleration, and angular acceleration. Then, the vehicle control unit 16 outputs a control signal corresponding to the set steering amount to the actuator (not shown) controlling the steering wheel of the vehicle 10 via the in-vehicle network 18. Furthermore, the vehicle control unit 16 calculates the fuel injection amount according to the set accelerator opening and outputs a control signal corresponding to this fuel injection amount to the vehicle 10's engine or other drive unit (not shown) via the in-vehicle network 18. Alternatively, the vehicle control unit 16 outputs a control signal corresponding to the set braking amount to the vehicle 10's brakes (not shown) via the in-vehicle network 18.

[0063] exist Figure 2 In this document, the map information storage device 11, the location calculation device 12, the object determination device 13, the driving lane planning device 14, the driving planning device 15, and the vehicle control device 16 are described as independent devices (e.g., ECU), but all or part of these devices may also constitute a single device.

[0064] Figure 3 This is an example of an operation flowchart related to the object determination process of the object determination device 13 in this embodiment. Hereinafter, refer to... Figure 3 This section explains the object determination process of the object determination device 13. The object determination device 13 performs object determination at predetermined periodic times according to... Figure 3The flowchart shown illustrates the object determination process. The period of the object determination step is preferably the same as or longer than the period of the object detection step. The previous object determination step was an example from the first detection step, and the current object determination step is an example from the second detection step.

[0065] First, the determination unit 231 determines whether the object detection information detected in the previous detection and the object detection information detected in the current detection detect the same object (step S101). The determination unit 231 uses the object determination information input from the object determination device 13 from the time of the last object determination to the time of the current object determination as the detection result for other objects in the current detection. In addition, the determination unit 231 uses the object determination information input from the object determination device 13 from the time of the object determination two years ago to the time of the last object determination as the detection result for other objects in the previous detection.

[0066] If the object recognition information contained in the previous detection result is included in the current detection result, the determination unit 231 determines that the object detection information detected in the previous detection and the object detection information detected in the current detection both detected the same object (step S101 - Yes).

[0067] On the other hand, if the object recognition information contained in the previous detection result is not included in the current detection result, or if the object recognition information not included in the previous detection result is included in the current detection result, the determination unit 231 determines that the same object was not detected (step S101 - No).

[0068] If it is determined that no identical object was detected (step S101 - No), the determination unit 231 determines, based on the map information, whether the specified area containing the position of the vehicle 10 at the time of this detection meets the specified terrain conditions (step S102). The specified terrain conditions are different when no object detected in the previous detection is detected in the current detection and when an object not detected in the previous detection is detected in the current detection.

[0069] The reasons why object recognition information included in the previous detection result is not included in the current detection result are considered in the following situations: there is a reason related to the object detection information detected in the current detection; other objects have moved out of the sensor's detection range. If no other objects are detected in the current detection because other objects have moved out of the sensor's detection range, the detection result is appropriate. On the other hand, if the reason is due to the object detection information detected in the current detection but no other objects are detected in the current detection, the detection result is invalid.

[0070] Furthermore, the inclusion of object recognition information in the current detection result, which was not included in the previous detection result, is considered in the following situations: new objects that should be tracked appear within the detection range of the object detection unit 230 (e.g., Figure 1 The detection range L); other objects (structures such as guardrails or side walls) that should not be tracked with attached object identification information are mistakenly detected as other objects that should be tracked. If other objects are detected during this detection because a new object that should be tracked appears within the detection range of the object determination device 13, the detection result is appropriate. On the other hand, if other objects (structures such as guardrails or side walls) that should not be tracked with attached object identification information are mistakenly detected as other objects that should be tracked, the detection result is invalid. In this case, the cause can be attributed to the object detection information detected during this detection.

[0071] Therefore, the designated area is preferably determined in a manner that allows for the determination of whether the test results are appropriate or invalid. For example, the designated area (e.g., Figure 1 The region M) preferably includes a detection range (e.g., the detection unit 230 is capable of detecting other objects) of the object detection unit 230. Figure 1 Detection range (L).

[0072] In addition, the determination unit 231 may, together with or in place of map information, determine whether a specified area containing the position of the vehicle 10 at the time of detection meets the specified terrain conditions based on road features such as lane markings or structures such as guardrails or side walls detected by the object detection unit 230.

[0073] If the specified terrain conditions are met in the specified area (step S102 - Yes), the determination unit 231 determines that other objects exist within the specified range from vehicle 10 (step S103).

[0074] The notification unit 232 notifies the driving lane planning device 14 of the determination result of the determination unit 231 (step S104), ending a series of processes. If no object detected in the previous detection is detected in the current detection, the notification unit 232 outputs (notifies) object determination information indicating that other objects detected in the previous detection exist within the detection range L from the vehicle 10, to the driving lane planning device 14, the driving planning device 15, and the vehicle control device 16, etc. This object determination information does not include information indicating the location of other objects or the driving lane.

[0075] Furthermore, if an object not detected in the previous detection is detected during the current detection, the notification unit 232 will also output (notify) object determination information indicating that the other objects detected during the current detection exist within a range L specified from the vehicle 10, to the driving lane planning device 14, driving planning device 15, and vehicle control device 16, etc. This object determination information includes information indicating the type of other detected objects, their object identification information, information indicating their position, and information indicating their speed, acceleration, and driving lane.

[0076] Furthermore, if the same object is detected in both the previous and current detections (step S101 - Yes), the notification unit 232 will also output (notify) object determination information indicating that other objects detected in the current detection exist within a specified range L from the vehicle 10, to the driving lane planning device 14, driving planning device 15, and vehicle control device 16, etc. This object determination information includes information indicating the type of other detected objects, their object identification information, information indicating their position, and information indicating their speed, acceleration, and driving lane.

[0077] On the other hand, if the specified area does not meet the specified terrain conditions (step S102 - No), the determination unit 231 determines that there are no other objects (step S105) and ends a series of processes.

[0078] The following is for reference Figures 4-6 The following describes the first action example of the object determination process of the object determination device 13.

[0079] exist Figure 4 In the first action example shown, with Figure 1 Similarly, vehicle 10 is traveling in lane 52 of road 50. In the previous detection results, vehicle 30 traveling in lane 53 was detected within the detection range L. Vehicle 30 was identified by the prescribed object recognition information.

[0080] like Figure 5 As shown, in the detection results of this test, no vehicle 30 identified by the prescribed object recognition information was detected within the detection range L. The object determination device 13 determined that the object detection information detected in the previous test and the object detection information detected in this test did not detect the same object.

[0081] exist Figure 5In the example shown, the region M containing the location of vehicle 10 at the time of this detection does not include lanes or roads outside the detection range L at the time of this detection, where vehicle 30 detected in the previous detection could have moved. Therefore, the reason why the object recognition information contained in the previous object determination information is not included in the current object determination information is considered to be due to the use of sensors to detect other objects. If vehicle 30 detected in the previous detection is not detected in the current detection, the detection result is invalid, so it is assumed that vehicle 30 detected in the previous detection exists within the detection range L starting from vehicle 10.

[0082] The object determination device 13 determines that the area M containing the location of the vehicle 10 at the time of the current detection meets the prescribed terrain conditions, and outputs (notifies) object determination information indicating that the vehicle 30 detected in the last detection exists within the detection range L from the vehicle 10, to the driving lane planning device 14, the driving planning device 15, and the vehicle control device 16, etc. This object determination information does not include information indicating the location of other objects or the driving lane.

[0083] When the driving lane planning device 14 generates a driving lane plan for vehicle 10 in a situation where the driving lane plan includes movement between driving lanes of vehicle 10, the driving lane plan starts with a predetermined time delay. The predetermined time is preferably a few times the period of the object detection time (for example, 2 to 5 times). Thus, if vehicle 30 is detected again in the next detection, a driving lane plan is generated in a manner corresponding to the detected vehicle 30.

[0084] In addition, the driving lane planning device 14 can also plan the vehicle 10 to move from driving lane 52 to driving lane 51. When the vehicle 30 identified by the object recognition information contained in the previous object determination information is driving in driving lane 53 which is not the destination of the vehicle 10, the start of the driving lane change operation is not changed.

[0085] On the other hand, Figure 6 In the example shown, the region M containing the location of vehicle 10 at the time of this detection includes lane 61 and road 60, which are both areas where vehicle 30, detected in the last detection, could move outside the detection range L at the time of this detection. Road 60 branches off from road 50 at branch position 61. Therefore, as a reason why vehicle 30, which was detected in the last detection, was not detected in the current detection, the scenario where vehicle 30 has moved outside the sensor detection range L is considered.

[0086] The object detection device 13 determines that the area M containing the location of vehicle 10 at the time of this detection does not meet the specified terrain conditions. Since the detection result of not detecting vehicle 30 at the time of this detection is correct, it is considered that vehicle 30 does not exist within the specified range L starting from vehicle 10 at the time of this detection.

[0087] The following is for reference Figures 7-9 The following describes the second action example of the object determination process of the object determination device 13.

[0088] exist Figure 7 In the second action example shown, vehicle 10 is traveling in lane 51 of road 50. In the detection results of the previous detection, no other objects identified by object recognition information were detected within the detection range L.

[0089] like Figure 8 As shown, in the detection results of this detection, other objects 31 identified by the prescribed object recognition information were detected within the detection range L. The object determination device 13 determined that the object detection information detected in the previous detection and the object detection information detected in this detection did not detect the same object.

[0090] exist Figure 8 In the example shown, the region M containing the position of vehicle 10 at the time of detection includes branch road 70, which branches off from road 50 from which vehicle 10 is traveling. Branch road 70 branches off from road 50 at branch position 71. Therefore, as a reason for including object identification information that was not included in the previous object determination information in the current object determination information, the possibility of mistakenly detecting other objects 31 (structures such as guardrails or side walls) that should not be tracked as other objects 31 that should be tracked is considered. For example, when vehicle 10 is about to branch off from road 50 to road 70 at branch position 71, the side wall of branch road 70 becomes non-parallel to the direction of travel of vehicle 10, so sometimes the side wall of branch road 70 is detected as a moving object with a relative speed to vehicle 10.

[0091] The object detection device 13 determines that the area M containing the location of the vehicle 10 at the time of this detection does not meet the specified terrain conditions. The detection result of other objects 31 that were not detected in the previous detection is invalid, so it is considered that other objects existing within the specified range L from the vehicle 10 at the time of this detection do not exist.

[0092] When the driving lane plan includes the vehicle 10 moving from the driving lane 51 of road 50 to the driving lane 72 of branch road 70, the driving lane planning device 14 generates a driving lane plan including the vehicle 10 exiting from road 50 to road 70.

[0093] On the other hand, Figure 9 In the example shown, the detection results during this detection showed that a vehicle 32 identified by the prescribed object recognition information was detected in the driving lane 52 within the detection range L.

[0094] The area M containing the location of vehicle 10 at the time of this detection does not include branch road 70 branching from road 50 from which vehicle 10 travels. During this detection, the new vehicle 32 that should be tracked appears within the detection range L of object detection unit 230. Since the detection result of detecting vehicle 32, which was not detected in the previous detection, is correct, it is assumed that the new vehicle 32 that should be tracked appears within the detection range L of object detection unit 230.

[0095] The object determination device 13 determines that the area M containing the position of vehicle 10 at the time of detection meets the prescribed terrain conditions. The object determination device 13 outputs (notifies) object determination information indicating that the detected vehicle 32 exists within a prescribed range L from vehicle 10 to the driving lane planning device 14, driving planning device 15, and vehicle control device 16, etc. This object determination information includes information indicating the type of detected vehicle 32, its object identification information, information indicating its position, and information indicating its speed, acceleration, and driving lane.

[0096] According to the object detection device of this embodiment described above, when no identical object is detected during the first detection and the second detection, the device can determine whether the same object exists based on the terrain conditions during the second detection, so that the accurate object detection result can be reflected in the vehicle control.

[0097] Next, refer to Figure 10 as well as Figure 11 The following describes a variation of the object detection device of this embodiment.

[0098] Figure 10 This is an example of an operation flowchart related to the object determination process of a variant of the object determination device of this embodiment. The object determination device 13, at object determination times having a predetermined period, performs... Figure 10 The flowchart shown illustrates the object determination process. The period for object determination is preferably longer than the period for object detection.

[0099] Figure 10 The processing in step S201 is the same as described above. Figure 3 The steps are the same as in step S101.

[0100] If the same object is not detected in the object detection information detected in the previous detection and the object detection information detected in the current detection (step S201 - No), the determination unit 231 determines whether other objects detected in the previous detection are detected in the current detection (step S202).

[0101] If no other object detected in the previous detection is detected in this detection (step S202 - No), the determination unit 231 determines that in this detection, the object is within a second detection range N (refer to) that is narrower than the detection range L. Figure 11 Check whether any other objects are detected that are different from the other objects identified by the specified object identification information detected in the last detection (step S203).

[0102] When other objects are detected in the second detection range (step S203 - Yes), the determination unit 231 determines that the object detection information detected in the previous detection and the object detection information detected in the current detection both detect the same object (step S204), and proceeds to step S207.

[0103] On the other hand, if no other object is detected in the second detection range (step S203 - No) or if no other object detected in the previous detection is detected in the current detection (step S202 - Yes), the process proceeds to step S205. Figure 10 The processing in steps S205 to S208 is the same as described above. Figure 3 Steps S102 to S105 are the same.

[0104] The following is for reference Figure 4 as well as Figure 11 The following describes an example of the object determination process of a modified object determination device 13.

[0105] like Figure 4 As shown, in the detection results of the previous detection, vehicle 30 traveling in lane 53 was detected within the detection range L. Vehicle 30 was identified by the prescribed object recognition information.

[0106] like Figure 11 As shown, in the detection results of this test, vehicle 30, which was identified by the prescribed object recognition information, was not detected within the detection range L. However, vehicle 33, which was different from vehicle 30 detected in the previous test, was detected within the second detection range N. Vehicle 33 was identified by object recognition information different from that of vehicle 30.

[0107] The second detection range N is preferably an area where sensors like LiDAR sensors 3a-3d, which detect other objects based on reflected waves, are prone to false detection or oversight of other objects, depending on the relationship between the vehicle 10's position and map information. For example, the area to the side of the vehicle 10 has a high volume of reflected waves from structures such as guardrails or walls, making it an area prone to false detection of other objects. Therefore, the side of the vehicle 10 can be considered the second detection range N. On the other hand, the area in front of and behind the vehicle is less likely to have reflected waves if there are no other vehicles present, making it an area where false detection of other objects is less probable.

[0108] exist Figure 11 In the example shown, the second detection range N is set on the right side of vehicle 10. The size of the second detection range N can be determined, for example, to include the extent of another vehicle (a regular car). Alternatively, the second detection range can also be set on the left side of vehicle 10. The second detection range N is contained within the detection range L.

[0109] Although vehicle 30 is missing, there is a high probability that vehicle 33 is the same vehicle as vehicle 30. Therefore, the object detection device 13 determines that the object detection information detected in the previous detection and the object detection information detected in the current detection both detected the same object.

[0110] The object identification device 13 adds object recognition information of vehicle 30 to vehicle 33. It outputs (notifies) the object identification information indicating that vehicle 30 (vehicle 33) detected in this detection exists within a specified range L from vehicle 10 to the driving lane planning device 14, driving planning device 15, and vehicle control device 16, etc. This object identification information includes information indicating the type of detected vehicle 30 (vehicle 33), its object recognition information, information indicating its position, and information indicating its speed, acceleration, and driving lane.

[0111] In this disclosure, the object determination apparatus, object determination computer program, and object determination method described above can be appropriately modified without departing from the spirit of this disclosure. Furthermore, the technical scope of this disclosure is not limited to these embodiments, but covers the invention as described in the claims and its equivalents.

[0112] For example, the first detection time is the last detection time, and the second detection time is the current detection time. However, the first and second detection times can also be specified time points that include time points from the past to the present.

[0113] Furthermore, in the above embodiments, object detection information output by the camera and the LiDAR sensor is used to detect other objects, but other objects can also be detected using object detection information output by either the camera or the LiDAR sensor.

[0114] Furthermore, in the above embodiments, the object detection unit, the determination unit, and the notification unit are assembled in one device, but the object detection unit, the determination unit, and the notification unit may also be assembled in different devices.

[0115] Explanation of reference numerals in the attached figures

[0116] 1. Vehicle Control System

[0117] Cameras 2a and 2b

[0118] 3a-3d LiDAR sensors

[0119] 4. Positioning information receiver

[0120] 5. Navigation device

[0121] 6 User Interface

[0122] 6a display device

[0123] 10 vehicles

[0124] 11. Map information storage device

[0125] 12 Position estimation device

[0126] 13 Object Detection Device

[0127] 14. Driving lane planning device

[0128] 15 Driving plan device

[0129] 16 Vehicle control devices

[0130] 18. In-vehicle network

[0131] 21 Communication Interface

[0132] 22 Memory

[0133] 23 processors

[0134] 230 Object Detection Department

[0135] 231 Judgment Department

[0136] 232 Notification Department

Claims

1. An object detection device, which determines whether an object exists within a predetermined range from the vehicle based on object detection information output from a sensor disposed on a vehicle, characterized in that, The object determination device has: The first determination unit determines whether the same object is detected in the first detection during the determination process with a predetermined period, from the object determination time two years ago to the object determination time last time, and in the second detection during the second detection later than the first detection, from the object determination time last time to the object determination time now. The second determination unit determines whether, in the case where the first determination unit determines that no identical object is detected and the object detected in the first detection is not detected in the second detection, the second determination unit determines whether the defined area including the position of the vehicle in the second detection does not include other lanes or other roads of the road outside the defined range in the second detection, which the object detected in the first detection could move to. Furthermore, in the case where the first determination unit determines that no identical object is detected and the object not detected in the first detection is detected in the second detection, the second determination unit determines whether the defined area including the position of the vehicle in the second detection does not include branch roads branching off from the driving road the vehicle is traveling on. The third determination unit determines that an object exists within a specified range from the vehicle if the first determination unit determines that the same object has been detected, or if the second determination unit determines that the specified area including the location of the vehicle at the time of the second detection does not include other lanes or other roads on the road that the object detected at the time of the first detection could move to outside the specified range at the time of the second detection, or if the specified area including the location of the vehicle at the time of the second detection does not include branch roads branching off from the driving road traveled by the vehicle. as well as The notification department notifies the third determination department of the determination result.

2. The object determination device as described in claim 1, comprising: The fourth determination unit, if the first determination unit determines that no identical object was detected, and the object detected in the first detection is not detected during the second detection, determines whether a second object different from the object detected during the first detection is detected during the second detection within a second range narrower than the predetermined range; and The fifth determination unit, upon detecting the second object, determines that the first object detection information and the second object detection information have detected the same object.

3. A non-transitory storage medium, which is readable by a computer and stores a computer program for object determination, wherein the computer program for object determination determines whether an object exists within a predetermined range from the vehicle based on object detection information output from sensors configured on the vehicle, and causes a processor to perform processing including the following operations: The system determines whether the same object is detected in the first detection during a predetermined periodic object determination process, from the time two before the last object determination to the time of the last object determination, and in the second detection during a later period, from the time two after the last object determination to the time of the current object determination. If it is determined that no identical object was detected, and the object detected in the first detection was not detected in the second detection, it is determined whether the defined area including the vehicle's position in the second detection does not include other lanes or other roads on roads outside the defined range in the second detection that the object detected in the first detection could move to. Furthermore, if it is determined that no identical object was detected, and the object not detected in the first detection was detected in the second detection, it is determined whether the defined area including the vehicle's position in the second detection does not include branch roads branching off from the driving road the vehicle was traveling on. If it is determined that the same object was detected, or if the defined area containing the location of the vehicle at the time of the second detection does not contain other lanes or other roads on the road that the object detected at the first detection could move to outside the defined range at the time of the second detection, or if the defined area containing the location of the vehicle at the time of the second detection does not contain branch roads branching off from the road the vehicle was traveling on, then it is determined that the object exists within a defined range from the vehicle. The judgment result will be announced.

4. An object determination method, which determines whether an object exists within a predetermined range from the vehicle based on object detection information output from sensors disposed on a vehicle, characterized in that... Performed by the object detection device: The system determines whether the same object is detected in the first detection during a predetermined periodic object determination process, from the time two before the last object determination to the time of the last object determination, and in the second detection during a later period, from the time two after the last object determination to the time of the current object determination. If it is determined that no identical object was detected, and the object detected in the first detection was not detected in the second detection, it is determined whether the defined area including the vehicle's position in the second detection does not include other lanes or other roads on roads outside the defined range in the second detection that the object detected in the first detection could move to. Furthermore, if it is determined that no identical object was detected, and the object not detected in the first detection was detected in the second detection, it is determined whether the defined area including the vehicle's position in the second detection does not include branch roads branching off from the driving road the vehicle was traveling on. If it is determined that the same object was detected, or if the defined area containing the location of the vehicle at the time of the second detection does not contain other lanes or other roads on the road that the object detected at the first detection could move to outside the defined range at the time of the second detection, or if the defined area containing the location of the vehicle at the time of the second detection does not contain branch roads branching off from the road the vehicle was traveling on, then it is determined that the object exists within a defined range from the vehicle. The judgment result will be announced.

Images