Object detection method and object detection device

Abstract

In this object detection method and this object detection device, a first image photographed by a first camera and a second image photographed by a second camera are acquired in a vehicle, in which the first camera and the second camera are disposed such that a part of the imaging range of the first camera and a part of the imaging range of the second camera overlap each other. When an object is detected from at least one of the first image and the second image, the moving speed of the object on the image from which the object has been detected is calculated. In the case where the object is detected from both the first image and the second image after the moving speed of the object has been calculated, the distance to the object from the vehicle is calculated on the basis of the object on the first image and the object on the second image. The relative speed of the object to the vehicle is estimated on the basis of the moving speed and the distance.

Description

[Technical Field]

[0001] The present invention relates to an object detection method and an object detection device. [Background technology]

[0002] A technology has been proposed in which, when tracking an object detected from imaging data, distance information of the object is generated using a stereo camera, and the three-dimensional movement amount and speed of the object are calculated based on the time-series imaging data of the object and the time-series distance information of the object (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-127478 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the technology described in Patent Document 1, if an object cannot be captured by some of the multiple imaging means constituting the stereo camera, the three-dimensional velocity of the object cannot be calculated, resulting in a delay in the start of observation of the object.

[0005] The present invention has been made in view of the above-mentioned problems, and an object detection method and an object detection device are provided that can suppress delays in starting observation of an object even when some of a plurality of image capturing means are unable to capture an image of the object. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, an object detection method and object detection device according to one aspect of the present invention acquire a first image captured by the first camera and a second image captured by the second camera on a vehicle in which a first camera and a second camera are arranged so that a portion of the imaging range of the first camera overlaps a portion of the imaging range of the second camera. When an object is detected in at least one of the first image or the second image, the movement speed of the object on the image in which the object is detected is calculated. After calculating the movement speed of the object, when the object is detected in both the first image and the second image, the distance from the vehicle to the object is calculated based on the object on the first image and the object on the second image. Then, the relative speed of the object with respect to the vehicle is estimated based on the movement speed and distance. [Effects of the Invention]

[0007] According to the present invention, even if some of the multiple imaging means are unable to capture an image of the object, it is possible to suppress delays in starting observation of the object. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the configuration of an object detection device according to one embodiment of the present invention. [Figure 2] FIG. 2 is a flowchart showing the processing of the object detection device according to one embodiment of the present invention. [Figure 3A] FIG. 3A is a diagram showing an example of a first region set in a first image. [Figure 3B] FIG. 3B is a diagram showing an example of a second region set in the second image. [Figure 4A] FIG. 4A is a diagram showing an example of a first region that is set based on an occlusion region. [Figure 4B] FIG. 4B is a diagram showing an example of a second region that is set based on an occlusion region. [Figure 5] FIG. 5 is a diagram showing an example of the positional relationship between an object, another vehicle, and an imaging unit mounted on the vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0009] Next, an embodiment of the present invention will be described in detail with reference to the drawings. In the description, the same components are designated by the same reference numerals and duplicated explanations will be omitted.

[0010] [Configuration of object detection device] An example configuration of an object detection device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of an object detection device 1 according to this embodiment. As shown in Fig. 1, the object detection device 1 includes a first camera 11, a second camera 12, and a controller 20.

[0011] The object detection device 1 may be mounted on a vehicle with an automatic driving function, or on a vehicle without an automatic driving function. The object detection device 1 may also be mounted on a vehicle that is capable of switching between automatic driving and manual driving. The automatic driving function may also be a driving assistance function that automatically controls only some of the vehicle control functions, such as steering control, braking force control, and driving force control, to assist the driver in driving. In this embodiment, the object detection device 1 will be described as being mounted on a vehicle with an automatic driving function.

[0012] 1, the object detection device 1 may control various actuators such as a steering actuator, an accelerator pedal actuator, and a brake actuator based on the recognition results (object position, speed, etc.) by the controller 20. This may enable highly accurate autonomous driving.

[0013] The first camera 11 and the second camera 12 are mounted on the vehicle and capture images of the surroundings of the vehicle. The first camera 11 acquires a first image. The second camera 12 acquires a second image. For example, the first camera 11 and the second camera 12 have imaging elements such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS).

[0014] The first camera 11 and the second camera 12 are arranged on the vehicle so that a part of the imaging range of the first camera 11 overlaps a part of the imaging range of the second camera 12. The first camera 11 and the second camera 12 are also installed apart from each other. There are no particular limitations on the installation locations of the first camera 11 and the second camera 12, but as an example, the first camera 11 and the second camera 12 may be installed on the front mirror of the vehicle.

[0015] The first camera 11 and the second camera 12 continuously capture images of the surroundings of the vehicle at a predetermined cycle. The first camera 11 and the second camera 12 detect objects around the vehicle (pedestrians, bicycles, motorbikes, other vehicles, etc.) and information about the surroundings of the vehicle (land markings, traffic lights, signs, crosswalks, intersections, etc.). The first image and the second image are output to the controller 20. The first image and the second image may be stored in a storage device (not shown), and the controller 20 may refer to the images stored in the storage device.

[0016] The controller 20 is a general-purpose computer equipped with a CPU (Central Processing Unit), memory, and input / output units. A computer program for causing the microcomputer to function as the object detection device 1 is installed in the microcomputer. By executing the computer program, the computer functions as multiple information processing circuits equipped in the object detection device 1. The controller 20 processes a first image captured by the first camera 11 and a second image captured by the second camera 12.

[0017] Although an example is shown here in which the multiple information processing circuits provided in the object detection device 1 are realized by software, it is of course also possible to configure the information processing circuits by providing dedicated hardware for executing each of the information processes described below.Furthermore, the multiple information processing circuits may be configured by individual hardware.

[0018] The controller 20 includes, as examples of a plurality of information processing circuits (information processing functions), an object detection unit 21, an occlusion region extraction unit 23, a measurement region setting unit 25, a movement speed calculation unit 27, a distance calculation unit 29, and a three-dimensional speed estimation unit 31. The controller 20 may also be expressed as an ECU (Electronic Control Unit).

[0019] The occlusion region extraction unit 23 extracts an occlusion region included in the first image and an occlusion region included in the second image based on the first image and the second image. Here, the occlusion region included in the first image is a partial region of the first image that corresponds to a range that can be captured by the first camera 11 out of a blind spot for the second camera 12 caused by objects around the vehicle. Also, the occlusion region included in the second image is a partial region of the second image that corresponds to a range that can be captured by the second camera 12 out of a blind spot for the first camera 11 caused by objects around the vehicle.

[0020] For example, the occlusion region extraction unit 23 performs stereo matching between the first image and the second image to extract the occlusion region. In stereo matching, pixels in the first image and pixels in the second image that correspond to the same corresponding points around the vehicle are searched for (corresponding point search). Then, if the occlusion region extraction unit 23 cannot find a pixel in the second image that corresponds to a corresponding point around the vehicle related to a pixel in the first image, it extracts the pixel in the first image as a pixel that constitutes the occlusion region included in the first image. Similarly, if the occlusion region extraction unit 23 cannot find a pixel in the first image that corresponds to a corresponding point around the vehicle related to a pixel in the second image, it extracts the pixel in the second image as a pixel that constitutes the occlusion region included in the second image.

[0021] Stereo matching makes it possible to obtain information on the distance and direction to corresponding points around the vehicle. Therefore, if stereo matching can be performed on the same corresponding points consecutively at multiple times, it is possible to obtain information on the trajectory of movement of the corresponding points between the multiple times in the vehicle's stationary frame. Therefore, it is possible to calculate the three-dimensional velocity (three-dimensional velocity) of the corresponding points in the vehicle's stationary frame, i.e., the relative velocity of the corresponding points with respect to the vehicle.

[0022] Examples of corresponding point search methods used in stereo matching include SAD (sum of absolute difference) and POC (phase only correlation). SAD evaluates the similarity between regions of a predetermined size using the value of the "sum of absolute values ​​of differences in pixel values," and extracts the pixel at the position where this value is smallest as the pixel at the position where similarity is highest. POC performs a Fourier transform on an image to separate it into an amplitude component corresponding to density and a phase component corresponding to shape, and obtains correlation using only the phase component. The position where this correlation is highest is then extracted as the pixel at the position where similarity is highest. Corresponding point search methods that can be used in this embodiment are not limited to the examples given here.

[0023] The measurement area setting unit 25 sets a first area that is a part of the first image. The measurement area setting unit 25 sets a second area that is a part of the second image based on the surrounding environment of the vehicle. The set first and second areas are, for example, rectangular areas.

[0024] For example, assume that first camera 11 is located to the right of second camera 12 with respect to the forward direction of the vehicle. In this case, measurement area setting unit 25 may set the first area so that the left edge of the first image coincides with the left edge of the first area. Furthermore, measurement area setting unit 25 may set the second area so that the right edge of the second image coincides with the right edge of the second area.

[0025] 3A is a diagram showing an example of a first region set in a first image. According to Fig. 3A, the left edge of the first image SC1 and the left edge of the first region Z1 coincide with each other.

[0026] 3B is a diagram showing an example of the second region set in the second image. According to Fig. 3B, the right edge of the second image SC2 and the right edge of the second region Z2 coincide with each other.

[0027] By setting the first area or the second area in this manner, an object that is likely to affect the traveling of the vehicle among objects that appear from a blind spot created by objects around the vehicle can be set as a detection target by the travel speed calculation unit 27, which will be described later. In particular, by setting the first area described above, an object that appears from the left side to the center in front of the vehicle based on the forward direction of the vehicle can be reliably set as a detection target. Furthermore, by setting the second area described above, an object that appears from the right side to the center in front of the vehicle based on the forward direction of the vehicle can be reliably set as a detection target.

[0028] The measurement region setting unit 25 may set the first region so that the left edge of the occlusion region included in the first image coincides with the left edge of the first region. The measurement region setting unit 25 may set the second region so that the right edge of the occlusion region included in the second image coincides with the right edge of the second region.

[0029] 4A is a diagram showing an example of a first region set based on an occlusion region. In FIG. 4A, an occlusion region C1 occurs to the right of the region in which the other vehicle V1 appears in the first image SC1. The left edge of the occlusion region C1 coincides with the left edge of the first region Z1.

[0030] 4B is a diagram showing an example of a second region set based on the occlusion region. In the second image SC2, an occlusion region C2 is present to the left of the region in which the other vehicle V1 appears. The right edge of the occlusion region C2 coincides with the right edge of the second region Z2.

[0031] By setting the first area or the second area in this way, an object appearing from a blind spot formed by objects around the vehicle can be detected by the travel speed calculation unit 27, which will be described later.

[0032] According to the above-described method for setting the first and second regions, an object coming out to the right from a blind spot caused by objects around the vehicle, based on the forward direction of the vehicle, will be captured first in the first image in the first region, while an object coming out to the left from a blind spot caused by objects around the vehicle will be captured first in the second image in the second region.

[0033] Alternatively, the measurement area setting unit 25 may set the first area or the second area based on the planned travel route of the vehicle.

[0034] For example, if it is known that the vehicle is planning to turn right based on the planned travel route of the vehicle, the measurement area setting unit 25 may set only the first area and not the second area. This is because when the vehicle turns right, it is considered that an object captured in the second area is unlikely to affect the travel of the vehicle.

[0035] Furthermore, when it is known based on the planned travel route of the vehicle that the vehicle is planning to turn left, the measurement area setting unit 25 may set only the second area and not the first area. This is because when the vehicle turns left, it is considered that an object captured in the first area is unlikely to affect the travel of the vehicle.

[0036] Furthermore, when it is known based on the planned travel route of the vehicle that the vehicle is scheduled to stop, the measurement area setting unit 25 may not set the first area or the second area that is set based on the occlusion area of ​​an object that is farther away from the vehicle than the stopping position of the vehicle. This is because it is considered unlikely that an object that appears from a blind spot caused by an object that is farther away from the vehicle than the stopping position of the vehicle will affect the travel of the vehicle.

[0037] Alternatively, the measurement area setting unit 25 may set the width of the first area or the width of the second area based on the distance to an object related to the occlusion area included in the first image or the second image.

[0038] The measurement area setting unit 25 may acquire information about objects in the occlusion area (information about distance, width, height, etc.) via other sensors (not shown). One example of a sensor that acquires information about objects in the occlusion area is a LIDAR (Laser Imaging Detection and Ranging). A LIDAR measures the distance and direction to an object or recognizes the shape of the object by emitting light (laser light) to an object around the vehicle and measuring the time it takes for the light (reflected light) to hit the object and bounce back. Furthermore, a LIDAR can also acquire the positional relationship of objects in three dimensions.

[0039] Alternatively, the measurement area setting unit 25 may acquire information about objects related to the occlusion area based on map information about the road on which the vehicle is traveling. The map information may include position information about traffic lights, information about the road on which the vehicle is traveling, and information about moving objects moving around the road. The measurement area setting unit 25 may acquire map information stored in a map database (not shown). The measurement area setting unit 25 may acquire map information from an external map data server using cloud computing. Alternatively, the measurement area setting unit 25 may acquire map information using vehicle-to-vehicle communication or road-to-vehicle communication.

[0040] Note that the measurement region setting unit 25 may set the width of the first region or the second region set for the occlusion region to be smaller as the distance to the object in the occlusion region increases. The greater the distance to the object in the occlusion region, the slower the apparent speed on the image of an object jumping out from the blind spot of the object in the occlusion region. Therefore, unless the width of the first region or the second region is set small, the range of processing by the movement speed calculation unit 27 (described later) increases, which may increase the calculation load. Therefore, by setting the width of the first region or the second region to be smaller as the distance to the object in the occlusion region increases, the calculation load can be reduced while the movement speed calculation unit 27 detects objects jumping out from the blind spot.

[0041] Conversely, the shorter the distance to an object in the occlusion region, the larger the width of the first or second region set by the measurement region setting unit 25 for the occlusion region. The shorter the distance to an object in the occlusion region, the faster the apparent speed of an object appearing in the image from the blind spot of the object. Therefore, if the width of the first or second region is not set to be large, a situation may arise in which an object appearing in the blind spot of the object is not included in the first or second region. Therefore, by setting the width of the first or second region to be large as the distance to an object in the occlusion region is small, an object appearing in the blind spot can be reliably detected by the movement speed calculation unit 27.

[0042] Furthermore, the measurement region setting unit 25 may set the width of the first region or the width of the second region based on the width of an object related to the occlusion region included in the first image or the second image.

[0043] For example, the measurement area setting unit 25 may set the width of the first area or the second area set for the occlusion area to be larger as the width of the object related to the occlusion area becomes larger. As the width of the object related to the occlusion area becomes larger, the width of the first area or the second area is set to be larger, assuming that the width of an object hidden in the blind spot of the object related to the occlusion area becomes larger. As a result, an object that appears from the blind spot can be reliably detected by the movement speed calculation unit 27.

[0044] The measurement region setting unit 25 may set the width of the first region or the second region set for the occlusion region to be smaller as the width of the object related to the occlusion region becomes smaller. The smaller the width of the object related to the occlusion region, the smaller the width of the object hidden in the blind spot of the object, so the measurement region setting unit 25 sets the width of the first region or the second region to be smaller. As a result, it is possible to reduce the calculation load while making the movement speed calculation unit 27 detect an object that has jumped out from the blind spot.

[0045] Alternatively, the measurement area setting unit 25 may set the width of the first area or the width of the second area based on a representative speed associated with a moving object on the road on which the vehicle is traveling.

[0046] For example, the measurement area setting unit 25 may acquire a typical speed associated with a moving object from among information about a road on which the vehicle is traveling or a moving object moving around the road. For example, if a "pedestrian" is registered as a moving object, the width of the first area or the width of the second area may be set based on the typical speed (walking speed) of the "pedestrian." Similarly, if a "bicycle," "motorcycle," "other vehicle," or the like is registered as a moving object, the width of the first area or the width of the second area may be set based on the typical speed of the registered moving object.

[0047] The typical speed of the moving object may be included in the map information, or may be a predetermined speed associated with the moving object in a database (not shown).

[0048] Furthermore, the measurement region setting unit 25 may set the height of the first region or the height of the second region based on the height of an object related to an occlusion region included in the first image or the second image. For example, the measurement region setting unit 25 may acquire the height of an object related to an occlusion region and set the apparent height of the object on the image as the height of the first region or the height of the second region. This allows an object that appears in a blind spot to be reliably detected by the movement speed calculation unit 27.

[0049] The object detection unit 21 detects an object that appears in at least one of the first image and the second image. For example, the object detection unit 21 detects a group of pixels included in an occlusion region.

[0050] Alternatively, the object detection unit 21 may perform stereo matching between the first image and the second image. In this case, the object detection unit 21 may detect a pixel of the first image when it is unable to search for a pixel of the second image corresponding to a corresponding point around the vehicle associated with a pixel of the first image. Furthermore, the object detection unit 21 may detect a pixel of the second image when it is unable to search for a pixel of the first image corresponding to a corresponding point around the vehicle associated with a pixel of the second image.

[0051] The object detected by the object detection unit 21 is registered in a list as a tracking target in the first image and the second image that are subsequently acquired.

[0052] The movement speed calculation unit 27 calculates the movement speed of an object on an image in which the object is captured. For example, the movement speed calculation unit 27 may calculate the movement speed of the object on the image by taking the difference between two or more first images arranged in chronological order. The movement speed calculation unit 27 may also calculate the movement speed of the object on the image by taking the difference between two or more second images arranged in chronological order.

[0053] Furthermore, the movement speed calculation unit 27 may calculate the movement speed only for the object included in the first area or the second area set by the measurement area setting unit 25. This reduces the number of objects to be processed in the movement speed calculation unit 27, thereby reducing the calculation load.

[0054] The movement speed calculation unit 27 may calculate the movement speed when the object transitions from a state in which it appears in both the first image and the second image to a state in which it appears in only one of the first image and the second image. When the movement speed calculation unit 27 calculates the movement speed, the calculation may be limited to a case in which the object appears in only one of the first image and the second image.

[0055] When the object transitions to a state in which it appears in both the first image and the second image, distance calculation unit 29 calculates the distance from the vehicle to the object based on the first image and the second image. More specifically, distance calculation unit 29 acquires the direction in which the object is located as seen from first camera 11 and the direction in which the object is located as seen from second camera 12. Then, distance calculation unit 29 calculates the distance from the vehicle to the object by triangulation based on these two directions and the baseline length between first camera 11 and second camera 12.

[0056] The three-dimensional speed estimation unit 31 estimates the relative speed of the object with respect to the vehicle based on the moving speed and distance. Note that the moving speed used to estimate the relative speed is the moving speed of the object on the image in which the object appears, as explained above by the moving speed calculation unit 27. Therefore, the three-dimensional speed estimation unit 31 converts the moving speed on the image into the change in the apparent direction of the object per unit time, and estimates the value obtained by multiplying the change in apparent direction by the distance to the object as the relative speed of the object with respect to the vehicle. In other words, it can be said that the three-dimensional speed estimation unit 31 estimates the relative speed of the object with respect to the vehicle under the constraint that the distance between the vehicle and the object does not change.

[0057] According to the "estimation" of the relative velocity by the above-mentioned three-dimensional velocity estimator 31, the component of the object's velocity parallel to the line connecting the vehicle and the object is not reflected in the estimated relative velocity. However, it is possible to estimate the relative velocity even when the object appears in only one of the first image and the second image. This makes it possible to detect the object and initiate various vehicle controls earlier.

[0058] Additionally, when an object appears consecutively in both the first image and the second image, the three-dimensional velocity estimation unit 31 calculates the relative velocity of the object with respect to the vehicle based on the stereo matching results. In this case, stereo matching can be performed consecutively on the same object at multiple times. Therefore, the three-dimensional velocity estimation unit 31 acquires information on the trajectory of the object's movement in the vehicle's stationary frame based on the distance and direction to the object at multiple times, and calculates the relative velocity of the corresponding point with respect to the vehicle based on the trajectory of movement.

[0059] According to the "calculation" of the relative velocity by the three-dimensional velocity estimation unit 31 described above, the component of the velocity of the object parallel to the line connecting the vehicle and the object is reflected in the calculated relative velocity.

[0060] Alternatively, the three-dimensional speed estimation unit 31 may calculate the absolute speed of an object relative to the road on which the vehicle is traveling, based on the traveling speed of the vehicle and the relative speed of the object relative to the vehicle. The three-dimensional speed estimation unit 31 may acquire information from a sensor that measures the absolute position of the vehicle, i.e., the position, attitude, and speed of the vehicle relative to a predetermined reference point, using a position detection sensor that measures the absolute position of the vehicle, such as a GPS (Global Positioning System) or odometry. The three-dimensional speed estimation unit 31 acquires the traveling speed of the vehicle via the sensor.

[0061] [Processing procedure of object detection device] Next, a processing procedure of the object detection device 1 according to this embodiment will be described with reference to the flowchart of Fig. 2. Fig. 2 is a flowchart showing the processing of the object detection device 1 according to this embodiment. The processing of the object detection device 1 shown in Fig. 2 is repeatedly executed at a predetermined cycle.

[0062] First, in step S101, first camera 11 captures a first image, and second camera 12 captures a second image.

[0063] In step S103, the occlusion region extraction unit 23 performs stereo matching between the first image and the second image.

[0064] In step S105, the measurement region setting unit 25 sets a first region and a second region.

[0065] In step S107, the object detection unit 21 detects an object that appears in at least one of the first image and the second image.

[0066] In step S109, the movement speed calculation unit 27 calculates the movement speed of the object on the image in which the object appears.

[0067] In step S111, object detection unit 21 determines whether the object appears in both the first image and the second image. If it is determined that the object does not appear in both the first image and the second image (the object appears in only one of the images) (NO in step S111), the process returns to step S101.

[0068] On the other hand, if it is determined that the object appears in both the first image and the second image (YES in step S111), then in step S113, object detection unit 21 determines whether the object appears in both the past first image and the past second image. Here, "past first image and second image" refers to the first image and second image acquired in the process executed one cycle before, among the processes shown in FIG. 2 that are repeatedly executed at predetermined intervals.

[0069] If it is determined that the object appears in both the past first and second images (YES in step S113), this corresponds to a case where the object appears in both the first and second images consecutively. Therefore, in step S115, the three-dimensional velocity estimation unit 31 calculates the relative velocity of the object with respect to the vehicle based on the stereo matching result. Then, the flowchart in FIG. 2 ends.

[0070] On the other hand, if it is determined that the object is not captured in either the past first image or the past second image (NO in step S113), this corresponds to a transition to a state in which the object is captured in both the first image and the second image. Therefore, in step S117, distance calculation unit 29 calculates the distance from the vehicle to the object based on the first image and the second image.

[0071] Then, in step S119, the three-dimensional velocity estimation unit 31 estimates the relative velocity of the object with respect to the vehicle based on the moving velocity and distance, after which the flowchart in FIG.

[0072] [Effects of the embodiment] As described in detail above, the object detection method and object detection device according to this embodiment acquires a first image captured by the first camera and a second image captured by the second camera on a vehicle in which the first camera and the second camera are arranged so that part of the imaging range of the first camera overlaps part of the imaging range of the second camera. When an object is detected in at least one of the first image or the second image, the movement speed of the object on the image in which the object is detected is calculated. After calculating the movement speed of the object, when the object is detected in both the first image and the second image, the distance from the vehicle to the object is calculated based on the object on the first image and the object on the second image. Then, the relative speed of the object with respect to the vehicle is estimated based on the movement speed and distance.

[0073] This allows for suppressing delays in starting observation of an object even when some of the multiple imaging means are unable to capture an image of the object. In particular, estimating the relative velocity based on the object's moving speed in the image has a faster processing speed than calculating the relative velocity based on the object captured in both the first and second images. This allows for earlier start of various vehicle controls for an object appearing in a blind spot defined by surrounding objects around the vehicle.

[0074] The fact that the timing for starting vehicle control can be advanced will be explained using Fig. 5. Fig. 5 is a diagram showing an example of the positional relationship between an object, another vehicle, and an imaging unit mounted on the vehicle. Fig. 5 shows the positional relationship as a bird's-eye view from above the vehicle, and shows positions P1, P2, and P3 of a first camera 11 and a second camera 12 that are provided spaced apart in the width direction of the vehicle (not shown), another vehicle V1, and an object appearing in a blind spot of the other vehicle V1. In addition, the imageable range AR1 of the first camera 11 and the imageable range AR2 of the second camera 12 are shown.

[0075] It is assumed that position P1 is in the blind spot of the other vehicle V1 as viewed from the second camera 12, and positions P2 and P3 are outside the blind spot of the other vehicle V1 as viewed from the second camera 12. It is also assumed that positions P1, P2, and P3 are outside the blind spot of the other vehicle V1 as viewed from the first camera 11.

[0076] At this time, assume that the object moves to positions P1, P2, and P3 in this order. In this case, when the object moves to position P1, it can be photographed by first camera 11, but cannot be photographed by second camera 12. Therefore, the object appears in the first image but not in the second image. Therefore, the object at position P1 is detected first.

[0077] Thereafter, when the object moves to position P2, the object can be photographed by both first camera 11 and second camera 12. Therefore, the distance from the vehicle (or first camera 11 and second camera 12) to position P2 can be calculated.

[0078] Since the moving speed of the object on the first image when the object moves from position P1 to position P2 can be calculated, the relative speed of the object at position P2 can be estimated based on the moving speed and distance.

[0079] On the other hand, to calculate the relative speed of an object moving from positions P1 to P3 in this order based on the stereo matching results, the object needs to move to position P3. This is because the first stereo matching is performed on the object when it moves to position P2 where it can be photographed by both first camera 11 and second camera 12, and then the second stereo matching is performed when the object moves to position P3.

[0080] As described above, the relative velocity of an object can be calculated based on the stereo matching results when the object moves to position P3, whereas the relative velocity of the object can be estimated when the object moves to position P2. Therefore, the timing of estimating the relative velocity of an object using the object detection method and object detection device according to this embodiment is earlier than the timing of calculating the relative velocity of an object based on the stereo matching results. This makes it possible to accelerate the timing of initiating various vehicle controls for an object that appears from a blind spot defined by surrounding objects around the vehicle.

[0081] Furthermore, the object detection method and object detection device according to this embodiment may set a first region that is a part of the first image and a second region that is a part of the second image, and calculate the moving speed of only the object included in the first region or the second region. This makes it possible to calculate the moving speed of the object while reducing the calculation load.

[0082] Furthermore, assuming that the first camera is located to the right of the second camera relative to the forward direction of the vehicle, the object detection method and object detection device according to this embodiment may set the first region so that the left edge of the first image coincides with the left edge of the first region. Also, the second region may be set so that the right edge of the second image coincides with the right edge of the second region. This makes it possible to more reliably detect objects that appear from blind spots created by objects around the vehicle and that are likely to affect the vehicle's driving, while reducing the computational load.

[0083] Furthermore, assuming that the first camera is located to the right of the second camera relative to the forward direction of the vehicle, the object detection method and object detection device according to this embodiment may set the first region so that the left edge of the occlusion region included in the first image coincides with the left edge of the first region. Furthermore, the second region may be set so that the right edge of the occlusion region included in the second image coincides with the right edge of the second region. This makes it possible to more reliably detect objects appearing from blind spots created by objects around the vehicle while reducing the computational load.

[0084] Furthermore, the first and second regions are not set in areas where objects appearing from blind spots around the vehicle are not captured, thereby reducing false detection of objects.

[0085] Furthermore, the object detection method and object detection device according to this embodiment may set the first area or the second area based on the planned driving route of the vehicle, thereby enabling objects that are likely to affect the driving of the vehicle to be detected preferentially among objects appearing from blind spots around the vehicle.

[0086] Furthermore, the object detection method and object detection device according to this embodiment may set the width of the first region or the width of the second region based on the distance to an object in an occlusion region included in the first image or the second image. This makes it possible to detect objects appearing from blind spots created by objects around the vehicle while reducing the calculation load.

[0087] Furthermore, the object detection method and object detection device according to this embodiment may set the width of the first region or the width of the second region based on the width of an object related to an occlusion region included in the first image or the second image. This makes it possible to detect objects appearing from blind spots created by objects around the vehicle while reducing the calculation load.

[0088] Furthermore, the object detection method and object detection device according to this embodiment may set the width of the first region or the width of the second region based on a representative speed associated with a moving object on a road on which the vehicle is traveling. This allows the width of the first region or the width of the second region to be set appropriately according to the speed of an object emerging from a blind spot created by objects around the vehicle. As a result, the object can be reliably detected while reducing the computational load.

[0089] Furthermore, the object detection method and object detection device according to this embodiment may set the height of the first region or the height of the second region based on the height of an object related to an occlusion region included in the first image or the second image. This makes it possible to detect objects appearing from blind spots created by objects around the vehicle while reducing the calculation load.

[0090] Furthermore, the object detection method and object detection device according to this embodiment may calculate the moving speed when the object transitions from a state in which it appears in both the first image and the second image to a state in which it appears in only one of the first image and the second image. This allows the object to continue to be a detection target even when the object transitions from a state in which it is captured by all of the multiple image capturing devices to a state in which it is captured by only some of the multiple image capturing devices.

[0091] Furthermore, the object detection method and object detection device according to this embodiment may calculate the absolute speed of an object relative to the road on which the vehicle is traveling, based on the vehicle's traveling speed and relative speed, thereby enabling various vehicle controls to be initiated based on the absolute speed of the object.

[0092] Each function described in the above embodiments may be implemented by one or more processing circuits, including programmed processors, electrical circuits, and even devices such as application specific integrated circuits (ASICs), circuit components arranged to perform the described functions.

[0093] Although the present invention has been described above based on the embodiments, it will be apparent to those skilled in the art that the present invention is not limited to these descriptions and that various modifications and improvements are possible. The descriptions and drawings that form part of this disclosure should not be understood as limiting the present invention. Various alternative embodiments, examples, and operating techniques will become apparent to those skilled in the art from this disclosure.

[0094] The present invention naturally includes various embodiments not described herein. Therefore, the technical scope of the present invention is defined only by the invention-specifying matters according to the scope of the claims that are appropriate from the above description. [Explanation of symbols]

[0095] 1. Object detection device 11 Camera 1 12 Second Camera 20 Controller 21 Object detection unit 23 Occlusion region extraction unit 25 Measurement area setting section 27 Movement speed calculation section 29 Distance calculation unit 31 Three-dimensional velocity estimation part

Claims

1. 1. An object detection method for controlling a controller that receives signals from a first camera and a second camera in a vehicle in which a first camera and a second camera are arranged so that a part of an imaging range of the first camera and a part of an imaging range of the second camera overlap, comprising: The controller acquiring a first image captured by the first camera; acquiring a second image captured by the second camera; When an object is detected from at least one of the first image and the second image, calculating a moving speed of the object on the image in which the object is detected; after calculating the moving speed of the object, if the object is detected from both the first image and the second image, calculating a distance from the vehicle to the object based on the object in the first image and the object in the second image; estimating a relative speed of the object with respect to the vehicle based on the moving speed and the distance; An object detection method characterized by:

2. The object detection method according to claim 1 , The controller setting a first region that is a part of the first image and a second region that is a part of the second image; Calculating the moving speed only for the object included in the first area or the second area. An object detection method characterized by:

3. The object detection method according to claim 2, the first camera is located to the right of the second camera with respect to the forward direction of the vehicle; The controller setting the first area so that the left edge of the first image coincides with the left edge of the first area; The second area is set so that the right edge of the second image coincides with the right edge of the second area. An object detection method characterized by:

4. The object detection method according to claim 2, the first camera is located to the right of the second camera with respect to the forward direction of the vehicle; The controller setting the first region so that the left edge of an occlusion region included in the first image coincides with the left edge of the first region; setting the second region so that the right edge of an occlusion region included in the second image coincides with the right edge of the second region; An object detection method characterized by:

5. The object detection method according to any one of claims 2 to 4, The controller sets the first area or the second area based on a planned travel route of the vehicle. An object detection method characterized by:

6. The object detection method according to any one of claims 2 to 4, The controller sets a width of the first region or a width of the second region based on a distance to an object related to an occlusion region included in the first image or the second image. An object detection method characterized by:

7. The object detection method according to any one of claims 2 to 4, The controller sets the width of the first region or the width of the second region based on the width of an object related to an occlusion region included in the first image or the second image. An object detection method characterized by:

8. The object detection method according to any one of claims 2 to 4, The controller sets the width of the first area or the width of the second area based on a representative speed associated with a moving object on a road on which the vehicle is traveling. An object detection method characterized by:

9. The object detection method according to any one of claims 2 to 4, The controller sets a height of the first region or a height of the second region based on a height of an object related to an occlusion region included in the first image or the second image. An object detection method characterized by:

10. The object detection method according to claim 1 , the controller calculates the moving speed when the object transitions from a state in which the object appears in both the first image and the second image to a state in which the object appears in only one of the first image and the second image. An object detection method characterized by:

11. The object detection method according to claim 1 , The controller calculates an absolute speed of the object relative to a road on which the vehicle is traveling, based on the traveling speed of the vehicle and the relative speed. An object detection method characterized by:

12. an object detection device in a vehicle in which a first camera and a second camera are arranged so that a part of an imaging range of the first camera and a part of an imaging range of the second camera overlap; and a controller that receives signals from the first camera and the second camera, The controller acquiring a first image captured by the first camera; acquiring a second image captured by the second camera; When an object is detected from at least one of the first image and the second image, Calculating a moving speed of the object on the image in which the object is detected; after calculating the moving speed of the object, if the object is detected from both the first image and the second image, calculating a distance from the vehicle to the object based on the object in the first image and the object in the second image; estimating a relative speed of the object with respect to the vehicle based on the moving speed and the distance; An object detection device characterized by:

Images