Object detection method, object detection device, object detection program, and recording medium in which object detection program is stored

Abstract

This object detection method includes detecting an object approaching a moving body from a plurality of images obtained by imaging the periphery of the moving body at a prescribed frame rate. When another object that partially conceals the object is present in the plurality of images, the object detection method includes generating a composite image by synthesizing the plurality of images without aligning the positions of the in-image objects between the plurality of images, and detecting the object from the generated composite image.

Description

Object detection method, object detection device, object detection program, and recording medium storing object detection program

[0001] The present disclosure relates to an object detection method, an object detection device, an object detection program, and a recording medium storing an object detection program.

[0002] In recent years, there has been active development of technology that makes it easier to see an object by combining multiple captured images. Patent Document 1 discloses an image processing device that combines multiple captured frame images to obtain an image with a wide dynamic range that is free of blown-out highlights and crushed shadows. In Patent Document 1, an image of a moving object that is the object is extracted from the multiple captured frame images, the amount of movement of the moving object image is calculated, and the multiple image data are aligned and combined.

[0003] Japanese Patent Application Laid-Open No. 2016-208083

[0004] In Patent Document 1, the imaging target is extracted from captured frame images. Therefore, if there is an object such as a wire mesh, a utility pole, or a roadside tree in front of the imaging target, part of the imaging target is hidden by the foreground object, so the imaging target cannot be correctly extracted from the frame image, or the foreground object is erroneously extracted. As a result, the amount of movement of the imaging target is not calculated correctly, and an image with a wide dynamic range cannot be obtained by image synthesis.

[0005] An object of the present disclosure is to provide an object detection method, an object detection device, an object detection program, and a recording medium storing an object detection program, which correctly detect objects that are partially obscured by other objects and that may collide with a moving body.

[0006] An object detection method according to one aspect of the present disclosure detects an object approaching a moving body from multiple images captured at a predetermined frame rate. When another object partially occludes the object in the multiple images, the object detection method generates a composite image by combining the multiple images without aligning the positions of the objects in the multiple images, and detects the object from the generated composite image.

[0007] According to another aspect of the present disclosure, an object detection device detects an object approaching a moving object from a plurality of images captured at a predetermined frame rate. The object detection device includes an image synthesis unit that, when another object that partially obscures the object is present in the plurality of images, synthesizes the plurality of images to generate a composite image without aligning the positions of the object in the plurality of images, and an object detection unit that detects the object from the generated composite image.

[0008] Another aspect of the present disclosure is an object detection program for causing a computer to execute each process of the object detection method described above.

[0009] Another aspect of the present disclosure is a computer-readable recording medium on which an object detection program is recorded for causing a computer to execute each process of the object detection method described above.

[0010] According to the present disclosure, it is possible to provide an object detection method, an object detection device, an object detection program, and a recording medium storing an object detection program, which correctly detect objects that are partially obscured by other objects and that may collide with a moving body.

[0011] Fig. 1 is a block diagram showing the configuration of a moving body 1 including an object detection device 2 according to a first embodiment. Fig. 2 is a top view showing an example of a change over time in the relative positional relationship between the moving body 1 and an object T with which the moving body 1 may collide. Fig. 3 is a graph showing the relationship between the moving speed of the object T in an image and the distance to an intersection point C of the moving body 1. Fig. 4 is a flowchart showing an example of an object detection method according to the first embodiment. Fig. 5 is a flowchart showing an example of a camera control method according to a second embodiment.

[0012] The embodiments will be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[0013] 1 , an object detection device 2 detects an object approaching the mobile body 1 from a plurality of images captured at a predetermined frame rate by a camera 4 mounted on the mobile body 1 around the mobile body 1. The objects approaching the mobile body 1 include objects that may collide with the mobile body 1. In the first embodiment, a case will be described in which the object detection device 2 is mounted on the mobile body 1, but the object detection device 2 does not have to be mounted on the mobile body 1. For example, the object detection device 2 may be disposed near an intersection point C, which will be described later. In this case, the object detection device 2 is connected to the mobile body 1 via road-to-vehicle communication so as to be capable of wireless communication.

[0014] The types of the moving body 1 and the target object include vehicles (land moving bodies) such as automobiles, motorcycles, tricycles, and bicycles, air moving bodies such as airplanes, helicopters, drones, and gliders, and maritime moving bodies such as ships, submarines, and yachts. In the first embodiment, the description will continue taking as an example a case where the moving body 1 and the target object are both vehicles.

[0015] The mobile body 1 is equipped with an object detection device 2, a vehicle speed measurement unit 3 that measures the traveling speed of the mobile body 1, a camera 4 that captures images of the surroundings of the mobile body 1, and a GPS (Global Positioning System) receiver (not shown) that receives signals from GPS satellites.

[0016] The camera 4 includes an imaging unit 12 that captures images multiple times at an arbitrary frame rate (fps) to generate multiple images, and a camera control unit 11 that controls the imaging unit 12. The camera 4 is attached, for example, to the windshield of the vehicle 1 facing forward, and is capable of capturing images of the surroundings of the vehicle 1 including the area ahead of the vehicle 1. The camera 4 is a digital camera that includes an image sensor such as a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS), a lens system that focuses light on the image sensor, and an image processing engine that generates image data from signals acquired by the image sensor.

[0017] The camera control unit 11 controls the exposure (electronic shutter speed, lens aperture, ISO sensitivity) of the imaging unit 12. The camera control unit 11 controls the frame rate and the number of times the camera 4 captures images continuously. The camera control unit 11 controls the frame rate and the number of times the camera 4 captures images based on an instruction signal received from the controller 5 (size change rate calculation unit 21), which will be described later. The camera 4 may be a high-speed camera capable of capturing images at a frame rate of 1000 fps or more. The angle of view or the focal length of the lens of the camera 4 may be constant or variable. If the angle of view or focal length is variable, the camera control unit 11 controls the angle of view or focal length.

[0018] The vehicle speed measurement unit 3 measures the traveling speed of the mobile object 1. For example, the vehicle speed measurement unit 3 may be a wheel speed sensor that detects the rotation speed of wheels provided on the vehicle (mobile object 1) as a signal.

[0019] The object detection device 2 can be realized using a microcomputer including a controller 5 serving as a central processing unit (CPU), a memory 6 including a random-access memory (RAM) and a read-only memory (ROM), and input / output units. For example, the microcomputer may be an electronic control unit (ECU) mounted on a vehicle (mobile object 1). A computer program (object detection program) for causing the microcomputer to function as the object detection device 2 is installed and executed on the microcomputer. This causes the microcomputer to function as the multiple information processing units (21-29) included in the object detection device 2. Note that while an example of implementing the object detection device 2 using software is shown here, it is also possible to configure the object detection device 2 using dedicated hardware for executing each information processing function. Dedicated hardware includes devices such as application-specific integrated circuits (ASICs) and conventional circuit components configured to perform the functions described in the embodiments. Furthermore, the multiple information processing units (21-29) included in the object detection device 2 may be configured as separate hardware. The object detection program can be recorded on a computer-readable recording medium.

[0020] As shown in FIG. 1, the object detection device 2 includes a controller 5, a memory 6, and an alarm unit 7.

[0021] The controller 5 has the above-mentioned multiple information processing units, including an object estimation unit 39, a size change rate calculation unit 21, a movement speed calculation unit 22, an image shift unit 23, an image integration unit 24, a gradation normalization unit 25, a pattern matching unit 26, a vertical edge extraction unit 27, a density calculation unit 28, and an occlusion determination unit 29.

[0022] The memory 6 stores map data indicating roads on which a moving body (1(to), 1(t1)) and an object (T(to), T(t1)) as shown in Figure 2 can travel. The symbols 1(to) and 1(t1) indicate the positions of the moving body 1 at different times. Similarly, the symbols T(to) and T(t1) indicate the positions of the object at different times, and an object whose position does not matter is called an "object T."

[0023] The alarm unit 7 issues an alarm buzzer to the user of the object detection device 2, such as a passenger of the moving body 1, to warn of the presence of an object T approaching the moving body 1 or of an object T that may collide with the moving body 1.

[0024] The object estimation unit 39 estimates the position on the map of an object approaching the mobile object 1 and the traveling speed of the object. For example, when the mobile object 1 is approaching an intersection, the object estimation unit 39 can estimate the position and traveling speed of an object that is assumed to be a potential collision among objects approaching the intersection, from the position on the map and traveling speed of the mobile object 1. The estimated position on the map of the object and the traveling speed of the object are temporarily stored in the memory 6.

[0025] The size change rate calculation unit 21 calculates the rate of change in size of the object T in multiple images captured at a predetermined frame rate using the camera 4. Since the object T approaching the moving body 1 approaches the moving body 1 between multiple image captures, the size of the object T in the images increases. The size change rate calculation unit 21 calculates the rate of change in size of the object T between the multiple images. Then, it calculates a predetermined frame rate and the number of multiple images so that the rate of change in size of the object T between the first and last images of the multiple images is equal to or less than a predetermined value (for example, 1%). The size change rate calculation unit 21 transmits an instruction signal indicating the calculated predetermined frame rate and the number of multiple images to the camera control unit 11.

[0026] For example, the size change rate calculation unit 21 reads the position and traveling speed of the mobile object 1 on the map. The position of the mobile object 1 on the map can be determined from a GPS signal. The traveling speed of the mobile object 1 is the traveling speed measured by the vehicle speed measurement unit 3. The size change rate calculation unit 21 reads map data of the road on which the mobile object 1 is traveling from the memory 6 based on the position of the mobile object 1 on the map. As an example, a case where the map data shown in FIG. 2 is read will be described. FIG. 2 shows, as an example of an object T approaching the mobile object 1, an object T (another vehicle) emerging from a side street (second road 42) that may collide with the mobile object 1 (host vehicle). The "side street" refers to a road that intersects with the main road (first road 41) and is narrower than the main road. The object T is included within the angle of view of the camera 4 mounted on the mobile object 1.

[0027] 2, a first road 41 on which the mobile body 1 is traveling intersects with a second road 42 at an intersection point C located ahead in the traveling direction of the mobile body 1. An object T traveling on the second road 42 toward the intersection point C is an object approaching the mobile body 1 and may collide with the mobile body 1, so it is desirable to detect the object T correctly.

[0028] The size change rate calculation unit 21 reads from the memory 6 the position on the map and the traveling speed of the object T traveling on the second road 42 toward the intersection point C, which have been estimated by the object estimation unit 39. Specifically, the size change rate calculation unit 21 reads from the memory 6 the estimated position on the map and the traveling speed of the object T traveling on the second road 42 toward the intersection point C, which is assumed to have a possibility of collision.

[0029] The size change rate calculation unit 21 calculates the relative speed of the object T with respect to the moving body 1 from the position and traveling speed of the object T on the map shown in FIG. 2 and the position and traveling speed of the moving body 1. From the relative speed of the object T with respect to the moving body 1, the rate of change per unit time of the distance between the moving body 1 and the object T can be calculated. The size change rate calculation unit 21 sets a predetermined frame rate and the number of multiple images according to the rate of change per unit time of the distance between the moving body 1 and the object T. This makes it possible to suppress the rate of change in size of the object T between the first image and the last image of the multiple images to a predetermined value (e.g., 1%) or less, so that when the multiple images are combined, blurring of the object T can be suppressed and a clear image of the object T can be generated.

[0030] The higher the frame rate, i.e., the shorter the image capture interval, the more images can be combined. Conversely, the lower the frame rate, i.e., the longer the image capture interval, the fewer images need to be combined.

[0031] The predetermined frame rate and the number of images may be set by other methods. For example, the size change rate calculation unit 21 may set the predetermined frame rate and the number of images based on the traveling speed of the moving object 1. As shown in FIG. 2 , when the first road 41, which is the main road, is prioritized over the second road 42, which is a side road, it is assumed that the traveling speed of the object T is sufficiently slower than the traveling speed of the moving object 1. In this case, the predetermined frame rate and the number of images may be set by regarding the relative speed of the object T with respect to the moving object 1 as the traveling speed of the moving object 1. This reduces the calculation load and speeds up the calculation process.

[0032] As another method, the size change rate calculation unit 21 may obtain the speed limit of the driving zone in which each of the moving body 1 and the object T is traveling from the map data read from the memory 6, and set the predetermined frame rate and the number of multiple images based on the speed limit. For example, the size change rate calculation unit 21 may set the predetermined frame rate and the number of multiple images based on the speed limit of the first road 41 and the second road 42. This makes it possible to calculate the relative speed of the object T with respect to the moving body 1 based on the respective traveling speeds of the moving body 1 and the object T estimated from the speed limit. Therefore, the predetermined frame rate and the number of multiple images can be set by a simpler method.

[0033] The movement speed calculation unit 22 calculates the movement speed and movement direction of the object T in multiple images, for example, from the estimated position and traveling speed of the object T on the map shown in Figure 2 and the position and traveling speed of the moving body 1 on the map.

[0034] With reference to Figure 3, changes in the estimated movement speed of the object T in the image will be described with respect to the estimated position on the map and travel speed of the object T. Figure 3 shows the movement speed of the object T in the image calculated by the travel speed calculation unit 22. The vertical axis indicates the movement speed of the object T in the image. A positive movement speed on the vertical axis indicates the movement speed in the left direction, and a negative movement speed on the vertical axis indicates the movement speed in the right direction. The travel speed of the moving object 1 is constant at 50 km / h, the estimated travel speed of the object T is constant at 40 km / h, and the frame rate of the camera 4 is 300 fps. The horizontal axis indicates the distance of the moving object 1 to the intersection point C.

[0035] The position of object T when moving body 1 reaches intersection point C is defined as the "intersection position." The intersection position is a relative position with respect to intersection point C. When the intersection position is a negative value (-2 m, -1 m in FIG. 3), moving body 1 arrives at intersection point C before object T. On the other hand, when the intersection position is a positive value (+2 m, +1 m in FIG. 3), object T arrives at intersection point C before moving body 1. When the intersection position is zero (±0 m in FIG. 3), moving body 1 and object T arrive at intersection point C at the same time. In this way, the graph in FIG. 3 shows the change in the movement speed of object T in the image for five objects T that have the same traveling speed but different intersection positions.

[0036] When the intersection position is a negative value, the object T moves leftward in the image. On the other hand, when the intersection position is a positive value, the object T moves rightward in the image. In both cases where the intersection position is a negative value and a positive value, the moving speed of the object T in the image increases as the distance to the intersection point C of the moving object 1 decreases.

[0037] In contrast, when the intersection position is zero, as shown in FIG. 2, the moving speed of the object T remains constant at zero regardless of the distance to the intersection point C of the moving object 1. In other words, the object T does not move in the image but is located at the same place in the image. The intersection position of zero corresponds to the fact that the first time it takes for the moving object 1 to reach the intersection point C is the same as the second time it takes for the object T to reach the intersection point C. Therefore, when there is a possibility that the object T will collide with the moving object 1, the object T will be located at the same place in the image.

[0038] FIG. 2 shows the positions of moving body 1(to) and object T(to) at time to, and the positions of moving body 1(t1) and object T(t1) at time t1, which is later than time to, when the intersection position in FIG. 3 is zero. Assume that the traveling speeds of moving body 1 and object T are constant, as in the case of FIG. 3. At any time to and any time t1, triangle A(to)B(to)C and triangle A(t1)B(t1)C are similar. Note that symbols A(to) and A(t1) indicate the center position of object T, and symbols B(to) and B(t1) indicate the position of camera 4 mounted on moving body 1. Therefore, angle θ(to) and angle θ(t1) are equal, and the angle θ at which object T is viewed from moving body 1 (camera 4) does not change.

[0039] The image shifting unit 23 calculates the amount of movement of the object T between the images when the multiple images are superimposed, based on the movement speed and movement direction of the object T estimated by the movement speed calculation unit 22. The amount of movement of the object includes the movement direction and movement distance of the object. The image shifting unit 23 can calculate the movement direction and movement distance of the object between the images based on the estimated movement speed and movement direction of the object and a predetermined frame rate of the camera 4. In this way, the movement speed calculation unit 22 and the image shifting unit 23 can calculate the amount of movement of the object T between the multiple images, based on the estimated position on the map and traveling speed of the object T.

[0040] For example, the image shifting unit 23 shifts the second image in the opposite direction of the movement direction relative to the first image by the movement distance and overlays them, thereby matching the position of the object T in the second image with the position of the object T in the first image. The same process is performed for the third and subsequent images, shifting them by the movement amount of the object T and overlaying them, thereby matching the positions of the object T. This makes it possible to overlay multiple images with the positions of the object T matched, so that the object T appears clearer through integration and gradation normalization, which will be described later.

[0041] The image integration unit 24 generates one integral image by integrating the multiple images superimposed by the image shift unit 23. The integration is performed on the entire image.

[0042] The gradation normalization unit 25 normalizes the gradation of the integral image by gamma correction. Specifically, the gradation normalization unit 25 compresses the integral image, whose gradation has increased through integration, to the same gradation as the image generated by the imaging unit 12. In other words, normalizing the gradation of an image is a process of scaling the pixel values ​​of the image to a certain range. Generally, pixel values ​​are normalized to a range from 0 to 1 or a range from 0 to 255. Multiple images are synthesized by the above process. The synthesis of multiple images includes integrating the multiple images and normalizing the gradation of the integral image. The synthesized multiple images are called a composite image.

[0043] The pattern matching unit 26 detects the object T by searching for a pattern image of the object T from within the entire composite image through pattern matching. The main pattern matching techniques that can be used are template matching and feature point matching. Template matching is a technique that calculates the similarity between a template image and a composite image and finds the position where they most closely match. Feature point matching is a technique that extracts feature points (e.g., corners or edges) and compares images based on these feature points. Typical feature point matching algorithms include SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), and ORB (Oriented FAST and Rotated BRIEF).

[0044] The vertical edge extraction unit 27 extracts multiple vertical edges extending in the vertical direction from at least one of the multiple images. Specifically, the vertical edge extraction unit 27 calculates the absolute value of the difference between pixel values ​​of adjacent pixels as edge strength to generate an edge image. The vertical edge extraction unit 27 performs a Hough transform on the edge image to extract vertical edge lines. The vertical edge extraction unit 27 calculates the number of pixels between the vertical edge lines. For example, the vertical edge extraction unit 27 can generate the edge image using a Sobel filter.

[0045] The density calculation unit 28 calculates the density and number of the extracted vertical edges. Specifically, it calculates the density of the vertical edges in the left-right direction and the number of consecutive vertical edges. The density of the vertical edges may be the average value of the density within a predetermined range.

[0046] The occlusion determination unit 29 determines whether the density and number of the plurality of vertical edges are equal to or greater than predetermined reference values. If the density and number of the plurality of vertical edges are equal to or greater than the predetermined reference values, the occlusion determination unit 29 determines that another object that partially occludes the target T is present in the plurality of images. On the other hand, if at least one of the density and number of the plurality of vertical edges is less than the predetermined reference value, the occlusion determination unit 29 determines that no other object that partially occludes the target T is present in the plurality of images.

[0047] Examples of other objects that partially occlude the object T include wire mesh, utility poles, and street trees. The contours of these other objects that partially occlude the object T include many vertical edges, so the presence of other objects can be determined based on the density and number of vertical edges.

[0048] As described above, when there is another object in the multiple images that partially obscures the object T, the object detection device 2 generates a composite image by combining the multiple images without aligning the position of the object T in the multiple images, and detects the object T from the generated composite image. On the other hand, when there is no other object in the multiple images that partially obscures the object T, the object detection device 2 aligns the position of the object in the multiple images, combines the aligned multiple images to generate a composite image, and detects the object T approaching the moving body 1 from the generated composite image.

[0049] When there is another object that partially obscures the object T, it is difficult to clearly see the object T located behind it in the image. For example, if there is an obstruction such as a wire mesh, a utility pole, or a roadside tree on the line connecting the camera B(t0) and the center A(to) of the object T(to) in Figure 2, it is difficult to clearly identify the object T(to) in the image captured by the camera B(t0). Therefore, it is difficult to correctly detect the object T from the image and accurately grasp the amount of movement of the object T, and it is difficult to match the position of the object T between multiple images.

[0050] In this embodiment, the amount of movement of the object T in the image is calculated based on the estimated position and traveling speed of the object T. Therefore, the estimated position of the object T in the image can be matched between multiple images. According to this embodiment, there is no need to detect the amount of movement of the object T from the image, and therefore the object T is not affected by other objects that partially obscure it.

[0051] Furthermore, in this embodiment, assuming an object T that may collide with the moving body 1, the object T does not move in images captured from the moving body 1 between predetermined frames, but remains in the same location in the images. Therefore, the object detection device 2 synthesizes multiple images without aligning the position of the object T in the images. In other words, the movement amount and movement speed of the object T are calculated as zero. This allows multiple images to be synthesized by matching the position of the object T that may collide with the moving body 1 and is partially occluded by another object. Even if the object T is partially occluded by another object, the object T that may collide with the moving body 1 can be clearly identified from the synthesized image. Unlike Patent Document 1, the object T is not extracted from the captured images, so there is no risk of erroneously calculating the movement amount of the object T.

[0052] On the other hand, if there is no other object that partially occludes the object T in the multiple images, the object estimation unit 39 can estimate various objects T with different intersection positions shown in Fig. 3, thereby aligning and synthesizing the positions of various objects T approaching the moving body 1. The various objects T approaching the moving body 1 are not limited to the object T (intersection position = 0) that may collide with the moving body 1, but also include objects T before and after it (intersection positions = +1, +2, -1, -2).

[0053] [Object Detection Method] An example of an object detection method according to the first embodiment will be described with reference to the flowchart in FIG. 4 . Here, as an example of an object detection method, an operation method of the object detection device 2 will be described. In step S01, the object detection device 2 reads speed information (traveling speed) of the mobile object 1 measured by the vehicle speed measurement unit 3, and also reads the position of the mobile object 1 on the map from the received GPS signal. Proceeding to step S02, the object estimation unit 39 estimates the position and traveling speed of the object T on the map. Specifically, the object estimation unit 39 estimates the position and traveling speed of the object T on the map that are assumed from the position and traveling speed of the mobile object 1 on the map. The estimated position and traveling speed on the map are stored in the memory 6.

[0054] Proceeding to step S03, the movement speed calculation unit 22 calculates the movement speed and movement direction of the object T in the multiple images from the position on the map and running speed of the object T estimated in step S02 and the position on the map and running speed of the moving body 1 read in step S01.

[0055] Proceeding to step S04, the object detection device 2 instructs the camera control unit 11 to control the imaging unit 12. Specifically, the object detection device 2 instructs the camera control unit 11 to control a predetermined frame rate and the number of times to capture images. Based on this instruction, the imaging unit 12 captures images a predetermined number of times at the predetermined frame rate to generate multiple images. The multiple images are temporarily stored in the memory 6, and the object detection device 2 reads the multiple images from the memory 6.

[0056] In step S05, the vertical edge extraction unit 27 extracts a plurality of vertical edges extending in the up-down direction from at least one of the plurality of images read in. In step S06, the density calculation unit 28 calculates the density and number of the extracted vertical edges.

[0057] In step S07, the occlusion determination unit 29 determines whether or not there is another object in the multiple images that partially occludes the target object. Specifically, it determines whether or not the density and number of multiple vertical edges are equal to or greater than predetermined reference values. If the determination is affirmative (YES in step S07), the process proceeds to step S09. If the determination is negative (NO in step S07), the process proceeds to step S08.

[0058] In step S08, the image shifting unit 23 calculates the amount of movement of the object T between the images when the multiple images are superimposed, based on the movement speed and movement direction of the object T calculated in step S03. The image shifting unit 23 shifts the multiple images by the amount of movement of the object T and superimposes them, thereby aligning the positions of the object T.

[0059] In step S09, the image integration unit 24 integrates the multiple images superimposed in step S08 to generate a single integral image. Then, in step S10, the gradation normalization unit 25 normalizes the gradation of the integral image. Specifically, the gradation normalization unit 25 compresses the integral image, the gradation of which has increased as a result of integration, to the same gradation as the image generated by the imaging unit 12.

[0060] The process proceeds to step S11, where the pattern matching unit 26 detects the object T by searching for a pattern image of the object T from within the entire composite image through pattern matching.

[0061] The process proceeds to step S12, and if an object T is detected (YES in step S12), the process proceeds to step S13, where the controller 5 activates the alarm unit 7. The alarm unit 7 issues a warning buzzer to a user of the object detection device 2, such as a passenger in the moving body 1. This makes it possible to warn of the presence of an object T approaching the moving body 1 or the presence of an object T that may collide with the moving body 1. On the other hand, if an object T is not detected (NO in step S12), the controller 5 does not activate the alarm unit 7, and the flowchart in FIG. 4 ends.

[0062] If the answer is YES in step S07, that is, if another object that partially occludes the object T is present in the multiple images, the process proceeds to step S09 without performing step S08. In other words, the object detection device 2 combines the multiple images without aligning the position of the object T in the multiple images. The amount of movement in the images of the object T that may collide with the moving object 1 can be estimated to be zero. Therefore, the object T that is partially occluded by another object and that may collide with the moving object 1 can be correctly detected.

[0063] The present disclosure can be embodied as an object detection program for causing a computer to execute each process of the object detection method described with reference to Fig. 4. The present disclosure can be embodied as a computer-readable recording medium on which the object detection program is recorded.

[0064] (Second embodiment) The frame rate and number of images taken by the camera 4 can be freely set according to the relative speed of the object T with respect to the moving body 1, thereby increasing the number of images to be synthesized while keeping the rate of change in the size of the object T below a predetermined value (e.g., 1%).

[0065] The object detection method according to the second embodiment will be described with reference to Fig. 5. Note that in the object detection method according to the second embodiment, the processing in step S04 in Fig. 4 is different, but the other steps S01 to S03 and S05 to S13 are the same as those in the first embodiment, and therefore will not be described again.

[0066] In the object detection method according to the second embodiment, steps S51 and S52 in FIG. 5 are performed instead of step S04 in FIG.

[0067] In step S51, the size change rate calculation unit 21 sets the frame rate of the camera 4 and the number of multiple images based on the traveling speed and position on the map of the moving body 1 read in step S01, and the position on the map and traveling speed of the object T estimated in step S02. In detail, the size change rate calculation unit 21 calculates the relative speed of the object T with respect to the moving body 1 from the traveling speed and position on the map of the moving body 1 and the position on the map and traveling speed of the object T. The size change rate calculation unit 21 calculates the rate of change per unit time of the distance between the moving body 1 and the object T from the relative speed of the object T with respect to the moving body 1. The frame rate of the camera 4 and the number of multiple images are set according to the rate of change per unit time of the distance between the moving body 1 and the object T. As a result, when images are captured at the set frame rate and number of multiple images, the rate of change in size of the object T between the first and last images of the multiple images can be kept to a predetermined value (e.g., 1%) or less.

[0068] Proceeding to step S52, the size change rate calculation unit 21 instructs the camera control unit 11 to control the imaging unit 12. Specifically, the size change rate calculation unit 21 instructs the camera control unit 11 to capture images at the set frame rate and the set number of images. Based on this instruction, the imaging unit 12 captures images at the set frame rate the set number of times and generates a plurality of images.

[0069] As described above, according to the second embodiment, it is possible to suppress the rate of change in size of the object T between the first and last images of the multiple images to a predetermined value (for example, 1%) or less by setting the frame rate and the number of images taken in consideration of the positional relationship and speed of the object T and the moving body 1. Therefore, it is possible to increase the number of multiple images to be synthesized while suppressing blurring of the object T in the synthesized image, and thus it is possible to obtain a clear image of the object T.

[0070] Although the embodiments have been described above, the descriptions and drawings that form part of this disclosure should not be understood to limit the present invention. Various alternative embodiments, examples, and operating techniques will become apparent to those skilled in the art from this disclosure.

[0071] The position on the map and the traveling speed of the object T to be read in step S03 of Fig. 4 can be arbitrarily selected, as shown in Fig. 3. On the other hand, the position on the map and the traveling speed of the object T to be read may be limited to the object T whose intersection position in Fig. 3 is zero. That is, the object T to be detected may be limited to the object T that arrives at the intersection point C at the same time as the moving object 1. In this case, the image shifting in step S08 is unnecessary, and the object detection device 2 can simply combine multiple images without aligning the positions of the objects in the multiple images. Accordingly, steps S01 to S03 for calculating the movement amount of the object T may also not be performed.

[0072] Furthermore, if it is determined in step S07 that no other object exists that partially occludes the object T, the pattern matching in step S11 may be performed without performing the image integration in step S09 and the tone normalization in step S10. Since no other object exists that partially occludes the object T, the object T can be accurately detected from the image before image synthesis.

[0073] REFERENCE SIGNS LIST 1 Vehicle (moving body) 2 Object detection device 3 Vehicle speed measurement unit 4 Camera 5 Controller 6 Memory 7 Alarm unit 21 Size change rate calculation unit 22 Travel speed calculation unit 23 Image shift unit 24 Image integration unit 25 Gradation normalization unit 26 Pattern matching unit 27 Vertical edge extraction unit 28 Density calculation unit 29 Occlusion determination unit 39 Object estimation unit 41 First road 42 Second road C Intersection point T Object

Claims

1. A method for detecting an object approaching a moving object from a plurality of images captured at a predetermined frame rate using a camera mounted on the moving object, wherein, if another object that partially obscures the object is present in the plurality of images, the plurality of images are synthesized to generate a composite image without aligning the position of the object in the plurality of images, and the object is detected from the synthesized image.

2. The object detection method according to claim 1, wherein, if there is no other object in the plurality of images that partially obscures the object, the position of the object in the plurality of images is aligned between the plurality of images, and the aligned plurality of images are combined to generate the combined image.

3. The object detection method according to claim 2, further comprising: estimating the position and traveling speed of the object on the map; calculating the amount of movement of the object between the plurality of images based on the estimated position and traveling speed of the object on the map; and overlaying the plurality of images with a shift equal to the amount of movement of the object to generate the composite image.

4. An object detection method according to any one of claims 1 to 3, comprising extracting a plurality of vertical edges extending in the vertical direction from at least one of the plurality of images, and determining that there is another object in the plurality of images that partially obscures the object if the density and number of the plurality of vertical edges are equal to or greater than predetermined reference values.

5. An object detection method according to any one of claims 1 to 4, wherein the predetermined frame rate and the number of the plurality of images are set so that the rate of change in size of the object between the first image and the last image in the plurality of images captured at the predetermined frame rate is equal to or less than a predetermined value.

6. A method for detecting an object according to any one of claims 1 to 5, wherein the predetermined frame rate and the number of the plurality of images are set according to the rate of change per unit time of the distance between the moving body and the object.

7. The object detection method according to any one of claims 1 to 5, wherein the predetermined frame rate and the number of the plurality of images are set based on the traveling speed of the moving object.

8. An object detection method according to any one of claims 1 to 5, wherein the moving body and the object are vehicles, the speed limit of the driving area in which each of the moving body and the object is driving is obtained from map data, and the predetermined frame rate and the number of the plurality of images are set based on the speed limit.

9. An object detection method according to any one of claims 1 to 8, wherein the moving body and the object are vehicles, a first road on which the moving body is traveling and a second road on which the object is traveling intersect in front of the moving body and the object, and a first time required for the moving body to reach the intersection point where the first road and the second road intersect is equal to a second time required for the object to reach the position.

10. The object detection method according to claim 9, wherein the amount of movement of the object between the plurality of images is estimated to be zero.

11. An object detection device that detects an object approaching a moving body from multiple images captured at a predetermined frame rate using a camera mounted on the moving body, the object detection device comprising: an image synthesis unit that, when another object that partially obscures the object is present in the multiple images, synthesizes the multiple images to generate a synthetic image without aligning the position of the object in the multiple images between the multiple images; and an object detection unit that detects the object from the generated synthetic image.

12. An object detection program for causing a computer to execute each process of the object detection method according to any one of claims 1 to 10.

13. A computer-readable recording medium having recorded thereon an object detection program for causing a computer to execute each process of the object detection method according to any one of claims 1 to 10.

Images