Object detection device

By combining ambient light images and point group information, the accuracy problem of the object detection device in the prior art when clusters are oversegmented and segmented is solved, higher-precision object detection is achieved, and the ability to determine the object size and distance is enhanced.

CN115176175BActive Publication Date: 2024-11-22DENSO CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202180015257.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2021-02-08
Filing Date
2021-02-16
Publication Date
2024-11-22
Estimated Expiration
2041-02-16

AI Technical Summary

Technical Problem

When existing object detection devices use point groups for clustering, it is difficult to correctly detect objects, especially when clusters are oversegmented or over-bound, resulting in a decrease in detection accuracy, especially when the last and the This time the consistency of the cluster is generated, which affects the accuracy of object recognition.

Method used

By combining ambient light images and point group information, a high-resolution irradiation and light receiving element array is used to generate ambient light images, distance images and reflection intensity images, clustering and identification of clusters, and using rectangular encirclement and overlap rate to determine the cluster Correctness and switch resolutions when necessary to correct cluster over-bound or segmentation.

Benefits of technology

It improves the accuracy and accuracy of object detection, can maintain consistent detection effects during the initial distance measurement and subsequent distance measurement, reduces misidentification and misdividing of object types, and enhances the ability to determine object size and distance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115176175B_ABST
    Figure CN115176175B_ABST
Patent Text Reader

Abstract

The object detection device of the present invention comprises an irradiation unit (2), a light receiving unit (3), and a detection unit (51 to 54, 61). The light receiving unit is configured to receive light reflected from the light irradiated by the irradiation unit, i.e., reflected light, and ambient light. The detection unit is configured to detect a specified object based on a point group and at least one image, wherein the point group is information based on the reflected light. The point group is a group of reflection points detected in the entire distance measurement area. The at least one image includes an image based on ambient light, i.e., an ambient light image, an image based on the distance from the object detected based on the reflected light, i.e., a distance image, and / or an image based on the reflection intensity of the reflected light, i.e., a reflection intensity image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to an object detection device. Background Art

[0002] Patent Document 1 describes an object recognition device that detects an object using a cluster generated by clustering a plurality of detection points detected by a laser radar. Specifically, the object recognition device determines a cluster representing an object by calculating the degree of consistency between a cluster generated last time and a cluster generated this time. At this time, the object recognition device calculates the degree of consistency from the cluster of the root node toward the cluster of the child node by utilizing the fact that the cluster has a tree structure.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2013-228259

[0004] However, as a result of the inventor's detailed research, the following problems were discovered. That is, when clustering is performed using only point groups as in the device described in Patent Document 1, it is difficult to detect objects in correct units. For example, in the case where a cluster smaller than the cluster that should have been generated is generated as a cluster with a root node for an object set as a detection target, it is difficult to detect a cluster larger than the cluster with the root node as an object, resulting in over-segmentation. In addition, for example, in the case where there is no cluster generated last time, that is, in the initial clustering, it is impossible to calculate the consistency between the cluster generated last time and the cluster generated this time, so it is difficult to determine the cluster representing the object, and the detection accuracy becomes low. Summary of the invention

[0005] One aspect of the present disclosure provides an object detection device capable of detecting an object with higher accuracy and in a correct unit.

[0006] One embodiment of the present disclosure is an object detection device, which includes an irradiation unit, a light receiving unit, and a detection unit. The irradiation unit is configured to irradiate light to a specified distance measurement area. The light receiving unit is configured to receive light reflected from the light irradiated by the irradiation unit, i.e., reflected light, and ambient light. The detection unit is configured to detect a specified object based on a point group as information based on the reflected light, and at least one image. The point group is a group of reflection points detected in the entire distance measurement area. The at least one image includes an image based on ambient light, i.e., an ambient light image, an image based on the distance to an object detected based on the reflected light, i.e., a distance image, and / or an image based on the reflection intensity of the reflected light, i.e., a reflection intensity image.

[0007] According to such a configuration, an object can be detected in a correct unit with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram showing the structure of an object detection device.

[0009] Figure 2FIG. 1 is an example of a schematic diagram showing an image object of a pedestrian.

[0010] Figure 3 This is an example of a schematic diagram showing a pedestrian portion of the point group.

[0011] Figure 4 FIG. 1 is an example of a schematic diagram showing an image object of a mixer truck.

[0012] Figure 5 This is an example of a schematic diagram showing a portion of a mixer truck in a point group.

[0013] Figure 6 This is a flowchart showing the first half of the object detection process according to the first embodiment.

[0014] Figure 7 This is a flowchart showing the second half of the object detection process according to the first embodiment.

[0015] Figure 8 This is a flowchart showing the first half of the object detection process according to the second embodiment.

[0016] Fig. 9 : is a flowchart showing the object detection process according to the modification example of the second embodiment. DETAILED DESCRIPTION

[0017] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings.

[0018] [1.Structure]

[0019] Figure 1 The object detection device 1 shown is mounted on a vehicle for use, and detects an object existing in front of the vehicle by irradiating light and receiving reflected light from an object that reflects the irradiated light.

[0020] like Figure 1 As shown, the object detection device 1 includes an irradiation unit 2 , a light receiving unit 3 , a storage unit 4 , and a processing unit 5 .

[0021] The irradiation unit 2 irradiates the laser beam to the distance measurement area in front of the vehicle. The distance measurement area is an area extending in a predetermined angle range in both the horizontal direction and the vertical direction. The irradiation unit 2 scans the laser beam in the horizontal direction.

[0022] The light receiving unit 3 detects the amount of incident light from the distance measurement area. The incident light detected by the light receiving unit 3 includes not only reflected light of the laser light emitted by the irradiating unit 2 and reflected by an object, but also ambient light such as reflected light of sunlight.

[0023] The ranging area is divided into a plurality of segmented areas. The ranging area can detect the amount of incident light in each of the plurality of segmented areas. When the ranging area is represented as a two-dimensional plane visually confirmed when observing the front (i.e., the irradiation direction of the laser) with the light receiving unit 3 as the viewpoint, the plurality of segmented areas correspond to the areas formed by dividing the two-dimensional plane into multiple segments in the horizontal and vertical directions. If observed as a three-dimensional space, the segmented areas are spatial areas having lengths along the straight line extending from the light receiving unit 3. For each segmented area, a correspondence is established with the horizontal angle and the vertical angle of the straight line mentioned above to determine.

[0024] In this embodiment, the distance measurement area is divided into smaller segments than those of conventional general LIDARs. For example, the number of segments in the distance measurement area is designed to be 500 segments in the horizontal direction and 100 segments in the vertical direction in the above-mentioned two-dimensional plane.

[0025] Each segmented area corresponds to one or more light receiving elements. According to the number of light receiving elements corresponding to a segmented area, the size of the segmented area (that is, the size of its area on the above-mentioned two-dimensional plane) changes. The fewer the number of light receiving elements corresponding to a segmented area, the smaller the size of a segmented area, and the higher the resolution. In order to achieve such a structure, the light receiving unit 3 has a light receiving element array in which a plurality of light receiving elements are arranged. The light receiving element array is composed of, for example, SPAD and other photodiodes. In addition, SPAD is the abbreviation of Single Photon Avalanche Diode.

[0026] As described above, the incident light includes reflected light and ambient light. In the light receiving waveform representing the relationship between time and the amount of incident light obtained by sampling the amount of incident light for a certain period starting from the laser irradiation timing, the reflected light formed by the laser irradiated by the irradiation unit 2 and reflected by the object is detected as a peak value that can be fully distinguished from the ambient light. The distance from the reflection point where the laser is reflected by the object is calculated based on the time from the irradiation timing of the laser by the irradiation unit 2 to the detection timing of the reflected light. Therefore, the three-dimensional position of the reflection point is determined based on the horizontal and vertical angles of the divided area and the distance from the object detection device 1. Since the three-dimensional position of the reflection point is determined in each divided area, the three-dimensional position of the point group as a group of detected reflection points is determined in the ranging area as a whole. That is, for the object that reflects the laser, its three-dimensional position and the horizontal and vertical dimensions in the three-dimensional space are determined. In addition, for the clustering and other processing described later, the three-dimensional position of the reflection point is converted into X, Y, and Z coordinate values.

[0027] Ambient light is detected as a light receiving waveform during a period in which reflected light is not detected. For example, a light receiving waveform after a period set for detecting reflected light of a laser may be detected as ambient light. As described above, after detecting the amount of incident light from each divided area, the light receiving unit 3 generates a grayscale image with a resolution of 500 pixels horizontally and 100 pixels vertically and multiple grayscale levels as an ambient light image based on the received ambient light. That is, the ambient light image becomes the same image as the one taken in front of the vehicle by a camera. In addition, since the angular position of each reflection point in the point group corresponds one-to-one to the position of each pixel in the ambient light image, the correspondence between the object identified by image analysis of the ambient light image and the object identified in the point group can be determined with high accuracy. It should be noted that the ambient light image is generated by the image generation unit 61 described later.

[0028] The storage unit 4 stores type information, distance thresholds, and size thresholds.

[0029] The so-called type information is the type of the object to be detected. The objects to be detected include pedestrians, preceding vehicles, and other objects that the driver should pay attention to while driving.

[0030] The so-called distance threshold is a threshold set for each type of object as a standard for the distance range of the object to be detected. For example, if the possibility of detecting a pedestrian at a position farther than a predetermined distance is extremely low due to important factors such as the performance of the object detection device 1 and the driving environment, the predetermined distance is set as the distance threshold for the pedestrian. In addition, the distance threshold used may also be changed according to the driving environment, etc.

[0031] The so-called size threshold is a threshold set for each type of object as a standard for the appropriate size of the object to be detected. For example, the upper limit of the height and width ranges that are likely to be pedestrians and the possibility of being pedestrians exceeding the ranges are extremely low are set as the size threshold for pedestrians. By not setting the lower limit, for example, even if only the upper body of a pedestrian is captured, it is possible to determine that it is a pedestrian.

[0032] The processing unit 5 is mainly composed of a well-known microcomputer having a CPU, ROM, RAM, flash memory, etc. (not shown). The CPU executes a program stored in the ROM as a non-transitory physical recording medium. By executing the program, a method corresponding to the program is executed. Specifically, the processing unit 5 executes the following method according to the program. Figure 6 as well as Figure 7 The object detection process shown. In addition, the processing unit 5 may include one microcomputer or a plurality of microcomputers.

[0033] The processing unit 5 includes a point cloud generation unit 51, a cluster generation unit 52, a recognition unit 53, an object detection unit 54, a switching unit 55, and an image generation unit 61 as functional modules implemented by the CPU executing a program, that is, virtual components. The method of implementing the functions of each unit included in the processing unit 5 is not limited to software, and part or all of the functions may also be implemented using one or more hardware. For example, when the above functions are implemented by an electronic circuit as hardware, the electronic circuit may also be implemented by a digital circuit, an analog circuit, or a combination thereof.

[0034] The point group generation unit 51 generates a point group based on the light receiving waveform. The so-called point group is a group of reflection points detected in the entire distance measurement area. The so-called reflection point means a point reflecting the laser from the irradiation unit 2, which is obtained in each of the above-mentioned divided areas. By changing the number of light receiving elements corresponding to a divided area, the point group resolution can be switched in the point group. The so-called point group resolution is the number of units (i.e., divided areas) for detecting multiple reflection points constituting the point group.

[0035] The cluster generation unit 52 generates a plurality of clusters by clustering the point cluster generated by the point cluster generation unit 51 .

[0036] The image generation unit 61 generates an ambient light image, a distance image, and a reflection intensity image. The distance image is an image in which each pixel represents the distance from the reflection point where the laser light emitted by the irradiation unit 2 is reflected by the object. The reflection intensity image is an image in which each pixel represents the intensity of the reflected light received by the light receiving unit 3 from the laser light emitted by the irradiation unit 2 and reflected by the object. The resolution of each image can be switched.

[0037] The recognition unit 53 analyzes the ambient light image and detects the portion of the object recognized as the detection target, that is, the image object, in the ambient light image. That is, the recognition unit 53 detects objects that match the type information stored in the storage unit 4 from the ambient light image. As a method for detecting the image object, for example, Deep Learning, machine learning, etc. can be used.

[0038] The object detection unit 54 detects, from the point group, a cluster corresponding to the image object detected by the recognition unit 53. The detection of the cluster corresponding to the image object will be described in detail later.

[0039] The switching unit 55 switches the resolution as a switching process. With respect to the resolution in the vertical direction, in the light receiving unit 3, the resolution is increased by reducing the number of light receiving elements used in one pixel and increasing the number of pixels in the vertical direction. The object detection device 1 is configured to be able to switch between a first resolution in which a first number of light receiving elements among the plurality of light receiving elements is set as one pixel, and a second resolution in which a second number of light receiving elements, which is smaller than the first number among the plurality of light receiving elements, is set as one pixel. In the present embodiment, the object detection device 1 is able to switch between a default resolution in which 24 light receiving elements in total, 6 in the horizontal direction and 4 in the vertical direction, are set as one pixel, and a high-level resolution in which 12 light receiving elements in total, 6 in the horizontal direction and 2 in the vertical direction, are set as one pixel. On the other hand, with respect to the resolution in the horizontal direction, in the irradiation unit 2, the resolution is increased by narrowing the interval of the scanning laser and increasing the number of pixels in the horizontal direction. In the present embodiment, in the case of a high-level resolution, an image with a resolution of 1000 pixels in the horizontal direction and 200 pixels in the vertical direction is generated as an ambient light image. In addition, in the present embodiment, the point group resolution is set to be consistent with the resolution of the above-mentioned image. Specifically, in the case of a high level, a point group is generated as a group of reflection points detected in 1000 divided areas in the horizontal direction and 200 divided areas in the vertical direction.

[0040] Here, use Figure 2 as well as Figure 3 , taking the case of a wall and a pedestrian approaching as an example. Figure 2 In the ambient light image, when a pedestrian 22 is detected as an image object separately from a wall 21, Figure 3 In the point group of , there is a case where the wall 23 and the pedestrian 24 are not distinguished and are detected as one cluster. Here, the state where a plurality of clusters that should be distinguished in units of objects are combined into one cluster is called cluster over-combination.

[0041] In addition, use Figure 4 as well as Figure 5 , take a mixer truck as an example. Figure 4 As shown, even if the mixer truck 25 is detected as an image object in the ambient light image, Figure 5 In the point group of , there is a case where the mixer truck is detected as two clusters with the front part 26 of the vehicle body and the tank part 27. Here, the state where one cluster that should be distinguished in units of objects is divided into multiple clusters is called cluster over-division.

[0042] That is, in Figure 2 to Figure 5 In the case shown, even if the object can be detected in the ambient light image in the correct unit, it may be difficult to detect the object in the point group in the correct unit.

[0043] On the contrary, even if an object can be detected in the point group in the correct unit, it is difficult to detect the object in the ambient light image in the correct unit. For example, in the ambient light image, a part of a wall with a spot pattern, an arrow painted on the road surface, etc. may be detected as an image object representing a pedestrian.

[0044] Therefore, the object detection device 1 according to the present embodiment executes an object detection process that improves the object detection accuracy by utilizing both the ambient light image and the point group.

[0045] [2. Processing]

[0046] use Figure 6 as well as Figure 7 The object detection process performed by the processing unit 5 of the object detection device 1 is described with reference to the flowchart. The object detection process is performed each time the ranging of the entire ranging area is completed. In addition, when the object detection process starts, the resolution is set to the default resolution. In addition, in the description of this process, when only the resolution is referred to, both the resolution of the image and the point group resolution of the point group are included.

[0047] First, in S101 , the processing unit 5 generates a point cloud. S101 corresponds to the processing of the point cloud generating unit 51 .

[0048] Next, in S102 , the processing unit 5 generates a plurality of clusters by clustering the point group. Each generated cluster has no type information as an initial value. S102 corresponds to the processing of the cluster generating unit 52 .

[0049] Next, in S103, the processing unit 5 detects image objects from the ambient light image. By detecting the image objects, the type of the image objects is also identified. When there are multiple objects to be detected in the ambient light image, the processing unit 5 detects multiple image objects from the ambient light image. The following processing is performed for each image object. In addition, although clusters are generated in S102, if image objects are not detected in S103, the processing unit 5 moves to S112 and detects each generated cluster as an object, and then ends. Figure 6 At this time, each generated cluster is detected as an object without category information. In addition, S103 corresponds to the processing of the recognition unit 53.

[0050] Next, in S104, the processing unit 5 detects the cluster corresponding to the image object marker. Specifically, first, the processing unit 5 surrounds the image object marker detected in S103 with a rectangle in the ambient light image. In addition, the processing unit 5 regards the point group as a two-dimensional plane having information on the angular position of each reflection point, and surrounds the multiple clusters generated in S102 with rectangles in the point group. Next, the processing unit 5 detects the rectangle of the cluster that overlaps with the rectangle of the image object marker, and detects the cluster as the cluster corresponding to the image object marker. Here, in the case where there are multiple rectangles of clusters that overlap with the rectangle of the image object marker, the rectangle of the cluster with the largest overlap rate with the rectangle of the image object marker is detected, and the cluster is detected as the cluster corresponding to the image object marker. That is, the processing unit 5 establishes the correspondence between the image object marker and the cluster. In addition, in the case where there is no rectangle of the cluster that overlaps with the rectangle of the image object marker, the image object marker is invalidated and the process ends. Figure 6 Object detection processing.

[0051] Next, in S105, the processing unit 5 determines whether the distance to the object indicated by the image object mark is appropriate. Specifically, when the distance to the object indicated by the image object mark is within the distance threshold, the processing unit 5 determines that the distance is appropriate. It is not possible to determine whether the distance to the object is appropriate using only the ambient light image, but it is possible to determine whether the distance to the object is appropriate by using a point group having information about the distance to the reflection point. That is, since the image object mark is associated with the cluster, for example, the distance between the center point of the image object mark corresponding cluster and the object detection device 1 can be used as the distance between the image object mark and the object detection device 1. It should be noted that in the following description, the so-called pixel or number of pixels of the image object mark corresponding cluster refers to the unit for detecting multiple reflection points constituting the point group, that is, the segmented area or the number of segmented areas.

[0052] When the processing unit 5 determines in S105 that the distance to the object indicated by the image object is appropriate, it moves to S106 to determine whether the size of the object indicated by the image object is appropriate. Specifically, when the size of the object indicated by the image object converges to the size threshold, the processing unit 5 determines that the size is appropriate. It is not possible to determine whether the size of the object is appropriate using only the ambient light image, but it is possible to determine whether the size of the object is appropriate by using a point group having information on the three-dimensional position of the reflection point. The size of the object indicated by the image object is inferred based on the portion in the point group corresponding to the image object. The so-called portion in the point group corresponding to the image object is the portion of the angular position corresponding to the position of each pixel of the image object in the point group. For example, when the number of pixels of the image object corresponding cluster is larger than the number of pixels of the image object, the size of the portion in the point group corresponding to the image object in the image object corresponding cluster is inferred as the size of the object indicated by the image object. In addition, for example, when the number of pixels of the cluster corresponding to the image object marker is smaller than the number of pixels of the image object marker, the size of the cluster obtained by multiplying the ratio of the number of pixels of the image object marker to the number of pixels of the cluster corresponding to the image object marker by the size of the cluster corresponding to the image object marker is inferred as the size of the object shown by the image object marker.

[0053] When the processing unit 5 determines in S106 that the size of the object indicated by the image object is appropriate, the process moves to S107 to determine whether the number of pixels of the image object is equal to the number of pixels of the cluster corresponding to the image object. Specifically, when the difference obtained by subtracting the number of pixels in the image object from the number of pixels in the cluster corresponding to the image object converges to an upper limit value or less and a lower limit value of a pixel number threshold value indicating a predetermined pixel number range, the processing unit 5 determines that the number of pixels of the image object is equal to the number of pixels of the cluster corresponding to the image object. For example, when the pixel number threshold value indicates a range of plus or minus 10 pixels, the upper limit value indicates plus 10 and the lower limit value indicates minus 10.

[0054] When the processing unit 5 determines in S107 that the number of pixels of the image object mark is different from the number of pixels of the cluster corresponding to the image object mark, the process moves to S108 to determine whether the clusters are over-connected. Whether the clusters are over-connected is determined based on whether there is a cluster in the part of the point group corresponding to the image object mark that is larger than the part of the point group corresponding to the image object mark, that is, an over-connected cluster. For example, when the difference between the number of pixels in the cluster corresponding to the image object mark and the number of pixels in the image object mark is larger than the upper limit of the pixel number threshold, it is determined that there is an over-connected cluster. When there is an over-connected cluster, the processing unit 5 determines that the cluster is over-connected.

[0055] If the processing unit 5 determines in S108 that the clusters are excessively combined, the process moves to S109 to determine whether the switching process has been performed. In this embodiment, the processing unit 5 determines whether the resolution has been switched. The resolution switching process is performed in S110 or S115 described later.

[0056] If the processing unit 5 determines in S109 that the resolution has not been switched, it proceeds to S110, performs a switching process, i.e., switches the resolution to a higher level, and then returns to S101. That is, the processing unit 5 executes the processes in S101 to S108 again in a state where the resolution of the image and the point group is higher.

[0057] On the other hand, if the processing unit 5 determines in S109 that the resolution has been switched, the process moves to S111. That is, the processing unit 5 executes the processes in S101 to S108 again in a state where the resolution of the image and the point group is higher, and moves to S111 if it is still determined that the clusters are excessively combined.

[0058] The processing unit 5 divides the over-joined cluster in S111. The processing unit 5 divides the over-joined cluster so that the shortest distance between the portion of the over-joined cluster corresponding to the image object, i.e., the target cluster, and the portion of the over-joined cluster other than the portion corresponding to the image object, i.e., the adjacent cluster, is larger than the largest distance between two adjacent points in the target cluster and is larger than the largest distance between two adjacent points in the adjacent cluster. In addition, the processing unit 5 may directly divide the portion of the over-joined cluster corresponding to the image object into one cluster.

[0059] Next, in S112, the processing unit 5 detects the partial clusters in the point group corresponding to the image object as objects. That is, when it is determined in S108 that the clusters are over-joined and the over-joined clusters are split in S111, the processing unit 5 detects the partial clusters in the over-joined clusters corresponding to the image object as objects with category information. In addition, in S112, the processing unit 5 detects the adjacent clusters split from the over-joined clusters as objects without category information. Then, the processing unit 5 ends. Figure 6 Object detection processing.

[0060] On the other hand, when the processing unit 5 determines in S108 that the clusters are not over-combined, the process moves to S113 to determine whether the clusters are over-divided. Whether the clusters are over-divided is determined based on whether there are two or more clusters in the portion of the point group corresponding to the image object. Specifically, when the difference between the number of pixels in the cluster corresponding to the image object and the number of pixels in the image object is smaller than the lower limit value in the pixel number threshold, and when there are more than one cluster in the portion of the point group corresponding to the image object in addition to the cluster corresponding to the image object, the processing unit 5 determines that the clusters are over-divided.

[0061] If the processing unit 5 determines in S113 that the cluster is over-divided, the process proceeds to S114 to determine whether the switching process has been performed. In the present embodiment, the processing unit 5 determines whether the resolution has been switched.

[0062] If the processing unit 5 determines in S114 that the resolution has not been switched, it moves to S115, performs the switching process, that is, switches the resolution to a higher level of resolution, and then returns to S101. That is, the processing unit 5 executes the processes in S101 to S108 and S113 again in a state where the resolution of the image and the point group is higher. It should be noted that S110 and S115 correspond to the processes of the switching unit 55.

[0063] On the other hand, if the processing unit 5 determines in S114 that the resolution has been switched, the process moves to S116. That is, the processing unit 5 executes the processes in S101 to S108 and S113 again in a state where the resolution of the image and the point group is higher, and moves to S116 if it is still determined that the cluster is over-divided.

[0064] After the processing unit 5 combines two or more clusters existing in the point group corresponding to the image object in S116, it moves to S112. That is, when two or more clusters are combined in S116, the processing unit 5 detects the combined cluster as an object having type information. Then, the processing unit 5 ends Figure 6 Object detection processing.

[0065] If the processing unit 5 determines in S113 that the cluster is not over-divided, the process proceeds to S112, where the image object corresponding cluster is detected as an object, and the process ends. Figure 6 In this case, the image object corresponding cluster is detected as an object without category information. In addition, even if the processing unit 5 determines in S113 that the cluster is not over-divided, if there are multiple cluster rectangles overlapping with the image object rectangle in S104, the cluster with the second largest overlap rate with the image object rectangle is regarded as the image object corresponding cluster, and the processing from S105 onwards is repeated.

[0066] On the other hand, if the processing unit 5 determines in S107 that the number of pixels of the image object mark is equal to the number of pixels of the image object mark corresponding cluster, the processing unit 5 moves to S112, detects the image object mark corresponding cluster, which is a cluster of part of the point group corresponding to the image object mark, as an object having type information, and then ends the processing. Figure 6 This means that the number of pixels in the cluster corresponding to the image object is almost the same as the number of pixels in the image object, and the cluster corresponding to the image object is neither over-divided nor over-joined. In other words, the clusters representing the object shown by the image object and the parts in the point group corresponding to the image object are detected in correct units.

[0067] On the other hand, if it is determined in S106 that the size of the object indicated by the image object mark is inappropriate, the processing unit 5 invalidates the object mark image. In addition, the processing unit 5 moves to S112 and detects the image object mark corresponding cluster as an object, and then ends. Figure 6 At this time, the image object corresponding cluster is detected as an object without category information.

[0068] In addition, if it is determined in S105 that the distance to the object indicated by the image object mark is inappropriate, the processing unit 5 also invalidates the object mark image. In addition, after the processing unit 5 moves to S112 and detects the image object mark corresponding cluster as an object, the processing unit 5 ends. Figure 6 In this case, the image object corresponding cluster is detected as an object without category information. In addition, S104 to S108, S111 to S113, and S116 correspond to the processing of the object detection unit 54.

[0069] [3. Effect]

[0070] According to the above-described detailed embodiments, the following effects can be obtained.

[0071] (3a) The object detection device 1 detects a specified object based on a point group and an ambient light image. According to such a structure, compared with the case where a specified object is detected in a point group without using an ambient light image, it is easier to identify the type and unit of the object in the point group. In addition, compared with the case where the object is detected by calculating the consistency between the cluster generated last time and the cluster generated this time, the object can be detected at the initial distance measurement with the same accuracy as the second and subsequent distance measurements. Therefore, according to the object detection device 1, the object can be detected in the correct unit with higher accuracy.

[0072] (3b) When the object detection device 1 determines that two or more clusters among the plurality of clusters generated by clustering the point group exist in a portion of the point group corresponding to the image object, the object detection device 1 detects the two or more clusters as one object. According to such a structure, even when the clusters in the point group are over-divided, the object detection device 1 can detect the object in the correct unit.

[0073] (3c) When the object detection device 1 determines that an over-connected cluster larger in size than a portion in the point group corresponding to the image object exists in a portion in the point group corresponding to the image object among a plurality of clusters generated by clustering the point group, the object detection device 1 detects the portion in the over-connected cluster corresponding to the image object as an object. According to such a structure, even when clusters are over-connected in the point group, the object detection device 1 can detect an object in a correct unit.

[0074] (3d) The object detection device 1 divides the over-connected cluster so that the shortest distance between the portion of the over-connected cluster corresponding to the image object, i.e., the object cluster, and the portion of the over-connected cluster other than the portion corresponding to the image object, i.e., the adjacent cluster, is larger than the largest distance between two adjacent points in the object cluster and is larger than the largest distance between two adjacent points in the adjacent cluster. According to such a structure, the object detection device 1 can detect the object in a correct unit, compared with the case where the portion of the over-connected cluster corresponding to the image object is directly divided and formed into one cluster.

[0075] (3e) When the object detection device 1 determines that the size of the object is within a predetermined size range based on the type of the object shown by the image object, the object detection device 1 detects a portion of the point group corresponding to the image object as the object. That is, the object detection device 1 verifies the probability of the object based on the size assumed for each type of object. At this time, the object detection device 1 uses the ambient light image to identify the type of the object and uses the point group to calculate the size of the object. By combining not only the ambient light image but also the point group, the object detection device 1 can reduce the misidentification of the type of the object.

[0076] (3f) When the object detection device 1 determines that the distance to the object is within a preset distance range based on the type of the object shown by the image object, the object detection device 1 detects a portion of the point group corresponding to the image object as the object. That is, the object detection device 1 verifies the probability of the object based on the assumed existence position for each type of object. At this time, the object detection device 1 uses the ambient light image to identify the type of the object and uses the point group to calculate the distance to the object. By combining not only the ambient light image but also the point group, the object detection device 1 can reduce the misidentification of the type of the object.

[0077] (3g) In the object detection device 1, the light receiving unit 3 has a plurality of light receiving elements. The object detection device 1 can switch the resolution between a first resolution in which a first number of light receiving elements among the plurality of light receiving elements is regarded as one pixel, and a second resolution in which a second number of light receiving elements among the plurality of light receiving elements, which is smaller than the first number, is regarded as one pixel. According to such a configuration, the object detection device 1 can detect an object with higher accuracy than a case in which the resolution cannot be switched between the first resolution and the second resolution.

[0078] In the present embodiment, the point cloud generation unit 51 , the cluster generation unit 52 , the recognition unit 53 , the object detection unit 54 , and the image generation unit 61 correspond to the processing as a detection unit.

[0079] [4. Second embodiment]

[0080] [4-1. Differences from the first embodiment]

[0081] Since the basic structure and process of the second embodiment are the same as those of the first embodiment, the description of the common structure and process will be omitted and the description will focus on the differences.

[0082] In the first embodiment, the object detection device 1 detects the image target only from the ambient light image in S103 of the object detection process. On the other hand, in the second embodiment, the object detection device 1 detects the image target from each of the ambient light image, the distance image, and the reflection intensity image. In addition, in the second embodiment, the object detection device 1 switches the resolution of the point group, the ambient light image, the distance image, and the reflection intensity image according to the external brightness.

[0083] [4-2. Processing]

[0084] use Figure 8 The object detection process executed by the processing unit 5 of the object detection device 1 according to the second embodiment will be described with reference to the flowchart of FIG.

[0085] In S201, the processing unit 5 determines whether the brightness of the outside is brighter than a predetermined threshold value. For example, when the intensity of the ambient light is equal to or greater than a predetermined threshold value, the processing unit 5 determines that the outside is bright.

[0086] In S202, the processing unit 5 generates a point group with a point group resolution corresponding to the brightness of the outside. Specifically, when the processing unit 5 determines in S201 that the brightness of the outside is brighter than the predetermined threshold, the point group resolution is generated relatively low compared to the case where the processing unit 5 determines in S201 that the brightness of the outside is not brighter than the predetermined threshold. On the other hand, when the processing unit 5 determines in S201 that the brightness of the outside is not brighter than the predetermined threshold, the point group resolution is generated relatively high. The point group resolution is consistent with the resolution of the distance image and the reflection intensity image generated in S203.

[0087] Next, in S102 , the processing unit 5 generates a plurality of clusters by clustering the point group.

[0088] Next, in S203, the processing unit 5 generates an image with a resolution corresponding to the brightness of the outside. Specifically, when the processing unit 5 determines in S201 that the brightness of the outside is brighter than the predetermined threshold, the processing unit 5 generates an ambient light image with a relatively high resolution, and generates a distance image and a reflection intensity image with a relatively low resolution, compared to a case where the processing unit 5 determines in S201 that the brightness of the outside is not brighter than the predetermined threshold. On the other hand, when the processing unit 5 determines in S201 that the brightness of the outside is not brighter than the predetermined threshold, the processing unit 5 generates an ambient light image with a relatively low resolution, and generates a distance image and a reflection intensity image with a relatively high resolution.

[0089] In addition, in S203, the processing unit 5 detects image targets from each of the ambient light image, the distance image, and the reflection intensity image, and integrates the image targets. The so-called integration is to generate an image target used in the processing after S203 based on the image targets detected using the three types of images. For example, when the processing unit 5 detects an image target from any image of the three types of images, it adopts it as the image target. In addition, the method of integrating the image targets is not limited to this. For example, the image target detected from only any one image of the three types of images may not be adopted as the image target. That is, in this case, it is considered that the image target is not detected and the processing continues. In addition, for example, the image target detected from only any two images of the three types of images may not be adopted as the image target. In addition, when different image targets are detected in the three types of images, the image target may be determined based on the priority determined in advance for each image, or as the image target obtained by integrating the image targets detected in the two images. After this S203, the processing enters S104. Processing of S104 to S106 Figure 6 The processing of S104 to S106 shown in is the same.

[0090] When the processing unit 5 determines in S106 that the size of the object indicated by the image object is appropriate, the process proceeds to S204 to determine whether the number of pixels of the image object corresponds to the number of pixels of the cluster corresponding to the image object.

[0091] In S107 of the first embodiment described above, the processing unit 5 compares the number of pixels of the image object corresponding cluster and the image object in order to compare the size of the image object and the image object corresponding cluster. However, in the second embodiment, since the point group resolution is different from the image resolution, a simple comparison cannot be performed. Therefore, based on the point group resolution of the point group generated in S202 and the resolution of the image generated in S203, the ratio of the point group resolution to the image resolution is calculated. For example, if it is assumed that the image resolution is 500 pixels horizontally and 200 pixels vertically, and the point group resolution is 1000 pixels horizontally and 200 pixels vertically, the area of ​​one pixel of the image is twice the area of ​​one pixel of the point group. In this case, if the number of pixels of the image object corresponding cluster is twice the number of pixels of the image object, it can be said that the size of the same range in the distance measurement area is obtained. In this way, the above ratio is calculated, and on the basis of considering the ratio, it is calculated whether the size of the image object and the size of the image object corresponding cluster are equal. The above method is an example, and when the number of pixels of the image object corresponding cluster and the image object are different, various methods that can compare their sizes can be used.

[0092] If the processing unit 5 determines in S204 that the number of pixels of the image object marker does not correspond to the number of pixels of the image object marker corresponding cluster, the process proceeds to S108. On the other hand, if the number of pixels of the image object marker corresponds to the number of pixels of the image object marker corresponding cluster, the process proceeds to S112. The processes in and after S108 are the same as those in Figure 7 The processes of S108 to S116 shown in FIG. 1 are the same, so the description thereof is omitted.

[0093] [4-3. Effect]

[0094] According to the second embodiment described in detail above, the following effects can be obtained.

[0095] (4a) When the object detection device 1 determines that the external brightness is brighter, the object is detected based on an ambient light image with a relatively high resolution, a distance image with a relatively low resolution, and a reflection intensity image, compared to a case where the external brightness is not brighter. According to such a structure, in the ambient light image, since the image object is detected from the high-resolution ambient light image, the image recognition accuracy is high. In addition, in the distance image and the reflection intensity image, since the SN is improved, the detection distance tends to be extended. Therefore, objects farther away can be detected. In addition, the so-called SN is the ratio of signal to noise.

[0096] (4b) When the object detection device 1 determines that the external brightness is not bright, it detects the object based on an ambient light image with a relatively low resolution, a distance image with a relatively high resolution, and a reflection intensity image, compared to a case where the external brightness is determined to be bright. According to such a structure, since the reliability of the ambient light image is low when the external brightness is not bright, it is difficult to affect the reliability even if the resolution of the ambient light image is reduced. Therefore, it is possible to suppress the processing load and generate the ambient light image. In addition, in the distance image and the reflection intensity image, since there is less noise when the intensity of the ambient light is low, the detection distance tends to be longer. Therefore, even if the resolution is increased, the decrease in the detection distance can be suppressed.

[0097] (4c) In the object detection device 1, the point group resolution is consistent with the resolution of the range image and the reflection intensity image. According to such a structure, since the angular position of each reflection point in the point group corresponds to the position of each pixel in the range image and the reflection intensity image, it is easy to obtain the correspondence between the object recognized by performing image analysis on the range image and the reflection intensity image and the object recognized in the point group.

[0098] (4d) In the object detection device 1, the processing unit 5 generates a point group with a point group resolution corresponding to the brightness of the outside in S202, and generates an image with a resolution corresponding to the brightness of the outside in S203. In addition, when the processing unit 5 determines in S108 that the clusters are excessively combined, it switches to a high level of point group resolution and resolution in S110. When the processing unit 5 determines in S113 that the clusters are excessively divided, it also switches to a high level of point group resolution and resolution in S115. According to such a structure, as in the first embodiment, it is possible to detect an object with higher accuracy.

[0099] In addition, in this embodiment, S201 corresponds to the processing as a determination unit.

[0100] [4-4. Modification of the Second Embodiment]

[0101] (i) In the above embodiment, an object is detected based on three types of images, namely, an ambient light image, a distance image, and a reflection intensity image. However, the number of types of images used is not limited thereto. For example, at least one of the ambient light image, the distance image, and the reflection intensity image may be used. In addition, at least one of the ambient light image, the distance image, and the reflection intensity image may be used.

[0102] (ii) In the above-mentioned embodiment, when it is determined that the brightness of the outside is brighter, the resolution of the ambient light image is relatively high, and the point group resolution of the point group is relatively low, compared with the case where the brightness of the outside is determined not to be brighter. In addition, when it is determined that the brightness of the outside is not brighter, the resolution of the ambient light image is relatively low, and the point group resolution of the point group is relatively high, compared with the case where the brightness of the outside is determined to be brighter. That is, when the point group resolution of the point group is relatively low, the resolution of the ambient light image is set to be relatively high, and when the point group resolution of the point group is relatively high, the resolution of the ambient light image is set to be relatively low. However, the method of setting the point group resolution and the resolution is not limited thereto. For example, the point group resolution of the point group may be kept constant, while the resolution of the ambient light image may be switched to be higher or lower, or the resolution of the ambient light image may be kept constant, while the point group resolution of the point group may be switched to be higher or lower. In addition, for example, when the point group resolution of the point group is low, the resolution of the ambient light image may be switched to be lower, and when the point group resolution of the point group is high, the resolution of the ambient light image may be switched to be higher.

[0103] In addition, similarly to the distance image and the reflection intensity image, for example, the point group resolution of the point group may be kept constant, while the resolution of the distance image and the reflection intensity image may be switched to a higher or lower level, or the resolution of the distance image and the reflection intensity image may be kept constant, while the resolution of the point group of the point group may be switched to a higher or lower level. In addition, for example, when the point group resolution of the point group is low, the resolution of the distance image and the reflection intensity image may be switched to a lower level, or when the point group resolution of the point group is high, the resolution of the distance image and the reflection intensity image may be switched to a higher level.

[0104] According to such a configuration, the point group resolution of the point group and the resolution of the image can be independently set to appropriate values.

[0105] (iii) In the above embodiment, the resolutions of the ambient light image, the distance image, and the reflection intensity image are different. That is, the object is detected based on the ambient light image having a third resolution, the distance image having a fourth resolution different from the third resolution, and the reflection intensity image. However, the resolutions of the ambient light image, the distance image, and the reflection intensity image may be consistent. With such a structure, it is easy to obtain the correspondence between the image objects detected from each image.

[0106] (iv) In the above embodiment, the point group resolution is consistent with the resolution of the range image and the reflection intensity image. However, the point group resolution may not be consistent with the resolution of the range image and the reflection intensity image, or may be consistent with the resolution of only one of the range image and the reflection intensity image.

[0107] (v) In the above embodiment, a point group and an image are generated with a resolution corresponding to the external brightness. However, the resolution of the point group and the image may be set according to conditions other than the external brightness. For example, the resolution may be set according to the time, whether the headlights are on, the attributes of the road being driven, etc.

[0108] (vi) In the above embodiment, the external brightness is determined based on the intensity of the ambient light. However, the method of determining the external brightness is not limited to this. For example, an illuminance sensor may be used.

[0109] (vii) In the above embodiment, the processing unit 5 performs segmentation when the clusters are excessively combined, and performs segmentation when the clusters are excessively divided, through the processing of S107 to S111 and S113 to S116. However, the processing unit 5 may not perform the above segmentation and segmentation of the clusters. For example, Fig. 9 As shown, after the processing unit 5 detects the image object corresponding cluster in S104, it determines in S205 whether the distance from the object indicated by the image object is appropriate. Figure 6 The determination is performed in the same manner as in S105.

[0110] Next, in S206, the processing unit 5 determines whether the size of the object indicated by the image object is appropriate. Figure 6 The determination is performed in the same manner as in S106.

[0111] Next, in S207, the processing unit 5 detects an object. At this time, if it is determined in S205 that the distance to the object indicated by the image object mark is appropriate, and in S206 that the size of the object indicated by the image object mark is appropriate, the image object mark corresponding cluster is detected as an object having type information. Then, the processing unit 5 ends. Fig. 9 Object detection processing.

[0112] On the other hand, in S207, if it is determined in S205 that the distance to the object indicated by the image object mark is inappropriate, or in S206 that the size of the object indicated by the image object mark is inappropriate, the image object mark corresponding cluster is detected as an object without type information. Fig. 9 Object detection processing.

[0113] In addition, back to Figure 7 For example, the processing unit 5 may skip the processing of S109 and S110 after determining whether the clusters are excessively combined in S108. In addition, the processing unit 5 may skip the processing of S114 and S115 after determining whether the clusters are excessively divided in S113. That is, the processing unit 5 may divide or combine the clusters without performing the switching process and detect them as objects with category information.

[0114] [5. Other embodiments]

[0115] As mentioned above, although embodiment of this disclosure was described, this disclosure is not limited to the said embodiment, and it goes without saying that various forms can be adopted.

[0116] (5a) In the above embodiment, a configuration including a SPAD as a light receiving element is exemplified, but any light receiving element may be used as long as it can detect a temporal change in the amount of incident light.

[0117] (5b) In the first embodiment described above, a structure using an ambient light image is illustrated. However, the type of image used is not limited thereto. For example, at least one of a distance image and a reflection intensity image may be used in addition to or instead of the ambient light image. In addition, the distance image and the reflection intensity image are both generated based on the number of segmented areas, so that the angular position of each reflection point in the point group corresponds one-to-one to the position of each pixel in the distance image and the reflection intensity image. Thus, the correspondence between the object identified by performing image analysis on the distance image and the reflection intensity image and the object identified in the point group can be determined with high accuracy.

[0118] Here, when the ambient light image is used, the detection performance is high in sunny daytime, etc., but the detection performance decreases at night, in tunnels, etc. On the other hand, when the distance image and the reflection intensity image are used, there are opposite characteristics. Therefore, the object detection device 1 can detect objects with higher accuracy and in correct units by using these images together.

[0119] (5c) In the above embodiment, when there is suspicion of over-segmentation or over-combination, the ranging is performed again in the entire ranging area after the resolution is switched. However, the range in which the ranging is performed again is not limited to this. The object detection device 1 may also perform ranging again only in a part of the ranging area, for example, only in the range suspected of over-combination or over-segmentation. In this way, excessive detection of objects in the range where resolution switching is not required is suppressed, and detection timing delay can be easily prevented.

[0120] (5d) In the above embodiment, a structure is illustrated in which the resolution is switched between the first resolution and the second resolution by switching the number of light receiving elements of each pixel. However, the object detection device 1 can also switch the resolution by switching the range of the ranging area. Specifically, the angle range in the horizontal direction of the laser irradiated by the irradiation unit 2 is switched. For example, the object detection device 1 does not change the number of divided areas, but switches the angle range from -60° to +60° to -20° to +20°. When the angle range is narrowed without changing the number of divided areas, the number of divided areas in the angle range becomes relatively large, and the resolution becomes relatively high. Therefore, a more detailed point group can be generated. In addition, in the ambient light image, since one-third of the range is represented without changing the number of pixels, the resolution is also relatively high.

[0121] In addition, the object detection device 1 improves the SN by switching the number of times the laser is irradiated to each divided area from a first irradiation number to a second irradiation number that is larger than the first irradiation number as a switching process, in addition to or instead of switching the resolution. The so-called number of times the laser is irradiated refers to the number of times the object detection device 1 irradiates the laser to each divided area during one round of ranging in the ranging area. In addition, in the above-mentioned embodiment, once is set as the first irradiation number, and the laser is irradiated to each divided area one by one. According to such a structure, for example, Figure 5 Thus, even when the mixer truck is detected as two clusters with the front portion 26 and the tank portion 27 in the vehicle body, the object detection device 1 can easily detect the portion of the vehicle body connecting the front portion 26 and the tank portion 27 by increasing the SN. Thus, the object detection device 1 can detect the mixer truck as one cluster instead of two clusters.

[0122] In addition, in the above-mentioned embodiment, the whole distance measurement area is set as the range of the object irradiated with laser light, but it is considered that when the number of times of irradiating laser light to each divided area is increased, the detection cycle becomes longer, and the timing of detecting the object is delayed. Therefore, the object detection device 1 can also switch from the first irradiation number to the second irradiation number only in a part of the range of the distance measurement area, for example, only in the range suspected of over-combination or over-division. In this way, the object detection device 1 can suppress the timing delay of detecting the object and detect the object in the correct unit with higher accuracy.

[0123] (5e) In the above embodiment, an example is given of a configuration in which only an upper limit value is set as a size threshold value. However, a lower limit value may be set as a size threshold value in addition to or instead of the upper limit value.

[0124] (5f) In the above-mentioned embodiment, the object detection device 1 determines that there is an over-combined cluster when the number of pixels of the cluster corresponding to the image object is more than the number of pixels of the image object by a predetermined number of pixels. However, for example, the object detection device 1 may also determine whether there is an over-combined cluster by comparing the number of points of all reflection points constituting the cluster existing in the part of the point group corresponding to the image object, that is, the overall number of points, and the number of points of the reflection points corresponding to the part of the image object, that is, the partial number of points. In addition, for example, the object detection device 1 may also determine that there is an over-combined cluster when the value obtained by dividing the overall number of points by the partial number of points is greater than a predetermined value greater than 1.

[0125] (5g) The function of one component in the above-mentioned embodiment may be dispersed into a plurality of components, or the functions of a plurality of components may be integrated into a single component. In addition, a part of the structure of the above-mentioned embodiment may be omitted. In addition, at least a part of the structure of the above-mentioned embodiment may be added to or replaced with the structure of another above-mentioned embodiment.

Claims

1. An object detection device, wherein: have: an irradiation unit configured to irradiate light toward a predetermined distance measurement area; a light receiving unit configured to receive ambient light and light reflected from the light irradiated by the irradiating unit, i.e., reflected light; as well as a detection unit configured to detect a predetermined object based on a point group and at least one image, wherein the point group is information based on the reflected light, The above point group is a group of reflection points detected in the entire ranging area. The at least one image includes an image based on the ambient light, i.e., an ambient light image, an image based on the distance from the object detected based on the reflected light, i.e., a distance image, and / or an image based on the reflection intensity of the reflected light, i.e., a reflection intensity image. The detection unit is capable of detecting the object based on the at least one image of the first resolution and the point group of the first point group resolution indicating the number of units for detecting the plurality of reflection points constituting the point group, and is capable of detecting the object based on the at least one image of the second resolution higher than the first resolution and the point group of the second point group resolution higher than the first point group resolution, The light receiving unit has a plurality of light receiving elements. The detection unit is capable of switching between the first resolution in which a first number of light-receiving elements among the plurality of light-receiving elements is regarded as one pixel, and the second resolution in which a second number of light-receiving elements among the plurality of light-receiving elements, which is smaller than the first number, is regarded as one pixel. The detection unit detects a portion recognized as the object, that is, an image object in the at least one image, and generates a plurality of clusters by clustering the point group. When the detection unit determines that two or more of the multiple clusters exist in the portion of the point group corresponding to the image object, or determines that a combined cluster of the multiple clusters that is larger in size than the portion of the point group corresponding to the image object exists in the portion of the point group corresponding to the image object, the detection unit switches the resolution of the at least one image from the first resolution to the second resolution, detects the image object, switches the point group resolution of the point group from the first point group resolution to the second point group resolution, and clusters the point group.

2. The object detection device according to claim 1, wherein: The detection unit can detect the object based on the point group and the at least one image having a resolution different from a point group resolution indicating the number of units for detecting the plurality of reflection points constituting the point group.

3. An object detection device, wherein: have: an irradiation unit configured to irradiate light toward a predetermined distance measurement area; a light receiving unit configured to receive ambient light and light reflected from the light irradiated by the irradiating unit, i.e., reflected light; as well as a detection unit configured to detect a predetermined object based on a point group and at least one image, wherein the point group is information based on the reflected light, The above point group is a group of reflection points detected in the entire ranging area. The at least one image includes an image based on the ambient light, i.e., an ambient light image, an image based on the distance from the object detected based on the reflected light, i.e., a distance image, and / or an image based on the reflection intensity of the reflected light, i.e., a reflection intensity image. The detection unit can detect the object based on the point group and the at least one image having a resolution different from a point group resolution indicating the number of units for detecting the plurality of reflection points constituting the point group.

4. The object detection device according to claim 3, wherein: The detection unit can detect the object based on the point group and the ambient light image having a resolution different from the point group resolution of the point group.

5. An object detection device, wherein: have: an irradiation unit configured to irradiate light toward a predetermined distance measurement area; a light receiving unit configured to receive ambient light and light reflected from the light irradiated by the irradiating unit, i.e., reflected light; as well as a detection unit configured to detect a predetermined object based on a point group and at least one image, wherein the point group is information based on the reflected light, The above point group is a group of reflection points detected in the entire ranging area. The at least one image includes an image based on the ambient light, i.e., an ambient light image, an image based on the distance from the object detected based on the reflected light, i.e., a distance image, and / or an image based on the reflection intensity of the reflected light, i.e., a reflection intensity image. The detection unit may detect the object based on at least one of the ambient light image having a third resolution and the distance image and the reflection intensity image having a fourth resolution different from the third resolution.

6. The object detection device according to claim 5, wherein: The point group resolution indicating the number of units for detecting the plurality of reflection points constituting the point group is consistent with at least one of the resolution of the range image and the reflection intensity image.

7. The object detection device according to claim 5 or 6, wherein: Also available: A brightness determination unit determines external brightness. The detection unit detects the object based on the ambient light image with relatively high resolution and at least one of the distance image and the reflection intensity image with relatively low resolution, when the brightness determination unit determines that the external brightness is brighter than the prescribed threshold value, compared with a case where the external brightness is not brighter than the prescribed threshold value.

8. An object detection device, wherein: have: an irradiation unit configured to irradiate light toward a predetermined distance measurement area; a light receiving unit configured to receive ambient light and light reflected from the light irradiated by the irradiating unit, i.e., reflected light; as well as a detection unit configured to detect a predetermined object based on a point group and at least one image, wherein the point group is information based on the reflected light, The above point group is a group of reflection points detected in the entire ranging area. The at least one image includes an image based on the ambient light, i.e., an ambient light image, an image based on the distance from the object detected based on the reflected light, i.e., a distance image, and / or an image based on the reflection intensity of the reflected light, i.e., a reflection intensity image. The irradiation unit is capable of switching the number of times of irradiating light to at least a part of the range in the distance measurement area between a first irradiation number and a second irradiation number that is larger than the first irradiation number. The detection unit detects a portion recognized as the object, that is, an image object in the at least one image, and generates a plurality of clusters by clustering the point group. The irradiation unit switches from the first irradiation frequency to the second irradiation frequency when it is determined that two or more of the plurality of clusters exist in the portion of the point group corresponding to the image object, or when it is determined that an over-connected cluster of the plurality of clusters that is larger in size than the portion of the point group corresponding to the image object exists in the portion of the point group corresponding to the image object, When the number of times of irradiating light is switched from the first number of times of irradiation to the second number of times of irradiation, the detection unit detects the image object target at the second number of times of irradiation and clusters the point group.

9. The object detection device according to any one of claims 1, 3, 5, and 8, wherein: The detection unit detects a portion recognized as the object, i.e., an image object marker, in the at least one image, and when it is determined that two or more clusters among a plurality of clusters generated by clustering the point group exist in a portion of the point group corresponding to the image object marker, the two or more clusters are detected as one of the object.

10. The object detection device according to any one of claims 1, 3, 5, and 8, wherein: The detection unit detects a portion recognized as the object, i.e., an image object mark, in the at least one image. When it is determined that an over-connected cluster larger in size than a portion in the point group corresponding to the image object mark among a plurality of clusters generated by clustering the point group exists in the portion in the point group corresponding to the image object mark, the portion in the over-connected cluster corresponding to the image object mark is separated and detected as the object.

11. The object detection device according to claim 10, wherein: The over-combined cluster is divided so that the shortest distance between the portion of the over-combined cluster corresponding to the image object, i.e., the object cluster, and the portion of the over-combined cluster other than the portion corresponding to the image object, i.e., the adjacent cluster, is larger than the maximum distance between two adjacent points in the object cluster, and is larger than the maximum distance between two adjacent points in the adjacent cluster.

12. The object detection device according to any one of claims 1, 3, 5, and 8, wherein: The detection unit detects the portion recognized as the object, i.e., the image object mark, in the at least one image. When it is determined that the size of the object is within a predetermined size range based on the type of the object indicated by the image object mark, the portion of the point group corresponding to the image object mark is detected as the object.

13. The object detection device according to any one of claims 1, 3, 5, and 8, wherein: The detection unit detects a portion recognized as the object, i.e., an image object marker, in the at least one image, and when it is determined that the distance to the object is within a predetermined distance range based on the type of the object indicated by the image object marker, the portion of the point group corresponding to the image object marker is detected as the object.