Road object detection system and road object detection method

Abstract

To provide an on-road object detection system capable of filtering a detection target or detection result according to a pavement type of a road on which a vehicle travels, a material of the road, and a road surface state of the road.SOLUTION: An on-road object detection system includes an image capturing device mounted on a vehicle, and a control device. The control device includes: a road surface state determination unit for determining at least one of a pavement type of a road on which the vehicle travels, a material of the road, and a road surface state of the road, from an image captured by the image capturing device; an on-road object detection unit for detecting damage to the road and an object on the road, from the image; a detection condition decision unit for deciding on a detection condition of the on-road object detection unit on the basis of a determination result of the road surface state determination unit; and a filtering unit for filtering a detection target or detection result of the on-road object detection unit on the basis of the detection condition.SELECTED DRAWING: Figure 1

Description

Technical Field

[0006] , , ,

[0005] ,

[0001] The present disclosure relates to a road object detection system and a road object detection method.

Background Art

[0002] There is known a road surface state detection system for a vehicle including an imaging device that images a road surface on which the vehicle travels, and means for recognizing the type of the state of the road surface on which the vehicle travels based on an image captured by the imaging device, the means calculating an index value representing the likelihood that the type of the state of the road surface in the image is each of a plurality of types of states of the road surface, and recognizing the type with the largest index value as the type of the state of the road surface in the image (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the road surface state detection system disclosed in Patent Document 1, damage to the road on which the vehicle travels and objects on the road cannot be detected. On the other hand, in a road object detection system, damage to the road on which the vehicle travels and objects on the road can be detected, but depending on the state of the road surface, damage to the road or objects on the road that cannot be detected may be erroneously detected.

[0005] In view of the above circumstances, at least one embodiment of the present invention aims to provide a road object detection system and a road object detection method capable of filtering a detection target or a detection result according to the type of paving, the material of the road, and the road surface state of the road on which the vehicle travels.

Means for Solving the Problems

[0006] (1) A road object detection system according to at least one embodiment of the present invention is a road object detection system comprising an imaging device mounted on a vehicle and a control device, wherein the control device comprises: a road surface condition determination unit that determines from an image captured by the imaging device at least one of the type of pavement of the road on which the vehicle travels, the material of the road, and the road surface condition of the road; a road object detection unit that detects damage to the road and objects on the road from the image; a detection condition determination unit that determines the detection conditions of the road object detection unit based on the determination result of the road surface condition determination unit; and a filtering unit that filters the detection target or detection result of the road object detection unit based on the detection conditions.

[0007] According to the configuration described in (1) above, the detection conditions of the road surface condition determination unit are determined based on the determination result of the road surface condition determination unit, and the detection targets or detection results of the road surface condition determination unit are filtered based on the detection conditions. Therefore, the detection targets or detection results can be filtered according to the type of pavement, road material, and road surface condition of the road on which the vehicle is traveling.

[0008] (2) In some embodiments, in the configuration of (1) above, the filtering unit has an object detection execution determination unit that determines whether the road damage and the objects on the road are the objects to be detected, and which conform to the detection conditions.

[0009] According to the configuration described in (2) above, road damage and objects on the road that meet the detection conditions are targeted for detection. Therefore, road damage and objects on the road that are suitable for the type of pavement, material, and surface condition of the road on which the vehicle travels can be targeted for detection.

[0010] (3) In some embodiments, in the configuration of (1) or (2) above, the filtering unit has a detection object filtering unit which determines the road damage and objects on the road that meet the detection conditions as the detection result.

[0011] According to the configuration described in (3) above, road damage and objects on the road that meet the detection conditions are detected, so the detection results can be tailored to the type of pavement, material, and surface condition of the road on which the vehicle travels.

[0012] (4) A road object detection method according to at least one embodiment of the present invention comprises: a road surface condition determination step in which an imaging device determines from an image captured by an imaging device at least one of the type of pavement of the road on which a vehicle is traveling, the material of the road, and the road surface condition of the road; a road object detection step in which damage to the road and objects on the road are detected from the image; a detection condition determination step in which detection conditions for the road object detection step are determined based on the determination result of the road surface condition determination step; and a filtering step in which the detection target or detection result of the road object detection step is filtered based on the detection conditions.

[0013] According to the method described in (4) above, the detection conditions of the road surface condition determination unit are determined based on the determination result of the road surface condition determination unit, and the detection targets or detection results of the road surface object detection step are filtered based on the detection conditions. Therefore, the detection targets or detection results can be filtered according to the type of pavement, road material, and road surface condition of the road on which the vehicle is traveling. [Effects of the Invention]

[0014] According to at least one embodiment of the present invention, the detection target or detection results can be filtered based on the type of pavement, road material, and road surface condition of the road on which the vehicle travels. [Brief explanation of the drawing]

[0015] [Figure 1] This is a block diagram schematically showing the configuration of the road object detection system according to Embodiment 1. [Figure 2] This diagram shows the type of pavement, material, and surface condition of the road on which the vehicle is traveling, as well as detectable road damage and objects on the road. [Figure 3]Figure 1 is a flowchart illustrating the processing steps of the road object detection system. [Figure 4] This is a block diagram schematically showing the configuration of the road object detection system according to Embodiment 2. [Figure 5] Figure 4 is a flowchart illustrating the processing steps of the road object detection system. [Modes for carrying out the invention]

[0016] Hereinafter, several embodiments of the present invention will be described with reference to the attached drawings. However, the dimensions, materials, shapes, relative arrangements, etc., of the components described as embodiments or shown in the drawings are not intended to limit the scope of the present invention, but are merely illustrative examples. For example, expressions describing relative or absolute arrangements such as "in a certain direction," "along a certain direction," "parallel," "orthogonal," "center," "concentric," or "coaxial" should not only strictly represent such arrangements, but also represent states of relative displacement with tolerances, or angles or distances that allow the same function to be obtained. Furthermore, expressions describing shapes such as square or cylindrical should not only represent geometrically precise square or cylindrical shapes, but also shapes including concave and concave parts, chamfered parts, etc., to the extent that the same effect can be obtained. On the other hand, expressions such as "equipped," "possess," "features," "includes," or "has" a single component are not exclusive expressions that exclude the existence of other components.

[0017] [Embodiment 1] [Configuration of the road object detection system] Figure 1 is a schematic block diagram showing the configuration of the road object detection system 1 according to Embodiment 1. As shown in Figure 1, the road object detection system 1 according to Embodiment 1 is a road object detection system comprising an imaging device 10 mounted on a vehicle and a control device 12.

[0018] The imaging device 10 is, for example, a camera for an advanced driver-assistance system (ADAS) or a camera for a surround view monitor used for parking assistance. The control device 12 is composed of a processor including an arithmetic unit, a register for storing instructions and information, and peripheral circuits, a memory such as a ROM (Read Only Memory) and a RAM (Random Acess Memory), and an input interface.

[0019] The control device 12 includes a road surface condition determination unit 14, a road object detection unit 16, a detection condition determination unit 18, and a filtering unit 20.

[0020] The road surface condition determination unit 14 is a part that determines at least one of the type of road pavement, the material of the road, and the road surface condition of the road on which the vehicle travels from the image captured by the imaging device 10. For example, an image acquired from the imaging device 10 by the image acquisition unit 22, cut out by the image processing unit 24, and reduced, enlarged, or color-converted is input to the road surface condition determination unit 14. The road surface condition determination unit 14 determines at least one of the type of road pavement, the material of the road, and the road surface condition of the road on which the vehicle travels, for example, using an inference model pre-learned using machine learning (particularly, a convolutional neural network (CNN)). Examples of the type of road pavement include asphalt, concrete, and tiles, and examples of the material of the road include soil, gravel, sand, and wood, but are not limited thereto. Examples of the road surface condition include a state where the road surface is covered with water and a state where the road surface is covered with snow.

[0021] The road object detection unit 16 is a part that detects damage to the road on which the vehicle travels and objects on the road from the image captured by the imaging device 10. Similar to the road surface state determination unit 14, for the road object detection unit 16, for example, an image acquired from the imaging device 10 by the image acquisition unit 22, cut out by the image processing unit 24, and reduced, enlarged, or color-converted is input. Similar to the road surface state determination unit 14, the road object detection unit 16 uses, for example, an inference model pre-trained using machine learning (especially CNN) to detect damage to the road on which the vehicle travels and objects on the road. Examples of road damage include holes (potholes) that occur in a part of the surface of an asphalt pavement, and examples of objects on the road include structures (speed bumps) that raise a part of the road and cause vertical vibrations to passing vehicles to prompt the driver to decelerate, people, and automobiles, but are not limited thereto.

[0022] The detection condition determination unit 18 is a part that determines the detection conditions of the road object detection unit 16 based on the determination result of the road surface state determination unit 14. As described above, the determination result of the road surface state determination unit 14 is at least one of the type of pavement of the road on which the vehicle travels, the material of the road, and the road surface state, and the detection conditions of the road object detection unit 16 are to limit the road damage and the objects on the road to be detected according to the determination result of the road surface state determination unit 14. The road damage and the objects on the road to be detected according to the determination result of the road surface state are predetermined for each of the type of pavement of the road, the material of the road, and the road surface state.

[0023] For example, as shown in Figure 2, if the type of pavement is tile, potholes cannot be detected. Therefore, if the road surface condition determination unit 14 determines that "the road pavement is tile," potholes are excluded, and detection is limited to road damage other than potholes and objects on the road. For example, in the example shown in Figure 2, the detection conditions are limited to speed bumps, people, and automobiles. Similarly, if the road material is soil, there are no potholes or speed bumps. Therefore, if the road surface condition determination unit 14 determines that "the road material is soil," potholes and speed bumps are excluded, and detection is limited to road damage other than potholes and objects on the road other than speed bumps. For example, in the example shown in Figure 2, the detection conditions are limited to people and automobiles. Similarly, if the road surface is covered with snow, the detection of potholes and speed bumps is likely to be a false positive. Therefore, if the road surface condition determination unit 14 determines that "the road surface is covered with snow," potholes and speed bumps are excluded, and detection is limited to road damage other than potholes and objects on the road other than speed bumps. For example, in the example shown in Figure 2, the detection conditions are limited to people and automobiles.

[0024] The filtering unit 20 is the part that filters the targets to be detected by the road object detection unit 16 based on the detection conditions. The filtering unit 20 has an object detection execution determination unit 26 that targets road damage and objects on the road that meet the detection conditions.

[0025] Therefore, in the example described above, if the road surface condition determination unit 14 determines that "the road pavement is tiled," potholes are excluded from the detection targets of the road object detection unit 16, and road damage other than potholes and objects on the road become the detection targets. Similarly, if the road surface condition determination unit 14 determines that "the road material is soil," potholes and speed bumps are excluded from the detection targets of the road object detection unit 16, and road damage other than potholes and objects on the road other than speed bumps become the detection targets. Similarly, if the road surface condition determination unit 14 determines that "the road surface is covered with snow," potholes and speed bumps are excluded from the detection targets of the road object detection unit 16, and road damage other than potholes and objects on the road other than speed bumps become the detection targets.

[0026] Information on road damage and objects on the road detected by the road object detection unit 16 is transmitted by the communication control unit 28 from the communication device 30 to a server (not shown) that manages the road object detection system 1. The communication device 30 is, for example, a 4G modem, 5G modem, WiFi modem mounted on a vehicle, or a smartphone connected via USB (Universal Serial Bus), but is not limited to these.

[0027] [Method for detecting objects on the road] Figure 3 is a flowchart showing the processing details (road object detection method) of the road object detection system 1 shown in Figure 1. As shown in Figure 3, the road object detection system 1 according to Embodiment 1 performs the following steps: road surface condition determination step (step S13), detection condition determination step (step S14), filtering step (step S15), and road object detection step (step S16).

[0028] In the road surface condition determination step (step S13), the type of pavement, the material of the road, and the road surface condition of the road on which the vehicle is traveling are determined from the image captured by the imaging device 10. In the road surface condition determination step (step S13), the image acquired from the imaging device 10 in the image acquisition step (step S11), and the image that has been cropped, reduced, enlarged, or color converted in the image processing step (step S12) are used. In the road surface condition determination step (step S13), for example, an inference model trained using machine learning (particularly CNN) is used to determine the type of pavement, the material of the road, and the road surface condition of the road on which the vehicle is traveling.

[0029] The detection condition determination step (step S14) is the part in which the detection conditions for the road object detection step (step S16) are determined based on the determination result of the road surface condition determination step (step S13). As described above, the determination result of the road surface condition determination step (step S13) is at least one of the type of pavement of the road on which the vehicle is traveling, the material of the road, and the road surface condition of the road. The detection conditions of the road object detection unit 16 are to limit the road damage and objects on the road to be detected based on the determination result of the road surface condition determination step (step S13).

[0030] In the filtering step (step S15), the objects to be detected in the road object detection step are filtered based on the detection conditions. The filtering step (step S15) targets road damage and objects on the road that meet the detection conditions.

[0031] In the road object detection step (step S16), the road damage and objects on the road determined in the filtering step (step S15) are used as inspection targets, and the road damage and objects on the road being traveled on are detected from the images captured by the imaging device 10. In the road object detection step (step S16), similar to the road surface condition determination step, for example, images acquired from the imaging device 10 in the image processing step (step S12), cropped, reduced, enlarged, or color converted in the image processing step (step S12) are used. In the road object detection step (step S16), for example, an inference model trained using machine learning (particularly CNN) is used to detect the road damage and objects on the road being traveled on.

[0032] Information on road damage and objects on the road detected in the road object detection step (step S16) is transmitted by the communication control unit 28 from the communication device 30 to a server (not shown) that manages the road object detection system 1.

[0033] [Effectiveness of road object detection systems] According to the road object detection system 1 of Embodiment 1, the detection conditions of the road object detection unit 16 are determined based on the determination result of the road surface condition determination unit 14, and the objects to be detected by the road object detection unit 16 are filtered based on the detection conditions. Therefore, the objects to be detected can be filtered according to the type of pavement, the material of the road, and the road surface condition of the road on which the vehicle is traveling.

[0034] [Embodiment 2] [Configuration of the road object detection system] Figure 4 is a schematic block diagram showing the configuration of the road object detection system 4 according to Embodiment 2. Components identical to those in the road object detection system 1 according to Embodiment 1 are denoted by the same reference numerals and their descriptions are omitted.

[0035] As shown in Figure 4, the filtering unit 40 of the road object detection system 4 according to Embodiment 2 is the part that filters the detection results of the road object detection unit 16 based on the detection conditions determined by the detection condition determination unit 18. The filtering unit 40 has a detection object filtering unit 42 that detects road damage and objects on the road that meet the detection conditions.

[0036] Therefore, for example, if the road surface condition determination unit 14 determines that "the road pavement is tiled," potholes are excluded from the detection results of the road object detection unit 16, and road damage other than potholes and objects on the road become the detection results. Similarly, if the road surface condition determination unit 14 determines that "the road material is soil," potholes and speed bumps are excluded from the detection results of the road object detection unit 16, and road damage other than potholes and objects on the road other than speed bumps become the detection results. Similarly, if the road surface condition determination unit 14 determines that "the road surface is covered with snow," potholes and speed bumps are excluded from the detection results of the road object detection unit 16, and road damage other than potholes and objects on the road other than speed bumps become the detection targets.

[0037] [Method for detecting objects on the road] Figure 5 is a flowchart showing the processing details (road object detection method) of the road object detection system 4 shown in Figure 4. Processing details that are the same as those of the road object detection system 1 according to Embodiment 2 are denoted by the same reference numerals and their explanations are omitted.

[0038] As shown in Figure 5, the road object detection system 4 according to Embodiment 2 performs a road condition determination step (step S13), a road object detection step (step S21), a detection condition determination step (step S14), and a filtering step (step S22).

[0039] In the road object detection step (step S21) according to Embodiment 2, all road damage and objects on the road that are targeted for detection by the road object detection system 4 according to Embodiment 2 are targeted for detection. In this respect, it differs from the road object detection step (step S16) according to Embodiment 1, which targets road damage and objects on the road that have been filtered in the filtering step (step S15). Furthermore, in the road object detection step (step S21) according to Embodiment 2, similar to the road object detection step (step S16) according to Embodiment 1, for example, a reasoning model trained using machine learning (particularly CNN) is used to detect road damage and objects on the road that a vehicle is traveling on.

[0040] In the filtering step (step S22) according to Embodiment 2, the detection results of the road object detection step (step S21) are filtered based on the detection conditions. In this respect, it differs from the filtering step (S15) according to Embodiment 1, which targets road damage and objects on the road that have been filtered based on the detection conditions. The filtering step (step S22) according to Embodiment 2 detects road damage and objects on the road that meet the detection conditions.

[0041] [Effectiveness of road object detection systems] According to the road object detection system 4 of Embodiment 2, the detection conditions of the road object detection unit 16 are determined based on the determination result of the road surface condition determination unit 14, and the detection results of the road object detection unit 16 are filtered based on the detection conditions. Therefore, the detection results can be filtered according to the type of pavement, the material of the road, and the road surface condition of the road on which the vehicle is traveling.

[0042] The present invention is not limited to the embodiments described above, and includes modified forms of the embodiments described above, as well as forms that combine these forms as appropriate. [Explanation of Symbols]

[0043] 1. Road Object Detection System 10 Imaging device 12 Control device 14. Road surface condition determination unit 16. Road object detection unit 18 Detection condition determination unit 20 Filtering section 22 Image acquisition unit 24 Image Processing Unit 26 Object detection execution determination unit 28 Communication Control Unit 30 Communication equipment 4. Roadside detection system 40 Filtering section 42 Detected object filtering unit

Claims

1. A road object detection system comprising an imaging device mounted on a vehicle and a control device, The control device is A road surface condition determination unit determines, from the image captured by the imaging device, at least one of the type of pavement on the road on which the vehicle is traveling, the material of the road, and the road surface condition of the road. From the aforementioned image, a road object detection unit detects damage to the road and objects on the road, A detection condition determination unit determines the detection conditions for the road object detection unit, based on the determination result of the road surface condition determination unit, which limits the road damage and objects on the road to be detected. A filtering unit that filters the objects to be detected or the results of the road object detection unit based on the aforementioned detection conditions, A road object detection system equipped with [specific features / equipment].

2. The road surface condition determination unit determines that the road surface condition is tiled, and the detection condition determination unit excludes potholes from the targets to be detected by the road surface object detection unit and determines the detection conditions, the road surface object detection system according to claim 1.

3. If the road surface condition determination unit determines that the material of the road is soil, the detection condition determination unit excludes potholes and speed bumps from the targets to be detected by the road surface object detection unit and determines the detection conditions, the road surface object detection system according to claim 1.

4. If the road surface condition determination unit determines that the road surface is covered with snow, the detection condition determination unit excludes potholes and speed bumps from the targets to be detected by the road surface object detection unit and determines the detection conditions, the road surface object detection system according to claim 1.

5. The filtering unit includes an object detection execution determination unit that determines whether the road damage and objects on the road that meet the detection conditions are the targets for detection. The road object detection system according to claim 1.

6. The filtering unit includes a detection object filtering unit that determines the detection results to include road damage and objects on the road that meet the detection conditions. A road object detection system according to claim 1 or 5.

7. A road surface condition determination step in which the imaging device determines, from the image it captures, at least one of the type of pavement on the road on which the vehicle is traveling, the material of the road, and the road surface condition of the road, A road object detection step is performed to detect damage to the road and objects on the road from the aforementioned image, A detection condition determination step is performed to determine the detection conditions for the road object detection step, based on the determination result of the road surface condition determination step, which limits the road damage and objects on the road to be detected, A filtering step that filters the objects to be detected or the results of the road object detection step based on the detection conditions, A road object detection method equipped with the following features.

Images