Puddle detection apparatus, puddle detection method and puddle detection program

Abstract

To provide techniques which suppress the influence of an environmental change and enable the detection of a puddle at high precision.SOLUTION: A puddle detection apparatus acquires plural images different in camera positions on a vehicle and including road surfaces, specifies road surface areas in each of the plural images, extracts a feature areas having an RGB value or a luminescence different from the road surfaces from the road surface areas, calculates a first variation of the RGB value or the luminescence of each road surface area between the plural images and a second variation of the RGB value or the luminescence of the feature area, and determines the feature area as a puddle based on a difference between the first variation and the second variation.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a puddle detection device, a puddle detection method, and a puddle detection program. [Background technology]

[0002] In simple vehicles such as senior cars and personal mobility devices, driving into a puddle can cause the tires to spin, so it is important to be able to recognize puddles in advance. As a method for recognizing puddles, Patent Document 1 describes a method in which a puddle on the road surface is photographed with a camera and detected based on the difference in brightness from the dry road surface. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-54290 Summary of the Invention [Problem to be solved by the invention]

[0004] Cited Document 1 detects puddles based on the brightness of the image. Therefore, in the case of Cited Document 1, puddles on the road surface are detected based on the difference in brightness of the road surface image captured by the camera, so when it is cloudy or in the shade, the brightness of the entire image changes and there is no difference in brightness between the puddle area and the other areas, which causes a problem that puddles cannot be correctly identified.

[0005] The technology disclosed herein aims to provide technology that enables accurate detection of puddles even when the environment outside the vehicle changes. [Means for solving the problem]

[0006] In order to solve the above problem, the puddle detection device of the present disclosure acquires a plurality of images including a road surface taken at different positions from a camera mounted on a vehicle, identifies a road surface area in each of the plurality of images, and extracts a feature having an RGB value or luminance different from that of the road surface from the road surface area. A first difference between the RGB values ​​or brightness of the road surface area among the plurality of images is calculated. a change amount of the characteristic region and a second change amount of the RGB value or luminance of the characteristic region; The characteristic region is determined to be a puddle based on a difference between the amount of change and the second amount of change.

[0007] In order to solve the above problems, the technology disclosed herein may be a puddle detection program recorded on a computer-readable recording medium, and the computer can read and execute the program from the recording medium to provide the function.

[0008] Here, the computer-readable recording medium is a hard disk drive (HDD). and ROM (Read Only Memory). Effect of the Invention

[0009] According to the technology disclosed in this document, the RGB value or brightness of a puddle changes depending on the shooting position. By utilizing this characteristic, it is possible to provide a technology that can extract road surface areas and feature areas from multiple images taken at different positions, calculate the amount of change in each area, and compare the amount of change in each area, thereby suppressing the effects of environmental changes such as cloudy weather and enabling accurate detection of puddles. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing the configuration of a puddle detection system according to the first embodiment. [Diagram 2] FIG. 2 is a diagram showing the configuration of the in-vehicle device. [Diagram 3]FIG. 3 is a diagram showing a puddle detection method according to the present embodiment. [Figure 4] FIG. 4 is an explanatory diagram of an extracted road surface area. [Diagram 5] FIG. 5 is an explanatory diagram of the extracted feature region. [Figure 6] FIG. 6 is a schematic explanatory diagram showing the amount of change in RGB values ​​or brightness between images taken at different positions when the feature region is an image of an object existing on the road surface. [Figure 7] FIG. 7 is a schematic explanatory diagram showing the amount of change in RGB value or brightness between images captured at different positions when the feature region is an image of a puddle. [Figure 8] FIG. 8 is a diagram showing the configuration of a puddle detection system according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The configurations of the following embodiments are merely examples, and the present invention is not limited to the configurations of the embodiments.

[0012] First Embodiment FIG. 1 is a diagram showing a configuration of a puddle detection system 100 according to the present embodiment. The puddle detection system 100 includes a vehicle 10, a camera 20 attached to the vehicle 10, and an in-vehicle device 30. The vehicle 10 in the present embodiment is a so-called senior car, but is not limited thereto and may be a moving body such as a bicycle or personal mobility. The in-vehicle device 30 is one form of a puddle detection device, and detects a puddle on a road surface based on an image captured by the camera 20. When the in-vehicle device 30 detects a puddle, for example, the in-vehicle device 30 notifies the user of a message indicating the presence of the puddle, so that the user can drive the vehicle 10 while avoiding the puddle. In the present embodiment, the puddle is a general term for a reflective object that has a large image change that may cause a hindrance to driving, for example. For example, the puddle may be ice, mud, or the like.

[0013] <Camera> The camera 20 captures an image of an area (detection target area) where puddles are to be detected, such as in front of, to the sides of, or behind the vehicle 10, and outputs the image. The camera 20 may be a dedicated camera provided for detecting puddles, or may also serve as a camera for a drive recorder or the like. The camera 20 of this embodiment is attached to the front of the vehicle 10 with the photographing lens facing the traveling direction of the vehicle 10. That is, the camera 20 of this embodiment captures an image in front of the vehicle 10.

[0014] The camera 20 captures images at regular time intervals and outputs images captured at different times. In this embodiment, the camera 20 captures a moving image, and outputs each frame constituting the moving image as an image captured at a different time.

[0015] <In-vehicle equipment> 2 is a diagram showing the configuration of the in-vehicle device 30. The in-vehicle device 30 is a computer having a control unit 332, a memory 333, an input / output IF (interface) 334, and a communication IF 335, which are interconnected by a connection bus 331. The control unit 332 processes input information and outputs the processing result, thereby controlling the entire device. The control unit 332 is also called a CPU (Central Processing Unit) or an MPU (Micro-processing unit). The control unit 332 is not limited to a single processor, and may have a multi-processor configuration. Also, the control unit 332 may have a multi-core configuration having multiple cores in a single chip connected by a single socket.

[0016] The memory 333 includes a main storage device and an auxiliary storage device. The main storage device is used, for example, as a working area for the control unit 332, a storage area for temporarily storing information processed by the control unit 332, and a buffer area for communication data. The main storage device is a storage medium for the control unit 332 to cache programs and data and to expand the working area. The main storage device is, for example, a RAM (Random Access Memory) or the like. The auxiliary storage device is a storage medium for storing programs executed by the control unit 332, data used in information processing, operation setting information, and the like. The auxiliary storage device is, for example, a HDD (Hard-disk Drive) or the like. .

[0017] The input / output IF 334 is an interface that inputs and outputs data between the in-vehicle device 30 and peripheral devices connected thereto. The input / output IF 334 inputs and outputs data between devices such as a disk drive that reads data from a storage medium such as a CD or DVD, an operation unit, a display device, a microphone, a speaker, and a sensor. The operation unit is an input unit such as a mouse, keyboard, or touch panel that inputs information to the in-vehicle device 30 by an operator's operation. The display device is an output unit that displays and outputs information such as processing results to the operator.

[0018] The communication IF 335 is an interface (communication module) that communicates with other devices via a communication line (network), and is also called a CCU (Communication Control Unit).

[0019] In the in-vehicle device 30, the control unit 332 executes an application program, so that the control unit 332 functions as each processing unit, such as the image acquisition unit 301, the detection processing unit 302, and the output control unit 303. That is, the control unit 332 can be used as each processing unit depending on the software being executed. However, a part or all of the above processing units may be formed of hardware such as a digital circuit. Also, at least a part of the above processing units may include an analog circuit. The control unit 332 may be configured such that one processor functions as multiple processing units, or may be configured to include multiple processors that function as one processing unit.

[0020] The image acquisition unit 301 acquires a moving image including a road surface from the camera 20 provided on the vehicle 10. As described above, the camera 20 captures a moving image of the detection target area. Since the images of each frame constituting the moving image have different shooting times, the position (shooting position) of the camera 20 that captured the road surface of the detection target area changes as the vehicle 10 moves. That is, the image acquisition unit 301 acquires a plurality of images including the road surface at different shooting positions. Note that the difference in the shooting positions in each frame differs depending on the moving speed of the vehicle 10, and the difference is small when the moving speed is low (in the case of low speed), and the difference is large when the moving speed is high (in the case of high speed). For this reason, the image acquisition unit 301 may change the time interval of the frames to be acquired, that is, the interval of the shooting times of the frames to be acquired, depending on the moving speed of the vehicle 10. Note that in the following description, among the plurality of frames (images), one that is shot earlier than the other frames (images) is simply referred to as the previous frame (image), and one that is shot later than the other frames (images) is simply referred to as the later frame (image). For example, two frames captured at different times are also called previous and next frames (images).

[0021] The detection processing unit 302 obtains the amount of change in the images from the multiple images (before and after images) acquired by the image acquisition unit 301, and detects puddles based on the magnitude of this amount of change. For example, the detection processing unit 302 identifies a road surface area in each of the multiple images, and further extracts a feature area having an RGB value or brightness different from that of the road surface from the road surface area. The unit 302 detects a change in RGB value or brightness of a road surface area between a plurality of images (a first change and the change in the RGB value or brightness of the feature region (also called the second change). The detection processing unit 302 then determines whether the characteristic region is a puddle or not based on the difference between the first amount of change and the second amount of change. As described below, a puddle region has a different amount of change in an image from the surroundings when images of previous and subsequent frames are compared, so the detection processing unit 302 can determine that the portion with the different amount of change is a puddle. The detection processing unit 302 may also extract a characteristic region having a different characteristic from the surroundings from the road surface region of each image, compare the first amount of change in the road surface region of each image with the second amount of change in the characteristic region of each image, and detect the characteristic region as a puddle region if the second amount of change is greater than the first amount of change.

[0022] The detection processing unit 302 compares a first image (e.g., a previous image) and a second image (e.g., a subsequent image) among the multiple images, and calculates the amount of change in RGB value or brightness. The amount of change is calculated by, for example, calculating the amount of change in RGB value or brightness between the first image and the second image. It can be calculated as the rate of change or similarity.

[0023] When determining the rate of change of pixel values ​​(RGB values), the detection processing unit 302 calculates, for example, the average value of the rate of change of each element (R value, G value, B value) that constitutes the RGB value. When the (R, G, B) values ​​are each 8 bits, for example, black (0, 0, 0) and white (255, 255, 255), the detection processing unit 302 Calculate the rate of change in RGB values ​​between the first and second images for each element, and Calculate the average rate of change of the elements.

[0024] For example, as shown in Table 1, the average RGB values ​​of the pixels in the road surface area are dark. When changing from gray(70,70,70) to gray(125,125,125), the rate of change is (|70-125| / 255+ If the average RGB value of the pixels in the characteristic region changes from blue (0,0,255) to green (0,255,0), the rate of change is (|0-0 | / 255+|0-225| / 255+|255-0| / 255) / 3=66.7%. [Table 1]

[0025] The amount of change is not limited to the rate of change of pixel values, and may be calculated based on the similarity between the first and second images. For example, the similarity is calculated by finding feature points from the first and second images, matching the feature points between the first and second images, and calculating the similarity. The similarity may also be calculated using an AI model.

[0026] The output control unit 303 outputs the result of puddle detection. For example, when a puddle is detected, the output control unit 303 notifies the user by displaying on the display device that a puddle is present. The above-mentioned display output may be, for example, a text display such as "There is a puddle ahead," a flashing light on the display screen, or a graphic display of the puddle in a moving image on the display screen. Not limited to this, when a puddle is detected, the output control unit 303 may The output control unit 303 may output a voice message from the speaker to inform the user that a puddle exists. The voice output from the speaker may be, for example, a voice saying, "There is a puddle ahead." In addition, when the vehicle 10 has a driving control unit that performs automatic driving, the output control unit 303 may output a control signal to the driving control unit to control the steering, driving force, etc. of the vehicle so as to avoid the puddle.

[0027] <How to detect puddles> Next, a puddle detection method executed by the control unit 332 of the in-vehicle device 30 will be described. Fig. 3 is a diagram showing a puddle detection method according to this embodiment. When the in-vehicle device 30 is powered on or when a start operation is performed by the user, the control unit 332 executes processing according to the puddle detection program of Fig. 3. Note that the control unit 332 repeatedly executes processing according to the puddle detection program of Fig. 3 until the in-vehicle device 30 is powered off or until an end operation is performed by the user.

[0028] In step S10, the control unit 332 acquires a plurality of images from the camera 20. For example, the control unit 332 acquires a moving image from the camera 20, and acquires a pair of frames, for example, the latest frame and the previous frame, from among a plurality of frames constituting the moving image. Here, the control unit 332 may acquire consecutive frames from one frame to the other, or may acquire a pair of frames with a time interval between the previous frame (first image) and the next frame (second image). The control unit 332 may also change this time interval according to the traveling speed of the vehicle 10. For example, when the traveling speed of the vehicle 10 is equal to or higher than a predetermined speed, the control unit 332 acquires the previous and next frames for each frame, that is, acquires consecutive frames from one frame to the other. Then, when the traveling speed of the vehicle 10 is lower than the predetermined speed, the control unit 332 increases the interval between the frames acquired until a sufficient amount of change is obtained. This allows the puddle area to be detected with high accuracy even when the traveling speed of the vehicle 10 is slow and the change in the shooting position accompanying the traveling of the vehicle 10 is small.

[0029] In step S20, the control unit 332 extracts a portion in which the road surface is captured as a road surface area from the multiple images acquired in step S10. For example, the control unit 332 extracts the road surface area from each image by semantic segmentation. In the semantic segmentation, machine learning is performed using training data including learning images and correct answer labels in advance to identify a specific area (the road surface area in this example) from an image, and an AI model is created. Then, the control unit 332 inputs the image acquired in step S10 into the AI ​​model to identify the road surface area. FIG. 4 is an explanatory diagram of the extracted road surface area. In FIG. 4, reference numeral 51 indicates an image acquired from the camera 20, and reference numeral 52 indicates the extracted road surface area.

[0030] In addition, since the road surface is located at the bottom of the image captured by the camera 20 and has a specific color such as black or gray, the control unit 332 may extract an area that satisfies conditions such as position and color in the image as the road surface area. Furthermore, if the in-vehicle device 30 has 3D data of the driving location, the control unit 332 can specify the shape of the road surface based on this 3D data, so may extract a portion of each image that matches this shape as the road surface area. In addition, the control unit 332 may calculate the shape, area, and position of the extracted road surface area.

[0031] In step S30, the control unit 332 extracts a feature region having characteristics different from those of the road surface from the road surface region obtained in step S20. For example, the control unit 332 detects edge lines having pixels or brightness different from those of the road surface from the road surface region, and extracts the region surrounded by these edge lines as a feature region. This allows the extraction of a portion of the road surface region where an object having a color or brightness different from that of the road surface exists. FIG. 5 is an explanatory diagram of the extracted feature region. In FIG. 5, reference numeral 52 denotes the road surface region extracted in step S20, reference numeral 54 denotes the road surface region. 3 shows a feature region extracted from the image. This feature region includes not only puddles but also objects present on the road surface, such as road signs, cracks in the road, manhole covers, and garbage. The control unit 332 may also calculate the shape, area, and position of the extracted feature region. The control unit 332 may then exclude regions that are unlikely to be puddles, such as feature regions whose area is equal to or smaller than a predetermined value, regions that are raised above the road surface, and regions with specific shapes such as road signs and manholes, thereby narrowing down the feature regions to be detected.

[0032] In step S40, the control unit 332 compares the road surface area extracted from the previous frame with the road surface area extracted from the subsequent frame of the previous and subsequent frames, and obtains a first amount of change in the road surface area. For example, the control unit 332 may obtain the number of pixels (number of pixels) whose values, such as color or brightness, have changed between the previous frame and the subsequent frame, and may determine the amount of change as the ratio (rate of change) of the number of changed pixels to the number of pixels in the entire area. Note that the control unit 332 may obtain the ratio of the number of changed pixels after performing image processing, such as binarization, on the road surface area.

[0033] The amount of change may also be the similarity between the previous and next frames. A publicly known algorithm may be used to calculate the similarity between images (frames). Furthermore, the control unit 332 may input the previous and next frames to an AI model that outputs the similarity, and use the similarity calculated by the AI ​​model as the amount of change.

[0034] In step S50, the control unit 332 compares the feature region extracted from the previous frame with the feature region extracted from the subsequent frame, among the previous and subsequent frames, to obtain a second change amount in the feature region. Note that the change amount in the feature region may be the proportion of changed pixels, a similarity calculated by a predetermined algorithm, a similarity calculated by an AI model, or the like, similar to the change amount in the road surface region.

[0035] In step S60, the control unit 332 compares the amount of change in the road surface area obtained in step S40 with the amount of change in the characteristic area obtained in step S50, and detects the characteristic area having a different amount of change from the amount of change in the road surface area as a puddle area. FIG. 7 is a schematic diagram showing the amount of change in RGB value or brightness between images taken at different positions when the feature region is an image of a puddle. 6 and 7, the reference symbol P1 indicates the shooting position of the previous frame (first image), and the reference symbol P2 indicates the shooting position of the subsequent frame (second image).

[0036] As shown in FIG. 6, when an object 541 existing on the road surface is photographed from a photographing position P1, an image 54A is extracted as the characteristic region, and when the object 541 is photographed from a photographing position P2, an image 54B is extracted as the characteristic region. When the photographing positions P1 and P2 are different in this way, the angle at which the characteristic region is photographed changes, and so the pixel values ​​and brightness of each image change. However, when the object 541 in the characteristic region is an object existing on the road surface, the same object is photographed even if the photographing position is different, so there is no significant change. Moreover, this change is approximately the same as the amount of change in the road surface region.

[0037] On the other hand, as shown in Figure 7, when puddle 542 is photographed from photographing position P1, image 54C is extracted as the characteristic region, and when puddle 542 is photographed from photographing position P2, image 54D is extracted as the characteristic region. In this way, when photographing positions P1, P2 are different, the angle at which the characteristic region is photographed changes, and so the pixel values ​​and brightness of each image change. And when puddle 542 is in the characteristic region, the image reflected in the puddle is not of an object on the road surface, but of an object that is located away from the puddle region, so even a slight change in the photographing position will result in an object in a significantly different position being reflected in the puddle, causing a large change in pixel values ​​and brightness. In this way, in the case of a puddle, the amount of change in the reflected image is large, so that a feature area with a larger amount of change than the amount of change in the road surface area can be detected as a puddle area. Moreover, the change in the reflected position in the puddle is not limited to sunny days, but also does not change even in low illuminance conditions such as cloudy days and dusk, so that a puddle area can be detected with high accuracy regardless of environmental changes. Moreover, when the amount of change is calculated based on the similarity, the road surface area has a high similarity between each image even when the shooting position is changed as shown in FIG. 6. That is, the amount of change is small. On the other hand, the puddle area has a low similarity between each image as shown in FIG. 7 when the shooting position is changed. That is, the amount of change is large. Therefore, when the similarity of the feature area is lower than that of the road surface area, that is, when the amount of change in the feature area is larger than that of the road surface area, the feature area can be detected as a puddle area.

[0038] In step S70, control unit 332 determines whether or not a puddle area has been detected. If control unit 332 has not detected a puddle area (S70, No), it ends the processing of the puddle detection program in FIG. 3, and if control unit 332 has detected a puddle area (S70, Yes), it proceeds to step S80.

[0039] In step S80, the control unit 332 outputs the detection result of the puddle area. For example, a voice message such as "A puddle has been detected" is output, and the position of the puddle is displayed on the display device. Furthermore, if the vehicle 10 has a driving control unit that performs automatic driving, the control unit 332 may output, for example, position information indicating the position of the puddle and a control signal including a command to drive the vehicle 10 while avoiding the position where the puddle is located, to the driving control unit of the vehicle 10.

[0040] In this embodiment, the in-vehicle device 30 is configured to perform the puddle determination in a stand-alone manner, but the present invention is not limited to this, and the in-vehicle device 30 may transmit captured images to an external server, which may then execute a process of determining whether or not a puddle exists according to a puddle detection program such as that shown in Fig. 3. In this case, the external server acquires multiple images from the in-vehicle device 30 via the network, performs a puddle determination process, and then transmits the determination result to the in-vehicle device 30, which then outputs the result to the user, etc.

[0041] <Effects of the embodiment> (1) As described above, the vehicle-mounted device (puddle detection device) 30 of this embodiment acquires multiple images including the road surface taken at different positions relative to the road surface from the camera 20 installed in the vehicle, and identifies a road surface area in each of the multiple images. Then, the vehicle-mounted device 30 extracts feature areas having RGB values ​​or brightness different from the road surface from the road surface area, and calculates the road surface area from the multiple images. The amount of change in the road surface area and the amount of change in the characteristic area are calculated, and the characteristic area is detected as a puddle based on the difference between these amounts.

[0042] When the shooting position is changed, the image reflected in a puddle changes significantly compared to the road surface other than the puddle. Therefore, the in-vehicle device 30 of this embodiment calculates the amount of change between images shot at different positions and determines the puddle based on this amount of change. In this way, the characteristic that the image of a puddle changes significantly depending on the shooting position does not depend on environmental conditions such as weather or road surface color, so it is less susceptible to environmental changes than determining a puddle simply based on the difference in brightness from a dry road surface. As a result, the in-vehicle device 30 of this embodiment can detect puddles with high accuracy while suppressing the effects of environmental changes.

[0043] (2) When the second amount of change in the characteristic area of ​​each image captured at different positions is greater than the first amount of change in the road surface area of ​​each image, the in-vehicle device 30 determines that the characteristic area is a puddle. The environmental conditions, such as the surrounding brightness, are substantially the same for the road surface area and the characteristic area. In other words, if the road surface area is bright, the characteristic area is also bright, and if the road surface area is dark, the characteristic area is also dark. For this reason, the in-vehicle device 30 determines that the first amount of change in the characteristic area of ​​each image captured at different positions is a puddle. The amount of change and the second amount of change are calculated, and it is determined whether the second amount of change is larger or smaller based on the first amount of change, so that the puddle area can be detected more accurately than if the puddle were simply determined based on whether it is larger than a threshold value.

[0044] (3) The vehicle-mounted device 30 detects, in the road surface area of ​​the image, an RGB value or brightness different from that of the road surface. The in-vehicle device 30 of the present embodiment detects edge lines having different degrees of contrast and extracts regions surrounded by the edge lines as feature regions. As a result, the in-vehicle device 30 of the present embodiment can identify portions of the road surface region that are candidates for puddle regions, and perform comparisons with these identified regions to accurately detect puddle regions.

[0045] (4) The vehicle-mounted device 30 determines the first change amount and the second change amount from the similarity between the multiple images to be compared. The road surface area has a high similarity between the images even when the image capturing positions are changed, whereas the puddle area has a low similarity between the images when the image capturing positions are changed. For this reason, the vehicle-mounted device 30 determines the first change amount and the second change amount from the similarity, When the similarity of the feature region is low compared to the road surface region and the second change amount is larger than the first change amount, the feature region can be determined to be a puddle region. In this way, since the determination based on similarity is made based on, for example, the distance or direction of the compared feature points, it is not easily affected by the brightness of the surroundings. Specifically, when the vehicle goes into the shadow of a building or the sun is hidden by clouds and the brightness of the surroundings changes, the RGB value or brightness changes, but the similarity does not change. Puddle areas can be detected with high accuracy.

[0046] (5) The in-vehicle device 30 acquires a plurality of images from the camera 20 provided on the vehicle, which are a moving image, and changes the time interval between frames of the moving image in accordance with the traveling speed of the vehicle. This allows the in-vehicle device 30 to appropriately control the image capture position and detect puddle areas with high accuracy. For example, when the traveling speed of the vehicle 10 is slow and the change in the capture position accompanying the traveling of the vehicle 10 is small, the in-vehicle device 30 sets a large time interval between images to be compared so that a sufficient change can be obtained, thereby enabling the in-vehicle device 30 to detect puddle areas with high accuracy even when the traveling speed is slow.

[0047] (6) The in-vehicle device 30 extracts the road surface area by semantic segmentation. As a result, the in-vehicle device 30 of the present embodiment can accurately detect puddle areas by eliminating areas other than the road surface from the captured image and suppressing the influence of objects that exist in areas other than the road surface, such as window glass.

[0048] (7) The in-vehicle device 30 obtains images taken at different positions as the vehicle moves by obtaining multiple images taken at different times from the camera 20. This allows the in-vehicle device 30 to obtain multiple images taken at different positions with a single camera 20, and detect puddle areas.

[0049] Second Embodiment 8 is a diagram showing the configuration of a puddle detection system 100 according to the second embodiment. Compared to the above-described first embodiment, this embodiment is different in the configuration in which a plurality of images are taken at different shooting positions relative to the road surface by a plurality of cameras 21, 22. Note that the same elements as those in the first embodiment are given the same reference numerals and will not be described again.

[0050] As shown in FIG. 8, in this embodiment, the multiple cameras 21, 22 are attached at different positions in the height direction of the vehicle 10. The multiple cameras 21, 22 are attached with their photographing lenses facing the front of the vehicle 10 so as to photograph the area ahead of the vehicle 10. However, the cameras 21, 22 may be arranged in other ways as long as they photograph approximately the same detection target area from different positions. For example, the cameras 21, 22 may be arranged at different positions in the width direction of the vehicle 10. Furthermore, the cameras 21 and 22 may be provided facing a detection target area other than the forward direction.

[0051] The in-vehicle device 30 of this embodiment acquires images from the cameras 21 and 22, respectively, and regards the image taken by the camera 21 and the image taken by the camera 22 as a pair of images. For example, when moving images are taken by the cameras 21 and 22, respectively, frames (images) taken at the same time are regarded as a pair of frames (images). There may be a slight error in the aforementioned same time.

[0052] The detection processing unit 302 compares a pair of frames (images) acquired by the image acquisition unit 301, and detects a puddle based on the amount of change in the frames. The detection method and the like are the same as in the above-mentioned embodiment. In this embodiment, the amount of change is obtained by comparing a pair of images captured by the two cameras 21 and 22, but it is also possible to install three or more cameras in different positions and compare images captured by each camera to obtain the amount of change.

[0053] Thus, according to this embodiment, by using multiple cameras installed at different positions, images captured at different positions relative to the road surface can be obtained even when the vehicle 10 is stopped, and puddles can be detected regardless of the speed of the vehicle 10. When the vehicle 10 is stopped, multiple cameras can be used to capture images captured at different positions, and when the vehicle 10 is moving, one camera can be left alone and the other cameras can be stopped, and multiple images captured at different times can be captured by one camera, as in the first embodiment. This can reduce the number of cameras operated while the vehicle is moving, thereby saving power. [Explanation of symbols]

[0054] 10: Vehicle 20, 21, 22: Camera 30:In-vehicle equipment 100: Puddle detection system 301: Image acquisition unit 302: Detection processing unit 303: Output control section 331: Connection bus 332: Control section 333: Memory 334: Input / Output IF 335: Communication IF 541 :object 542: Puddle

Claims

1. Multiple images including the road surface are acquired from different positions using a camera installed in the vehicle, Identifying a road surface area in each of the plurality of images; Extracting a feature area having an RGB value or brightness different from that of the road surface from the road surface area; Among the plurality of images, A first change amount of the RGB value or luminance of the road surface area; a second change amount of the RGB value or luminance of the characteristic region; The feature region is determined to be a puddle based on a difference between the first change amount and the second change amount. Puddle detector.

2. The puddle detection device according to claim 1 , wherein the characteristic region is determined to be a puddle when the second amount of change is greater than the first amount of change.

3. In the road surface region of the image, an edge line having an RGB value or brightness different from that of the road surface 2. The puddle detection device according to claim 1, wherein an area surrounded by the edge lines is extracted as the characteristic area.

4. The puddle detection device according to claim 1 , wherein the first amount of change and the second amount of change are determined from a similarity between the plurality of images being compared.

5. 2. The puddle detection device according to claim 1, wherein the plurality of images are moving images, and the time interval between frames of the moving images is changed according to the traveling speed of the vehicle.

6. The puddle detection device according to claim 1 , wherein the road surface area is extracted by semantic segmentation.

7. The puddle detection device according to claim 1 , wherein the plurality of images are acquired from the camera at different shooting times, thereby acquiring images whose shooting positions are changed as the vehicle moves.

8. The puddle detection device according to claim 1 , wherein the plurality of images are acquired from a plurality of cameras mounted at different positions on the vehicle.

9. The puddle detection device described in claim 8, wherein when the vehicle is stopped, the multiple images are acquired from multiple cameras mounted at different positions at the same time, and when the vehicle is moving, the multiple images are acquired using only one camera at different times.

10. Multiple images including the road surface are acquired from different positions using a camera installed in the vehicle, Identifying a road surface area in each of the plurality of images; A feature area having an RGB value or brightness different from that of the road surface is extracted from the road surface area. 、 Among the plurality of images, A first change amount of the RGB value or luminance of the road surface area; a second change amount of the RGB value or luminance of the characteristic region; determining that the feature region is a puddle based on a difference between the first amount of change and the second amount of change; A computer implemented method for detecting puddles.

11. Multiple images including the road surface are acquired from different positions using a camera installed in the vehicle, Identifying a road surface area in each of the plurality of images; A feature area having an RGB value or brightness different from that of the road surface is extracted from the road surface area. 、 Among the plurality of images, A first change amount of the RGB value or luminance of the road surface area; a second change amount of the RGB value or luminance of the characteristic region; determining that the feature region is a puddle based on a difference between the first amount of change and the second amount of change; A puddle detection program to be run on a computer.

Images