Information processing device, information processing method, and program

Abstract

An information processing device (100) comprises: an acquiring unit (110) that acquires a first video and a second video captured with a closer focal point than the first video; a rotating unit (120) that rotates the second video through 180 degrees; a detecting unit (130) that detects a first feature quantity from the first video and detects a second feature quantity from the second video rotated through 180 degrees; a calculating unit (140) that calculates a correlation between the first feature quantity and the second feature quantity; and a determining unit (150) that determines whether or not water droplets appear in the second video on the basis of the calculated correlation.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] Conventionally, there are devices for detecting water droplets such as raindrops. Patent Document 1 discloses a water droplet detection device that detects water droplets using a light receiving unit that detects only light that travels through a light-transmitting plate at a refraction angle within a predetermined range relative to the normal. Patent Document 2 discloses a rain sensor that includes an imaging device that captures an image of the outside of the vehicle cabin through the front window from inside the cabin of the vehicle and detects the presence or absence of rain based on image information of the image captured by the imaging device. Patent Document 3 discloses a device that detects water droplets using a polarization imaging element.

[0003] Japanese Patent Laid-Open No. 10-31078 Japanese Patent Laid-Open No. 2006-292543 Japanese Patent Laid-Open No. 2015-180864

[0004] The present disclosure provides an information processing device and the like that can easily detect water droplets.

[0005] An information processing device according to one aspect of the present disclosure includes an acquisition unit that acquires a first image and a second image captured at a closer focus than the first image, a rotation unit that rotates the second image by 180 degrees, a detection unit that detects a first feature amount from the first image and a second feature amount from the second image rotated by 180 degrees, a calculation unit that calculates a correlation between the first feature amount and the second feature amount, and a determination unit that determines whether or not water droplets are reflected in the second image based on the calculated correlation.

[0006] An information processing method according to one aspect of the present disclosure acquires a first image and a second image captured at a closer focus than the first image, rotates the second image by 180 degrees, detects a first feature from the first image, detects a second feature from the second image rotated by 180 degrees, calculates a correlation between the first feature and the second feature, and determines whether water droplets are visible in the second image based on the calculated correlation.

[0007] A program according to one aspect of the present disclosure is a program for causing a computer to execute the information processing method.

[0008] According to an information processing device according to an aspect of the present disclosure, water droplets can be easily detected.

[0009] Fig. 1 is a block diagram showing a configuration of an information processing system according to an embodiment. Fig. 2 is a diagram showing a specific example of a video according to an embodiment. Fig. 3 is a diagram showing a specific example of a video according to an embodiment. Fig. 4 is a flowchart showing an information processing method according to an embodiment.

[0010] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0011] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims of the present disclosure are described as optional components.

[0012] Furthermore, in this specification, a numerical range such as 180 degrees is not an expression that expresses only a strict meaning, but also an expression that means that it includes a substantially equivalent range, for example, a difference of a few percent, such as 5%.

[0013] Furthermore, unless otherwise specified, ordinal numbers such as "first" and "second" do not refer to the number or order of components, but are used to avoid confusion and distinguish between components of the same type.

[0014] (Embodiment) [Configuration] FIG. 1 is a block diagram showing the configuration of an information processing system 10 according to this embodiment.

[0015] The information processing system 10 is a system for detecting water droplets. Specifically, the information processing system 10 detects water droplets adhering to a light-transmitting member such as a window glass based on an image obtained from an imaging device 200 that captures an image through the light-transmitting member. In this embodiment, the information processing system 10 detects raindrops adhering to the windshield using an image generated by the imaging device 200, which is installed inside a vehicle, capturing an image of the area ahead of the vehicle through the windshield.

[0016] The information processing system 10 includes an information processing device 100 and an imaging device 200 .

[0017] The information processing device 100 is a computer that detects water droplets. Specifically, the information processing device 100 detects the presence or absence of water droplets on a light-transmitting member such as a window glass based on an image obtained from a camera that captures an image through the light-transmitting member. In this embodiment, the information processing device 100 detects raindrops on the windshield using an image generated by an imaging device 200 installed inside a vehicle capturing an image of the area in front of the vehicle through the windshield.

[0018] The information processing device 100 is realized, for example, by a communication interface for communicating with the imaging device 200, a non-volatile memory storing a program, a volatile memory serving as a temporary storage area for executing the program, an input / output port for transmitting and receiving signals, and a processor for executing the program. The communication interface may be realized, for example, by an antenna and a wireless communication circuit for enabling wireless communication, or by a connector to which a communication line is connected for wired communication. The information processing device 100 is mounted, for example, on a vehicle, but may also be located outside the vehicle.

[0019] The imaging device 200 is a camera that generates a video (more specifically, an image) that is used by the information processing device 100 to detect water droplets. In this embodiment, the imaging device 200 is disposed inside a vehicle, and generates a video by capturing an image of the area ahead of the vehicle through a windshield that the vehicle is equipped with.

[0020] The information processing system 10 may include one or more imaging devices 200, and may include one imaging device 200 or multiple imaging devices 200. For example, the information processing system 10 may include a separate imaging device that captures (generates) the first video and a separate imaging device that captures the second video.

[0021] The vehicle may be, for example, a motorcycle or an automobile, or may be a mobile robot such as an AMR (Autonomous Mobile Robot). The vehicle may be operated by a driver on board the vehicle, may be remotely controlled, or may be capable of autonomous driving.

[0022] Next, the configuration of the information processing device 100 will be specifically described.

[0023] The information processing device 100 includes an acquisition unit 110 , a rotation unit 120 , a detection unit 130 , a calculation unit 140 , a determination unit 150 , a control unit 160 , and a storage unit 170 .

[0024] The acquisition unit 110 is a processing unit that acquires, from the imaging device 200, the video captured (generated) by the imaging device 200. Specifically, the acquisition unit 110 acquires the first video and the second video from the imaging device 200.

[0025] The first video and the second video are videos (images) captured with the focal point (focus) of the imaging device 200 set at different positions. Specifically, the second video is captured from the same position as the imaging device 200 when the first video was captured, but with a focus closer than that of the first video. In other words, the acquisition unit 110 acquires the first video and the second video captured at a closer focus than that of the first video.

[0026] For example, the first image is an image captured over a wider range and at a far focus than the second image, and the second image is an image captured over a narrower range and at a near focus than the first image. A wide range means, for example, that the space captured in the first image is a wider range in real space than the space captured in the second image.

[0027] For example, the first image is an image of the second position captured from the first position through the light-transmitting member, and the second image is an image captured from the first position with the focus on the light-transmitting member.

[0028] The light-transmitting member is a member that transmits light detected by the imaging device 200. The wavelength of the light is, for example, in the visible range, but may be any wavelength and is not particularly limited. The light-transmitting member is, for example, window glass. In this embodiment, the light-transmitting member is a windshield provided on a vehicle.

[0029] Also, for example, the first position is inside the vehicle, and the second position is far in front of the vehicle.

[0030] The term "far ahead of the vehicle" refers to a location outside the vehicle and in front of the vehicle. The distance from the vehicle indicated by the far ahead of the vehicle may be any distance as long as it is outside the vehicle.

[0031] The light-transmitting member may also be a window glass of a building. In this case, the first position is, for example, inside the building, and the second position is, for example, outside the building. The light-transmitting member may also be a side window or rear window of the vehicle instead of the windshield of the vehicle. In this case, the second position may be a distant side window or a distant rear window of the vehicle.

[0032] For example, the second image is an image showing a portion of the vehicle's windshield, and the first image is an image showing the scenery seen through the windshield from inside the vehicle so that the entire windshield is captured.

[0033] When the first video and the second video are stored in the storage unit 170 , the acquisition unit 110 may acquire the first video and the second video from the storage unit 170 .

[0034] The rotation unit 120 is a processing unit that rotates the image. Specifically, the rotation unit 120 rotates the second image by 180 degrees. For example, the rotation unit 120 rotates the second image by 180 degrees around the center of the second image in the image coordinate system as the rotation axis.

[0035] The image coordinate system is a two-dimensional Cartesian coordinate system consisting of an x-axis and a y-axis corresponding to the image acquired from the imaging device 200. Specifically, the image coordinate system is a coordinate system in which the position of the upper left pixel of the image when it is assumed that the entire image is displayed on a display device such as a display (not shown) is the origin, the right direction as viewed from that pixel is the positive x-axis direction, and the downward direction as viewed from that pixel is the positive y-axis direction.

[0036] The detection unit 130 is a processing unit that detects feature amounts from the video. Specifically, the detection unit 130 detects a first feature amount from the first video, and detects a second feature amount from the second video rotated 180 degrees. The first feature amount is a feature amount detected (extracted) from the first video. The second feature amount is a feature amount detected from the second video.

[0037] The feature amount is, for example, information indicating the contour (edge) of an object or the like shown in the video. The feature amount is, for example, information including the coordinates of each of a plurality of feature points that make up the contour. Specifically, the feature amount is, for example, information indicating the shape of an object or the like shown in the video, and pixel values ​​that indicate the color or brightness of each pixel in the video. For example, the detection unit 130 detects edges in the first video as first feature amounts and edges in the second video as second feature amounts. In this way, the detection unit 130 detects the shapes of objects included in the first video and the second video, and positions where color changes significantly, such as boundaries between areas with and without clouds, as edges.

[0038] The calculation unit 140 is a processing unit that calculates the correlation (also referred to as a correlation value or a correlation coefficient) between the first feature amount and the second feature amount. That is, the calculation unit 140 calculates the correlation between the first image and the second image rotated 180 degrees. In other words, the calculation unit 140 calculates the degree of similarity between the first image and the second image rotated 180 degrees.

[0039] The correlation calculation method may be determined arbitrarily and is not particularly limited. Examples of the correlation calculation method include Sum of Absolute Difference (SAD), Sum of Squared Difference (SSD), and Normalized Cross-Correlation (NCC).

[0040] The determination unit 150 is a processing unit that determines whether water droplets are reflected in the second image based on the correlation calculated by the calculation unit 140. That is, the determination unit 150 detects water droplets reflected in the second image. For example, the determination unit 150 determines whether water droplets are attached to a light-transmitting member (e.g., a windshield provided on a vehicle) by determining whether water droplets are reflected in the second image based on the correlation calculated by the calculation unit 140. For example, if the determination unit 150 determines that the correlation is greater than a predetermined threshold, it determines that water droplets are reflected in the second image, and if the determination unit 150 determines that the correlation is equal to or less than the predetermined threshold, it determines that water droplets are not reflected in the second image. The predetermined threshold may be set arbitrarily and is not particularly limited.

[0041] The control unit 160 is a processing unit that executes processing based on the determination result of the determination unit 150. For example, when the determination unit 150 determines that water droplets are attached to the windshield of the vehicle, the control unit 160 operates the wipers of the vehicle. As a result, the control unit 160 removes water droplets from the windshield with the wipers. The water droplets are, for example, raindrops. Therefore, the information processing device 100 can detect when raindrops have adhered to the windshield of the vehicle due to rain, and automatically remove the raindrops attached to the windshield with the wipers.

[0042] The control unit 160 may notify the user of the determination result by outputting information indicating the determination result to a display device such as a display or an audio device such as a speaker.

[0043] Each processing unit, such as the acquisition unit 110, the rotation unit 120, the detection unit 130, the calculation unit 140, the determination unit 150, and the control unit 160, is realized by, for example, a memory that stores a control program and a processor, such as a CPU (Central Processing Unit), that executes the control program. The processor of each processing unit may be realized by a single memory and processor, or may be realized by separate memories and processors.

[0044] The storage unit 170 is a storage device that stores various types of information. For example, the storage unit 170 stores images captured by the imaging device 200 and information indicating a predetermined threshold value. The storage unit 170 is realized by, for example, a semiconductor memory or a hard disk drive (HDD).

[0045] [Specific Example] Next, a specific example of the processing executed by each processing unit will be described. In the specific example described below, the first image is an image of the distant area ahead of the vehicle captured from inside the vehicle through the windshield, and the second image is an image captured from inside the vehicle with the focus focused on the windshield. Furthermore, the determination unit 150 determines whether water droplets are attached to the windshield by determining whether water droplets are captured in the second image.

[0046] 2 and 3 are diagrams showing specific examples of images according to the embodiment. Image 300 shown in FIG. 2 and image 301 shown in (a) of FIG. 3 are examples of the first image. Image 310 shown in (b) of FIG. 3 is an example of the second image. Image 311 shown in (c) of FIG. 3 is an image obtained by rotating image 310 by 180 degrees, and is an example of the second image rotated by 180 degrees.

[0047] The imaging device 200 repeatedly generates a first image by, for example, repeatedly capturing an image of the distant area ahead of the vehicle through the windshield from inside the vehicle. As a result, for example, a scene seen from inside the vehicle is captured as the first image, as in image 300.

[0048] Furthermore, the imaging device 200 repeatedly generates the second video by, for example, focusing on the windshield and repeatedly capturing an image, thereby capturing the second video in which, for example, a portion of the windshield is enlarged.

[0049] For example, if there are no raindrops on the windshield, only the scenery seen from inside the vehicle is captured as the first image, as in image 300. In such a case, the second image is an image in which a portion of the windshield is enlarged (for example, an image in which the scenery is captured as is, but out of focus).

[0050] On the other hand, for example, when raindrops are attached to the windshield, the raindrops W are captured as a first image in the scenery seen from inside the car, as in image 301. For example, when the area where the raindrops W are attached is captured as a second image, the raindrops W are widely reflected, as in image 310. In addition, due to light refraction, a portion of the scenery reflected in image 301 is rotated 180 degrees and reflected in the portion of the raindrops W in image 310. Therefore, as shown in FIG. 3C, the portion of the raindrops W in image 311, which is obtained by rotating image 310 180 degrees, is a portion of image 301 (in the example shown in FIG. 3, the portion surrounded by the dashed-dotted circle shown in FIG. 3A). It becomes an image similar to that.

[0051] Therefore, the information processing device 100 can determine whether or not water droplets are reflected in the image 311 by detecting the feature amounts of the image 301 and the image 311 and calculating the correlation.

[0052] Because the first video is captured so that the entire area in front of the vehicle is captured, even if raindrops W are captured in part of the first video, the area where the raindrops W are captured will only appear slightly blurred, making it difficult to detect that part. On the other hand, from the second video alone, it may be difficult to determine whether the scenery captured in the second video is rotated 180 degrees relative to the actual scenery. Therefore, the information processing device 100 detects water droplets by comparing the first video with the second video rotated 180 degrees.

[0053] [Processing Procedure] Next, a processing procedure of the information processing device 100 according to the embodiment will be described.

[0054] 4 is a flowchart showing an information processing method according to an embodiment. For example, the information processing device 100 includes a processor and a memory, and the processor performs the following processes using the memory.

[0055] First, the information processing device 100 acquires a first video and a second video captured at a closer focus than the first video (S110). Specifically, the acquisition unit 110 included in the information processing device 100 acquires the first video and the second video from the imaging device 200. For example, the information processing device 100 controls the imaging device 200 to cause the imaging device 200 to capture images so as to generate the first video and the second video.

[0056] The first video is, for example, the image 301. The second video is, for example, the image 310.

[0057] Next, the information processing device 100 rotates the acquired second video by 180 degrees (S120). For example, the information processing device 100 rotates the second video around the center of the second video in the image coordinate system as the rotation axis, thereby changing the position of each pixel in the second video in the image coordinate system.

[0058] Next, the information processing device 100 detects features from the first video and the second video rotated 180 degrees (S130). Specifically, the information processing device 100 detects a first feature from the first video and a second feature from the second video rotated 180 degrees.

[0059] The second video image rotated by 180 degrees is, for example, image 311.

[0060] Next, the information processing device 100 calculates the correlation between the detected first feature amount and the detected second feature amount (S140).

[0061] Next, the information processing device 100 determines whether or not water droplets are captured in the second image based on the calculated correlation (S150).

[0062] For example, the information processing device 100 performs processing based on the determination result of step S150. For example, when the information processing device 100 detects raindrops on the windshield of the vehicle, if the information processing device 100 determines that water droplets are reflected in the second image, it determines that raindrops are on the windshield and operates the wipers of the vehicle to remove the raindrops from the windshield.

[0063] Note that the imaging device 200 may generate multiple second images by capturing images at different imaging positions. In this case, multiple second images may be acquired in step S110. In this case, steps S120 to S150 may be performed for each of the multiple second images. For example, the information processing device 100 may operate the wipers if it determines that water droplets are captured in any of the multiple second images, or may operate the wipers if it determines that water droplets are captured in a predetermined number or more of the multiple second images. Of course, the information processing device 100 may not operate the wipers if it determines that water droplets are not captured in any of the multiple second images, or may not operate the wipers if it determines that water droplets are captured in only less than a predetermined number of the multiple second images. The predetermined number may be set arbitrarily and is not particularly limited.

[0064] [Effects, etc.] As described above, information processing device 100 according to this embodiment includes acquisition unit 110 that acquires a first image and a second image captured at a closer focus than the first image, rotation unit 120 that rotates the second image by 180 degrees, detection unit 130 that detects a first feature amount from the first image and detects a second feature amount from the second image rotated by 180 degrees, calculation unit 140 that calculates the correlation between the first feature amount and the second feature amount, and determination unit 150 that determines whether or not water droplets are reflected in the second image based on the calculated correlation.

[0065] When the portion of the image where the water droplets appear is enlarged, it may appear as if the entire image has been rotated 180 degrees. Therefore, for example, a first image captured at a distant object can be compared with a second image captured at a closer distance than when the first image was captured, by rotating one of the first and second images by 180 degrees.

[0066] For example, the second image is an image showing a portion of the vehicle's windshield, and the first image is an image showing the scenery seen through the windshield from inside the vehicle, with the entire windshield captured. In the comparison process between the first and second images, feature quantities such as the shape of objects and the color of the images (pixel values ​​indicating color) are compared between the first and second images. If water droplets are captured in the second image, at least a portion of the first image will be similar to at least a portion of the second image rotated 180 degrees (specifically, the portion showing the water droplets), resulting in a high correlation. In other words, if water droplets are captured in the second image, the first feature quantity and the second feature quantity will be similar. In this way, the information processing device 100, for example, rotates the second image 180 degrees and determines that water droplets are captured in the second image if there is a correlation with the first image captured at a greater distance from the imaging device 200 than the second image. Therefore, this allows water droplets to be easily detected from the first and second images.

[0067] In this embodiment, the first image is an image of a distant area ahead of the vehicle captured from inside the vehicle through the windshield, and the second image is an image captured from inside the vehicle with the focus on the windshield. The water droplets are, for example, raindrops. For example, when the determination unit 150 determines that water droplets are captured in the second image, that is, when the determination unit 150 determines that raindrops are attached to the windshield, the control unit 160 included in the information processing device 100 operates the windshield wipers included in the vehicle.

[0068] This makes it possible to easily determine whether or not raindrops are present on the windshield, and if it is determined that raindrops are present on the windshield, the raindrops can be automatically removed by the wipers.

[0069] These comprehensive or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM that stores the computer program, or may be realized by any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0070] For example, the present disclosure may be realized as an information processing method executed by information processing device 100, which includes acquiring a first image and a second image captured at a closer focus than the first image (S110), rotating the second image 180 degrees (S120), detecting a first feature from the first image and detecting a second feature from the second image rotated 180 degrees (S130), calculating a correlation between the first feature and the second feature (S140), and determining whether water droplets are present in the second image based on the calculated correlation (S150). Furthermore, for example, the present disclosure may be realized as a program for causing a computer to execute the information processing method. Furthermore, for example, the present disclosure may be realized as a computer-readable non-transitory recording medium storing the program.

[0071] (Modification) Note that the configuration of the information processing system 10 and the processes executed by the information processing system 10 are not limited to those described above.

[0072] For example, when water droplets such as raindrops adhere to the windshield, the water droplets tend to become circular. Therefore, for example, the detection unit 130 may cut out a portion of the first image into a circular shape, then cut out a portion of the second image rotated 180 degrees into a circular shape, detect a first feature from the circularly cut out first image, and detect a second feature from the circularly cut out and rotated 180 degrees second image. For example, a portion of the second image rotated 180 degrees may be cut out into a circular shape along the outer edge of the raindrop W shown in FIG. 3C.

[0073] The circle may be a perfect circle or an ellipse. The cutout position is determined, for example, so that the center of the video is the center of the circle, but may be determined arbitrarily. The size of the circle is determined, for example, so that the diameter of the circle is equal to or smaller than the short side of the rectangular video, but may be determined arbitrarily.

[0074] Furthermore, the process of cutting out a portion of the second video into a circular shape may be performed after or before rotating the second video by 180 degrees. If the process of cutting out a portion of the second video into a circular shape is performed before rotating the second video by 180 degrees, the circularly cut second video is rotated by 180 degrees and then the process of detecting the feature amount is performed.

[0075] Furthermore, for example, if a circular portion such as the outer edge of the raindrop W can be detected by performing image processing on the second video, the detected circular portion may be cut out.

[0076] Also, for example, the first video and the second video are video images captured by the imaging device 200, that is, by one imaging device 200. However, the first video and the second video may be video images captured by different imaging devices. In other words, the first video and the second video may be captured by separate imaging devices.

[0077] Furthermore, for example, the second image is an image captured so that a portion of the windshield is visible. Therefore, in order to determine whether or not water droplets are present across the entire windshield, multiple second images captured at different positions are required. Therefore, for example, the second image may be an image captured by scanning. In other words, the imaging device that captures the second image may be, for example, an imaging device that can capture images while changing the imaging position by focusing on the windshield. The imaging device may include a drive mechanism such as a motor that changes the position and / or orientation of the imaging unit that captures the images.

[0078] (Other Embodiments) As described above, the embodiments have been described as examples of the technology according to the present disclosure. However, the technology according to the present disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made as appropriate. For example, the following modifications are also included in one embodiment of the present disclosure.

[0079] For example, when the present disclosure is realized as a program (software), each step is performed by running the program using hardware resources such as a computer's CPU, memory, input / output circuits, etc. In other words, each step is performed by the CPU acquiring data from memory or input / output circuits, etc., performing calculations, and outputting the calculation results to memory or input / output circuits, etc.

[0080] Furthermore, the computer that executes this program may be a single computer or multiple computers, and may perform centralized processing or distributed processing.

[0081] In the above embodiment, each component included in information processing device 100 may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0082] Some or all of the functions of the information processing device 100 according to the above-described embodiment are typically realized as an LSI, which is an integrated circuit. These may be individually integrated into single chips, or some or all of them may be integrated into a single chip. Furthermore, the integrated circuit is not limited to an LSI, and may be realized using a dedicated circuit or a general-purpose processor. It is also possible to use an FPGA (Field Programmable Gate Array), which can be programmed after LSI manufacturing, or a reconfigurable processor, which allows the connections and settings of circuit cells within an LSI to be reconfigured.

[0083] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that each component included in the information processing device 100 may be integrated using that technology.

[0084] Furthermore, for example, in the above-described embodiment, the processing performed by a specific processing unit may be performed by another processing unit, the order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

[0085] In addition, this disclosure also includes forms obtained by making various modifications to the embodiments that a person skilled in the art would think of, and forms realized by arbitrarily combining the components and functions in each embodiment within the scope of the present disclosure.

[0086] (Additional Notes) The above description of the embodiments discloses the following techniques.

[0087] (Technology 1) An information processing device comprising: an acquisition unit that acquires a first image and a second image captured at a closer focus than the first image; a rotation unit that rotates the second image by 180 degrees; a detection unit that detects a first feature amount from the first image and detects a second feature amount from the second image rotated by 180 degrees; a calculation unit that calculates a correlation between the first feature amount and the second feature amount; and a determination unit that determines whether or not water droplets are reflected in the second image based on the calculated correlation.

[0088] The image of the portion of the entire image that shows water droplets may appear as if a portion of the entire image had been rotated 180 degrees. Therefore, for example, a first image captured from a distant object and a second image captured at a closer focus than the first image are rotated 180 degrees, and the features of the first image and the second image are compared. If water droplets are captured in the second image, at least a portion of the first image will be similar to at least a portion of the second image rotated 180 degrees, resulting in a high correlation. Therefore, this allows water droplets to be easily detected from two images captured at different focal lengths.

[0089] (Technology 2) The information processing device described in Technology 1, wherein the determination unit determines that water droplets are reflected in the second image if it determines that the correlation is greater than a predetermined threshold, and determines that water droplets are not reflected in the second image if it determines that the correlation is equal to or less than the predetermined threshold.

[0090] This allows water droplets to be easily detected.

[0091] (Technology 3) The information processing device according to Technology 1 or 2, wherein the detection unit detects an edge in the first image as the first feature amount and detects an edge in the second image as the second feature amount.

[0092] This makes it possible to easily detect water droplets in cases where an object appears in the first and second images, based on the shape of the object appearing in the images detected by edge detection.

[0093] (Technology 4) An information processing device described in any one of Technologies 1 to 3, wherein the detection unit cuts out a portion of the first video into a circle, cuts out a portion of the second video rotated 180 degrees into a circle, detects the first feature from the first video cut out into a circle, and detects the second feature from the second video cut out into a circle and rotated 180 degrees.

[0094] For example, water droplets on a windshield can have a circular shape. Therefore, this method allows for the extraction of a portion of the image where the water droplets rotate the image 180 degrees and for comparison of the feature values. This reduces the number of locations in the image where feature values ​​are detected, thereby reducing the amount of processing. Furthermore, if the image is rotated 180 degrees by water droplets, the feature values ​​can be detected using the image that includes many of the 180-degree rotated portions, allowing for accurate detection of water droplets.

[0095] (Technology 5) An information processing device according to any one of techniques 1 to 4, wherein the first image and the second image are images captured by different imaging devices.

[0096] This makes it possible to obtain the first and second images captured with simple processing and detect water droplets, compared to capturing the first and second images using the same imaging device with different focal lengths.

[0097] (Technology 6) An information processing device according to any one of technologies 1 to 5, wherein the second image is an image captured by scanning.

[0098] The second image is captured at a closer focus than the first image, and therefore captures a narrower range than the first image. Therefore, for example, when detecting water droplets adhering anywhere on the entire windshield, the second image will only include a portion of the windshield, so multiple images are required. Therefore, by capturing the second image through scanning, that is, by capturing multiple images with different focal positions when each image is captured, water droplets can be easily detected over a wide range.

[0099] (Technology 7) An information processing device described in any one of Technologies 1 to 6, wherein the first image is an image of a second position captured from a first position through a light-transmitting member, the second image is an image captured from the first position with the focus on the light-transmitting member, and the determination unit determines whether water droplets are attached to the light-transmitting member by determining whether water droplets are reflected in the second image based on the correlation.

[0100] This allows water droplets adhering to the light transmitting member to be easily detected.

[0101] (Technology 8) An information processing device according to Technology 7, wherein the light-transmitting member is a windshield of a vehicle, the first position is inside the vehicle, the second position is far ahead of the vehicle, and the information processing device includes a control unit that operates wipers equipped on the vehicle when the determination unit determines that water droplets are attached to the windshield.

[0102] This makes it easy to determine whether or not water droplets are adhering to the windshield, and if it is determined that water droplets are adhering to the windshield, the water droplets can be removed by the wipers.

[0103] (Technology 9) An information processing method that acquires a first image and a second image captured at a closer focus than the first image, rotates the second image by 180 degrees, detects a first feature from the first image, and detects a second feature from the second image rotated by 180 degrees, calculates a correlation between the first feature and the second feature, and determines whether or not water droplets are reflected in the second image based on the calculated correlation.

[0104] This provides the same effects as the information processing device described in Technology 1.

[0105] (Technology 10) A program for causing a computer to execute the information processing method described in Technology 9.

[0106] This provides the same effects as the information processing device described in Technology 1.

[0107] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0108] The present disclosure is applicable to devices for detecting raindrops, etc.

[0109] REFERENCE SIGNS LIST 10 Information processing system 100 Information processing device 110 Acquisition unit 120 Rotation unit 130 Detection unit 140 Calculation unit 150 Determination unit 160 Control unit 170 Storage unit 200 Imaging device 300, 301, 310, 311 Image W Raindrop

Claims

1. An information processing device comprising: an acquisition unit that acquires a first image and a second image captured at a closer focus than the first image; a rotation unit that rotates the second image by 180 degrees; a detection unit that detects a first feature amount from the first image and detects a second feature amount from the second image rotated by 180 degrees; a calculation unit that calculates the correlation between the first feature amount and the second feature amount; and a determination unit that determines whether or not water droplets are visible in the second image based on the calculated correlation.

2. The information processing device of claim 1, wherein the determination unit determines that water droplets are present in the second image if it determines that the correlation is greater than a predetermined threshold, and determines that water droplets are not present in the second image if it determines that the correlation is equal to or less than the predetermined threshold.

3. The information processing device according to claim 1, wherein the detection unit detects an edge in the first image as the first feature amount, and detects an edge in the second image as the second feature amount.

4. The information processing device of claim 1, wherein the detection unit: cuts out a portion of the first image into a circle; cuts out a portion of the second image rotated 180 degrees into a circle; detects the first feature from the first image cut out into a circle; and detects the second feature from the second image cut out into a circle and rotated 180 degrees.

5. The information processing device according to claim 1, wherein the first image and the second image are images captured by different imaging devices.

6. The information processing device according to claim 1, wherein the second image is an image captured by scanning.

7. An information processing device according to any one of claims 1 to 6, wherein the first image is an image of a second position captured from a first position through a light-transmitting member, the second image is an image captured from the first position with the focus on the light-transmitting member, and the determination unit determines whether water droplets are attached to the light-transmitting member by determining whether water droplets are reflected in the second image based on the correlation.

8. The information processing device according to claim 7, wherein the light-transmitting member is a windshield of a vehicle, the first position is inside the vehicle, the second position is far ahead of the vehicle, and the information processing device includes a control unit that operates wipers equipped on the vehicle when the determination unit determines that water droplets are attached to the windshield.

9. An information processing method comprising: acquiring a first image and a second image captured at a closer focus than the first image; rotating the second image by 180 degrees; detecting a first feature from the first image and a second feature from the second image rotated by 180 degrees; calculating a correlation between the first feature and the second feature; and determining whether or not water droplets are captured in the second image based on the calculated correlation.

10. A program for causing a computer to execute the information processing method according to claim 9.

Images