Information processing apparatus, information processing method, and program

Abstract

An information processing apparatus (100) includes: an acquisition unit (110) that acquires a first image and a second image that is captured at a close focus point compared to the first image; a rotation unit (120) that rotates the second image by 180 degrees; a detection unit (130) that detects a first feature quantity from the first image and a second feature quantity from the second image that is rotated by 180 degrees; a calculation unit (140) that calculates a correlation of the first feature quantity and the second feature quantity; and a determination unit (150) that determines whether a water droplet is present in the second image based on the calculated correlation.

Description

Technical Field

[0001] This disclosure relates to information processing apparatus, information processing methods, and procedures. Background Technology

[0002] Previously, devices for detecting water droplets such as raindrops have existed. Patent Document 1 discloses a water droplet detection device that uses a light-receiving part to detect water droplets, wherein the light-receiving part only detects light traveling within a light-transmitting plate at a refraction angle relative to the normal within a predetermined range. Patent Document 2 discloses a rain sensor that includes a camera device that captures an image of the exterior of a vehicle through the front window from inside the vehicle, and detects whether rainfall has occurred based on image information from the image captured by the camera device. Patent Document 3 discloses a device for detecting water droplets using a polarizing imaging element.

[0003] Existing technical documents Patent documents Patent Document 1: Japanese Patent Application Publication No. 10-31078 Patent Document 2: Japanese Patent Application Publication No. 2006-292543 Patent Document 3: Japanese Patent Application Publication No. 2015-180864 Summary of the Invention

[0004] The problem that the invention aims to solve This disclosure provides an information processing device, etc., capable of easily detecting water droplets.

[0005] Methods used to solve problems An information processing apparatus according to one aspect of this disclosure includes: an acquisition unit that acquires a first image and a second image captured at near focus relative to the first image; a rotation unit that rotates the second image by 180 degrees; a detection unit that detects a first feature quantity from the first image and detects a second feature quantity from the second image rotated by 180 degrees; a calculation unit that calculates the correlation between the first feature quantity and the second feature quantity; and a determination unit that determines whether water droplets are reflected in the second image based on the calculated correlation.

[0006] One aspect of the information processing method disclosed herein involves acquiring a first image and a second image captured at near focus relative to the first image, rotating the second image by 180 degrees, detecting a first feature quantity from the first image, detecting a second feature quantity from the rotated second image, calculating the correlation between the first feature quantity and the second feature quantity, and determining, based on the calculated correlation, whether water droplets are reflected in the second image.

[0007] One aspect of this disclosure is a program for a computer to perform the aforementioned information processing method.

[0008] Invention Effects An information processing device or the like according to one aspect of this disclosure can easily detect water droplets. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating the structure of the information processing system in the implementation method.

[0010] Figure 2 This is a diagram showing a specific example of an embodiment.

[0011] Figure 3 This is a diagram showing a specific example of an embodiment.

[0012] Figure 4 This is a flowchart illustrating the information processing method for implementing the method. Detailed Implementation

[0013] The embodiments will now be described in detail with reference to the accompanying drawings.

[0014] Furthermore, the embodiments described below are general or specific examples. The numerical values, shapes, materials, constituent elements, the arrangement and connection methods of the constituent elements, the steps, and the order of the steps shown in the following embodiments are examples and are not intended to limit this disclosure. In addition, constituent elements in the following embodiments that are not described in the independent claims of this disclosure are described as arbitrary constituent elements.

[0015] Furthermore, in this specification, numerical ranges such as 180 degrees do not merely represent a strict meaning, but rather imply substantially equivalent ranges, including, for example, a difference of approximately 5%.

[0016] In addition, unless otherwise stated, ordinal numbers such as “first” and “second” do not refer to the quantity or order of constituent elements, but are used to distinguish them in order to avoid confusion with the same constituent elements.

[0017] (Implementation Method) [structure] Figure 1 This is a block diagram illustrating the structure of the information processing system 10 in this embodiment.

[0018] The information processing system 10 is a system for detecting water droplets. Specifically, the information processing system 10 detects water droplets adhering to a light-transmitting component, such as a glass window, based on an image obtained from a camera device 200 that captures images through the vehicle's windshield. In this embodiment, the information processing system 10 detects raindrops adhering to the windshield by using an image generated by a camera device 200 disposed inside the vehicle that captures images of the area in front of the vehicle through the vehicle's windshield.

[0019] The information processing system 10 includes an information processing device 100 and a camera device 200.

[0020] The information processing device 100 is a computer that detects water droplets. Specifically, the information processing device 100 detects the presence or absence of water droplets adhering to a light-transmitting component, such as a glass window, based on an image obtained from a camera that captures images of the area in front of the vehicle through the windshield of the vehicle. In this embodiment, the information processing device 100 detects raindrops adhering to the windshield by using an image generated by a camera device 200 disposed inside the vehicle that captures images of the area in front of the vehicle through the windshield.

[0021] The information processing device 100 is implemented, for example, by a communication interface for communicating with the camera device 200, a non-volatile memory storing a program, a volatile memory serving as a temporary storage area for executing the program, input / output ports for transmitting and receiving signals, and a processor for executing the program. The communication interface can be implemented, for example, via an antenna and wireless communication circuit capable of wireless communication, or via a connector for wired communication. The information processing device 100 is mounted in a vehicle, for example, but can also be disposed outside the vehicle.

[0022] The camera device 200 is a camera that generates images (more specifically, pictures) used by the information processing device 100 to detect water droplets. In this embodiment, the camera device 200 is disposed inside a vehicle and generates an image by photographing the front of the vehicle through the windshield of the vehicle.

[0023] Furthermore, the information processing system 10 may have one or more camera devices 200, or it may have only one camera device 200. For example, the information processing system 10 may have a camera device for capturing (generating) a first image and a camera device for capturing a second image.

[0024] In addition, the aforementioned vehicles can be motorcycles or cars, but they can also be mobile robots such as AMRs (Autonomous Mobile Robots). Furthermore, the vehicles can be operated by a driver riding in them, can be remotely operated, and can also drive autonomously.

[0025] Next, the structure of the information processing device 100 will be described in detail.

[0026] The information processing device 100 includes an acquisition unit 110, a rotation unit 120, a detection unit 130, a calculation unit 140, a determination unit 150, a control unit 160, and a storage unit 170.

[0027] The acquisition unit 110 is a processing unit that acquires images captured (generated) by the camera device 200. Specifically, the acquisition unit 110 acquires a first image and a second image from the camera device 200.

[0028] The first image and the second image are images (pictures) captured by setting the focus of the imaging device 200 at different positions. Specifically, the second image is an image captured from the same position of the imaging device 200 when the first image was captured, but with a closer focus than the first image. That is, the acquisition unit 110 acquires the first image and the second image captured at a closer focus than the first image.

[0029] For example, the first image is taken with a wider field of view and a farther focus compared to the second image, while the second image is taken with a smaller field of view and a closer focus compared to the first image. "Wide field of view" means, for example, that the space depicted in the first image is a larger area in actual space compared to the space depicted in the second image.

[0030] For example, the first image is an image obtained by taking a picture of a second position from a first position via a light-transmitting component. Alternatively, for example, the second image is an image obtained by taking a picture of a first position focused on the light-transmitting component.

[0031] A light-transmitting component is a component that allows light detected by the camera device 200 to pass through. The wavelength of this light may be, for example, in the visible region, but can be arbitrary and not particularly limited. A light-transmitting component is, for example, a glass window. In this embodiment, the light-transmitting component is the windshield of a vehicle.

[0032] Additionally, for example, the first position is the interior of the vehicle, and the second position is the far front of the vehicle.

[0033] Furthermore, "far ahead of the vehicle" refers to the exterior of the vehicle and the area in front of it. The distance from the vehicle indicated by "far ahead of the vehicle" can be any distance from the exterior of the vehicle.

[0034] Alternatively, the light-transmitting component can be a glass window of a building. In this case, the first location is, for example, inside the building, and the second location is, for example, outside the building. Alternatively, the light-transmitting component may not be the windshield of a vehicle, but rather a side window or rear window. In this case, the second location can be either a distant view to the side of the vehicle or a distant view to the rear of the vehicle.

[0035] For example, the second image is an image showing a portion of the vehicle's windshield, while the first image is an image showing the view seen from inside the vehicle through the windshield by photographing the entire windshield.

[0036] Furthermore, if the storage unit 170 stores the first image and the second image, the acquisition unit 110 can also acquire the first image and the second image from the storage unit 170.

[0037] The rotation unit 120 is a processing unit that rotates the image. Specifically, the rotation unit 120 rotates the second image by 180 degrees. For example, the rotation unit 120 rotates the second image by 180 degrees with the center of the second image in the image coordinate system as the rotation axis.

[0038] The image coordinate system is a two-dimensional orthogonal coordinate system consisting of the x-axis and y-axis corresponding to the image acquired from the imaging device 200. Specifically, the image coordinate system refers to the following coordinate system: the position of the upper left pixel of the image, assuming that the entire image is displayed on a display device such as a monitor (not shown), is set as the origin; the right side when viewed from that pixel is set as the positive x-axis; and the lower side when viewed from that pixel is set as the positive y-axis.

[0039] The detection unit 130 is a processing unit that detects feature quantities from images. Specifically, the detection unit 130 detects a first feature quantity from a first image and a second feature quantity from a second image rotated 180 degrees. The first feature quantity is a feature quantity detected (extracted) from the first image. The second feature quantity is a feature quantity detected from the second image.

[0040] Feature quantities, for example, are information representing the outlines (edges) of objects reflected in an image. Feature quantities, for example, include information containing the coordinates of multiple feature points constituting the outline. Specifically, feature quantities include information representing the shape of objects reflected in the image, and pixel values ​​representing the color or brightness of each pixel in the image. For example, the detection unit 130 detects edges in the first image as first feature quantities and edges in the second image as second feature quantities. Thus, the detection unit 130 detects edges as the shapes of objects contained in the first and second images, and as locations where there are significant color changes, such as the boundary between areas with clouds and areas without clouds.

[0041] The calculation unit 140 is a processing unit that calculates the correlation (also called correlation value or correlation coefficient) between the first feature quantity and the second feature quantity. That is, the calculation unit 140 calculates the correlation between the first image and the second image rotated by 180 degrees. In other words, the calculation unit 140 calculates the degree of similarity between the first image and the second image rotated by 180 degrees.

[0042] The relevant calculation methods can be arbitrarily determined without any particular limitation. Examples of relevant calculation methods include SAD (Sum of Absolute Difference), SSD (Sum of Squared Difference), or NCC (Normalized Cross-Correlation).

[0043] The determination unit 150 is a processing unit that determines whether water droplets are reflected in the second image based on the correlation calculated by the calculation unit 140. That is, the determination unit 150 detects water droplets reflected in the second image. For example, the determination unit 150 determines whether water droplets are reflected in the second image based on the correlation calculated by the calculation unit 140, thereby determining whether water droplets are attached to a light-transmitting component (e.g., the windshield of a vehicle). For example, if the determination unit 150 determines that the correlation is greater than a predetermined threshold, it determines that water droplets are reflected in the second image; if the determination unit determines that the correlation is less than the predetermined threshold, it determines that water droplets are not reflected in the second image. The predetermined threshold can be arbitrarily determined and is not particularly limited.

[0044] The control unit 160 is a processing unit that performs processing based on the determination result of the determination unit 150. For example, if the determination unit 150 determines that water droplets are attached to the windshield of the vehicle, the control unit 160 activates the windshield wipers. Thus, the control unit 160 removes the water droplets from the windshield using the wipers. The water droplets are, for example, raindrops. Therefore, according to the information processing device 100, for example, when raindrops are attached to the windshield of a vehicle due to rain, it is possible to detect this and automatically remove the raindrops attached to the windshield using the wipers.

[0045] In addition, the control unit 160 can notify the user of the determination result by outputting information indicating the determination result to a display device such as a monitor or an audio device such as a speaker.

[0046] Each processing unit, such as the acquisition unit 110, rotation unit 120, detection unit 130, arithmetic unit 140, determination unit 150, and control unit 160, is implemented, for example, by a memory storing the control program and a processor such as a CPU (Central Processing Unit) executing the control program. The processor of each processing unit can be implemented by a single memory and processor, or they can be implemented by different memories and processors.

[0047] Storage unit 170 is a storage device that stores various types of information. Storage unit 170 may store, for example, images captured by camera device 200 and information indicating predetermined thresholds. Storage unit 170 may be implemented using, for example, a semiconductor memory or an HDD (Hard Disk Drive).

[0048] [Specific example] Next, specific examples of the processing performed by each processing unit will be explained. Furthermore, in the specific examples described below, the first image is an image obtained from inside the vehicle, taken through the windshield, showing a distant view in front of the vehicle, and the second image is an image taken from inside the vehicle, focused on the windshield. Additionally, the determination unit 150 determines whether water droplets are attached to the windshield by determining whether water droplets are reflected in the second image.

[0049] Figure 2 and Figure 3 These are diagrams showing specific examples of images illustrating the implementation methods. Figure 2 Image 300 and shown Figure 3 Image 301 shown in (a) is an example of the first image. Figure 3 Image 310 shown in (b) is an example of the second image. Figure 3 Image 311 shown in (c) is an image after rotating image 310 by 180 degrees, and is an example of a second image rotated by 180 degrees.

[0050] The camera device 200 generates a first image by repeatedly taking pictures of the distant view in front of the vehicle from inside the vehicle through the windshield. Thus, for example, as in image 300, the scenery seen from inside the vehicle is captured as a first image.

[0051] Furthermore, the camera device 200 repeatedly generates a second image by focusing on the windshield and repeatedly taking pictures. Thus, for example, a second image is captured, such as a magnified version of a portion of the windshield.

[0052] For example, if raindrops are not adhering to the windshield, as in image 300, only the view seen from inside the vehicle is captured as the first image. In such a case, the second image becomes an image that is like a magnified version of a portion of the windshield (e.g., an out-of-focus image that directly reflects the scenery).

[0053] On the other hand, for example, when raindrops adhere to the windshield, as in image 301, a first image is captured showing the raindrops W reflected in the scenery seen from inside the car. For example, when the area where the raindrops W adhere is captured as a second image, as in image 310, a larger portion of the raindrops W is reflected. Furthermore, in image 310, the portion of the raindrops W reflected in image 301 due to light refraction, etc., is rotated 180 degrees and reflected. Therefore, as... Figure 3 As shown in (c), the portion of raindrop W in image 311 obtained by rotating image 310 by 180 degrees becomes a part of image 301 (in Figure 3 In the example shown, Figure 3The image is similar to the part enclosed by the single-dotted circle shown in (a).

[0054] Therefore, by detecting the feature quantities of image 301 and image 311 and performing correlation calculations, the information processing device 100 can determine whether water droplets are reflected in image 311.

[0055] Since the first image is captured as a whole of the area in front of the vehicle, it is difficult to detect even if a raindrop W is reflected in part of the first image, because the area reflected by the raindrop W is only slightly blurred. On the other hand, sometimes it is not known whether the scenery or other elements reflected in the second image have been rotated 180 degrees relative to the actual scenery or other elements, based solely on the second image. Therefore, the information processing device 100 detects water droplets by comparing the first image with the second image that has been rotated 180 degrees.

[0056] [Processing Order] Next, the processing sequence of the information processing device 100 in the embodiment will be described.

[0057] Figure 4 This is a flowchart illustrating an information processing method implemented in this way. For example, the information processing apparatus 100 includes a processor and a memory, and the processor uses the memory to perform the following processing.

[0058] First, the information processing device 100 acquires a first image and a second image captured at a closer focus than the first image (S110). Specifically, the acquisition unit 110 of the information processing device 100 acquires the first image and the second image from the imaging device 200. For example, the information processing device 100 controls the imaging device 200 to capture images in a manner that generates the first image and the second image.

[0059] Furthermore, the first image is, for example, image 301. The second image is, for example, image 310.

[0060] Next, the information processing device 100 rotates the acquired second image by 180 degrees (S120). For example, the information processing device 100 changes the position of each pixel in the second image in the image coordinate system by rotating the second image about the center of the second image in the image coordinate system as the rotation axis.

[0061] Next, the information processing device 100 detects feature quantities from the first image and the second image rotated by 180 degrees respectively (S130). Specifically, the information processing device 100 detects a first feature quantity from the first image and a second feature quantity from the second image rotated by 180 degrees.

[0062] In addition, a second image rotated 180 degrees is, for example, image 311.

[0063] Next, the information processing device 100 performs a calculation on the correlation between the detected first feature and the detected second feature (S140).

[0064] Next, the information processing device 100 determines whether a water droplet is reflected in the second image based on the calculated correlation (S150).

[0065] For example, the information processing device 100 performs processing based on the determination result of step S150. For example, if raindrops are detected adhering to the windshield of the vehicle, and it is determined that water droplets are reflected in the second image, the information processing device 100 determines that raindrops are adhering to the windshield and removes the raindrops from the windshield by activating the windshield wipers of the vehicle.

[0066] Furthermore, the camera device 200 can also generate multiple second images by changing the shooting position. In this case, multiple second images can also be acquired in step S110. Additionally, in this case, steps S120 to S150 can be performed on each of the multiple second images separately. For example, the information processing device 100 can activate the windshield wipers if it determines that water droplets are reflected in one of the multiple second images, or if it determines that water droplets are reflected in a predetermined number or more of the multiple second images. Of course, the information processing device 100 can also deactivate the windshield wipers if it determines that no water droplets are reflected in any of the multiple second images, or if it determines that water droplets are reflected only in a predetermined number of the multiple second images. The predetermined number can be arbitrarily determined and is not particularly limited.

[0067] [Effects, etc.] As described above, the information processing apparatus 100 of this embodiment includes: an acquisition unit 110 that acquires a first image and a second image captured at near focus relative to the first image; a rotation unit 120 that rotates the second image by 180 degrees; a detection unit 130 that detects a first feature quantity from the first image and a second feature quantity from the second image rotated by 180 degrees; a calculation unit 140 that performs a correlation calculation on the first feature quantity and the second feature quantity; and a determination unit 150 that determines whether water droplets are reflected in the second image based on the calculated correlation.

[0068] If the portion of an image showing water droplets is magnified, it may appear as if a part of the entire image has been rotated 180 degrees. Therefore, for example, by rotating one of the first and second images by 180 degrees, one can compare the first image obtained by shooting from a distance with the second image obtained by focusing on a close-up shot compared to the case of shooting the first image.

[0069] For example, the second image is an image reflecting a portion of the vehicle's windshield, while the first image is an image reflecting the scenery seen from inside the vehicle through the windshield, captured as a whole. In the comparison processing of the first and second images, features such as the shape of objects reflected in each image and the color (pixel values ​​representing color) are compared. If a water droplet is reflected in the second image, at least a portion of the first image becomes the same as at least a portion of the second image rotated 180 degrees (specifically, the portion reflecting the water droplet), thus the correlation is high. That is, if a water droplet is reflected in the second image, the first feature is similar to the second feature. Thus, if the information processing device 100 rotates the second image by 180 degrees and there is a correlation with the first image captured from a distance relative to the second image as seen from the camera device 200, it is considered that a water droplet is reflected in the second image. Therefore, water droplets can be easily detected from the first and second images.

[0070] In this embodiment, the first image is an image taken from inside the vehicle through the windshield, showing a distant view in front of the vehicle, and the second image is an image taken from inside the vehicle, focused on the windshield. Water droplets are, for example, raindrops. For instance, the control unit 160 of the information processing device 100 activates the vehicle's windshield wipers when the determination unit 150 determines that water droplets are reflected in the second image, that is, when it determines that raindrops are attached to the windshield.

[0071] Therefore, it is possible to easily determine whether there are raindrops attached to the windshield, and if it is determined that there are raindrops attached to the windshield, the raindrops can be automatically removed by the wipers.

[0072] Furthermore, these general or specific methods can be implemented by systems, methods, integrated circuits, computer programs, or non-transitory recording media such as computer-readable CD-ROMs storing the computer programs, or by any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0073] For example, this disclosure can also be implemented as an information processing method executed by an information processing device 100, including: acquiring a first image and a second image captured at near focus relative to the first image (S110); rotating the second image by 180 degrees (S120); detecting a first feature quantity from the first image and detecting a second feature quantity from the second image rotated by 180 degrees (S130); performing a correlation calculation on the first feature quantity and the second feature quantity (S140); and determining whether a water droplet is reflected in the second image based on the calculated correlation (S150). Alternatively, for example, this disclosure can also be implemented as a program for a computer to execute the information processing method. Alternatively, for example, this disclosure can also be implemented as a computer-readable non-transitory recording medium storing the program.

[0074] (Modified Example) Furthermore, the structure of the information processing system 10 and the processing performed by the information processing system 10 are not limited to the above.

[0075] For example, when raindrops or other water droplets adhere to the windshield, the droplets tend to form a circle. Therefore, for example, the detection unit 130 cuts a portion of the first image into a circle, and cuts a portion of the second image rotated 180 degrees into a circle, detecting a first feature from the first image cut into a circle, and detecting a second feature from the second image cut into a circle and rotated 180 degrees. For example, along... Figure 3 As shown in (c), the outer edge of the raindrop W is cut out in a circular shape from a portion of the second image that has been rotated 180 degrees.

[0076] Furthermore, the circle can be a perfect circle or an ellipse. Additionally, the cutting position can be determined, for example, by making the center of the image the center of the circle, but it can also be determined arbitrarily. Furthermore, the size of the circle can be determined, for example, by making the diameter of the circle below the shorter side of the rectangular image, but it can also be arbitrary.

[0077] Furthermore, the process of circularly cropping a portion of the second image can be performed either before or after the 180-degree rotation. If the circular cropping of the second image is performed before the 180-degree rotation, the feature detection process is performed after rotating the circularly cropped second image by 180 degrees.

[0078] Alternatively, for example, image processing can be performed on the second image, and if a circular portion can be detected as the outer edge of the raindrop W, then the detected circular portion can be cut out.

[0079] Alternatively, for example, the first image and the second image may be images captured by the camera device 200, i.e., a single camera device 200. However, the first image and the second image may also be images captured by different camera devices. That is, the first image and the second image may be captured by their respective different camera devices.

[0080] Furthermore, for example, the second image is an image captured by showing a portion of the windshield. Therefore, in order to determine whether water droplets are attached to the entire windshield, multiple second images at different positions need to be captured. Therefore, for example, the second image could also be a scanned image. That is, the camera device for capturing the second image could be, for example, a camera device capable of focusing on the windshield while changing the shooting position to capture images. This camera device can have a drive mechanism such as a motor to change the position and / or posture of the camera unit performing the shooting.

[0081] (Other implementation methods) As described above, embodiments have been illustrated as examples of the technology disclosed herein. However, the technology disclosed herein is not limited thereto and can also be applied to embodiments with appropriate modifications, substitutions, additions, omissions, etc. For example, the following variations are also included in one embodiment of this disclosure.

[0082] For example, in the case where this disclosure is implemented by a program (software), each step is executed by utilizing the computer's hardware resources such as the CPU, memory, and input / output circuits. That is, the CPU retrieves data from the memory or input / output circuits, performs calculations, and outputs the calculation results to the memory or input / output circuits, thereby executing each step.

[0083] Furthermore, the program can be executed by a single computer or multiple computers. That is, it can be processed centrally or distributed.

[0084] Furthermore, in the above embodiments, each component included in the information processing apparatus 100 can be constructed by dedicated hardware or implemented by executing software programs suitable for each component. Each component can also be implemented by a program execution unit such as a CPU or processor reading and executing software programs recorded on a recording medium such as a hard disk or semiconductor memory.

[0085] The functions of the information processing apparatus 100 described above are typically implemented, in whole or in part, as integrated circuits, i.e., LSIs. They can be implemented as a single chip, or as a single chip including some or all of the functions. Furthermore, integrated circuit implementation is not limited to LSIs; it can also be implemented using dedicated circuits or general-purpose processors. Alternatively, FPGAs (Field Programmable Gate Arrays) capable of being programmed after LSI fabrication, or reconfigurable processors capable of reconfiguring the connections and settings of the circuit units within the LSI, can be utilized.

[0086] Furthermore, if an integrated circuit technology emerges to replace LSI due to advancements in semiconductor technology or other derived technologies, then of course that such technology can also be used to integrate the various components included in the information processing device 100.

[0087] Alternatively, for example, in the above embodiments, the processing performed by the specific processing unit may be executed by other processing units. Furthermore, the order of multiple processes may be changed, or multiple processes may be executed in parallel.

[0088] Furthermore, this disclosure also includes various modifications to the implementation methods that can be conceived by those skilled in the art, and methods that arbitrarily combine the constituent elements and functions of each implementation method without departing from the spirit of this disclosure.

[0089] (Postscript) The following technology has been disclosed through the above description of the embodiments.

[0090] (Technology 1) An information processing apparatus comprising: an acquisition unit for acquiring a first image and a second image captured at a near-focus relative to the first image; a rotation unit for rotating the second image by 180 degrees; a detection unit for detecting a first feature quantity from the first image and detecting a second feature quantity from the second image rotated by 180 degrees; a calculation unit for calculating the correlation between the first feature quantity and the second feature quantity; and a determination unit for determining, based on the calculated correlation, whether a water droplet is reflected in the second image.

[0091] The portion of an image showing a water droplet might be an image in which a part of the entire image has been rotated 180 degrees. Therefore, for example, given a first image captured from a distance and a second image captured at close range (compared to the case of capturing the first image), the second image is rotated 180 degrees, and the feature values ​​of the first and second images are compared. Thus, if a water droplet is reflected in the second image, at least a portion of the first image becomes identical to at least a portion of the second image rotated 180 degrees, resulting in a higher correlation. Therefore, it is possible to easily detect water droplets based on two images captured at different focal lengths.

[0092] (Technology 2) According to the information processing apparatus of Technology 1, the determination unit determines that water droplets are reflected in the second image when it determines that the correlation is greater than a predetermined threshold, and the determination unit determines that water droplets are not reflected in the second image when it determines that the correlation is less than the predetermined threshold.

[0093] Therefore, water droplets can be detected easily.

[0094] (Technology 3) In the information processing apparatus according to Technology 1 or 2, the detection unit detects the edge in the first image as the first feature quantity and detects the edge in the second image as the second feature quantity.

[0095] Therefore, when an object is reflected in the first and second images, water droplets can be easily detected based on the shape of the object reflected in the image detected by edge detection.

[0096] (Technology 4) In the information processing apparatus according to any one of Technologies 1 to 3, the detection unit cuts out a portion of the first image in a circular shape and cuts out a portion of the second image rotated by 180 degrees in a circular shape, the detection unit detects the first feature quantity from the first image cut out in a circular shape, and detects the second feature quantity from the second image cut out in a circular shape and rotated by 180 degrees.

[0097] For example, water droplets adhering to a windshield may be round. Therefore, it is possible to crop the portion of the image rotated 180 degrees due to the water droplet and compare the feature values. Thus, the area in the image where feature values ​​are detected can be reduced, thereby reducing the processing load. Furthermore, if an image is projected that has been rotated 180 degrees due to the water droplet, a large number of images containing the 180-degree rotated portion can be used to detect the feature values, thus enabling high-precision detection of the water droplet.

[0098] (Technology 5) The information processing apparatus according to any one of Technologies 1 to 4, wherein the first image and the second image are images captured by different camera devices.

[0099] Therefore, compared to using the same camera device to change the focal length to capture the first and second images, it is possible to detect water droplets by simply processing the captured first and second images.

[0100] (Technology 6) In any one of the technology 1 to 5, the second image is an image captured by scanning.

[0101] Because the second image was captured at a closer focus than the first image, it covers a smaller area. Therefore, in cases such as detecting water droplets adhering to a specific point on the windshield, the second image would only show a portion of the windshield, necessitating multiple images. Thus, by setting the second image as an image captured through scanning—that is, by setting multiple images with different focal points at the time of capture—it is possible to easily detect water droplets over a large area.

[0102] (Technology 7) The information processing apparatus according to any one of Technologies 1 to 6, wherein the first image is an image obtained by taking a picture of a second position from a first position via a light-transmitting member, the second image is an image taken from the first position focused on the light-transmitting member, and the determination unit determines whether water droplets are reflected in the second image based on the correlation, thereby determining whether water droplets are attached to the light-transmitting member.

[0103] Therefore, it is possible to easily detect water droplets adhering to light-transmitting components.

[0104] (Technology 8) According to the information processing device of Technology 7, the light-transmitting component is the windshield of a vehicle, the first position is the interior of the vehicle, the second position is the far front of the vehicle, the information processing device includes a control unit, and the control unit causes the windshield wipers of the vehicle to operate when the determination unit determines that water droplets are attached to the windshield.

[0105] Therefore, it is easy to determine whether there are water droplets attached to the windshield, and if it is determined that there are water droplets attached to the windshield, the water droplets can be removed by the wipers.

[0106] (Technology 9) An information processing method, acquiring a first image and a second image captured at near focus relative to the first image, rotating the second image by 180 degrees, detecting a first feature quantity from the first image, and detecting a second feature quantity from the second image rotated by 180 degrees, calculating the correlation between the first feature quantity and the second feature quantity, and determining whether water droplets are reflected in the second image based on the calculated correlation.

[0107] Thus, it achieves the same effect as the information processing device described in Technique 1.

[0108] (Technology 10) A program for a computer to perform the information processing method described in Technology 9.

[0109] Thus, it achieves the same effect as the information processing device described in Technique 1.

[0110] Furthermore, these general or specific methods can be implemented through systems, methods, integrated circuits, computer programs, or non-transitory recording media such as computer-readable CD-ROMs, or through any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0111] Industrial applicability This disclosure can be applied to devices used for detecting raindrops, etc.

[0112] Explanation of reference numerals in the attached figures 10 Information Processing System 100 Information Processing Device 110 obtained department 120 rotating part 130 Testing Department 140 Computing Unit 150 Judgment Department 160 Control Department 170 Storage Division 200 camera devices Images 300, 301, 310, and 311 W raindrops

Claims

1. An information processing device, have: The acquisition unit acquires a first image and a second image captured at near focus compared to the first image; The rotating part rotates the second image by 180 degrees. The detection unit detects a first feature from the first image and a second feature from the second image that has been rotated 180 degrees. The arithmetic unit performs calculations on the correlation between the first feature quantity and the second feature quantity; as well as The determination unit, based on the calculated correlation, determines whether water droplets are reflected in the second image.

2. The information processing device according to claim 1, If the determination unit determines that the correlation is greater than a predetermined threshold, it determines that water droplets are reflected in the second image. If the determination unit determines that the correlation is below the predetermined threshold, it determines that no water droplets are reflected in the second image.

3. The information processing device according to claim 1, The detection unit detects edges in the first image as the first feature quantity and detects edges in the second image as the second feature quantity.

4. The information processing device according to claim 1, The detection unit cuts out a portion of the first image in a circular shape, and also cuts out a portion of the second image, which has been rotated 180 degrees, in a circular shape. The detection unit detects the first feature quantity from the first image cut out in a circle, and detects the second feature quantity from the second image cut out in a circle and rotated 180 degrees.

5. The information processing device according to claim 1, The first image and the second image were captured by different camera devices.

6. The information processing apparatus according to claim 1, The second image is a scanned image.

7. The information processing apparatus according to any one of claims 1 to 6, The first image is an image obtained by capturing a second position from a first position through a light-transmitting component. The second image is an image captured by focusing on the light-transmitting component from the first position. The determination unit determines whether water droplets are reflected in the second image based on the correlation, thereby determining whether water droplets are attached to the light-transmitting component.

8. The information processing apparatus according to claim 7, The light-transmitting component is the vehicle's windshield. The first location is inside the vehicle. The second position is in the far front of the vehicle. The information processing device includes a control unit, which activates the vehicle's windshield wipers when the determination unit determines that water droplets are attached to the windshield.

9. An information processing method, Acquire a first image and a second image captured at near focus compared to the first image. Rotate the second image 180 degrees. A first feature is detected from the first image, and a second feature is detected from the second image rotated 180 degrees. Calculate the correlation between the first feature and the second feature. Based on the calculated correlation, it is determined whether water droplets are reflected in the second image.

10. A program for a computer to perform the information processing method of claim 9.

Images