Image processing apparatus and image processing method

Abstract

The purpose of the present invention is to detect generation of a light glare line regardless of the direction in which the light glare line extends. An image processing device according to the present invention comprises a light point detection unit, a light glare line candidate extraction unit, and a light glare line presence / absence determination unit. The light point detection unit detects, from an image obtained by a camera attached in a vehicle, a light point having a luminance value of not less than a prescribed value. The light glare line candidate extraction unit extracts a light glare line candidate that extends from the light point which has been detected by the light point detection unit. When the difference in angle between a portion of the light glare line candidate which extends toward one side from the light point and a portion of the light glare line candidate which extends toward another side from the light point is within a prescribed range, the light glare line presence / absence determination unit determines that the light glare line candidate is a light glare line.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus and an image processing method.

Background Art

[0002] In recent years, for the automatic driving of vehicles (automobiles), a technique for recognizing an image captured using a camera (in-vehicle camera) has been widely used. The camera is installed, for example, inside the windshield. Therefore, the image processing apparatus may not be able to recognize the surrounding environment of the vehicle properly due to dirt on the windshield. For example, when there is a wiper smear on the windshield or when there is a scratch on the windshield, when strong light hits during imaging, a streak of light extending along the smear or scratch (a light streak) is captured in the image.

[0003] Patent Document 1 describes an object recognition apparatus and a dirt detection method using a captured image of a stereo camera that captures an image outside the vehicle through a window glass from inside the vehicle cabin of a vehicle such as an automobile. The object recognition apparatus described in Patent Document 1 detects an edge extending in a direction intersecting the wiping direction of the wiper, and in combination with the light source detection result, determines that the front windshield is in a dirty state when an edge is continuously detected.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the object recognition apparatus described in Patent Document 1 needs to preset a specific direction intersecting the wiping direction of the wiper. Therefore, the specific direction had to be set for each vehicle with a different wiper wiping direction.

[0006] Furthermore, when the windshield is wiped using a method other than the wipers, streaks of light captured due to remaining streaks or scratches on the windshield may not have edges in the direction intersecting the wiper's wiping direction. For this reason, the object recognition device described in Patent Document 1 was unable to detect light streaks extending in directions other than the wiper's wiping direction.

[0007] The objective of this invention is to provide an image processing apparatus and an image processing method that can detect the generation of light rays regardless of the direction in which they extend, taking into consideration the above-mentioned problems. [Means for solving the problem]

[0008] To solve the above problems and achieve the objective, an image processing apparatus according to one aspect of the present invention comprises a light spot detection unit, a light streak candidate extraction unit, and a light streak presence / absence determination unit. The light spot detection unit detects light spots with a brightness value of a predetermined value or higher from an image acquired by a camera mounted inside a vehicle. The light streak candidate extraction unit extracts light streak candidates extending from the light spots detected by the light spot detection unit. The light streak presence / absence determination unit determines that a light streak exists if the difference in angle between a portion of the light streak candidate extending from one light spot and a portion of the light streak candidate extending from the light spot to the other is within a predetermined range. [Effects of the Invention]

[0009] According to one aspect of the present invention, the generation of light rays can be detected regardless of the direction in which they extend. Furthermore, issues, configurations, and effects other than those mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram showing the configuration of a vehicle equipped with an image processing device according to one embodiment. [Figure 2] This is a diagram illustrating the operation of a vehicle's windshield wipers. [Figure 3] A and B are diagrams illustrating images of the state in which light rays are generated. [Figure 4]This flowchart shows an example of a light streak detection process according to one embodiment. [Figure 5] A and B are diagrams illustrating a binarized image according to one embodiment. [Figure 6] A and B are diagrams illustrating cropped images according to one embodiment. [Figure 7] This is an illustrative diagram illustrating the angle of a search point relative to a light point according to one embodiment. [Figure 8] This figure shows the results of the angle voting according to one embodiment. [Figure 9] The image shows a cropped image according to one embodiment, with a straight line drawn at the angle that maximizes the number of votes. [Modes for carrying out the invention]

[0011] 1. Embodiment An embodiment of the image processing apparatus and image processing method will be described below with reference to Figures 1 to 9. Common parts in each figure are denoted by the same reference numerals.

[0012] [Example of vehicle configuration] First, the configuration of a vehicle equipped with an image processing device according to one embodiment will be described with reference to Figure 1. Figure 1 is a schematic diagram showing the configuration of a vehicle equipped with an image processing device.

[0013] As shown in Figure 1, vehicle (automobile) 1 is equipped with an on-board camera 10, an image processing device 20, and a vehicle control device 30. The image processing device 20 recognizes objects outside the vehicle from images acquired (captured) by the on-board camera 10. The recognition results from the image processing device 20 are sent to the vehicle control device 30. The vehicle control device 30 implements driver assistance functions such as ACC (Adaptive cruise control) and AEB (Autonomous Emergency Braking).

[0014] The in-vehicle camera 10 is composed of a plurality of imaging devices arranged at a predetermined interval in the vehicle width direction inside the windshield. The plurality of imaging devices are installed in a predetermined direction around the vehicle 1, for example, facing the front of the vehicle 1. Hereinafter, the in-vehicle camera 10 will be described as a stereo camera composed of two cameras, a left camera 11 and a right camera 12. Note that the in-vehicle camera may be composed of three or more cameras.

[0015] The left camera 11 and the right camera 12 of the in-vehicle camera 10 image the front of the vehicle 1. The image captured by the in-vehicle camera 10 is supplied to the image processing device 20.

[0016] The image processing device 20 includes hardware such as a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and an electronic circuit, and software that performs image recognition processing in cooperation with the hardware. That is, the CPU reads out various processing programs stored in the ROM and expands them in the RAM, and controls the image recognition processing according to the expanded programs.

[0017] The image processing device 20 performs image correction processing such as camera calibration and distortion correction on the captured image. Then, it recognizes objects such as objects and people from the image on which the image correction processing has been performed. Further, the image processing device 20 detects light streaks from the captured image. It has a light point detection unit 21, a light streak candidate extraction unit 22, a light streak presence / absence determination unit 23, and a recognition unit 24.

[0018] The light point detection unit 21 analyzes the pixel values (luminance values) in the image captured by the in-vehicle camera 10 using a known technique. Then, the light point detection unit 21 detects the position (light point) of the light source on the image. The light point is, for example, the highest one among the points with a luminance of 200 or more, or one point is detected from a strong light source such as a headlight or a streetlight. Note that the light point is not limited to one point, and a plurality of light points may be selected (detected). The light point detection unit 21 outputs the detection result to the light streak candidate extraction unit 22.

[0019] The light streak candidate extraction unit 22 extracts straight lines that are candidates for light streaks based on the light points acquired by the light point detection unit 21. Normally, wiper residue that causes light streaks is often arc-shaped, but it can be approximated as a straight line. Therefore, the light streak candidate extraction unit 22 extracts one of the straight lines passing through the light source as a candidate light streak line. The light streak candidate extraction unit 22 outputs the extraction result to the light streak presence / absence determination unit 23.

[0020] The light streak presence / absence determination unit 23 determines the presence or absence of light streaks by determining whether they are point-symmetric with respect to the light streak candidate line obtained by the light streak candidate extraction unit 22, and by using other auxiliary information. Other auxiliary information includes the thickness of the light streak candidate line, which will be described later, and vehicle information such as the speed of vehicle 1 and whether or not the wipers are being used.

[0021] The light streak detection unit 23 outputs to the recognition unit 24 that it has detected light streaks, for example, when light streaks are present in multiple consecutive frames. When the recognition unit 24 receives notification from the light streak detection unit 23 that light streaks have been detected, it outputs to the vehicle control device 30 to stop the driving assistance function of the light streak detection unit 23.

[0022] The recognition unit 24 detects (recognizes) objects such as vehicles and pedestrians in the surrounding environment (in this case, in front) from the image captured by the in-vehicle camera 10. The recognition unit 24 outputs the object recognition result to the vehicle control device 30.

[0023] The vehicle control device 30 comprises hardware such as a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and electronic circuits, as well as software that works in cooperation with the hardware to perform vehicle control processing, including driver assistance functions. Specifically, the CPU reads various processing programs stored in the ROM, loads them into the RAM, and controls the vehicle control processing according to the loaded programs.

[0024] The vehicle control device 30 acquires the object recognition results from the recognition unit 24 as information necessary for autonomous driving, and controls the driving state of the vehicle 1. For example, the vehicle control device 30 controls braking to avoid contact with objects or people in front of the vehicle 1, driving at a speed that follows another vehicle in front of the vehicle 1, and steering to ensure that the vehicle stays within its lane.

[0025] [Wiper operation] Next, the operation of the wipers will be explained with reference to Figure 2. Figure 2 illustrates the wiper operation of vehicle 1.

[0026] As shown in Figure 2, vehicle 1 is equipped with wipers 101 and 102. The wipers 101 and 102 wipe the windshield 100 in front of the left and right cameras 11 and 12. When the wipers 101 and 102 are operating, a thin streak of unwiped areas may occur, as shown by the dashed line.

[0027] Wiper streaks do not create light streaks in the images captured by the left and right cameras 11 and 12 during the daytime when the outside world is bright, or at night when there is no light source in the outside world. However, at night when there is a light source in the outside world, the light from the light source is extended by the streaks, causing light streaks to appear.

[0028] [Evolution of light rays] Next, we will explain the image from which light streaks are generated, referring to Figure 3. Figure 3 is an image illustrating the state in which light rays are generated.

[0029] Figure 3A is a schematic diagram showing a state where there is no dirt or scratches such as wiper streaks, and no light streaks appear in nighttime images. As shown in Figure 3A, if there is no dirt or scratches such as wiper streaks, light streaks will not appear in the captured image even if there is a light source.

[0030] Figure 3B is a schematic diagram showing the appearance of light streaks in nighttime images due to streaks left behind by the wipers. As shown in Figure 3B, objects that do not have strong light sources, such as pedestrians, are captured without problems. However, light sources such as car headlights, brake lights, and streetlights cause light to be stretched, resulting in light streaks. Note that light streaks caused by scratches on the windshield or streaks left behind by sources other than the wipers extend at a different angle (direction) than light streaks caused by streaks left behind by the wipers.

[0031] [Photon Beam Detection Processing] Next, the light streak detection process performed by the image processing device 20 will be explained with reference to Figure 4. Figure 4 is a flowchart showing an example of the light vein detection process.

[0032] First, the light spot detection unit 21 of the image processing device 20 detects light spots from the image sent from the in-vehicle camera 10 (S101). In step S101, the light spot detection unit 21 analyzes the pixel values ​​(luminance values) in the image using known techniques and detects the position of the light source (light spot) on the image.

[0033] Next, the light spot detection unit 21 determines whether or not there is a light spot (S102). If it is determined in step S102 that there is no light spot (S102 is a NO determination), the light spot detection unit 21 (image processing device 20) terminates the light streak detection process.

[0034] In step S102, if it is determined that a light spot exists (if S102 is a YES determination), the light spot detection unit 21 sends the detection result to the light streak candidate extraction unit 22. The light streak candidate extraction unit 22 performs image binarization processing based on the brightness value of the detected light spot (S103).

[0035] Figure 5A shows the image before binarization. The optical vein candidate extraction unit 22 performs binarization on the image shown in Figure 5A, using the brightness values ​​of the detected optical spots as a reference. Specifically, when the brightness of an optical spot is p and the binarization range of the brightness value is ±d, the pixel value is set to 1 if the brightness value of a pixel is greater than or equal to p and less than or equal to p+d, and to 0 otherwise. This enables the image to be binarized.

[0036] Figure 5B shows the image after binarization. As shown in Figure 5B, in the image after binarization, the pixel values ​​of light sources such as brake lights and streetlights, and light streaks, become 1 (white), while the pixel values ​​of other dark areas become 0 (black).

[0037] Referring again to Figure 4, we will continue the explanation of the light streak detection process. Next, the light streak candidate extraction unit 22 performs contour tracing on the binarized image to extract the image of the light streak portion (S104). In the contour tracing process, first, pixels are searched vertically or horizontally from the position of the light point to find pixels where the pixel value changes from 1 to 0. Then, using the pixels where the pixel value changes from 1 to 0 as a reference, the contour tracing process is performed again.

[0038] Contour tracking can be performed using a known algorithm. The contour tracking search is performed while simultaneously updating the minimum and maximum values ​​of the pixel's vertical (Y-axis) and horizontal (X-axis) directions. Once contour tracking is complete, the optical vein candidate extraction unit 22 extracts a binarized image from the minimum and maximum values ​​of the vertical and horizontal directions.

[0039] Figure 6A shows an image after contour tracking processing. As shown in Figure 6A, the contours of the light rays are detected in the image after contour tracking processing. Figure 6B is an image of a candidate light ray extracted from the binarized image.

[0040] Referring again to Figure 4, we will continue the explanation of the light streak detection process. Next, the light streak candidate extraction unit 22 calculates the angle θ that the light streak makes with the light streak for all points whose pixel values ​​are 1, excluding the light streak, in the extracted binarized image.

[0041] Figure 7 is an illustrative diagram illustrating the angle between the light point and the search point. Specifically, Figure 7 shows the angle θ between the light point and the search point. When the coordinates of the light point on the image are (xi, yi) and the point for calculating the angle is (xp, yp), the angle θ [deg] is calculated by equation (1). However, θ is assumed to be 0 ≤ θ < 180. [Mathematics 1] TIFF0007853456000001.tif17150

[0042] For example, if a single extracted binarized image (a candidate for a light streak) contains the brake lights of a preceding vehicle on both sides, then there are two light sources for that single candidate light streak. In this way, if there are other light points besides the target light point for a single candidate light streak, the angle of the search point cannot be calculated correctly. Therefore, if there are two or more light points in the extracted binarized image, the angle of the search point is not calculated. In other words, the extracted binarized image is excluded from the candidates for light streaks, and the determination of whether or not it is a light streak is not made.

[0043] Figure 8 shows the results of the angle voting. As shown in Figure 8, by performing the angle calculation using equation (1) described above for all points with a pixel value of 1 (search points) excluding light points, voting results can be obtained with angle on the horizontal axis and the number of votes on the vertical axis.

[0044] The angle with the highest number of votes in the voting results is the angle of the straight line that will become the candidate for a light streak (the candidate light streak line). Whether or not this candidate light streak line is a light streak is determined by the process described later. Note that although light streaks caused by wiper streaks are arc-shaped, they can be approximated as straight lines in the image. Therefore, the candidate light streak is approximated as a straight line centered on the light point to determine whether or not it is a light streak.

[0045] In the example shown in Figure 8, the angle that yields the maximum number of votes is 160 degrees. Therefore, in the subsequent processing, we determine whether a 160-degree straight line passing through a light point (a candidate line for a light streak) is a light streak or not. Figure 9 shows the extracted image with a 160-degree straight line drawn on it.

[0046] In this embodiment, the angle range θ was set to 0 ≤ θ < 180 in the angle voting process. However, since the wiper wiping angle generally does not change, the range of angles to be voted on may be narrowed by learning using information from multiple frames (images). For example, if the angle is consistently determined to be 100° in multiple frames (e.g., multiple images acquired over 10 seconds), the angle calculation can be performed in the range of 80 ≤ θ < 130 instead of 0 ≤ θ < 180. This can reduce the processing load.

[0047] Alternatively, the system can be trained using information from multiple frames (images) to set the target angles for voting, and votes will be excluded from angles other than those set. This increases the proportion of the maximum number of votes (angle votes) in the total number of votes. Therefore, the accuracy of the angle of the candidate light ray line can be improved using other auxiliary information (information from multiple frames (images)).

[0048] Referring again to Figure 4, we will continue the explanation of the light streak detection process. Next, the light streak presence / absence determination unit 23 performs a light streak presence / absence determination process (S106). In step S106, the light streak presence / absence determination unit 23 determines whether or not a light streak has occurred by determining whether or not the candidate light streak line is point-symmetric with respect to the light point. The light streak presence / absence determination unit 23 then outputs the determination result to the recognition unit 24.

[0049] It is determined whether the difference between the distance from the light point to the point furthest upward (or leftward) and the distance from the light point to the point furthest downward (or rightward) among the points with a pixel value of 1 on the candidate light ray line is less than or equal to a predetermined threshold (for example, 10 pixels).

[0050] When the difference in distance from the light point to each point furthest away is less than or equal to a threshold, the light streak presence / absence determination unit 23 determines that the candidate light streak line is point-symmetric with respect to the light point and determines that a light streak is present. On the other hand, when the difference in distance from the light point to each point furthest away is not less than or equal to a threshold, the light streak presence / absence determination unit 23 determines that the candidate light streak line is not point-symmetric with respect to the light point and determines that a light streak is not present.

[0051] Furthermore, in the process of determining the presence or absence of light rays, in addition to determining whether the candidate light ray line is point-symmetric with respect to the light point, the determination may also be made by combining information about the candidate light ray, such as the ratio of pixels with a pixel value of 1 to the entire cropped image, the length of the cropped image, the ratio of angle votes to the total number of votes for the candidate light ray line, and the thickness of the candidate light ray line, with at least one piece of information about vehicle 1, such as the speed of vehicle 1 and whether or not the wipers are being used. This can suppress misjudgments of whether or not a light ray is present.

[0052] For example, if the ratio of pixels with pixel value 1 to the entire cropped image is smaller than a predetermined value, it is possible that the light is not extending in a streaky pattern. Therefore, the light streaking presence / absence determination unit 23 determines that no light streaks are present if the ratio of pixels with pixel value 1 to the entire cropped image is smaller than a predetermined value.

[0053] Furthermore, if the cropped image is smaller than a predetermined length (for example, a 40-pixel square), the symmetry (point symmetry) may be misjudged. Therefore, the light streak presence / absence determination unit 23 determines that no light streaks are present if the cropped image is smaller than a predetermined size.

[0054] Furthermore, if the thickness of a candidate light streak line exceeds a predetermined specific value, it is often not a light streak but rather an object such as a guide rail reflecting light. Therefore, the light streak presence / absence determination unit 23 determines that no light streaks are present if the thickness of the candidate light streak line or the candidate light streak (cropped image) exceeds a predetermined value.

[0055] Next, the recognition unit 24 determines whether the light streak presence / absence determination unit 23 has determined that there are light streaks (S107). If the light streak presence / absence determination unit 23 determines that there are light streaks in step S107 (if S107 is a YES determination), the recognition unit 24 adds "1" to the pause (HALT) occurrence counter (S108).

[0056] On the other hand, in step S107, if the light streak presence / absence determination unit 23 determines that there are no light streaks (if S107 is a NO determination), the recognition unit 24 subtracts "1" from the pause occurrence counter (S109).

[0057] After processing in step S108 or step S109, the recognition unit 24 determines whether the counter value of the pause occurrence counter is equal to or greater than a reference value (S110). If in step S110 it is determined that the counter value of the pause occurrence counter is not equal to or greater than a reference value (if S110 is a NO determination), the recognition unit 24 (image processing device 20) terminates the light streak detection process.

[0058] On the other hand, in step S110, if the recognition unit 24 determines that the counter value of the pause occurrence counter is equal to or greater than the reference value (if S110 is determined to be YES), the recognition unit 24 outputs a driver assistance function temporary suspension (HALT) command to the vehicle control device 30 (S111). When the vehicle control device 30 receives the driver assistance function temporary suspension command, it temporarily suspends the driver assistance function. After the processing in step S111, the image processing device 20 terminates the light streak detection process.

[0059] Furthermore, if multiple light spots are detected in step S101, steps 103 to S106 are performed for each detected light spot. As mentioned above, if there are two or more light spots in a candidate light streak, the determination of whether or not that candidate light streak is a light streak is not performed.

[0060] In this embodiment, the driving assistance function is temporarily suspended when a predetermined number (above a reference value) of frames (images) are affected by the occurrence of light streaks. The predetermined number (above a reference value) may be changed, for example, according to the speed of vehicle 1. Furthermore, the image processing device (recognition unit) according to the present invention may output a command to temporarily suspend the driving assistance function when a light streak occurs in one frame (image).

[0061] The image processing device 20 of this embodiment determines whether or not a light streak is a light streak by determining whether the candidate line of the light streak extends point-symmetrically above and below or left and right with respect to the light point. This makes it possible to determine whether or not a candidate light streak is a light streak regardless of the direction in which it extends. Therefore, it can detect not only light streaks caused by streaks left behind by wipers or dirt, but also light streaks caused by scratches on the windshield or dirt left behind by methods other than wipers.

[0062] In this embodiment, the occurrence of a light streak was determined by determining whether the candidate light streak line extends point-symmetrically. However, the image processing apparatus according to the present invention is not limited to determining whether a light streak has occurred using a candidate light streak line. The image processing apparatus according to the present invention may determine that a light streak is a light streak if the difference in angle between the portion of the candidate light streak extending from the light point to one side and the portion of the candidate light streak extending from the light point to the other side is within a predetermined range. In other words, it is possible to determine whether a candidate light streak is a light streak even if it does not satisfy the symmetry of having approximately equal distances from the light point.

[0063] For example, the image processing apparatus according to the present invention may detect a first point and a second point flanking a light point from a candidate light streak (an image extracted from a binarized image), and determine that a light streak has occurred if the difference between the angle of the line connecting the first point and the light point and the angle of the line connecting the second point and the light point is within a predetermined range. In this case, the first point may be set to the point furthest upward (or to the left) from the light point, and the second point may be set to the point furthest downward (or to the right) from the light point.

[0064] 2. Summary (1) The image processing apparatus 20 according to the above embodiment comprises a light spot detection unit 21, a light streak candidate extraction unit 22, and a light streak presence / absence determination unit 23. The light spot detection unit 21 detects light spots with a brightness value of a predetermined value or higher from an image acquired by an in-vehicle camera 10 installed inside the vehicle 1. The light streak candidate extraction unit 22 extracts light streak candidates extending from the light spots detected by the light spot detection unit 21. The light streak presence / absence determination unit 23 determines that a light streak is present if the difference in angle between a portion of the light streak candidate extending from one light spot and a portion of the light streak candidate extending from the other light spot is within a predetermined range. This allows for the determination of whether a candidate light streak is a light streak, regardless of the direction in which it extends. As a result, it can detect not only light streaks caused by streaks left behind by the wipers, but also light streaks caused by scratches on the windshield or streaks left behind by means other than the wipers. Furthermore, it eliminates the need to set the direction in which the light streaks extend for each vehicle with different wiper wiping directions, making it applicable to a variety of vehicles.

[0065] (2) The light streak presence / absence determination unit 23 according to the above embodiment determines that a candidate light streak is a light streak if it has a point-symmetric shape with respect to the light point. This improves the accuracy of light vein detection.

[0066] (3) The optical vein candidate extraction unit 22 according to the above embodiment performs a binarization process that binarizes the region close to the brightness value of the optical spot and the other region, and a contour tracing process that searches for pixels whose pixel values ​​switch in the image obtained by the binarization process and traces the contour of the region close to the brightness value of the optical spot. The optical vein candidate extraction unit 22 also performs an extraction process that creates an extracted image of the optical vein candidate from the contour tracing results, and an angle voting process that determines the angle θ of the optical vein candidate in the range of 0 ≤ θ < 180. This allows the light streak presence / absence determination unit 23 to easily determine whether or not a candidate light streak has a point-symmetric shape around the light point.

[0067] (4) The light streak presence / absence determination unit 23 according to the above embodiment determines whether the candidate light streak line obtained by the angle voting process has a point-symmetric shape centered on the light point. This allows for easy determination of whether or not a line is point-symmetrical by calculating the distance from both ends of a candidate light streak line to the light point.

[0068] (5) Furthermore, if there are two or more light points in the light streak candidate according to the above embodiment, angle voting processing is not performed, and it is not determined whether or not the light streak candidate is a light streak. This prevents misdetection of the angle of candidate light rays.

[0069] (6) Furthermore, the optical vein presence / absence determination unit 23 according to the above embodiment learns the target angle of the optical vein and narrows the range of angle θ of the optical vein candidate. This reduces the processing load on the image processing device.

[0070] (7) The light streak presence / absence determination unit 23 according to the above embodiment determines whether a light streak candidate is a light streak by considering at least one of the information regarding the light streak candidate and the information regarding the vehicle 1. This helps to suppress misidentification of whether or not something is a photon beam.

[0071] (8) Furthermore, the optical streak presence / absence determination unit 23 according to the above embodiment determines that an optical streak candidate is not an optical streak if its thickness is greater than or equal to a predetermined specific value. This prevents false detection of objects that are not light beams as light beams.

[0072] (9) Furthermore, the light streak presence / absence determination unit 23 according to the above embodiment detects a first point and a second point located on either side of the light point from the light streak candidate, and determines that it is a light streak if the difference between the angle of the line connecting the first point and the light point and the angle of the line connecting the second point and the light point is within a predetermined range. This makes it possible to determine whether a candidate light streak is actually a light streak, even without obtaining the candidate light streak line itself.

[0073] (10) In the image processing method according to the above embodiment, the light spot detection unit 21 detects light spots with a brightness value of a predetermined value or higher from an image acquired by an in-vehicle camera 10 installed inside the vehicle 1. The light streak candidate extraction unit 22 extracts light streak candidates that extend from the light spots detected by the light spot detection unit 21. The light streak presence / absence determination unit 23 then determines that it is a light streak if the difference in angle between the portion of the light streak candidate extending from one light spot and the portion of the light streak candidate extending from the other light spot is within a predetermined range. This allows for the determination of whether a candidate light streak is a light streak, regardless of the direction in which it extends. As a result, it can detect not only light streaks caused by streaks left behind by the wipers, but also light streaks caused by scratches on the windshield or streaks left behind by means other than the wipers. Furthermore, it eliminates the need to set the direction in which the light streaks extend for each vehicle with different wiper wiping directions, making it applicable to a variety of vehicles.

[0074] The present invention is not limited to the embodiments described above and shown in the drawings, and various modifications can be made without departing from the spirit of the invention as described in the claims.

[0075] Furthermore, the embodiments described above are explained in detail for the purpose of clearly illustrating the present invention, and are not necessarily limited to those comprising all the described configurations. It is also possible to replace parts of the configuration of one embodiment with those of another embodiment, and to add configurations from other embodiments to the configuration of one embodiment. Additionally, it is possible to add, delete, or replace parts of the configuration of each embodiment with those of other embodiments.

[0076] In this specification, although terms such as "parallel" and "orthogonal" are used, these do not mean only strictly "parallel" and "orthogonal," but may also refer to states that are "approximately parallel" or "approximately orthogonal," which include "parallel" and "orthogonal" and are within a range in which they can perform their functions. [Explanation of Symbols]

[0077] 1...Vehicle, 10...In-vehicle camera, 11...Left camera, 12...Right camera, 20...Image processing device, 21...Light point detection unit, 22...Light streak candidate extraction unit, 23...Light streak presence / absence determination unit, 24...Recognition unit, 30...Vehicle control device, 100...Windshield, 101...Wiper

Claims

1. A light spot detection unit detects light spots with a brightness value above a predetermined value from images acquired by a camera installed inside the vehicle, A light streak candidate extraction unit extracts light streak candidates extending from the light points detected by the light point detection unit, The system includes a light streak presence / absence determination unit that determines if a light streak is present if the difference in angle between a portion of the light streak candidate extending from one light point to the other light point is within a predetermined range. Image processing device.

2. The light streak presence / absence determination unit determines that the light streak candidate is a light streak if it has a point-symmetric shape centered on the light point. The image processing apparatus according to claim 1.

3. The aforementioned light muscle candidate extraction unit is A binarization process that binarizes the region close to the brightness value of the aforementioned light point and the other regions, A contour tracking process is performed to search for pixels whose pixel values ​​switch in the image obtained by the binarization process and to perform contour tracking of the region close to the brightness value of the light point. The process involves creating an extracted image of a candidate optical vein from the results of the contour tracing, An angle voting process is performed to determine the angle θ of the candidate light streak within the range of 0 ≤ θ < 180. The image processing apparatus according to claim 2.

4. The light streak presence / absence determination unit determines whether the candidate light streak line obtained by the angle voting process has a point-symmetric shape centered on the light point. The image processing apparatus according to claim 3.

5. If there are two or more of the aforementioned light points in the candidate light streak, the angle voting process is not performed, and it is not determined whether or not the candidate light streak is a light streak. The image processing apparatus according to claim 3.

6. The light streak presence / absence determination unit learns the target angle of the light streak and narrows the range of angle θ of the candidate light streak. The image processing apparatus according to claim 3.

7. The light streak presence / absence determination unit determines whether the light streak candidate is a light streak by considering at least one of the information regarding the light streak candidate and the information regarding the vehicle. The image processing apparatus according to claim 1.

8. The light streak presence / absence determination unit determines that a candidate light streak is not a light streak if its thickness is greater than or equal to a predetermined specific value. The image processing apparatus according to claim 1.

9. The light streak presence / absence determination unit detects a first point and a second point located on either side of the light point from the light streak candidate, and determines that a light streak exists if the difference between the angle of the line connecting the first point and the light point and the angle of the line connecting the second point and the light point is within a predetermined range. The image processing apparatus according to claim 1.

10. The light spot detection unit detects light spots with a brightness value above a predetermined value from images acquired by a camera mounted inside the vehicle. The light streak candidate extraction unit extracts light streak candidates extending from the light points detected by the light point detection unit. The light streak presence / absence determination unit determines that a light streak exists if the difference in angle between the portion of the light streak candidate extending from the light point to one side and the portion of the light streak candidate extending from the light point to the other side is within a predetermined range. Image processing methods.

Images