Vehicle control system and vehicle control method

Abstract

The present invention provides a vehicle control system and a vehicle control method equipped with an image processing device capable of detecting the generation of light rays regardless of the direction in which they extend. [Solution] The vehicle control system comprises a camera mounted inside the vehicle, an image processing device that recognizes an object from the image acquired by the camera and determines the presence or absence of light rays, and a vehicle control device. The vehicle control device controls the vehicle's driving state based on the object recognition result acquired from the image processing device, and temporarily suspends the driving assistance function when it receives a command to temporarily suspend the driving assistance function from the image processing device.

Description

Technical Field

[0004]

[0001] The present invention relates to a vehicle control system and a vehicle control method.

Background Art

[0002] In recent years, for the automatic driving of vehicles (automobiles), a technology for recognizing an image captured using a camera (in-vehicle camera) has been widely used. The camera is installed, for example, inside the windshield. Therefore, there are cases where the image processing device cannot recognize the surrounding environment of the vehicle normally due to the dirt on the windshield. For example, when there is a wiping residue of the wiper on the windshield or when there is a scratch on the windshield, when strong light hits during imaging, a streak of light (light streak) extending along the wiping residue or scratch is imaged.

[0003] Patent Document 1 describes an object recognition device and a dirt detection method using an imaging image of a stereo camera that images the outside of a vehicle through a window glass from inside the vehicle cabin of a vehicle such as an automobile. The object recognition device described in Patent Document 1 detects an edge extending in a direction intersecting the wiping direction of the wiper, and in combination with the light source detection result, determines that the front windshield is in a dirty state when the edge is continuously detected.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the object recognition device described in Patent Document 1 needs to preset a specific direction intersecting the wiping direction of the wiper. Therefore, the specific direction had to be set for each vehicle with a different wiping direction of the wiper.

[0006] Furthermore, when the windshield is wiped using a method other than the wipers, streaks of light captured due to remaining streaks or scratches on the windshield may not have edges in the direction intersecting the wiper's wiping direction. For this reason, the object recognition device described in Patent Document 1 was unable to detect light streaks extending in directions other than the wiper's wiping direction.

[0007] The objective of this invention is to provide a vehicle control system and a vehicle control method equipped with an image processing device that can detect the generation of light rays regardless of the direction in which they extend, taking into consideration the above-mentioned problems. [Means for solving the problem]

[0008] To solve the above problems and achieve the objective, a vehicle control system according to one aspect of the present invention comprises a camera mounted inside the vehicle, an image processing device that recognizes an object from an image acquired by the camera and determines the presence or absence of light rays, and a vehicle control device. The vehicle control device controls the driving state of the vehicle based on the object recognition result acquired from the image processing device, and temporarily suspends the driving assistance function when it receives a command to temporarily suspend the driving assistance function from the image processing device. [Effects of the Invention]

[0009] According to one aspect of the present invention, the generation of light rays can be detected regardless of the direction in which they extend. Furthermore, issues, configurations, and effects other than those mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram showing the configuration of a vehicle equipped with an image processing device according to one embodiment. [Figure 2] This is a diagram illustrating the operation of a vehicle's windshield wipers. [Figure 3] A and B are diagrams illustrating images of the state in which light rays are generated. [Figure 4] This flowchart shows an example of a light streak detection process according to one embodiment. [Figure 5] A and B are diagrams illustrating a binarized image according to one embodiment. [Figure 6] A and B are diagrams illustrating cropped images according to one embodiment. [Figure 7] This is an illustrative diagram illustrating the angle of a search point relative to a light point according to one embodiment. [Figure 8] This figure shows the results of the angle voting according to one embodiment. [Figure 9] The image shows a cropped image according to one embodiment, with a straight line drawn at the angle that maximizes the number of votes. [Modes for carrying out the invention]

[0011] 1. Embodiment Hereinafter, an embodiment of the image processing apparatus and image processing method will be described with reference to Figures 1 to 9. Common parts in each figure are denoted by the same reference numerals.

[0012] [Example of vehicle configuration] First, the configuration of a vehicle equipped with an image processing device according to one embodiment will be described with reference to Figure 1. Figure 1 is a schematic diagram showing the configuration of a vehicle equipped with an image processing device.

[0013] As shown in Figure 1, vehicle (automobile) 1 is equipped with an on-board camera 10, an image processing device 20, and a vehicle control device 30. The image processing device 20 recognizes objects outside the vehicle from images acquired (captured) by the on-board camera 10. The recognition results from the image processing device 20 are sent to the vehicle control device 30. The vehicle control device 30 implements driver assistance functions such as ACC (Adaptive cruise control) and AEB (Autonomous Emergency Braking).

[0014] The in-vehicle camera 10 is composed of a plurality of imaging devices arranged at a predetermined interval in the vehicle width direction inside the windshield. The plurality of imaging devices are installed in a predetermined direction around the vehicle 1, for example, facing the front of the vehicle 1. Hereinafter, the in-vehicle camera 10 will be described as a stereo camera composed of two cameras, a left camera 11 and a right camera 12. Note that the in-vehicle camera may be composed of three or more cameras.

[0015] The left camera 11 and the right camera 12 of the in-vehicle camera 10 image the front of the vehicle 1. The image captured by the in-vehicle camera 10 is supplied to the image processing device 20.

[0016] The image processing device 20 includes hardware such as a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and electronic circuits, and software that performs image recognition processing in cooperation with the hardware. That is, the CPU reads out various processing programs stored in the ROM and expands them in the RAM, and controls the image recognition processing according to the expanded programs.

[0017] The image processing device 20 performs image correction processing such as camera calibration and distortion correction on the captured image. Then, it recognizes objects such as objects and people from the image on which the image correction processing has been performed. In addition, the image processing device 20 detects light streaks from the captured image. It has a light point detection unit 21, a light streak candidate extraction unit 22, a light streak presence / absence determination unit 23, and a recognition unit 24.

[0018] The light point detection unit 21 analyzes the pixel values (luminance values) in the image captured by the in-vehicle camera 10 using a known technique. Then, the light point detection unit 21 detects the position of the light source (light point) on the image. The light point is, for example, the highest one among the points with a luminance of 200 or more, or one point is detected from a strong light source such as a headlight or a streetlight. Note that the light point is not limited to one point, and a plurality of light points may be selected (detected). The light point detection unit 21 outputs the detection result to the light streak candidate extraction unit 22.

[0019] The light streak candidate extraction unit 22 extracts straight lines that are candidates for light streaks based on the light points acquired by the light point detection unit 21. Normally, wiper residue that causes light streaks is often arc-shaped, but it can be approximated as a straight line. Therefore, the light streak candidate extraction unit 22 extracts one of the straight lines passing through the light source as a candidate light streak line. The light streak candidate extraction unit 22 outputs the extraction result to the light streak presence / absence determination unit 23.

[0020] The light streak presence / absence determination unit 23 determines the presence or absence of light streaks by determining whether they are point-symmetric with respect to the light streak candidate line obtained by the light streak candidate extraction unit 22, and by using other auxiliary information. Other auxiliary information includes the thickness of the light streak candidate line, which will be described later, and vehicle information such as the speed of vehicle 1 and whether or not the wipers are being used.

[0021] The light streak detection unit 23 outputs to the recognition unit 24 that it has detected light streaks, for example, when light streaks are present in multiple consecutive frames. When the recognition unit 24 receives notification from the light streak detection unit 23 that light streaks have been detected, it outputs to the vehicle control device 30 to stop the driving assistance function of the light streak detection unit 23.

[0022] The recognition unit 24 detects (recognizes) objects such as vehicles and pedestrians in the surrounding environment (in this case, in front) from the image captured by the in-vehicle camera 10. The recognition unit 24 outputs the object recognition result to the vehicle control device 30.

[0023] The vehicle control device 30 comprises hardware such as a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and electronic circuits, as well as software that works in cooperation with the hardware to perform vehicle control processing, including driver assistance functions. Specifically, the CPU reads various processing programs stored in the ROM, loads them into the RAM, and controls the vehicle control processing according to the loaded programs.

[0024] The vehicle control device 30 acquires the object recognition results from the recognition unit 24 as information necessary for autonomous driving, and controls the driving state of the vehicle 1. For example, the vehicle control device 30 controls braking to avoid contact with objects or people in front of the vehicle 1, driving at a speed that follows another vehicle in front of the vehicle 1, and steering to ensure that the vehicle stays within its lane.

[0025] [Wiper operation] Next, the operation of the wipers will be explained with reference to Figure 2. Figure 2 illustrates the wiper operation of vehicle 1.

[0026] As shown in Figure 2, vehicle 1 is equipped with wipers 101 and 102. The wipers 101 and 102 wipe the windshield 100 in front of the left and right cameras 11 and 12. When the wipers 101 and 102 are operating, a thin streak of unwiped area may occur, as shown by the dashed line.

[0027] Wiper streaks do not create light streaks in the images captured by the left and right cameras 11 and 12 during the daytime when the outside world is bright, or at night when there is no light source in the outside world. However, at night when there is a light source in the outside world, the light from the light source is extended by the streaks, causing light streaks to appear.

[0028] [Evolution of photon rays] Next, we will explain the image from which light streaks are generated, referring to Figure 3. Figure 3 is an image illustrating the state in which light rays are generated.

[0029] Figure 3A is a schematic diagram showing a state where there is no dirt or scratches such as wiper streaks, and no light streaks appear in nighttime images. As shown in Figure 3A, if there is no dirt or scratches such as wiper streaks, light streaks will not appear in the captured image even if there is a light source.

[0030] Figure 3B is a schematic diagram showing the appearance of light streaks in nighttime images due to streaks left behind by the wipers. As shown in Figure 3B, objects that do not have strong light sources, such as pedestrians, are captured without problems. However, light sources such as car headlights, brake lights, and streetlights cause light to be stretched, resulting in light streaks. Note that light streaks caused by scratches on the windshield or streaks left behind by sources other than the wipers extend at a different angle (direction) than light streaks caused by streaks left behind by the wipers.

[0031] [Photon Beam Detection Processing] Next, the light streak detection process performed by the image processing device 20 will be explained with reference to Figure 4. Figure 4 is a flowchart showing an example of the light vein detection process.

[0032] First, the light spot detection unit 21 of the image processing device 20 detects light spots from the image sent from the in-vehicle camera 10 (S101). In step S101, the light spot detection unit 21 analyzes the pixel values ​​(luminance values) in the image using known techniques and detects the position of the light source (light spot) on the image.

[0033] Next, the light spot detection unit 21 determines whether or not there is a light spot (S102). If it is determined in step S102 that there is no light spot (S102 is a NO determination), the light spot detection unit 21 (image processing device 20) terminates the light streak detection process.

[0034] In step S102, if it is determined that a light spot exists (if S102 is a YES determination), the light spot detection unit 21 sends the detection result to the light streak candidate extraction unit 22. The light streak candidate extraction unit 22 performs image binarization processing based on the brightness value of the detected light spot (S103).

[0035] Figure 5A shows the image before binarization. The optical vein candidate extraction unit 22 performs binarization on the image shown in Figure 5A, using the brightness values ​​of the detected optical spots as a reference. Specifically, when the brightness of an optical spot is p and the binarization range of the brightness value is ±d, the pixel value is set to 1 if the brightness value of a pixel is greater than or equal to p and less than or equal to p+d, and to 0 otherwise. This enables the image to be binarized.

[0036] Figure 5B shows the image after binarization. As shown in Figure 5B, in the image after binarization, the pixel values ​​of light sources such as brake lights and streetlights, and light streaks, become 1 (white), while the pixel values ​​of other dark areas become 0 (black).

[0037] Referring again to Figure 4, we will continue the explanation of the light streak detection process. Next, the light streak candidate extraction unit 22 performs contour tracing on the binarized image to extract the image of the light streak portion (S104). In the contour tracing process, first, pixels are searched vertically or horizontally from the position of the light point to find pixels where the pixel value changes from 1 to 0. Then, using the pixels where the pixel value changes from 1 to 0 as a reference, the contour tracing process is performed again.

[0038] Contour tracking can be performed using a known algorithm. The contour tracking search is performed while simultaneously updating the minimum and maximum values ​​of the pixel in the vertical (Y-axis) and horizontal (X-axis) directions. Once contour tracking is complete, the optical vein candidate extraction unit 22 extracts a binarized image from the minimum and maximum values ​​in the vertical and horizontal directions.

[0039] Figure 6A shows an image after contour tracking processing. As shown in Figure 6A, the contours of the light rays are detected in the image after contour tracking processing. Figure 6B is an image of a candidate light ray extracted from the binarized image.

[0040] Referring again to Figure 4, we will continue the explanation of the light streak detection process. Next, the light streak candidate extraction unit 22 calculates the angle θ that the light streak makes with the light streak for all points with a pixel value of 1, excluding the light streak, in the extracted binarized image.

[0041] Figure 7 is an illustrative diagram illustrating the angle between the light point and the search point. Specifically, Figure 7 shows the angle θ between the light point and the search point. When the coordinates of the light point on the image are (xi, yi) and the point for calculating the angle is (xp, yp), the angle θ [deg] is calculated by equation (1). However, θ is assumed to be 0 ≤ θ < 180. [Mathematics 1] TIFF2026110633000002.tif19170

[0042] For example, if a single extracted binarized image (a candidate for a light streak) contains the brake lights of a preceding vehicle on both sides, then there are two light sources for that single candidate light streak. In this way, if there are other light points besides the target light point for a single candidate light streak, the angle of the search point cannot be calculated correctly. Therefore, if there are two or more light points in the extracted binarized image, the angle of the search point is not calculated. In other words, the extracted binarized image is excluded from the candidates for light streaks, and the determination of whether or not it is a light streak is not made.

[0043] Figure 8 shows the results of the angle voting. As shown in Figure 8, by performing the angle calculation using equation (1) described above for all points with a pixel value of 1 (search points) excluding light points, voting results can be obtained with angle on the horizontal axis and the number of votes on the vertical axis.

[0044] The angle with the highest number of votes in the voting results is the angle of the straight line that will become the candidate for a light streak (the candidate light streak line). Whether or not this candidate light streak line is a light streak is determined by the process described later. Note that although light streaks caused by wiper streaks are arc-shaped, they can be approximated as straight lines in the image. Therefore, the candidate light streak is approximated as a straight line centered on the light point to determine whether or not it is a light streak.

[0045] In the example shown in Figure 8, the angle with the maximum number of votes is 160 degrees. Therefore, in the subsequent processing, we determine whether a 160-degree straight line passing through the light point (a candidate line for a light streak) is a light streak or not. Figure 9 shows the extracted image with a 160-degree straight line drawn on it.

[0046] In this embodiment, the angle range θ was set to 0 ≤ θ < 180 in the angle voting process. However, since the wiper wiping angle generally does not change, the range of angles to be voted on may be narrowed by learning using information from multiple frames (images). For example, if the angle is consistently determined to be 100° in multiple frames (e.g., multiple images acquired over 10 seconds), the angle calculation can be performed in the range of 80 ≤ θ < 130 instead of 0 ≤ θ < 180. This can reduce the processing load.

[0047] Alternatively, the system can be trained using information from multiple frames (images) to set the target angles for voting, and votes will be excluded from angles other than those set. This increases the proportion of the maximum number of votes (angle votes) in the total number of votes. Therefore, the accuracy of the angle of the candidate light ray line can be improved using other auxiliary information (information from multiple frames (images)).

[0048] Referring again to Figure 4, we will continue the explanation of the light streak detection process. Next, the light streak presence / absence determination unit 23 performs a light streak presence / absence determination process (S106). In step S106, the light streak presence / absence determination unit 23 determines whether or not a light streak has occurred by determining whether or not the candidate light streak line is point-symmetric with respect to the light point. The light streak presence / absence determination unit 23 then outputs the determination result to the recognition unit 24.

[0049] It is determined whether the difference between the distance from the light point to the point furthest upward (or leftward) and the distance from the light point to the point furthest downward (or rightward) among the points with a pixel value of 1 on the candidate light ray line is less than or equal to a predetermined threshold (for example, 10 pixels).

[0050] When the difference in distance from the light point to each point furthest away is less than or equal to a threshold, the light streak presence / absence determination unit 23 determines that the candidate light streak line is point-symmetric with respect to the light point and determines that a light streak is present. On the other hand, when the difference in distance from the light point to each point furthest away is not less than or equal to a threshold, the light streak presence / absence determination unit 23 determines that the candidate light streak line is not point-symmetric with respect to the light point and determines that a light streak is not present.

[0051] Furthermore, in the process of determining the presence or absence of light rays, in addition to determining whether the candidate light ray line is point-symmetric with respect to the light point, the determination may also be made by combining information about the candidate light ray, such as the ratio of pixels with a pixel value of 1 to the entire cropped image, the length of the cropped image, the ratio of angle votes to the total number of votes for the candidate light ray line, and the thickness of the candidate light ray line, with at least one piece of information about vehicle 1, such as the speed of vehicle 1 and whether or not the wipers are being used. This can suppress misjudgments of whether or not a light ray is present.

[0052] For example, if the ratio of pixels with pixel value 1 to the entire cropped image is smaller than a predetermined value, it is possible that the light is not extending in a streaky pattern. Therefore, the light streaking presence / absence determination unit 23 determines that no light streaks are present if the ratio of pixels with pixel value 1 to the entire cropped image is smaller than a predetermined value.

[0053] Furthermore, if the cropped image is smaller than a predetermined length (for example, a 40-pixel square), the symmetry (point symmetry) may be misjudged. Therefore, the light streak presence / absence determination unit 23 determines that no light streaks are present if the cropped image is smaller than a predetermined size.

[0054] Furthermore, if the thickness of a candidate light streak line exceeds a predetermined specific value, it is often not a light streak but rather an object such as a guide rail reflecting light. Therefore, the light streak presence / absence determination unit 23 determines that no light streaks are present if the thickness of the candidate light streak line or the candidate light streak (cropped image) exceeds a predetermined value.

[0055] Next, the recognition unit 24 determines whether the light streak presence / absence determination unit 23 has determined that there is a light streak (S107). In step S107, if the light streak presence / absence determination unit 23 determines that there is a light streak (if S107 is a YES determination), the recognition unit 24 adds "1" to the pause (HALT) occurrence counter (S108).

[0056] On the other hand, in step S107, if the light streak presence / absence determination unit 23 determines that there are no light streaks (if S107 is a NO determination), the recognition unit 24 subtracts "1" from the pause occurrence counter (S109).

[0057] After processing in step S108 or step S109, the recognition unit 24 determines whether the counter value of the pause occurrence counter is equal to or greater than a reference value (S110). If in step S110 it is determined that the counter value of the pause occurrence counter is not equal to or greater than a reference value (if S110 is a NO determination), the recognition unit 24 (image processing device 20) terminates the light streak detection process.

[0058] On the other hand, in step S110, if the recognition unit 24 determines that the counter value of the pause occurrence counter is equal to or greater than the reference value (if S110 is determined to be YES), the recognition unit 24 outputs a driver assistance function temporary suspension (HALT) command to the vehicle control device 30 (S111). When the vehicle control device 30 receives the driver assistance function temporary suspension command, it temporarily suspends the driver assistance function. After the processing in step S111, the image processing device 20 terminates the light streak detection process.

[0059] Furthermore, if multiple light spots are detected in step S101, steps 103 to S106 are performed for each detected light spot. As mentioned above, if there are two or more light spots in a candidate light streak, the determination of whether or not that candidate light streak is a light streak is not performed.

[0060] In this embodiment, the driving assistance function is temporarily suspended when a predetermined number (above a reference value) of frames (images) are affected by the occurrence of light streaks. The predetermined number (above a reference value) may be changed, for example, according to the speed of vehicle 1. Furthermore, the image processing device (recognition unit) according to the present invention may output a command to temporarily suspend the driving assistance function when a light streak occurs in one frame (image).

[0061] The image processing device 20 of this embodiment determines whether or not a light streak is a light streak by determining whether the candidate line of the light streak extends point-symmetrically above and below or left and right with respect to the light point. This makes it possible to determine whether or not a candidate light streak is a light streak regardless of the direction in which it extends. Therefore, it can detect not only light streaks caused by streaks left behind by wipers or dirt, but also light streaks caused by scratches on the windshield or dirt left behind by methods other than wipers.

[0062] In this embodiment, the occurrence of a light streak was determined by determining whether the candidate light streak line extends point-symmetrically. However, the image processing apparatus according to the present invention is not limited to determining whether a light streak has occurred using a candidate light streak line. The image processing apparatus according to the present invention may determine that a light streak is a light streak if the difference in angle between the portion of the candidate light streak extending from the light point to one side and the portion of the candidate light streak extending from the light point to the other side is within a predetermined range. In other words, it is possible to determine whether a candidate light streak is a light streak even if it does not satisfy the symmetry of having approximately equal distances from the light point.

[0063] For example, the image processing apparatus according to the present invention may detect a first point and a second point flanking a light point from a candidate light streak (an image extracted from a binarized image), and determine that a light streak has occurred if the difference between the angle of the line connecting the first point and the light point and the angle of the line connecting the second point and the light point is within a predetermined range. In this case, the first point may be set to the point furthest upward (or to the left) from the light point, and the second point may be set to the point furthest downward (or to the right) from the light point.

[0064] 2. Summary (1) The image processing apparatus 20 according to the above embodiment comprises a light spot detection unit 21, a light streak candidate extraction unit 22, and a light streak presence / absence determination unit 23. The light spot detection unit 21 detects light spots with a brightness value of a predetermined value or higher from an image acquired by an in-vehicle camera 10 installed inside the vehicle 1. The light streak candidate extraction unit 22 extracts light streak candidates extending from the light spots detected by the light spot detection unit 21. The light streak presence / absence determination unit 23 determines that a light streak is present if the difference in angle between a portion of the light streak candidate extending from one light spot and a portion of the light streak candidate extending from the other light spot is within a predetermined range. This allows for the determination of whether a candidate light streak is a light streak, regardless of the direction in which it extends. As a result, it can detect not only light streaks caused by streaks left behind by the wipers, but also light streaks caused by scratches on the windshield or streaks left behind by means other than the wipers. Furthermore, it eliminates the need to set the direction in which the light streaks extend for each vehicle with different wiper wiping directions, making it applicable to a variety of vehicles.

[0065] (2) The light streak presence / absence determination unit 23 according to the above embodiment determines that a candidate light streak is a light streak if it has a point-symmetric shape with respect to the light point. This improves the accuracy of light vein detection.

[0066] (3) The optical vein candidate extraction unit 22 according to the above embodiment performs a binarization process that binarizes the region close to the brightness value of the optical spot and the other region, and a contour tracing process that searches for pixels whose pixel values ​​switch in the image obtained by the binarization process and traces the contour of the region close to the brightness value of the optical spot. The optical vein candidate extraction unit 22 also performs an extraction process that creates an extracted image of the optical vein candidate from the contour tracing results, and an angle voting process that determines the angle θ of the optical vein candidate in the range of 0 ≤ θ < 180. This allows the light streak presence / absence determination unit 23 to easily determine whether or not a candidate light streak has a point-symmetric shape around the light point.

[0067] (4) The light streak presence / absence determination unit 23 according to the above embodiment determines whether the candidate light streak line obtained by the angle voting process has a point-symmetric shape centered on the light point. This allows for easy determination of whether or not a line is point-symmetrical by calculating the distance from both ends of a candidate light streak line to the light point.

[0068] (5) Furthermore, if there are two or more light points in the light streak candidate according to the above embodiment, angle voting processing is not performed, and it is not determined whether or not the light streak candidate is a light streak. This prevents misdetection of the angle of candidate light rays.

[0069] (6) Furthermore, the optical streak presence / absence determination unit 23 according to the above embodiment learns the target angle of the optical streak and narrows the range of angle θ of the candidate optical streak. This reduces the processing load on the image processing device.

[0070] (7) The light streak presence / absence determination unit 23 according to the above embodiment determines whether a light streak candidate is a light streak by considering at least one of the information regarding the light streak candidate and the information regarding the vehicle 1. This helps to suppress misidentification of whether or not something is a light ray.

[0071] (8) Furthermore, the optical streak presence / absence determination unit 23 according to the above embodiment determines that an optical streak candidate is not an optical streak if its thickness is greater than or equal to a predetermined specific value. This prevents false detection of objects that are not light beams as light beams.

[0072] (9) Furthermore, the light streak presence / absence determination unit 23 according to the above embodiment detects a first point and a second point located on either side of the light point from the light streak candidate, and determines that it is a light streak if the difference between the angle of the line connecting the first point and the light point and the angle of the line connecting the second point and the light point is within a predetermined range. This makes it possible to determine whether a candidate light streak is actually a light streak without obtaining the candidate light streak line itself.

[0073] (10) In the image processing method according to the above embodiment, the light spot detection unit 21 detects light spots with a brightness value of a predetermined value or higher from an image acquired by an in-vehicle camera 10 installed inside the vehicle 1. The light streak candidate extraction unit 22 extracts light streak candidates that extend from the light spots detected by the light spot detection unit 21. The light streak presence / absence determination unit 23 then determines that it is a light streak if the difference in angle between the portion of the light streak candidate extending from one light spot and the portion of the light streak candidate extending from the other light spot is within a predetermined range. This allows for the determination of whether a candidate light streak is a light streak, regardless of the direction in which it extends. As a result, it can detect not only light streaks caused by streaks left behind by the wipers, but also light streaks caused by scratches on the windshield or streaks left behind by means other than the wipers. Furthermore, it eliminates the need to set the direction in which the light streaks extend for each vehicle with different wiper wiping directions, making it applicable to a variety of vehicles.

[0074] The present invention is not limited to the embodiments described above and shown in the drawings, and various modifications can be made without departing from the spirit of the invention as described in the claims.

[0075] Furthermore, the embodiments described above are explained in detail for the purpose of clearly illustrating the present invention, and are not necessarily limited to those comprising all the described configurations. It is also possible to replace parts of the configuration of one embodiment with those of another embodiment, and to add configurations from other embodiments to the configuration of one embodiment. Additionally, it is possible to add, delete, or replace parts of the configuration of each embodiment with those of other embodiments.

[0076] In this specification, although terms such as "parallel" and "orthogonal" are used, these do not mean only strictly "parallel" and "orthogonal," but may also refer to states that are "approximately parallel" or "approximately orthogonal," which include "parallel" and "orthogonal" and are within a range in which they can perform their functions. [Explanation of Symbols]

[0077] 1...Vehicle, 10...In-vehicle camera, 11...Left camera, 12...Right camera, 20...Image processing device, 21...Light point detection unit, 22...Light streak candidate extraction unit, 23...Light streak presence / absence determination unit, 24...Recognition unit, 30...Vehicle control device, 100...Windshield, 101...Wiper

Claims

1. A camera installed inside the vehicle, An image processing device that recognizes an object from an image acquired by the aforementioned camera and determines the presence or absence of light rays, A vehicle control device that controls the vehicle's driving state based on the object recognition result obtained from the image processing device, and temporarily suspends the driving assistance function when it receives a command to temporarily suspend the driving assistance function from the image processing device, A vehicle control system equipped with the following features.

2. The vehicle control device controls at least one of the following as a driving assistance function: braking to avoid contact with an object or person in front of the vehicle, driving at a speed that follows another vehicle in front of the vehicle, and steering to maintain the vehicle's lane on the road. The vehicle control system according to claim 1.

3. The image processing device adds a value to the pause occurrence counter when it determines that a light streak has occurred, subtracts a value from the pause occurrence counter when it determines that a light streak has not occurred, and outputs the driver assistance function pause command when the counter value of the pause occurrence counter is equal to or greater than a reference value. The vehicle control system according to claim 1 or claim 2.

4. The aforementioned reference value is changed according to the speed of the vehicle. The vehicle control system according to claim 3.

5. The system recognizes objects from images acquired by cameras installed inside the vehicle and determines the presence or absence of light rays. Based on the acquired object recognition results, the vehicle's driving state is controlled, and if a driver assistance function temporary suspension command is issued, the driver assistance function is temporarily suspended. Vehicle control method.

Images