Sensing system
The sensing system uses image processing to detect water droplets on the cover glass by analyzing contrast changes in consecutive frames, addressing accuracy reduction and cost issues associated with traditional raindrop sensors, ensuring effective droplet removal and maintaining system performance.
Patent Information
- Application Number
- JP2023559928
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-11-12
- Filing Date
- 2022-11-11
- Publication Date
- 2026-11-19
- Estimated Expiration
- 2042-11-11
AI Technical Summary
Water droplets on the cover glass of image sensors in automotive sensing systems reduce detection accuracy, necessitating additional raindrop sensors that increase cost and cannot be mounted on the optical axis, and even without such sensors, accuracy degradation is predicted, posing a functional safety concern.
A sensing system that includes an image sensor and an arithmetic processing device to detect water droplets by analyzing changes in contrast in consecutive frames using image processing, determining the presence or absence of water droplets based on contrast analysis in specific determination regions.
Accurately detects water droplets on the cover glass, enabling effective activation of water droplet removal mechanisms and maintaining detection accuracy, thereby enhancing functional safety and reducing system complexity and cost.
Smart Images

Figure 0007954158000001 
Figure 0007954158000002 
Figure 0007954158000003
Abstract
Description
[Technical Field]
[0001] This disclosure relates to a sensing system for automobiles. [Background technology]
[0002] Object recognition systems are used to sense the position and type of objects around a vehicle for autonomous driving and automatic control of headlamp beam pattern. These systems include sensors and processing units that analyze the sensor outputs. The sensors are selected from options such as cameras, LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), millimeter-wave radar, and ultrasonic sonar, taking into account the application, required accuracy, and cost.
[0003] Optical sensors (hereinafter simply referred to as sensors) can be broadly divided into passive sensors and active sensors. Passive sensors detect light emitted by an object or light reflected by an object from ambient light, and the sensor itself does not emit light. Active sensors, on the other hand, illuminate an object with illumination light and detect the reflected light. Active sensors mainly consist of a light source (illuminator) that illuminates the object with light and an optical sensor that detects the reflected light from the object. Active sensors have the advantage of being more resistant to disturbances than passive sensors by matching the wavelength of the illumination light with the sensitivity wavelength range of the sensor. [Overview of the Initiative] [Problems that the invention aims to solve]
[0004] When water droplets adhere to the cover glass surface of an image sensor, detection accuracy decreases. Therefore, sensing systems that need to maintain detection accuracy even in bad weather require measures such as adding a water droplet removal mechanism, like a wiper. In this case, in order to properly trigger the water droplet removal mechanism to activate the raindrop removal operation, it is necessary to detect the presence of water droplets.
[0005] As a technique for detecting water droplets, a raindrop sensor is known, but adding a raindrop sensor to a sensing system causes an increase in cost. In addition, the raindrop sensor cannot be mounted on the optical axis of the image sensor because it interferes with sensing.
[0006] Even when no water droplet removal mechanism is provided, if water droplets can be detected, a decrease in the accuracy of the sensing system can be predicted, which is preferable from the viewpoint of functional safety.
[0007] The present disclosure has been made in view of the above problems, and one exemplary object of an aspect thereof is to provide a sensing system capable of detecting raindrops by image processing. [Means for Solving the Problems]
[0008] A sensing system according to an aspect of the present disclosure includes an image sensor, and an arithmetic processing device that processes an image generated by the image sensor. The arithmetic processing device determines the presence or absence of water droplets based on a change in the same determination region in two consecutive frames of the image.
[0009] Note that any combination of the above constituent elements, and modifications in which constituent elements and expressions are mutually replaced among methods, apparatuses, systems, and the like are also effective as aspects of the present invention or the present disclosure. Furthermore, the description in this section (Means for Solving the Problems) does not describe all essential features of the present invention, and therefore a sub-combination of these described features may also constitute the present invention. [Effect of the Invention]
[0010] According to an aspect of the present disclosure, water droplets can be detected. [Brief Description of the Drawings]
[0011] [Figure 1] 1 is a block diagram of a sensing system according to an embodiment. [Figure 2]This diagram illustrates the effect of water droplets on image data. [Figure 3] Figures 3(a) and 3(b) illustrate the water droplet detection using the sensing system shown in Figure 1. [Figure 4] Figures 4(a) and 4(b) show examples of judgment region A. [Figure 5] This diagram illustrates another example of judgment region A. [Figure 6] This is a block diagram of the sensing system. [Figure 7] This diagram illustrates the operation of a ToF camera. [Figure 8] Figures 8(a) and 8(b) illustrate images obtained by a ToF camera. [Figure 9] This diagram illustrates the detection of water droplets based on a range image. [Figure 10] This figure shows a sensing system equipped with a water droplet removal device. [Figure 11] This diagram shows a vehicle light fixture with a built-in sensing system. [Figure 12] This is a block diagram showing a vehicle light fixture equipped with a sensing system. [Modes for carrying out the invention]
[0012] (Summary of the embodiment) This section outlines some exemplary embodiments of the present disclosure. This outline is intended to provide a basic understanding of the embodiments and to simplify some concepts of one or more embodiments, serving as a prelude to the more detailed descriptions that follow. It does not limit the scope of the invention or disclosure. Furthermore, this outline is not a comprehensive overview of all possible embodiments and does not limit the essential components of the embodiments. For convenience, "one embodiment" may refer to one or more embodiments (examples or variations) disclosed herein.
[0013] A sensing system according to one embodiment includes an image sensor and a processing unit that processes images generated by the image sensor. The processing unit determines the presence or absence of water droplets based on changes in the same determination area in two consecutive frames of the image.
[0014] The inventors have recognized that when water droplets adhere to the cover glass on the optical axis (field of view) of an image sensor, the scattering caused by the water droplets reduces the contrast of the image. When comparing two temporally adjacent frames, it is rare for the position of the same object in the image to differ significantly, and it is rare for the brightness of the object to differ significantly. Therefore, when comparing the same judgment area in two frames, the contrast is likely to be substantially the same. In other words, if a significant decrease in contrast is detected within the same judgment area, it can be presumed to be due to the adhesion of water droplets. Thus, with the above configuration, the adhesion of water droplets can be determined by image processing.
[0015] Whether or not "contrast has decreased" can be determined by generating a contrast indicator value for the judgment area of each frame and making a determination based on the relative or absolute relationship between the two indicator values.
[0016] In one embodiment, the determination area may be defined at a predetermined position. This predetermined position may be defined as an area where the same object is always visible, or where nothing is visible. Such areas are suitable as determination areas because there are few factors that change the contrast other than the presence of water droplets. For example, the upper part of the image corresponds to empty space (sky), and the lower part of the image corresponds to the road surface. By defining the determination area as an area where the same subject is always visible, it becomes easier to detect contrast due to the presence of water droplets.
[0017] In one embodiment, the processing unit may divide each of the two frames into a plurality of determination regions and determine whether water droplets are present in each determination region.
[0018] In one embodiment, the processing unit may detect objects in the image and define a determination area that includes the detected objects. This allows water droplets to be detected in the area where a person or vehicle is visible. Furthermore, since the contrast change due to the presence or absence of water droplets is greater in the areas where people or vehicles are visible, this is expected to improve the accuracy of water droplet detection.
[0019] In one embodiment, the processing unit may generate two histograms for the determination regions of two frames. By generating histograms, changes in contrast can be detected.
[0020] In one embodiment, the processing unit may determine that a water droplet has adhered when the frequency of a predetermined low-luminance interval in the histogram of a given frame decreases compared to the frequency of the same interval in the histogram of the previous frame. When a water droplet adheres, the scattered light from the water droplet causes the pixel values in areas where no object exists to increase. Therefore, a water droplet can be detected based on the frequency of the low-luminance interval.
[0021] In one embodiment, the processing unit may determine that a water droplet has adhered when the median of a predetermined interval in the histogram of a given frame increases compared to the median of a predetermined interval in the histogram of the previous frame. When a water droplet adheres, the scattered light from the droplet increases the pixel values in areas where no object exists, causing the median to shift towards the high-brightness side. Therefore, a water droplet can be detected based on the change in the median of the histogram.
[0022] In one embodiment, the processing unit may determine that a water droplet has adhered when the spatial frequency of the determination area of a given frame is lower than the spatial frequency of the determination area of the previous frame. When scattering occurs due to a water droplet, the outline of the object's image becomes blurred and the contrast decreases. This blurring of the outline manifests as a decrease in spatial frequency. Therefore, by monitoring the spatial frequency, the adhesion of a water droplet can be detected.
[0023] In one embodiment, the system may further include an illumination device that irradiates the field of view with illumination light. The sensing system may also be a Time of Flight (ToF) camera capable of dividing the field of view into multiple ranges in the depth direction and generating multiple range images corresponding to multiple ranges by changing the time difference between light emission and image acquisition for each range.
[0024] In one embodiment, the processing unit may compare the determination area of each of the multiple range images in a given frame with the determination area of each of the multiple range images in a subsequent frame, and if a decrease in contrast is detected in two or more range images, it may determine that water droplets have adhered to the image. The effect of water droplets appears in multiple range images corresponding to multiple ranges. Therefore, by making the detection of a decrease in contrast in two or more range images a condition for water droplet detection, the detection accuracy can be improved.
[0025] Preferred embodiments will be described below with reference to the drawings. The same or equivalent components, members, and processes shown in each drawing will be denoted by the same reference numerals, and redundant descriptions will be omitted where appropriate. Furthermore, the embodiments are illustrative and not limiting, and not all features or combinations thereof described in the embodiments are necessarily essential to the disclosure.
[0026] Figure 1 is a block diagram of a sensing system 400 according to an embodiment. The sensing system 400 includes a visible light camera, an infrared camera, a stereo camera, a ToF camera, a LiDAR, etc. In this embodiment, the sensing system 400 is an active sensor and includes an illumination device 410, an image sensor 420, a sensing controller 430, and a processing unit 440.
[0027] The illumination device 410 includes a semiconductor light-emitting element such as a laser diode or a light-emitting diode (LED), and irradiates the field of view with illumination light L1. The wavelength of the illumination light L1 is not particularly limited and may be infrared light, visible light, or white light.
[0028] The image sensor 420 is sensitive to the same wavelength as the illumination light L1. The image sensor 420 receives reflected light L2, which is the light reflected from the illumination light L1 by an object OBJ within the sensing range (field of view) of the sensing system 400, and generates image data IMG.
[0029] The sensing controller 430 provides integrated control of the sensing system 400. Specifically, it synchronizes the emission of light from the lighting device 410 with the sensing performed by the image sensor 420.
[0030] The processing unit 440 processes the image data IMG generated by the image sensor 420.
[0031] A cover glass 422 is provided on the optical axis of the image sensor 420. The cover glass 422 is a component that is directly exposed to the external environment and may be a part of the image sensor 420 or a separate component independent of the image sensor 420.
[0032] The arithmetic processing unit 440 has the function of detecting water droplets WD attached to the cover glass 422 by processing image data IMG.
[0033] The processing unit 440 processes two temporally consecutive frames F of the image data IMG. i ,F i+1 If a decrease in contrast is detected in the same area (called the judgment area), it is determined that a water droplet (WD) is present.
[0034] The above is the configuration of the sensing system 400. The operation thereof will be described next. FIG. 2 is a diagram for explaining the influence of water droplets on image data. FIG. 2 shows two pieces of image data obtained by photographing the same scene. The left side is the image data before water droplets adhere, and the right side is the image data after water droplets adhere. When water droplets adhere, reflected light from an object or light emitted by the object is scattered by the water droplets, so that the brightness of a portion where no object exists increases. That is, the water droplets lower the contrast of the image compared to the case where no water droplets are present.
[0035] In addition, when water droplets adhere, the outline of the object appears blurred. That is, when attention is paid to a narrow region including the object, the water droplets lower the contrast of the image compared to the case where no water droplets are present.
[0036] FIGS. 3(a) and 3(b) are diagrams for explaining water droplet detection by the sensing system 400 of FIG. 1. FIG. 3(a) shows two temporally consecutive frames F i , F i+1 is shown. Each frame F i , F i+1 shows determination regions A having the same size and the same position i , A i+1 is shown. In the previous frame F i , no water droplets adhere to the determination region A i , and it is assumed that water droplets WD adhere to the determination region A i+1 of the subsequent frame F i+1 . However, since the water droplets WD are transparent, the water droplets WD do not appear in the image itself.
[0037] FIG. 3(b) shows two frames F i , F i+1 and the histogram of the determination regions A i , A i+1 is shown respectively.
[0038] The arithmetic processing device 440 processes the two frames F i , F i+1 of the determination regions A i , A i+1Regarding this, two histograms HIST i HIST i+1 This generates a histogram. When light is scattered by water droplets, the amount of light incident on pixels that were originally receiving strong light decreases, and the amount of light incident on pixels that were not originally receiving light increases. As a result, the frequency of low-tone intervals (dark areas) with small luminance values decreases, and the frequency of high-tone intervals (bright areas) with large luminance values, or intermediate-tone intervals, increases. This causes a shift to a flat or high-luminance biased frequency distribution, resulting in a decrease in contrast.
[0039] The processing unit 440 determines the presence or absence of water droplets (WD) by detecting changes in the histogram, i.e., a decrease in contrast.
[0040] For example, the arithmetic processing unit 440 may monitor the histogram of the low-gradation section DARK and detect the adhesion of water droplets based on its changes.
[0041] For example, water droplet adhesion can be detected by using a monitoring interval consisting of multiple adjacent bins within a low-gradation interval (DARK), such as three adjacent bins, and comparing the sum of the frequencies within the monitoring interval. For example, a histogram (HIST). i+1 The sum of the frequencies in the monitoring interval and the histogram (HIST) i It may also be determined that a water droplet has adhered when the difference in the sum of the frequencies in the monitoring interval exceeds a threshold.
[0042] Alternatively, the arithmetic processing unit 440 may detect the presence of water droplets by comparing the median values of the monitoring intervals.
[0043] Water droplets may be detected based on the high-tone interval BRIGHT rather than the low-tone interval DARK. For example, multiple adjacent bins within the high-tone interval BRIGHT can be used as monitoring intervals, and the presence of water droplets can be detected by comparing the sum of the frequencies in these monitoring intervals. For example, histogram HIST i+1 The sum of the frequencies in the monitoring interval and the histogram (HIST) iThe device may determine that a water droplet has adhered when the difference in the sum of the frequencies within the monitoring interval exceeds a threshold. Alternatively, the processing unit 440 may detect the adhesion of a water droplet by comparing the median values within the monitoring interval.
[0044] Next, we will explain judgment region A. Figures 4(a) and 4(b) show examples of judgment region A.
[0045] As shown in Figure 4(a), the judgment area A may be fixed at a predetermined position within the frame. This predetermined position may be defined as an area where the same object is always visible, or where nothing is visible. For example, a portion Ax at the top of the image corresponds to empty space (sky), and the possibility of cars or pedestrians being visible is low. Therefore, it can be said that there are few factors that change the contrast other than the presence of water droplets. Thus, it can be said to be suitable as the judgment area.
[0046] Furthermore, a portion of the lower part of the image (Ay) corresponds to the road surface, making it unlikely that cars or pedestrians will be visible. Therefore, it can be said that there are few factors other than water droplets that would alter the contrast. Consequently, it can be said that this area is suitable for judgment.
[0047] In this way, by using the area where the same subject is always visible as the detection area, it becomes easier to detect contrast caused by water droplets.
[0048] Conversely, the judgment region A may be dynamically changed. Figure 4(b) shows a dynamically changing judgment region Az. The arithmetic processing unit 440 may detect objects OBJ1 and OBJ2 included in the frame and define judgment regions Az1 and Az2 to include the detected objects OBJ1 and OBJ2. This makes it possible to detect water droplets in the area where objects of interest, such as people or vehicles, are captured. Furthermore, since the change in contrast due to the presence or absence of water droplets is greater in the areas where people or vehicles are captured, it is expected that the accuracy of water droplet detection can be improved.
[0049] Figure 5 illustrates another example of judgment region A. The processing unit 440 divides the image data IMG (frame) into multiple parts, and each of the divided regions is used as judgment region A1~A n The arithmetic processing unit 440 determines multiple decision areas A1 to A n A judgment is made for each case. For example, if a decrease in contrast is detected in more than a predetermined number of areas, it may be determined that water droplets are present.
[0050] Let's describe a specific example of the sensing system 400. One suitable application for the sensing system 400 is a ToF camera.
[0051] Figure 6 is a block diagram of the ToF camera 400B. The ToF camera 400B has multiple N (N≧2) ranges RNG1~RNG in the depth direction of its field of view. N Image acquisition is performed by dividing the image into sections.
[0052] The ToF camera 400B includes a lighting device 410, an image sensor 420, a camera controller 430, and a processing unit 440.
[0053] The lighting device 410 emits pulsed illumination light L1 in front of the vehicle in synchronization with the light emission timing signal S1 provided by the camera controller 430. The illumination light L1 is preferably infrared light, but is not limited to infrared light and may be visible light having a predetermined wavelength.
[0054] The image sensor 420 is configured to enable exposure control synchronized with the shooting timing signal S2 provided by the camera controller 430, and to generate a range image IMG. The image sensor 420 is sensitive to the same wavelength as the illumination light L1 and captures the reflected light (backlight) L2 reflected by the object OBJ.
[0055] The camera controller 430 maintains predetermined flash timings and exposure timings for each range RNG. iWhen taking a picture, a flash timing signal S1 and a shooting timing signal S2 are generated based on the flash timing and exposure timing corresponding to that range, and the picture is taken. The ToF camera 400B has multiple ranges RNG1~RNG N Multiple range images corresponding to IMG1~IMG N It is possible to generate the i-th range image IMG i This includes the corresponding range RNG i The objects contained within will be captured in the image.
[0056] Figure 7 illustrates the operation of the ToF camera 400B. Figure 7 shows the i-th range RNG. i The process of measuring is shown. The illumination device 410 emits light during the emission period τ1 between times t0 and t1, in synchronization with the light emission timing signal S1. At the top, a diagram of light rays is shown with time on the horizontal axis and distance on the vertical axis. Range RNG from ToF camera 400B. i The distance to the boundary just before it is d. MINi、 Range RNG i The distance to the far end boundary is d. MAXi Let's assume that.
[0057] Light that departed from the lighting device 410 at a certain time traveled a distance d MINi The round-trip time T is the time it takes for the reflected light to reach the image sensor 420 and return to it. MINi teeth, T MINi = 2 × d MINi / c Therefore, c is the speed of light.
[0058] Similarly, light that departed from the lighting device 410 at a certain time travels a distance d MAXi The round-trip time T is the time it takes for the reflected light to reach the image sensor 420 and return to it. MAXi teeth, T MAXi = 2 × d MAXi / c That is the case.
[0059] Range RNG iWhen you want to photograph an object OBJ contained within, the camera controller 430 sets the time t2 = t0 + T MINi Exposure begins at time t3 = t1 + T MAXi A shooting timing signal S2 is generated to end the exposure. This constitutes one exposure operation.
[0060] i-th range RNG i When taking a picture, multiple flashes and exposures are repeated, and the measurement results are accumulated in the image sensor 420.
[0061] Figures 8(a) and 8(b) illustrate images obtained by the ToF camera 400B. In the example in Figure 8(a), an object (pedestrian) OBJ1 is in range RNG1, and an object (vehicle) OBJ3 is in range RNG3. Figure 8(b) shows multiple range images IMG1 to IMG3 obtained in the situation in Figure 8(a). When range image IMG1 is captured, the image sensor is exposed only by reflected light from range RNG1, so range image IMG1 captures the object image OBJ1 of the pedestrian OBJ1.
[0062] When capturing the range image IMG2, the image sensor is exposed by reflected light from the range RNG2, and therefore no object images are captured in the range image IMG2.
[0063] Similarly, when capturing the range image IMG3, the image sensor is exposed by reflected light from range RNG3, so only the object image OBJ3 is captured in the range image IMG3. In this way, the ToF camera 400B allows for the separation and capture of objects for each range.
[0064] Return to Figure 6. The processing unit 440 processes multiple range images IMG1~IMG N Based on this, the presence of water droplets is detected.
[0065] Figure 9 illustrates the detection of water droplets based on a range image. The processing unit 440 processes multiple range images IMG1~IMG of a given frame.N Each judgment area and multiple range images from the following frame IMG1~IMG N Each judgment region is compared and referenced. Then multiple range images IMG1~IMG N If a decrease in contrast within the detection area is detected in two or more of the images, it is determined that water droplets have adhered to the image.
[0066] The effect of water droplets is not limited to a specific range image, but is observed across multiple range images (IMG1-IMG). N It is highly likely that this will occur in multiple of these images. Therefore, by making the detection of a decrease in contrast in two or more range images a condition for water droplet detection, the detection accuracy can be improved.
[0067] The detection result from the arithmetic processing unit 440 can be used as a trigger to start the operation of the water droplet removal device.
[0068] Figure 10 shows a sensing system equipped with a water droplet removal device. The sensing system 400 includes a water droplet removal device 460. The water droplet removal device 460 is a blower type and removes water droplets by blowing air onto the cover glass 422. The water droplet removal device 460 includes a blower 462 and a duct 464. Alternatively, the airflow generated by the blower 462 may be directed directly onto the cover glass 422.
[0069] When the processing unit 440 detects a water droplet through image processing, it outputs a trigger TRIG to the water droplet removal device 460. In response to this trigger TRIG, the water droplet removal device 460 blows air onto the cover glass 422. The configuration of the water droplet removal device 460 is not limited to a blower type, and other means such as a wiper may also be used.
[0070] In Figure 10, the sensing system 400 is mounted on the underside of the headlamp 600 body, but the camera of the sensing system 400 may be built into the headlamp 600. In this case, water droplets adhering to the cover glass 602 of the headlamp 600 will be the target of detection by the sensing system 400.
[0071] (Application) Figure 11 shows a vehicle lighting fixture 200 incorporating a sensing system 400. The vehicle lighting fixture 200 comprises a housing 210, an outer lens 220, high-beam and low-beam lighting units 230H / 230L, and a sensing system 400. The lighting units 230H / 230L and the sensing system 400 are housed in the housing 210.
[0072] Furthermore, some parts of the sensing system 400, such as the image sensor 420 and the processing unit 440, may be installed outside the vehicle lighting fixture 200, for example, behind the rearview mirror.
[0073] Figure 12 is a block diagram showing a vehicle lighting fixture 200 equipped with a sensing system 400. The vehicle lighting fixture 200, together with the vehicle-side ECU 304, constitutes a lighting system 310. The vehicle lighting fixture 200 includes a light source 202, a lighting circuit 204, and an optical system 206.
[0074] The processing unit 40 is configured to identify the type of object based on the image obtained by the sensing system 400.
[0075] The arithmetic processing unit 40 can be implemented as a combination of a processor (hardware) such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or microcontroller, and a software program executed by the processor (hardware). The arithmetic processing unit 40 may also be a combination of multiple processors. Alternatively, the arithmetic processing unit 40 may consist only of hardware.
[0076] Information regarding the object OBJ detected by the arithmetic processing unit 40 may be used for controlling the light distribution of the vehicle lighting fixture 200. Specifically, the lighting fixture-side ECU 208 generates an appropriate light distribution pattern based on the information regarding the type and location of the object OBJ generated by the arithmetic processing unit 40. The lighting circuit 204 and optical system 206 operate to obtain the light distribution pattern generated by the lighting fixture-side ECU 208.
[0077] Furthermore, the information regarding the object OBJ detected by the processing unit 40 may be transmitted to the vehicle-side ECU 304. The vehicle-side ECU may perform autonomous driving based on this information.
[0078] The embodiments described above are illustrative, and it will be understood by those skilled in the art that various modifications are possible in combinations of their components and processing steps. Such modifications will be described below.
[0079] (Variation 1) The sensing system 400 is not limited to a ToF camera, but may also be a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging). Alternatively, the sensing system 400 may be a single-pixel imaging device (quantum radar) that utilizes correlation calculation.
[0080] (Modification 2) In this embodiment, a decrease in contrast was detected by a change in the histogram, but the means for determining a decrease in contrast are not limited to this. The processing unit 440 may determine that a water droplet has adhered when the spatial frequency of the determination area of a given frame is lower than the spatial frequency of the determination area of the previous frame.
[0081] (Variation 3) In this embodiment, a portion of the image data is used as the determination area, but the entire image data may also be used as the determination area.
[0082] Those skilled in the art will understand that the embodiments are illustrative, and that various modifications exist for each component and combination of processing steps, and that such modifications are also included within the scope of this disclosure or the present invention. [Industrial applicability]
[0083] This disclosure relates to a sensing system for automobiles. [Explanation of Symbols]
[0084] OBJ...Object, S1...Light emission timing signal, S2...Shooting timing signal, 200...Vehicle lighting fixture, 202...Light source, 204...Lighting circuit, 206...Optical system, 310...Lighting system, 304...Vehicle-side ECU, 400...Sensing system, 410...Illumination device, 420...Image sensor, 430...Sensing controller, 440...Processing unit, 460...Water droplet removal device.
Claims
1. Image sensor and A processing unit for processing images generated by the image sensor, Equipped with, The processing unit determines the presence of water droplets based on changes in the same determination area in two consecutive frames of the image. The aforementioned processing unit generates two histograms for the determination regions of the two frames, The aforementioned processing unit is a sensing system characterized in that it determines that water droplets have adhered when the frequency of a predetermined low-luminance interval in the histogram of a given frame decreases compared to the frequency of the same interval in the histogram of the previous frame.
2. The sensing system according to claim 1, characterized in that the determination area is located at a predetermined position.
3. The sensing system according to claim 1, characterized in that the processing unit divides each of the two frames into a plurality of determination regions and determines whether water droplets are attached to each determination region.
4. The sensing system according to claim 1, characterized in that the processing unit detects an object included in the image and defines the determination region to include the detected object.
5. The sensing system according to claim 1, characterized in that the processing unit determines that a water droplet has adhered when the median of a predetermined interval in the histogram of a certain frame increases compared to the median of a predetermined interval in the histogram of the previous frame.
6. The sensing system according to any one of claims 1 to 4, characterized in that the processing unit determines that a water droplet has adhered when the spatial frequency of the determination area of a given frame is lower than the spatial frequency of the determination area of the previous frame.
7. It is further equipped with an illumination device that projects illumination light into the field of view, The sensing system is a ToF camera capable of dividing the field of view into multiple ranges in the depth direction and generating multiple range images corresponding to the multiple ranges by changing the time difference between light emission and imaging for each range. This is the sensing system according to any one of claims 1 to 4.
8. The sensing system according to claim 7, characterized in that the processing unit compares the determination area of each of the plurality of range images in a given frame with the determination area of each of the plurality of range images in a subsequent frame, and if a decrease in contrast is detected in two or more range images, it determines that water droplets have adhered to the image.
Citation Information
Patent Citations
Moving object identification system
JP2005005799A
Camera device and wiper control method
JP2007159021A
Adjusting device of imaging means and object detecting apparatus
JP2009177311A
Lens attached substance detection apparatus, lens attached substance detection method, and vehicle system
JP2014030188A
Image processing apparatus, imaging apparatus, and image processing method and program
JP2018032984A
