Window fog detector

Abstract

To reduce the load of image processing when detecting fogging on a vehicle window on the basis of a camera image.SOLUTION: A window fog detector includes a fog determination unit 15 that determines the fog state of the window by performing image processing on an image generated by a camera 2 that captures the outside of a vehicle through a window 20 of the vehicle 1. The fog determination unit limits an image area used for determining the fog state so as to reduce the load of image processing.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a window fog detection device. [Background technology]

[0002] Patent Document 1 describes determining the fogging state of a window based on an image generated by a camera that captures an image of the outside of a vehicle through the vehicle window. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-004254 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, when image processing is performed to determine the fogging state of the windows, the processing load on the entire vehicle tends to increase. This is more noticeable when the number of pixels of the camera image for which image processing is performed is large. Various controls are performed in the vehicle to ensure safe driving, and it is desirable to reduce the load of such image processing from the viewpoints of preventing system failures, reducing power consumption, etc.

[0005] In view of the above-mentioned problems, an object of the present invention is to reduce the load of image processing when detecting fogging on vehicle windows based on camera images. [Means for solving the problem]

[0006] The gist of the present disclosure is as follows.

[0007] (1) A window fogging detection device comprising: a fogging determination unit that determines a fogging state of a window by performing image processing on an image generated by a camera that photographs the outside of the vehicle through the window of the vehicle; and the fogging determination unit limits an image area used for determining the fogging state so as to reduce the load of the image processing.

[0008] (2) A window fogging detection device as described in (1) above, wherein the limited image area includes an area near the edge of the window.

[0009] (3) The window fogging detection device according to (1) or (2) above, further comprising a load calculation unit that calculates a load of the image processing, and the fogging determination unit limits an image area used for determining the fogging state when the load of the image processing for the entire area of ​​the image is higher than a predetermined threshold.

[0010] (4) The window fogging detection device according to (3) above, wherein the fogging determination unit narrows the image area used for determining the fogging state as the load of the image processing for the entire area of ​​the image increases.

[0011] (5) The window fogging detection device according to (1) or (2) above, wherein the fogging determination unit switches the image area used to determine the fogging state between the entire area and a partial area. Effect of the Invention

[0012] According to the present invention, when fogging on vehicle windows is detected based on camera images, the load of image processing can be reduced. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating a schematic view of a portion of a configuration of a vehicle provided with a window fogging detection device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing a schematic view of the interior of a vehicle. [Diagram 3] FIG. 2 is a functional block diagram of a processor of the ECU. [Figure 4]4 is a flowchart showing a control routine for a fogging detection process. [Diagram 5] 11A and 11B are diagrams illustrating an example of control for switching an image area used for determining a cloudy state between the entire area and a part of the area. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, like components are designated by like reference numerals.

[0015] FIG. 1 is a diagram that shows a schematic view of a part of the configuration of a vehicle 1 provided with a window fogging detection device according to an embodiment of the present invention.

[0016] 1, a vehicle 1 includes a camera 2, a heater 3, a human machine interface (HMI) 4, and an electronic control unit (ECU) 10. The camera 2, the heater 3, and the HMI 4 are electrically connected to the ECU 10 via an in-vehicle network or the like that complies with a standard such as CAN (Controller Area Network).

[0017] FIG. 2 is a diagram showing the inside of the vehicle 1. As shown in FIG. 2, the camera 2 is provided inside the vehicle 1 (vehicle interior) and captures the outside of the vehicle 1 through a window 20 of the vehicle 1. The camera 2 repeatedly captures the outside of the vehicle 1 at a predetermined capture period (for example, 1 / 30 to 1 / 10 seconds) to generate an image. In this embodiment, the camera 2 is a front camera that captures the view ahead of the vehicle 1, and the window 20 of the vehicle 1 located within the capture range of the camera 2 is the windshield. In this case, the camera 2 is disposed in front of the driver inside the vehicle 1 (for example, the back of the room mirror, the inner surface of the windshield, etc.). The image generated by the camera 2 is transmitted to the ECU 10 and is used, for example, for controlling a driving assistance function (collision avoidance function, lane departure prevention function, etc.) in the vehicle 1.

[0018] The heater 3 (not shown in FIG. 2) is provided on a window 20 of the vehicle 1 and heats the window 20. The heater 3 is a transparent thin-film member made of, for example, carbon nanotubes, and is a heating element that generates heat when electricity is applied. The heater 3 is disposed on the inner surface or inside of the window 20 of the vehicle 1. The heater 3 may be an electric heating wire disposed inside the window 20, a defroster that blows warm air to the window 20, or the like. The ECU 10 controls the heater 3.

[0019] The HMI 4 transmits information between the vehicle 1 and the driver of the vehicle 1. The HMI 4 has an output unit (e.g., a display, a speaker, a vibration unit, etc.) that provides information to the driver of the vehicle 1, and an input unit (e.g., a touch panel, an operation button, an operation switch, a microphone, etc.) into which information is input by the driver of the vehicle 1. The output of the ECU 10 is notified to the driver of the vehicle 1 via the HMI 4, and the input from the driver of the vehicle 1 is transmitted to the ECU 10 via the HMI 4.

[0020] The ECU 10 executes various controls of the vehicle 1. As shown in Fig. 1, the ECU 10 includes a communication interface 11, a memory 12, and a processor 13. The communication interface 11 and the memory 12 are connected to the processor 13 via signal lines. Note that, although one ECU 10 is provided in this embodiment, multiple ECUs may be provided for each function.

[0021] The communication interface 11 has an interface circuit for connecting the ECU 10 to an in-vehicle network. The ECU 10 is connected to other in-vehicle devices via the communication interface 11.

[0022] The memory 12 includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 12 stores computer programs, data, etc. that are used when the processor 13 executes various processes.

[0023] The processor 13 includes one or more central processing units (CPUs) and their peripheral circuits, and may further include other arithmetic circuits such as a logic unit, a numerical calculation unit, or a graphics processing unit.

[0024] In this embodiment, the ECU 10 functions as a window fogging detection device that detects fogging on the windows 20 of the vehicle 1. Fogging on the windows 20 of the vehicle 1 occurs due to condensation or the like, and blocks the driver's view of the vehicle 1.

[0025] 3 is a functional block diagram of the processor 13 of the ECU 10. In this embodiment, the processor 13 has a fogging determination unit 15, a load calculation unit 16, and a heater control unit 17. The fogging determination unit 15, the load calculation unit 16, and the heater control unit 17 are each a functional module realized by the processor 13 of the ECU 10 executing a program stored in the memory 12 of the ECU 10. Note that each of these functional modules may be realized by a dedicated arithmetic circuit provided in the processor 13.

[0026] As described above, the camera 2 captures the outside of the vehicle 1 through the window 20 of the vehicle 1. That is, the capturing range of the camera 2 includes the window 20 of the vehicle 1. For this reason, when the window 20 of the vehicle 1 becomes fogged up, an abnormality (e.g., whitening of the image) occurs in the image generated by the camera 2. Therefore, the fog determination unit 15 performs image processing on the image generated by the camera 2 to determine the fog state of the window 20.

[0027] However, when image processing for determining the fogging state of the window 20 is performed, the processing load on the entire vehicle 1 tends to increase. For this reason, in this embodiment, the fogging determination unit 15 limits the image area used for determining the fogging state of the window 20 so as to reduce the load of image processing for determining the fogging state of the window 20. That is, the fogging determination unit 15 determines the fogging state of the window 20 by performing image processing on a partial area of ​​the image generated by the camera 2 (hereinafter referred to as the "camera image"). This makes it possible to reduce the load of image processing when detecting fogging on the window of the vehicle 1 based on the camera image. For example, the fogging determination unit 15 limits the image area used for determining the fogging state of the window 20 to the upper half, lower half, right half, left half, etc. of the camera image.

[0028] The load calculation unit 16 calculates the load of image processing for determining the fogging state of the window 20. When the load of image processing is low, there is less need to limit the image area used for determining the fogging state of the window 20 compared to when the load of image processing is high. For this reason, in this embodiment, when the load of image processing for the entire area of ​​the camera image is higher than a predetermined threshold, the fogging determination unit 15 limits the image area used for determining the fogging state of the window 20. This makes it possible to avoid limiting the image area more than necessary, and thus to prevent fogging of the window 20 from being overlooked.

[0029] Incidentally, when fogging occurs on the window 20 of the vehicle 1, the fogging typically starts from the edge of the window 20 (the connection portion between the window 20 and the body of the vehicle 1). Based on this phenomenon, in this embodiment, the image area limited by the fogging determination unit 15 includes an area close to the edge of the window 20, and does not include an area far from the edge of the window 20. For example, when the upper half of the camera image is closer to the edge of the window 20 (for example, the connection portion between the windshield and the roof of the vehicle 1) than the lower half of the camera image, the fogging determination unit 15 limits the image area used to determine the fogging state of the window 20 to the upper half of the camera image. This makes it possible to reduce the load of image processing while suppressing a decrease in the detection accuracy of fogging on the window 20.

[0030] The heater control unit 17 controls the supply of electricity to the heater 3, and controls the operating state of the heater 3. When the fogging determination unit 15 determines that fogging has occurred on the window 20 of the vehicle 1, the heater control unit 17 supplies power to the heater 3 to operate the heater 3. This makes it possible to heat the window 20 and eliminate fogging on the window 20.

[0031] The process flow of the above-mentioned control will be described below with reference to Fig. 4. Fig. 4 is a flowchart showing a control routine of the fogging detection process. This control routine is repeatedly executed at predetermined execution intervals by the processor 13 of the ECU 10 while the ignition switch of the vehicle 1 is turned on.

[0032] First, in step S101, the fogging determination unit 15 acquires a camera image.

[0033] Next, in step S102, the load calculation unit 16 calculates the load L of image processing for the entire area of ​​the camera image. For example, the load calculation unit 16 calculates the load L of image processing for the entire area of ​​the camera image based on the number of pixels, brightness, contrast, etc. of the camera image. Note that the load calculation unit 16 may calculate the load L of image processing for the entire area of ​​the camera image based on the number of targets recognized in the camera image (number of recognized targets), the power consumption or usage rate of the processor 13 (e.g., an image processing chip), etc.

[0034] Next, in step S103, the fogging determination unit 15 determines whether the load L calculated by the load calculation unit 16 is higher than a predetermined threshold Lth. The threshold Lth is determined in advance as a value at which a reduction in the processing load by degenerate control of the system is required. The threshold Lth may be changed according to the system load in the vehicle 1 (e.g., the processor usage rate of the ECU 10, etc.). In this case, the higher the system load, the lower the threshold Lth is set.

[0035] If it is determined in step S103 that the load L is higher than the threshold value Lth, the control routine proceeds to step S104. In step S104, the fogging determination unit 15 limits the image area used to determine the fogging state of the window 20. For example, the fogging determination unit 15 limits the image area used to determine the fogging state of the window 20 to the upper half of the camera image, the upper 1 / 3 of the camera image, or the like. Note that the fogging determination unit 15 may narrow the image area used to determine the fogging state as the load L increases. This makes it possible to suppress fluctuations in the load of image processing for determining the fogging state of the window 20. On the other hand, if it is determined in step S103 that the load L is equal to or lower than the threshold value Lth, the control routine skips step S104 and proceeds to step S105.

[0036] In step S105, the fogging determination unit 15 determines the fogging state of the window 20 based on the camera image. When the determination in step S103 is positive, the fogging determination unit 15 performs image processing on the image area limited in step S104, i.e., a part of the area of ​​the camera image, and when the determination in step S103 is negative, the fogging determination unit 15 performs image processing on the entire area of ​​the camera image. For example, when a white area is detected in the image, the fogging determination unit 15 determines that fogging has occurred on the window 20. Note that the fogging determination unit 15 may determine the fogging state of the window 20 based on multiple camera images captured in succession. In this case, in step S104, the image area is limited for each of the multiple camera images.

[0037] Next, in step S106, the heater control unit 17 determines whether or not fogging has occurred on the window 20 based on the determination result by the fogging determination unit 15. If it is determined that fogging has not occurred on the window 20, this control routine ends. On the other hand, if it is determined that fogging has occurred on the window 20, this control routine proceeds to step S107.

[0038] In step S107, the heater control unit 17 activates the heater 3 to heat the window 20 with the heater 3.

[0039] Next, in step S108, the fogging determination unit 15 notifies the driver of the vehicle 1 of poor visibility ahead of the camera 2 via the HMI 4. After step S108, this control routine ends.

[0040] It is also possible to omit step S107, and in step S108, the fogging determination unit 15 prompts the driver of the vehicle 1, via the HMI 4, to perform an operation to defog the window 20.

[0041] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the claims. For example, the window of the vehicle 1 for which the fogging state is determined may be a rear window, a side window, etc.

[0042] Also, the fogging determination unit 15 may always use a limited image area of ​​the camera image to determine the fogging state of the window 20. In this case, the fogging determination unit 15 may change the position of the limited image area every time it determines the fogging state.

[0043] Furthermore, the fogging determination unit 15 may temporarily limit the image area used to determine the fogging state regardless of the load of image processing. That is, the fogging determination unit 15 may switch the image area used to determine the fogging state between the entire area and a part of the area. This can reduce the load of image processing while suppressing a decrease in the detection accuracy of fogging on the window 20.

[0044] Fig. 5 is a diagram showing an example of such control. In Fig. 5, the upper arrow indicates the timing for determining the cloudy state using the image area of ​​the first area (upper half of the camera image), and the lower arrow indicates the timing for determining the cloudy state using the image area of ​​the second area (lower half of the camera image). The determination cycle for the first area is 1 / 2 the determination cycle for the second area. When the upper arrow and the lower arrow overlap, the entire area of ​​the camera image is used to determine the cloudy state, and in the example of Fig. 5, the cloudy state determination unit 15 alternately switches the image area used to determine the cloudy state between the entire area and a part of the area.

[0045] Furthermore, the fogging determination unit 15 may limit the image area used for the determination not only when determining the occurrence of a fogging state on the window 20, but also when determining the elimination of the fogging state on the window 20 due to the operation of the heater 3. In this case, for example, the fogging determination unit 15 limits the image area used for the determination of the elimination of the fogging state to an area far from the heater 3. In other words, the image area limited by the fogging determination unit 15 includes an area far from the heater 3, but does not include an area close to the heater 3. This makes it possible to prevent the window 20 from being determined to have been defogging before it is completely removed.

[0046] Furthermore, a computer program that causes a computer to realize the functions of each unit of the processor 13 of the ECU 10 may be provided in a form stored in a computer-readable recording medium. The computer-readable recording medium is, for example, a magnetic recording medium, an optical recording medium, or a semiconductor memory. [Explanation of symbols]

[0047] 1 vehicle 2 Camera 10 Electronic Control Unit (ECU) 13 Processors 15 Cloudiness determination section 20. Windows

Claims

1. a fogging determination unit that determines a fogging state of a window by performing image processing on an image generated by a camera that photographs the outside of the vehicle through the window of the vehicle; The window fogging detection device, wherein the fogging determination unit limits an image area used for determining the fogging state so as to reduce a load on the image processing.

2. The window fogging detection device of claim 1 , wherein the limited image area includes an area near an edge of the window.

3. A load calculation unit that calculates a load of the image processing, 3. The window fogging detection device according to claim 1, wherein the fogging determination unit limits an image area used for determining the fogging state when a load of the image processing for the entire area of ​​the image is higher than a predetermined threshold.

4. The window fogging detection device according to claim 3 , wherein the fogging determination unit narrows the image area used for determining the fogging state as the load of the image processing for the entire area of ​​the image increases.

5. The window fogging detection device according to claim 1 , wherein the fogging determination unit switches an image area used for determining the fogging state between an entire area and a part of the image area.

Images