Cleaning notification device, cleaning notification method, and cleaning notification program

Abstract

To provide a washing notification device, a washing notification method and a washing notification program which allow a user to easily recognize a notification when notifying that washing of an imaging instrument is executed.SOLUTION: A washing notification device comprises: a detection unit which detects the sight line of a user in a vehicle when washing of an imaging instrument that is mounted on the vehicle and images the outside of the vehicle is performed; and a processing unit which executes processing about the execution notification to the effect that washing of the imaging instrument is executed when the user is looking at a first display instrument that displays a photographed image captured by the imaging instrument.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a cleaning notification device, a cleaning notification method, and a cleaning notification program for notifying a user that an image capturing device associated with a digital mirror has been cleaned. [Background technology]

[0002] Patent Document 1 discloses a cleaning device that, when an automatic cleaning condition is met, executes a process of cleaning an image capturing device associated with an electronic mirror mounted on a vehicle with a cleaning liquid. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-115929 Summary of the Invention [Problem to be solved by the invention]

[0004] In order to allow a user to recognize that the photographing equipment has been cleaned, the user may be notified that the cleaning has been performed. However, if the display area for the notification is limited or if the timing of the notification overlaps with other notifications, the user may not be able to recognize that the cleaning has been performed.

[0005] An object of the present invention is to provide a cleaning notification device, a cleaning notification method, and a cleaning notification program that allow a user to easily recognize a notification that a photographing device has been cleaned. [Means for solving the problem]

[0006] The cleaning notification device described in claim 1 includes a detection unit that detects the line of sight of a user riding in a vehicle when a photographing device mounted on the vehicle for photographing the outside of the vehicle has been cleaned, and a processing unit that executes processing related to an implementation notification indicating that the photographing device has been cleaned when the user is looking at a first display device that displays an image captured by the photographing device.

[0007] The cleaning notification device described in claim 1 detects the line of sight of a user in a vehicle when cleaning of a lens of a photographing device related to an electronic mirror mounted on a vehicle is performed, and when it detects that the user is looking at a display device related to the electronic mirror, it performs processing related to the cleaning notification. In other words, according to the cleaning notification device, when the user is looking at a display device related to the electronic mirror, the priority of the cleaning notification is increased. This allows the user to easily recognize the notification when cleaning of the photographing device is performed.

[0008] The cleaning notification device according to claim 2 is the cleaning notification device according to claim 1, wherein the processing unit increases the priority of the implementation notification when the user is not looking at the first display device and a predetermined condition is satisfied.

[0009] According to the cleaning notification device of claim 2, other notifications can be given priority.

[0010] The cleaning notification device according to claim 3 is the cleaning notification device according to claim 2, wherein the predetermined condition is a case where the number of times the cleaning notification has not been issued is equal to or greater than a threshold value.

[0011] According to the cleaning notification device of claim 3, even if other notifications exist, the cleaning notification can be notified.

[0012] The cleaning notification device according to a fourth aspect of the present invention is the cleaning notification device according to the second or third aspect, wherein, when the cleaning notification is notified, the processing unit sets a priority of the cleaning notification to an initial value.

[0013] According to the cleaning notification device of claim 4, even if the priority of the implementation notification is increased, after the implementation notification is notified, other notifications can be given priority.

[0014] The cleaning notification device of claim 5 is the cleaning notification device of claim 3 or claim 4, in which the processing unit counts the number of times the implementation notification has not been notified when there is another notification that is notified at the same trigger as the implementation notification and the other notification has a higher priority than the implementation notification.

[0015] According to the cleaning notification device of claim 5, the conditions under which the cleaning notification is not issued can be made clearer, making it possible to control the notification.

[0016] The cleaning notification device according to claim 6 is the cleaning notification device according to claim 5, wherein the processing unit notifies the implementation notification when the priority of the implementation notification is higher than the priority of the other notifications, or when the other notifications do not exist.

[0017] According to the cleaning notification device of claim 6, the conditions for issuing a notification to notify the user to perform cleaning can be made clearer, making it possible to control the notification.

[0018] The cleaning notification device of claim 7 is the cleaning notification device of any one of claims 1 to 6, wherein the processing unit notifies by displaying the execution notification on a second display device that indicates that the cleaning has been performed.

[0019] According to the cleaning notification device of claim 7, the cleaning notification can be notified without blocking the display of the display device related to the digital mirror.

[0020] The cleaning notification method described in claim 8 detects the line of sight of a user riding in a vehicle when cleaning of a photographing device mounted on the vehicle for photographing the exterior of the vehicle is performed, and when the user is looking at a first display device that displays an image captured by the photographing device, executes processing related to an implementation notification indicating that the photographing device has been cleaned.

[0021] The cleaning notification method described in claim 8 detects the line of sight of a user in a vehicle when cleaning of a lens of a photographing device related to an electronic mirror mounted on a vehicle is performed, and performs processing related to a cleaning notification when it is detected that the user is looking at a display device related to the electronic mirror. In other words, according to the cleaning notification method, when the user is looking at a display device related to the electronic mirror, the priority of the cleaning notification is increased. This allows the user to easily recognize the notification when cleaning of the photographing device is performed.

[0022] The cleaning notification program described in claim 9 detects the line of sight of a user riding in a vehicle when cleaning of a photographing device mounted on the vehicle for photographing the exterior of the vehicle has been performed, and when the user is looking at a first display device that displays an image captured by the photographing device, causes a computer to execute a process related to an implementation notification indicating that the photographing device has been cleaned.

[0023] A computer that executes the cleaning notification program according to claim 9 detects the line of sight of a user in a vehicle when cleaning of a lens of a photographing device related to an electronic mirror mounted on a vehicle is performed, and when it detects that the user is looking at a display device related to the electronic mirror, performs processing related to the cleaning notification. In other words, according to the computer, when the user is looking at a display device related to the electronic mirror, the priority of the cleaning notification is increased. This allows the user to easily recognize the notification when cleaning of the photographing device is performed. Effect of the Invention

[0024] According to the present invention, when a user is notified that the photographing device has been cleaned, the user can easily recognize the notification. [Brief description of the drawings]

[0025] [Figure 1] 1 is a block diagram showing a hardware configuration of a vehicle according to an embodiment of the present invention; [Diagram 2]FIG. 2 is a block diagram showing a functional configuration of the vehicle-mounted device according to the embodiment; [Diagram 3] 5 is a flowchart showing a process for notifying an implementation notification executed in the vehicle-mounted device of the present embodiment; [Figure 4] 5 is a flowchart showing a flow of a process for determining whether or not to notify an implementation notification, which is executed in the vehicle-mounted device of the present embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] The cleaning notification device of the present invention will be described. The cleaning notification device notifies a user that a photographing device associated with an electronic mirror mounted on a vehicle has been cleaned (hereinafter, referred to as "cleaning notification").

[0027] (vehicle) As shown in FIG. 1, a vehicle 10 according to this embodiment includes an in-vehicle device 20, a plurality of ECUs (Electronic Control Units) 22, and a plurality of in-vehicle devices 24.

[0028] The vehicle-mounted device 20 includes a central processing unit (CPU) 20A, a read only memory (ROM) 20B, a random access memory (RAM) 20C, an in-vehicle communication interface (I / F) 20D, a wireless communication I / F 20E, and an input / output interface (input / output I / F) 20F. The CPU 20A, the ROM 20B, the RAM 20C, the in-vehicle communication I / F 20D, the wireless communication I / F 20E, and the input / output I / F 20F are connected to each other so as to be able to communicate with each other via an internal bus 20G.

[0029] The CPU 20A is a central processing unit that executes various programs and controls each part. That is, the CPU 20A reads out the programs from the ROM 20B and executes the programs using the RAM 20C as a working area.

[0030] The ROM 20B stores various programs and various data. In the present embodiment, the ROM 20B stores a cleaning notification program 100 for notifying the user of the cleaning notification. When the cleaning notification program 100 is executed, the vehicle-mounted device 20 detects the cleaning of the electronic mirror and executes a process for notifying the user of the cleaning notification. The RAM 20C temporarily stores programs or data as a working area.

[0031] The in-vehicle communication I / F 20D is an interface for connecting to each ECU 22. The interface uses a communication standard based on the CAN protocol. The in-vehicle communication I / F 20D is connected to the external bus 20H.

[0032] The wireless communication I / F 20E is a wireless communication module for communicating with an external device. The wireless communication module uses communication standards such as 5G, LTE, and Wi-Fi (registered trademark). The wireless communication I / F 20E is connected to a network N.

[0033] The input / output I / F 20F is an interface for communicating with an in-vehicle camera 26, a monitor 27, a HUD (Head Up Display) 28, and a speaker 29 mounted on the vehicle 10. The in-vehicle camera 26, the monitor 27, the head up display (HUD) 28, and the speaker 29 may be directly connected to the internal bus 20G. The monitor 27 and the HUD 28 according to this embodiment are examples of a "second display device."

[0034] The ECU 22 includes at least an electronic mirror ECU 22A and a washing control ECU 22B.

[0035] The electronic mirror ECU 22A is connected to a camera 24A and a display unit 24B. The electronic mirror ECU 22A controls the display of an image captured by the camera 24A on the display unit 24B. When the electronic mirror ECU 22A receives an instruction to issue an implementation notification from the vehicle-mounted device 20, the electronic mirror ECU 22A controls the display of an icon indicating the implementation notification on the display unit 24B. The camera 24A according to this embodiment is an example of an "imaging device", and the display unit 24B is an example of a "first display device".

[0036] The camera 24A is installed, for example, on the left and right front side doors and the rear bumper of the vehicle 10. The camera 24A is composed of a left rear outer camera that photographs the outside of the vehicle 10 and the left rear of the vehicle 10, a right rear outer camera that photographs the outside of the vehicle 10 and the right rear of the vehicle 10, and a rear outer camera that photographs the outside of the vehicle 10 and the rear of the vehicle 10. In this embodiment, the camera 24A is described as being the left rear outer camera, the right rear outer camera, and the rear outer camera. However, this is not limited to this. The camera 24A may be a camera that monitors the surrounding situation. For example, the camera 24A may be a back camera that is installed on the upper part of the rear gate and has an optical axis direction tilted toward the ground side more than the rear outer camera, or may be installed on the front bumper and include a front camera that photographs the front of the vehicle 10.

[0037] The display unit 24B is a display device corresponding to each of the left rear outer camera, the right rear outer camera, and the rear outer camera constituting the camera 24A. For example, the display device corresponding to the right rear outer camera is installed at the right end of the instrument panel, and the display device corresponding to the left rear outer camera is installed at the center of the instrument panel. In addition, the display device corresponding to the rear outer camera is arranged at the front upper part of the vehicle 10 so that the screen faces the vehicle rear direction. In addition, when the display unit 24B receives an instruction to perform the execution notification, it displays an icon indicating that the cleaning has been performed. Note that the display unit 24B according to the present embodiment has been described as displaying the images captured by the corresponding camera 24A. However, this is not limited to this. For example, the images captured by the left rear outer camera and the right rear outer camera may be displayed on the display device corresponding to the rear outer camera, and the images of the front camera and the back camera (not shown) may be displayed on the display device corresponding to the rear outer camera.

[0038] The cleaning control ECU 22B is connected to the pump driving unit 24C. When the cleaning control ECU 22B detects dirt adhering to the camera 24A, it drives the pump driving unit 24C and controls the cleaning of the camera 24A associated with the electronic mirror. When the cleaning control ECU 22B performs cleaning, it transmits a signal indicating a performance notification (hereinafter referred to as an "execution signal") to the in-vehicle device 20.

[0039] The pump driving unit 24C drives the pump in response to an instruction from the cleaning control ECU 22B, and sprays the cleaning liquid onto the lens portion of the camera 24A to clean the camera 24A.

[0040] The in-vehicle camera 26 is an imaging device that captures an image of a user (driver) on board the vehicle 10.

[0041] The monitor 27 is a liquid crystal monitor provided on an instrument panel, dashboard, or the like of the vehicle 10 for displaying various information.

[0042] The HUD 28 is a display provided on an instrument panel, dashboard, or the like of the vehicle 10, and displays various information on a small panel.

[0043] The speaker 29 is a device that is provided on an instrument panel, a center console, a front pillar, a dashboard, or the like, and outputs audio.

[0044] As shown in FIG. 2, in the vehicle-mounted device 20 of this embodiment, a CPU 20A executes a cleaning notification program 100 to function as a detection unit 200, an acquisition unit 210, a detection unit 220, a determination unit 230, and a processing unit 240.

[0045] The detection unit 200 detects that the camera 24A associated with the digital mirror has been cleaned. Specifically, the detection unit 200 detects an execution signal transmitted from the cleaning control ECU 22B to detect that cleaning has been performed.

[0046] When the acquisition unit 210 detects the implementation signal, it acquires the image captured by the in-vehicle camera 26.

[0047] The detection unit 220 uses the captured image acquired by the acquisition unit 210 to detect the line of sight of the user (driver).

[0048] The determination unit 230 determines whether the user is looking at the display unit 24B, using the user's line of sight detected by the detection unit 220. The determination unit 230 also determines whether the number of times that the implementation notification was not notified (hereinafter, referred to as the "number of non-notifications") is equal to or greater than a threshold. Here, the threshold may be determined in advance, or may be set by the user.

[0049] The processing unit 240 performs a process of increasing the priority of the implementation notification when the user is looking at the display unit 24B. For example, when the processing unit 240 sets the priority "1" for the implementation notification, if the user is looking at the display unit 24B, the processing unit 240 sets the priority to "2" which indicates a higher priority than the currently set priority.

[0050] In addition, when there is another notification different from the implementation notification at the same time of notifying the implementation notification, the processing unit 240 compares the priority of the implementation notification with the priority of the other notification, and notifies the notification with the higher priority. Here, the processing unit 240 transmits an instruction to the electronic mirror ECU 22A to display the implementation notification or the other notification, and causes the display unit 24B to display it.

[0051] When an implementation notification is not notified due to another notification, the processing unit 240 counts the number of non-notifications. When the number of non-notifications is equal to or greater than a threshold, the processing unit 240 performs processing to increase the priority of the implementation notification.

[0052] If the priority of the implementation notification is higher than other notifications, or if there are no other notifications to be notified at the same time, the processing unit 240 transmits an instruction to display the implementation notification and notifies the implementation notification. Here, when the processing unit 240 notifies the implementation notification, it initializes the priority of the implementation notification and the number of non-notifications.

[0053] (Flow of Control) The flow of each process executed by the cleaning notification device of this embodiment will be described with reference to the flowcharts of Figures 3 and 4. Each process in the vehicle-mounted device 20 is executed by the CPU 20A of the vehicle-mounted device 20 functioning as a detection unit 200, an acquisition unit 210, a detection unit 220, a determination unit 230, and a processing unit 240. The process of issuing an execution notification shown in Figure 3 is executed, for example, when the engine of the vehicle 10 is driven and an instruction to issue an execution notification is input.

[0054] In step S100, the CPU 20A sets initial values ​​for the priority of the implementation notification and the number of non-notifications. For example, the CPU 20A sets the priority to "1" and the number of non-notifications to "0."

[0055] In step S101, the CPU 20A determines whether or not an implementation signal has been detected. If an implementation signal has been detected (step S101: YES), the CPU 20A proceeds to step S102. On the other hand, if an implementation signal has not been detected (step S101: NO), the CPU 20A waits until an implementation signal is detected.

[0056] In step S102, CPU 20A acquires an image captured by in-vehicle camera .

[0057] In step S103, CPU 20A detects the user's line of sight from the acquired captured image.

[0058] In step S104, the CPU 20A uses the detected user's line of sight to determine whether the user is looking at the display unit 24B (digital mirror). If the user is looking at the display unit 24B (step S104: YES), the CPU 20A proceeds to step S105. On the other hand, if the user is not looking at the display unit 24B (step S104: NO), the CPU 20A proceeds to step S106.

[0059] In step S105, the CPU 20A performs a process of increasing the priority of the implementation notification.

[0060] In step S106, the CPU 20A determines whether the number of non-notifications is equal to or greater than the threshold. If the number of non-notifications is equal to or greater than the threshold (step S106: YES), the CPU 20A proceeds to step S105. On the other hand, if the number of non-notifications is not equal to or greater than the threshold (the number of non-notifications is less than the threshold) (step S106: NO), the CPU 20A proceeds to step S107.

[0061] In step S107, CPU 20A performs a notification determination process, which will be described in detail later with reference to FIG.

[0062] In step S108, the CPU 20A judges whether or not to end the process of notifying the implementation notification. If the process of notifying the implementation notification is to be ended (step S108: YES), the CPU 20A ends the process of notifying the implementation notification. On the other hand, if the process of notifying the implementation notification is not to be ended (step S108: NO), the CPU 20A proceeds to step S101 and judges whether or not the implementation signal has been detected.

[0063] Next, the notification determination process will be described with reference to Fig. 4. The notification determination process shown in Fig. 4 is executed, for example, when an instruction to determine whether or not to notify is input after an implementation signal is detected.

[0064] In step S200, the CPU 20A determines whether or not there is another notification that is to be notified at the same time as the implementation notification. If there is no other notification (step S200: YES), the CPU 20A proceeds to step S201. On the other hand, if there is another notification (step S107: NO), the CPU 20A proceeds to step S203.

[0065] In step S201, the CPU 20A transmits an instruction to display an implementation notification to the electronic mirror ECU 22A, and notifies the implementation notification.

[0066] In step S202, CPU 20A initializes the priority and the non-notification count.

[0067] In step S203, the CPU 20A determines whether the priority in the implementation notification is higher than the priority in the other notifications. If the priority in the implementation notification is higher (step S203: YES), the CPU 20A proceeds to step S201. On the other hand, if the priority in the implementation notification is not higher (the priority in the implementation notification is lower than the priority in the other notifications) (step S203: NO), the CPU 20A proceeds to step S204.

[0068] In step S204, CPU 20A adds 1 to the non-notification count.

[0069] (summary) The vehicle-mounted device 20 as a cleaning notification device of this embodiment detects the line of sight of the user in the vehicle when the lens of the photographing device related to the electronic mirror mounted on the vehicle is cleaned, and when it detects that the user is looking at the display device related to the electronic mirror, it performs processing related to the cleaning notification. In other words, according to the cleaning notification device, when the user is looking at the display device related to the electronic mirror, the priority of the cleaning notification is increased. This allows the user to easily recognize the notification when the photographing device is cleaned.

[0070] As described above, according to this embodiment, when the photographing device has been cleaned, the user can easily recognize the notification.

[0071] [remarks] In the above embodiment, the vehicle-mounted device 20 is described as being equipped with a cleaning notification device. However, the present invention is not limited to this. The cleaning notification device may be equipped in an external device such as a center server. For example, the vehicle-mounted device 20 may transmit an implementation signal and a captured image of the user to the center server, receive a decision from the center server, and notify the implementation notification in accordance with the decision.

[0072] In the above embodiment, the implementation notification is displayed on the display unit 24B associated with the electronic mirror. However, the embodiment is not limited to this. The implementation notification may be displayed on the monitor 27 and the HUD 28, or may be notified as a voice using the speaker 29. Also, the state of the user may be determined using a captured image, and the means of notification may be changed according to the state of the user. For example, when the user is looking at the display unit 24B, the implementation notification may be displayed on the display unit 24B, and when the user is not looking at the display unit 24B, the notification may be displayed on the monitor 27 and the HUD 28. Also, when the user is not driving the vehicle 10 due to automatic driving or the like, the notification may be made by the speaker 29.

[0073] In the above embodiment, the vehicle-mounted device 20 performs the notification determination process. However, the present invention is not limited to this. The electronic mirror ECU 22A may perform the notification determination process. For example, the vehicle-mounted device 20 may transmit an instruction for the implementation notification and the priority of the notification to the electronic mirror ECU 22A, and the electronic mirror ECU 22A may receive the instruction and the priority and determine whether or not to notify according to the priority. When the electronic mirror ECU 22A does not notify the implementation notification, the electronic mirror ECU 22A may transmit a signal indicating that the notification was not made to the vehicle-mounted device 20, and when the vehicle-mounted device 20 receives the signal, the vehicle-mounted device 20 may count the number of non-notifications.

[0074] In the above embodiment, various processes executed by the CPU 20A by reading the software (program) may be executed by various processors other than the CPU. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. In addition, each of the above-mentioned processes may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, etc.). In addition, the hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0075] In the above embodiment, each program has been described as being prestored (installed) in a non-transitory computer-readable recording medium. For example, the washing notification program 100 in the vehicle-mounted device 20 is prestored in the ROM 20B. However, the present invention is not limited to this, and each program may be provided in a form recorded in a non-transitory recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. Also, the program may be downloaded from an external device via a network.

[0076] The process flow described in the above embodiment is merely an example, and unnecessary steps may be deleted, new steps may be added, or the process order may be changed, without departing from the spirit of the invention. [Explanation of symbols]

[0077] 20 Vehicle-mounted device (cleaning notification device) 24A Camera (photographic equipment) 24B Display section (1st display device) 27 Monitor (second display device) 28 HUD (second display device) 100 Cleaning Notification Program 200 Detection unit 220 Detection unit 230 Judgment section 240 Processing section

Claims

1. a detection unit that detects a line of sight of a user riding in a vehicle when an image capturing device mounted on the vehicle and capturing an image of the outside of the vehicle is cleaned; a processing unit that, when it is detected by the detection unit that the user is looking at a first display device that displays an image captured by the photographing device, executes a process related to a notification indicating that the photographing device has been cleaned in accordance with a priority of the notification, and causes the first display device to display the notification; Equipped with The processing unit of the cleaning notification device increases the priority of the implementation notification when the detection unit detects that the user is not looking at the first display device and the number of times the implementation notification has not been notified reaches or exceeds a threshold value.

2. The processing unit sets a priority of the implementation notification to an initial value when the implementation notification is notified. The cleaning notification device of claim 1 .

3. The processing unit counts the number of times the implementation notification was not notified when there is another notification triggered by the same trigger as the implementation notification and the priority of the other notification is higher than the implementation notification. The cleaning notification device of claim 2.

4. The processing unit notifies the implementation notification when the priority of the implementation notification is higher than the priority of the other notifications, or when the other notifications do not exist. The cleaning notification device of claim 3.

5. The processing unit notifies the user by displaying the execution notice on a second display device that displays the fact that the cleaning has been performed. The cleaning notification device of claim 1 .

6. Detecting a line of sight of a user in a vehicle when a photographing device mounted on the vehicle for photographing the outside of the vehicle is cleaned; When it is determined as a result of detection of the user's line of sight that the user is looking at a first display device that displays an image captured by the photographing device, a process related to a notification indicating that the photographing device has been cleaned is executed in accordance with a priority of the notification, and the notification is displayed on the first display device. The computer executes the process, A cleaning notification method in which, when it is determined as a result of gaze detection that the user is not looking at the first display device and the number of times the notification has not been sent reaches or exceeds a threshold, the priority of the notification is increased.

7. Detecting a line of sight of a user in a vehicle when a photographing device mounted on the vehicle for photographing the outside of the vehicle is cleaned; when it is determined as a result of detection of the user's gaze that the user is looking at a first display device that displays an image captured by the photographing device, execute a process related to an execution notification indicating that the photographing device has been cleaned in accordance with a priority of the execution notification, and cause the first display device to display the execution notification. The process is executed by a computer, A cleaning notification program that increases the priority of the implementation notification when it is determined as a result of gaze detection that the user is not looking at the first display device and the number of times the implementation notification has not been notified exceeds a threshold value.

Images