Notification system, notification device, notification method, and imaging system

Abstract

To provide a notification system capable of improving vehicle security, a notification device, a notification method, and an imaging system.SOLUTION: The notification system includes: at least one camera that outputs an image captured at least a portion of the surroundings of a vehicle; an object detection unit that detects objects around the vehicle; a notification unit that outputs detection notification that is information that notifies that the object has been detected by the object detection unit to an external unit: and a control unit that is configured so as to, when receiving an instruction from the external unit regarding imaging by the imaging unit after outputting the detection notification, execute imaging using at least one camera following the instruction.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present disclosure relates to a notification system, a notification device, a notification method, and an imaging system. [Background technology]

[0002] There is known a technology for capturing and recording images of the inside and outside of a vehicle using a camera or the like attached to the vehicle body (for example, Patent Document 1). In the above-mentioned conventional technology, when a moving object is detected around the vehicle, the camera or the like starts capturing images in response to the detection, and further starts saving the data output by the camera (in other words, recording).

[0003] However, capturing and recording images when an object is detected, as in conventional technology, places a heavy burden on the vehicle's battery and consumes memory capacity to store the data. Therefore, depending on the memory capacity, there is a risk that evidence video that proves a crime will be overwritten and lost, or the battery will run out at the time when evidence video should be captured.

[0004] In addition, the conventional technology does not provide a way for the vehicle owner to know when an object is detected. Therefore, if the vehicle is stolen, for example, even if a video is recorded as evidence, it is meaningless. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2021-44661 A Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present disclosure is to provide a notification system, a notification device, a notification method, and an imaging system that can improve the crime prevention capabilities of a vehicle. [Means for solving the problem]

[0007] The notification system according to the present disclosure includes at least one camera that outputs an image capturing at least a portion of the surroundings of the vehicle, an object detection unit that detects objects around the vehicle, a notification unit that outputs a detection notification to an external device, which is information notifying that an object has been detected by the object detection unit, and a control unit that, upon receiving an instruction from the external device regarding imaging by the imaging unit after outputting the detection notification, performs imaging by the at least one camera in accordance with the instruction. Effect of the Invention

[0008] The notification system, notification device, notification method, and imaging system according to the present disclosure can improve the crime prevention capabilities of a vehicle. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a vehicle according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a vehicle according to the embodiment. [Diagram 3] FIG. 3 is a block diagram illustrating an example of a configuration of a notification system according to an embodiment. [Figure 4] FIG. 4 is a block diagram illustrating an example of a functional unit included in the control unit. [Diagram 5] FIG. 5 is a block diagram showing an example of the configuration of a user terminal. [Figure 6] FIG. 6 is a block diagram illustrating an example of a functional unit included in the control unit. [Figure 7] FIG. 7 is a diagram showing an example of the detection range of the surroundings monitor. [Figure 8] FIG. 8 is a diagram showing an example of the detection range when there are walls to the rear and to the left of a parked vehicle. [Figure 9] FIG. 9 is a sequence diagram illustrating an example of a flow of processes performed by the notification system. [Figure 10]FIG. 10 is a sequence diagram illustrating an example of the flow of processing performed by the notification system. [Figure 11] FIG. 11 is a sequence diagram illustrating an example of the flow of processing performed by the notification system. [Figure 12] FIG. 12 is a sequence diagram illustrating an example of a flow of processes performed by the notification system. [Figure 13] FIG. 13 is a sequence diagram illustrating an example of a flow of processes performed by the notification system. [Figure 14] FIG. 14 is a block diagram illustrating an example of a configuration of a notification system according to an embodiment. [Figure 15] FIG. 15 is a sequence diagram illustrating an example of the flow of processing performed by the notification system. [Figure 16] FIG. 16 is a sequence diagram illustrating an example of the flow of processing performed by the notification system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The first embodiment will be described with reference to the drawings.

[0011] (Vehicle configuration example) 1 and 2 are diagrams showing an example of a vehicle 1 according to an embodiment, in which FIG. 1 is a right side view and FIG. 2 is a plan view. The vehicle 1 of this embodiment includes a vehicle body 10, a windshield 11, a rear windshield 12, and a pair of left and right side mirrors 13. The vehicle 1 is also provided with a front camera 271 and a rear camera 272 for capturing images of the outside of the vehicle, and an interior camera 273 for capturing images of the inside of the vehicle. Here, the outside of the vehicle is the outside of the vehicle 1, and the inside of the vehicle is the inside of the vehicle 1. The vehicle 1 is also provided with a front camera 371, a rear camera 372, a right side camera 381, and a left side camera 382 for capturing images of the surroundings of the vehicle 1. The above-mentioned cameras 271 to 273, 371, 372, 381, and 382 are examples of an imaging unit that outputs an image capturing at least a part of the surroundings of the vehicle. Moreover, the above-mentioned cameras 271 to 273, 371, 372, 381, and 382 are an example of at least one camera that outputs an image capturing at least a part of the surroundings of the vehicle. Here, "at least a portion of the surroundings of the vehicle" means a portion of the detection target area (or monitoring target area) set around the vehicle, and the camera as a whole is capable of acquiring an image capturing the entire detection target area (or monitoring target area).

[0012] The front camera 271 is disposed inside the vehicle 1 (in other words, inside the vehicle) and above the windshield 11, and captures an image of the outside of the windshield 11, i.e., the front of the vehicle 1. The rear camera 272 is disposed inside the vehicle and below the rear windshield 12, and captures an image of the outside of the rear windshield 12, i.e., the rear of the vehicle 1. The interior camera 273 is disposed on the ceiling inside the vehicle and near the windshield 11, and captures an image of the interior of the vehicle.

[0013] The cameras 371, 372, 381, and 382 for capturing images of the surroundings of the vehicle 1 are wide-angle cameras that capture images of the area below the mounting positions at a wide angle. In other words, the cameras 371, 372, 381, and 382 are downward imaging cameras that capture images of the area below the mounting positions. The front camera 371 is disposed at the front of the vehicle body 10. The rear camera 372 is disposed at the rear of the vehicle body 10. The right side camera 381 is disposed on the right side mirror 13. The left side camera 382 is disposed on the left side mirror 13. In this case, more preferably, the imaging area of ​​the front camera 371 partially overlaps with the imaging area of ​​the right side camera 381 and the imaging area of ​​the left side camera 382, ​​and the imaging area of ​​the rear camera 372 partially overlaps with the imaging area of ​​the right side camera 381 and the imaging area of ​​the left side camera 382. As a result, no blind spots are generated, and it becomes possible to capture images all around the vehicle 1.

[0014] (Example of a notification system configuration) 3 is a block diagram showing an example of the configuration of a notification system. The notification system of this embodiment includes a drive recorder unit 2, a surroundings monitor 3, and a communication unit 4. The communication unit 4 is an example of a notification device. The vehicle 1 further includes an electronic mirror 15 and a display device 16. The drive recorder unit 2, the surroundings monitor 3, and the communication unit 4 are each an example of a first device, a second device, or a third device.

[0015] The drive recorder unit 2 includes an MCU (Micro Controller Unit) 21, an image processing unit 22, a CAN driver 23, I / Fs (interfaces) 241, 261 to 263, deserializers 251 to 253, a memory 29, etc. The surroundings monitor 3 includes an ECU (Electronic Control Unit) 31, etc. The communication unit 4 includes an ECU 41, etc.

[0016] The MCU 21 comprehensively controls each part of the drive recorder unit 2. The ECU 31 comprehensively controls each part of the surroundings monitor 3. The ECU 41 comprehensively controls each part of the communication unit 4. In addition, the MCU 21 and the ECUs 31 and 41 execute software programs embedded in the memory to realize various functional parts (see FIG. 4, described later) as the control parts 20, 30 and 40.

[0017] The image processing unit 22 is formed of, for example, a SOC (System on a Chip), and performs image processing of videos or images captured by the cameras 271-273. The image processing unit 22 also transfers the video signals received from the deserializers 251-253 to the communication unit 4 via the I / F 241. Data communication for this transfer is performed, for example, via Ethernet (registered trademark). Hereinafter, there may be cases where video and images are referred to as "video" or "image" without distinction.

[0018] The CAN driver 23 mediates serial communication between the MCU 21 and other devices using the CAN (Controller Area Network) protocol. The CAN driver 23 and the surroundings monitor 3, the surroundings monitor 3 and the communication unit 4, and the CAN driver 23 and the electronic mirror 15 and the display device 16 are each connected by wire.

[0019] The I / F 241 and the I / Fs 261 to 263 connect the communication unit 4 and the cameras 271 to 273 to the deserializers 251 to 253, and mediate communication between the communication unit 4 and the cameras 271 to 273 and the image processing unit 22.

[0020] When the deserializers 251-253 receive a control signal from the image processing unit 22, they receive the video transmitted from the cameras 271-273. At this time, the deserializers 251-253 convert the video transmitted as a serialized video signal into parallel data. The deserializers 251-253 transmit the parallelized video as a video signal to the image processing unit 22. The deserializers 251-253 are an example of a receiving section.

[0021] The drive recorder unit 2 is housed, for example, in a console box of the vehicle 1. The drive recorder unit 2 records (in other words, records) the video or images captured by the front camera 271, the rear camera 272, and the interior camera 273 by storing (in other words, saving) them in the memory 29. The memory 29 is, for example, a storage device such as a micro SD, an eMMC (embedded multi media card), or an SSD (solid state drive). The drive recorder unit 2 may record the video or images captured by the front camera 371, the rear camera 372, the right side camera 381, and the left side camera 382 (described later) by storing them in the memory 29 and receiving them from the surroundings monitor 3 (described later).

[0022] The surroundings monitor 3 is stored, for example, in a console box of the vehicle 1. The surroundings monitor 3 outputs videos or images captured by the front camera 371, the rear camera 372, the right side camera 381, and the left side camera 382. The surroundings monitor 3 also combines the images from the front camera 371, the rear camera 372, the right side camera 381, and the left side camera 382, ​​and outputs a video of the vehicle 1 and its surroundings as viewed from above.

[0023] The communication unit 4 enables data communication with an external device (for example, a user terminal 5) using an LTE communication network, a wireless LAN (Local Area Network), or the like.

[0024] The electronic mirror 15 is configured with a display device such as a liquid crystal display. The electronic mirror 15 plays the role of a mirror for checking the rear of the vehicle 1 by displaying an image of the rear of the vehicle 1 captured by the rear camera 272.

[0025] The display device 16 is a display device such as a panel-type liquid crystal display that is fitted in an instrument panel, etc. The display device 16 displays the surroundings monitor 3, the video processed by the drive recorder unit 2, etc.

[0026] The electronic mirror 15 and the display device 16 receive and display images transmitted from the drive recorder unit 2 (or the rear camera 272) and the surroundings monitor 3.

[0027] 4 is a block diagram showing an example of functional units included in the control units 20, 30, 40. The control unit 30 of the surroundings monitor 3 includes an object detection unit 301, a notification output unit 302, an instruction reception unit 303, an image capture control unit 304, a data output unit 305, etc. The control unit 20 of the drive recorder unit 2 includes an instruction reception unit 203, an image capture control unit 204, a data output unit 205, etc. The control unit 40 of the communication unit 4 includes a notification transfer unit 402, an instruction transfer unit 403, a data transfer unit 405, etc.

[0028] The object detection unit 301 detects a moving object within one or more detection ranges around the vehicle 1. In other words, the object detection unit 301 executes a detection process to detect objects around the vehicle. When a moving object is captured in the images output by the cameras 371, 372, 381, and 382, ​​the object detection unit 301 outputs a signal indicating that the object has been detected. This signal indicating that an object has been detected may be called a detection signal.

[0029] The notification output unit 302 is an example of a notification unit that outputs a detection notification, which is information notifying that an object has been detected by the object detection unit 301, to an external device (e.g., the user terminal 5). In other words, the notification unit executes a notification step of outputting a detection notification, which is information notifying that an object has been detected by the object detection unit 301, to an external device (e.g., the user terminal 5). When the object detection unit 301 outputs a detection signal, the notification output unit 302 notifies the communication unit 4 of a detection notification indicating that fact. In this embodiment, the notification output unit 302 configures a notification unit together with the notification transfer unit 402. The notification output unit 302 may output, in response to the detection of an object by the object detection unit 301, an image capture instruction for instructing at least one camera to capture an image to the drive recorder unit 2. The notification output unit 302 is an example of an instruction output unit.

[0030] When the instruction receiving unit 303 receives an instruction transferred by the communication unit 4, the instruction receiving unit 303 receives the instruction and performs processing according to the instruction. The processing includes, for example, transferring the instruction to the drive recorder unit 2, transmitting the instruction to the imaging control unit 304, and storing the setting of whether or not to operate the cameras 371, 372, 381, and 382 when parking.

[0031] The imaging control unit 304 performs control such as issuing instructions to the cameras 371, 372, 381, and 382 to start and end imaging.

[0032] The data output section 305 outputs data of the images captured by the cameras 371 , 372 , 381 , and 382 to the communication unit 4 .

[0033] When the instruction receiving unit 203 receives an instruction transferred by the communication unit 4 via the instruction receiving unit 303, the instruction receiving unit 203 receives the instruction and performs processing according to the instruction. The processing includes, for example, transmitting an instruction to the imaging control unit 204 and storing settings for the cameras 271 to 273 as to whether or not to operate them when parking.

[0034] The imaging control unit 204 performs control such as issuing instructions to the cameras 271 to 273 to start and end imaging.

[0035] The data output section 205 outputs data of the images captured by the cameras 271 to 273 to the communication unit 4. The data output section 205 is an example of an output section. The data output section 205 is also an example of an image output section.

[0036] The notification transfer unit 402 is an example of a notification unit that outputs a detection notification, which is information notifying that an object has been detected by the object detection unit 301, to an external device (e.g., the user terminal 5). The notification transfer unit 402 transfers the detection notification received from the surroundings monitor 3 to a predetermined external device (e.g., the user terminal 5).

[0037] The instruction transfer unit 403 is an example of a receiving unit that receives an instruction to start capturing images of at least a part of the surroundings of the vehicle 1 from an external device (e.g., user terminal 5) after the notification output unit 302 and the notification transfer unit 402 (in other words, the notification unit) output a notification to the external device (e.g., user terminal 5). The instruction to start capturing images of at least a part of the surroundings of the vehicle 1 is, in other words, an instruction to start capturing images by at least one camera. The instruction transfer unit 403 transfers the instruction received from the external device (e.g., user terminal 5) to the surroundings monitor 3.

[0038] The data transfer unit 405 transfers the data received from the surroundings monitor 3 and the drive recorder unit 2 to a predetermined external device (for example, the user terminal 5).

[0039] In the first embodiment, the instruction transfer unit 403, the instruction receiving unit 303, 203, the imaging control unit 304, 204, the data output unit 305, 205, and the data transfer unit 405 cooperate to function as a control unit that executes the following steps. That is, even if an object is detected by the object detection unit 301, the instruction transfer unit 403, the instruction receiving unit 303, 203, the imaging control unit 304, 204, the data output unit 305, 205, and the data transfer unit 405 do not immediately start imaging by the imaging unit (at least a part of the cameras 271 to 273, 371, 372, 381, 382), but execute a step of starting imaging by the imaging unit according to an instruction received from an external device (user terminal 5) after a detection notification is output by a notification unit (at least one of the notification output unit 302 and the notification transfer unit 402). In other words, the instruction transfer unit 403, the instruction receiving unit 303, 203, the imaging control unit 304, 204, the data output unit 305, 205, and the data transfer unit 405 cooperate to execute an imaging start step of starting imaging by at least one camera according to an instruction received from an external device. The instruction transfer unit 403, the instruction receiving unit 303, 203, the imaging control unit 304, 204, the data output unit 305, 205, and the data transfer unit 405 do not all cooperate with each other, but at least two or more of them may cooperate with each other to function as a control unit.

[0040] 5 is a block diagram showing an example of the configuration of the user terminal 5. The user terminal 5 is, for example, a smartphone or a tablet-type terminal device. A user who possesses the user terminal 5 is a user of the notification system of this embodiment, and is, for example, the owner (in other words, the possessor) of the vehicle 1.

[0041] The user terminal 5 includes a central processing unit (CPU) 51, a read only memory (ROM) 52, a random access memory (RAM) 53, a communication unit 54, a display unit 55, an operation unit 56, a storage unit 59, and the like.

[0042] The CPU 51 is an example of a processor, and controls each part of the user terminal 5. The ROM 52 stores various programs. The RAM 53 is a workspace for developing programs and various data. The CPU 51, the ROM 52, and the RAM 53 are connected via a bus or the like, and constitute a control unit 50 having a computer configuration.

[0043] The communication section 54 is a communication interface that connects the control section 50 and an external device (eg, the communication unit 4) so ​​that they can communicate with each other.

[0044] The display unit 55 has a display device such as an LCD (Liquid Crystal Display) and displays various information under the control of the CPU 51. The operation unit 56 has an input device such as a touch panel that is provided over the surface of the display unit 55 and outputs operation contents input via the input device to the CPU 51.

[0045] Furthermore, the user terminal 5 may include an audio output unit such as a speaker or a buzzer, and output audio under the control of the CPU 51.

[0046] The storage unit 59 has a storage medium such as an SSD (Solid State Drive) or a flash memory, and maintains the stored contents even when the power is cut off. The storage unit 59 stores programs executable by the CPU 51 and various setting information. The programs stored in the storage unit 59 also include a program 591 that realizes a dedicated application (abbreviation of application program) corresponding to the notification system of this embodiment.

[0047] The control unit 50 executes various processes by the CPU 51 operating in accordance with programs stored in the ROM 52 or the storage unit 59 and loaded into the RAM 53. In addition, the control unit 50 thereby realizes various functional units (see FIG. 6, described later).

[0048] 6 is a block diagram showing an example of functional units included in the control unit 50. The control unit 50 of the user terminal 5 includes a communication control unit 501, a display control unit 502, an input receiving unit 503, an instruction output unit 504, and the like.

[0049] The communication control unit 501 controls communication between the user terminal 5 and an external device (for example, the communication unit 4) via the communication unit 54. The display control unit 502 controls the display content of the display unit 55.

[0050] The input receiving unit 503 receives the operation contents inputted via the above-mentioned input device. The operation contents are, for example, instructions to the notification system (more specifically, the surroundings monitor 3 and the drive recorder unit 2). The instruction output unit 504 outputs the above-mentioned instructions to the communication unit 4.

[0051] 7 is a diagram showing an example of the detection range of the surroundings monitor 3. Detection ranges A, B, C, and D of the surroundings monitor 3 correspond to the ranges captured by the front camera 371, the rear camera 372, the right side camera 381, and the left side camera 382, ​​respectively.

[0052] Here, the conditions for executing notification by the notification transfer unit 402 are set in advance. The settings have default values, which may be changeable as appropriate. Changes to the settings can be accepted via an external device (e.g., the user terminal 5). The settings are stored in, for example, the communication unit 4. The settings may be stored in the surroundings monitor 3, the drive recorder unit 2, or the user terminal 5.

[0053] The conditions for the notification transfer unit 402 to execute notification are, for example, as follows. Condition 1: The number of times that the surrounding monitor 3 detects an object before executing one notification. Condition 2: Time period during which object detection and notification are performed by the surrounding monitor 3

[0054] Condition 1 is preferably used, for example, when the parking location of the vehicle 1 is a place with a relatively large number of people. In such a case, it is expected that the number of object detections will be large. For this reason, if a notification is given every time an object is detected, there is a concern that the frequency of notifications will be too high. In such a case, the number of object detections by the surroundings monitor 3 before a single notification is issued is set to, for example, 10 times. This makes it possible to control the notification to be issued when an object is detected 10 times.

[0055] Unlike the case described above, for example, if the parking area is a place with average or less than average foot traffic, it is preferable to set the number of times that the surrounding monitor 3 detects an object before issuing a single notification to a small value (for example, once).

[0056] Regarding condition 2, for example, notifications may be sent only at night because the owner is at home during the day, or object detection and notifications may not be necessary when the car is parked in a garage at home, and object detection and notifications may only be sent when the owner is out and about and using a parking lot outside the home. It is also possible to set the number of object detections by the surroundings monitor 3 before one notification is issued depending on the time period.

[0057] In addition to the above conditions 1 and 2, the following can also be set in advance. Condition 3: Selection and specification of the detection range for object detection by the surroundings monitor 3 Condition 4: Selection and specification of the imaging unit (cameras 271 to 273, 371, 372, 381, 382) to be operated by instruction from the user terminal 5

[0058] Here, Fig. 8 is a diagram showing an example of the detection range when there is a wall 6 behind and to the left of the parked vehicle 1. For example, when the vehicle 1 is parked in a parking space as shown in Fig. 8, the rear and left sides of the vehicle 1 are protected by the wall 6, so object detection in the detection ranges B and D is unnecessary. In such a case, therefore, only the detection ranges A and C can be selected as the targets for object detection. That is, in such a case, the front camera 371 and the right side camera 381 (or the front camera 271) are selected as the imaging units to be operated.

[0059] In addition to the above conditions 1 to 4, the following can also be set in advance. Condition 5: Whether or not recording of the image output by the specified imaging unit is required when imaging starts However, the default value of the above condition 5 may be “No.” That is, when imaging starts, saving of image data output by the imaging unit in the memory 29 does not start.

[0060] Similarly to conditions 1 and 2, conditions 3 to 5 also have default values, and these values ​​may be changeable as appropriate. Changes in the settings can be accepted via an external device (e.g., the user terminal 5). The settings are stored in, for example, the communication unit 4. The settings may be stored in the surroundings monitor 3, the drive recorder unit 2, or the user terminal 5.

[0061] In such a configuration, the notification system of this embodiment performs processing according to the flow illustrated below: Fig. 9 is a sequence diagram showing an example of the flow of processing performed by the notification system.

[0062] First, when the engine of the vehicle 1 is turned off (step S1), the power supply to each part of the vehicle 1 is generally stopped, but the surroundings monitor 3 operates by receiving power from a battery equipped in the vehicle 1, and the control unit 30 (specifically, the object detection unit 301) performs object detection (step S2).

[0063] When the movement of an object is detected within the designated detection range, the control unit 30 outputs a detection notification, which is information indicating that an object has been detected, to the communication unit 4 (step S3). Regarding the transmission of information, the notification output from the surroundings monitor 3 to the communication unit 4 is performed via CAN (i.e., wired communication).

[0064] When the control unit 40 of the communication unit 4 receives the detection notification, it transfers the notification to the user terminal 5 (step S4). Regarding the transmission of information, the notification is sent from the communication unit 4 to the user terminal 5, for example, via an LTE line (i.e., wireless data communication). The notification system of this embodiment uses the above-mentioned notification to warn the user of the theft of the vehicle 1 and asks for instructions on crime prevention.

[0065] When the control unit 50 of the user terminal 5 receives the notification, it causes a banner indicating the content of the notification to be displayed on the display unit 55. At that time, the control unit 50 may operate a speaker or the like provided in the user terminal 5 to emit a notification sound. Furthermore, if the screen of the user terminal 5 is in a locked state, the control unit 50 may notify the user by displaying a window-like pop-up on the lock screen.

[0066] The user terminal 5 starts the dedicated application in response to the user's operation on the display of the notification (step S5). The control unit 50, which executes the dedicated application, causes the display unit 55 to display a screen including details of the notification content and operators for receiving various instructions from the user. Examples of the operators include operators for instructing the imaging unit to start imaging or recording, and operators for selecting a camera to operate.

[0067] When the operation section 56 receives an operation on the operator, the control section 50 transmits the received instruction to the communication unit 4 (step S6).

[0068] When the communication unit 4 receives the instruction from the user terminal 5, it transfers the instruction to the drive recorder unit 2 (step S7).

[0069] When the drive recorder unit 2 receives the above-mentioned instruction, it starts capturing images using at least some of the cameras 271 to 273 (step S8). The cameras to be operated follow a predetermined setting. Note that, in the default setting, recording is not performed at this timing.

[0070] The drive recorder unit 2 outputs the video data output by the specified camera to the communication unit 4 (step S9). The video data output here is preferably not recorded video data but real-time video data.

[0071] When the communication unit 4 receives the video data from the drive recorder unit 2, the communication unit 4 transfers the video data to the user terminal 5 (step S10).

[0072] When the user terminal 5 receives the video data from the communication unit 4, the dedicated application plays back the video data. In conjunction with the playback, the dedicated application displays on the display unit 55 a screen including an operator for receiving an operation to instruct the execution of saving of the video data, and receives an operation on the operator through the operation unit 56. When the operator is operated, the user terminal 5 transmits the instruction to the communication unit 4. The communication unit 4 that has received the instruction transfers the instruction to the drive recorder unit 2. The drive recorder unit 2 that has received the instruction stores in the memory 29 the video data output by the designated imaging unit.

[0073] The above-mentioned object detection to notification (steps S2 to S4) are performed in accordance with the provisions of conditions 1 to 4. In addition, whether or not to save (in other words, record) the video data output by the designated camera (step S8) also follows a predetermined setting.

[0074] According to such an embodiment, the notification system can warn the owner of the vehicle 1 of the possibility of harm being done to the vehicle 1 by notifying the owner of the vehicle 1 via the user terminal 5. In addition, the notification system can avoid consumption of the capacity of the memory 29 and the remaining battery power due to recording immediately upon detection of an object. This can prevent the evidence images stored in the memory 29 from being erased by overwriting with recording that is not desired by the user. In addition, the notification system can receive instructions from the user who has received the notification, and store the image data output by the imaging unit when the user judges it necessary. This can leave an appropriate evidence image in the memory 29. In this way, according to this embodiment, the crime prevention of the vehicle 1 can be improved.

[0075] Furthermore, in the notification system of this embodiment, the cameras 371, 372, 381, and 382 for the surroundings monitor 3 are used for object detection, which makes it easier to detect a person approaching the vehicle 1 and eliminates the need to add a separate device.

[0076] In addition, in the notification system of this embodiment, the cameras 271-273 for the drive recorder unit 2 are used for imaging and recording after notifying the user terminal 5 of object detection, making it easier to image and record the faces of people approaching the vehicle 1.

[0077] In this embodiment, the drive recorder unit 2, the surroundings monitor 3, and the communication unit 4 are exemplified as separate devices. However, the present invention is not limited to this example, and for example, two or more of the drive recorder unit 2, the surroundings monitor 3, and the communication unit 4 may be included in the same device. In that case, the various functional units shown in this embodiment may be configured to be realized by a common ECU.

[0078] Furthermore, a case will be described where the user does not notice (or notices late) the notification displayed by the user terminal 5. If there is no response to the notification from the user terminal 5 even after a predetermined time has elapsed since the notification system output the notification, the notification system operates according to a predetermined setting, for example, as follows (1) to (3) below.

[0079] If a certain time has passed since the notification was output and no instructions have been received, the notification system: (1) Sleep mode or (2) Even if the setting value of condition 5 is “No”, image data output by the imaging unit that meets conditions 3 and 4 is stored in memory 29. Or, (3) Provide further notice.

[0080] The operation settings exemplified above in (1) to (3) have default values, similar to conditions 1 to 5, and changes to the values ​​are accepted from the user terminal 5 or the like.

[0081] In this embodiment, the object detection unit is realized by the control unit 30 of the surroundings monitor 3, and performs object detection based on the outputs of the cameras 371, 372, 381, and 382, ​​but the present invention is not limited to this. For example, the object detection unit may perform object detection using sonar instead of a camera.

[0082] Furthermore, in the first embodiment, it is more effective to limit notifications based on condition 1 and the like on the vehicle 1 side (e.g., communication unit 4) rather than on the user terminal 5 in order to reduce battery consumption in the vehicle 1. Although it is possible to limit the notification frequency by a dedicated app on the user terminal 5, it is preferable to reduce the number of times notifications are issued by the notification system, which also saves power by refraining from communication.

[0083] Furthermore, in implementing the present invention, an operation for terminating detection by the surroundings monitor 3 may be accepted from the user by displaying an operator on the user terminal 5. Also, a dedicated app may be configured to display an operator for accepting this operation on the user terminal 5 when the number of notifications within a predetermined time period exceeds a threshold.

[0084] Next, a modification of the embodiment will be described.

[0085] The second embodiment will be described with reference to Fig. 10. Since the second embodiment is a modified example of the first embodiment, the description of the same parts as in the first embodiment will be omitted and the same reference numerals will be used.

[0086] 10 is a sequence diagram showing an example of the flow of processing performed by the notification system. Steps S1 to S6 and S10 are the same as those in the first embodiment.

[0087] When the communication unit 4 receives the instruction from the user terminal 5 in step S6, it transfers the instruction to the surroundings monitor 3 (step S72).

[0088] When the surroundings monitor 3 receives the above-mentioned instruction, it starts capturing images using at least some of the cameras 371, 372, 381, and 382 (step S82). The cameras to be operated follow a predetermined setting. Note that, in the default setting, recording is not performed at this timing.

[0089] The surroundings monitor 3 outputs the video data output by the specified camera to the communication unit 4 (step S92). The subsequent processing is the same as in the first embodiment.

[0090] According to this embodiment, substantially the same effects as those of the first embodiment can be obtained.

[0091] Moreover, according to the second embodiment, the cameras 371, 372, 381, and 382 for the surroundings monitor 3 are used not only for object detection but also for capturing images after notifying the user terminal 5 of the object detection. With this configuration, it is not necessary to start the cameras 271 to 273 separately for capturing images after the notification. Therefore, it is possible to reduce consumption of the remaining battery power compared to the case where the separate cameras 271 to 273 are started after the notification based on the output of the cameras 371, 372, 381, and 382. As a result, it is expected that the operation time will be longer than that of the first embodiment.

[0092] Next, a third embodiment will be described with reference to Fig. 11. Since the third embodiment is a modified example of the first embodiment, the description of the same parts as in the first embodiment will be omitted and the same reference numerals will be used.

[0093] 11 is a sequence diagram showing an example of the flow of processing performed by the notification system. Steps S1 to S5 are the same as those in the first embodiment.

[0094] In step S5, the user terminal 5 starts a dedicated application in response to a user's operation on the display of the notification. The control unit 50 executing the dedicated application causes the display unit 55 to display a screen including details of the notification content and operators for receiving various instructions from the user. Here, in this embodiment, the control unit 55 displays an operator for instructing execution of a warning.

[0095] When the operation section 56 accepts an operation on the above-mentioned operator, the control section 50 transmits a warning instruction to the communication unit 4 (step S63).

[0096] When the communication unit 4 receives the warning instruction from the user terminal 5, it transfers the warning instruction to the vehicle 1 (step S73).

[0097] When the vehicle 1 receives the warning instruction, it executes a warning action (step S83). The warning action is intended to intimidate a person approaching the vehicle 1, who may have been detected by the object detection unit 301, with the intention of harming the vehicle 1, for crime prevention purposes, and examples of the warning action include the following. (Example 1) Turn on the headlights, interior lights, and other lighting devices equipped on vehicle 1 (Example 2) Activating the horn of vehicle 1 (Example 3) Turning on the screen of the electronic mirror 15 or the display device 16

[0098] In addition, when executing the above-mentioned warning action, the vehicle 1 is provided with a control unit that comprehensively controls the behavior of the vehicle 1 including the operation of the headlights, interior lights, horn, electronic mirror 15, and display device 16, and the control unit receives the above-mentioned warning instruction from the notification system. Note that the above-mentioned (Example 3) warning action can also be executed via the MCU 21 of the drive recorder unit 2, for example.

[0099] As described above, in the third embodiment, after the notification unit outputs a notification, the control unit receives an instruction from an external device (user terminal 5) to perform a warning action of activating at least one of the lighting, the horn, and the display device provided in the vehicle 1, and executes the warning action according to the instruction. Such a notification system can improve crime prevention.

[0100] Next, a fourth embodiment will be described with reference to Fig. 12. Since the fourth embodiment is a modified example of the first embodiment, the description of the same parts as in the first embodiment will be omitted and the same reference numerals will be used.

[0101] 12 is a sequence diagram showing an example of the flow of processing performed by the notification system. In this embodiment, steps S6 and S7 performed in the first embodiment are omitted, and an image output by an imaging unit (at least one of cameras 271 to 273) is transferred to user terminal 5 together with the output of the notification in step S3.

[0102] That is, the surroundings monitor 3 outputs the notification in step S3 and transmits an instruction to capture an image to the drive recorder unit 2 (step S74). In other words, the notification output unit 302 of the surroundings monitor 3 outputs an image capture instruction to capture an image by at least one camera to the drive recorder unit 2 in response to the detection of an object by the object detection unit 301. The subsequent processing is the same as in the first embodiment.

[0103] Next, a fifth embodiment will be described with reference to Fig. 13. Since the fifth embodiment is a modified example of the second embodiment, the description of the same parts as in the second embodiment will be omitted and the same reference numerals will be used.

[0104] FIG. 13 is a sequence diagram illustrating an example of a flow of processes performed by the notification system. In the fifth embodiment, steps S6 and S72 performed in the second embodiment are omitted, and the surroundings monitor 3 starts capturing an image by the imaging unit (at least one of the cameras 371, 372, 381, and 382) together with outputting the notification in step S3 (step S82). In other words, the control unit 30 of the surroundings monitor 3 executes capturing an image by at least one camera in response to detection of an object by the object detection unit 301. Then, the surroundings monitor 3 outputs the image data output by the designated camera to the communication unit 4 (step S92). The image data output here is real-time image. The subsequent processing is the same as in the second embodiment. In the fifth embodiment, the output image data may be displayed in real time on the user terminal 5 (step S11).

[0105] According to the fourth and fifth embodiments, the user can visually check the situation of the vehicle 1 at an earlier timing. In the fourth and fifth embodiments, steps S3 to S5 may be omitted. In other words, the notification system according to the fourth and fifth embodiments may be an imaging system.

[0106] Next, a sixth embodiment will be described with reference to Figures 14 and 15. Since the sixth embodiment is a modified example of the first embodiment, the description of the same parts as in the first embodiment will be omitted and the same reference numerals will be used.

[0107] 14 is a block diagram showing an example of the configuration of a notification system according to the sixth embodiment. The drive recorder unit 200 differs from the drive recorder unit 2 of the first embodiment in that it further includes an I / F (interface) 291 and a deserializer 281.

[0108] The I / F 291 connects the surrounding monitor 3 and the deserializer 281 , and mediates communication between the surrounding monitor 3 and the image processing unit 22 . When the deserializer 281 receives a control signal from the image processing unit 22, it receives the video transmitted from the surrounding monitor 3. At this time, the deserializer 281 converts the video transmitted as a serialized video signal into parallel data. Furthermore, the deserializer 281 transmits the video, which has become parallel data, to the image processing unit 22 as a video signal. The deserializer 281 is an example of a receiving unit.

[0109] In such a configuration, the notification system of the sixth embodiment performs processing in the following exemplary flow: Fig. 15 is a sequence diagram showing an example of the flow of processing performed by the notification system. Steps S1 to S6 are the same as those in the first embodiment.

[0110] When the communication unit 4 receives the instruction from the user terminal 5 in step S6, it transfers the instruction to the surroundings monitor 3 (step S75). The communication unit 4 also transfers the instruction to the drive recorder unit 2 (step S76).

[0111] When the surroundings monitor 3 receives the above-mentioned instruction, it starts capturing images using at least some of the cameras 371, 372, 381, and 382 (step S82). The surroundings monitor 3 transmits the video data output by the designated camera to the drive recorder unit 2 (step S93).

[0112] The drive recorder unit 2 records the video data received from the surroundings monitor 3 (step S83). The drive recorder unit 2 outputs the video data received from the surrounding monitor 3 to the communication unit 4 (step S94). After completing recording of the video data received from the surrounding monitor 3, the drive recorder unit 2 may output the video data to the communication unit 4, or may record the video data and output the video data to the communication unit 4 in parallel. The subsequent step S10 is the same as in the first embodiment.

[0113] According to the sixth embodiment, substantially the same effects as those of the first embodiment can be obtained. Moreover, according to the sixth embodiment, the cameras 371, 372, 381, and 382 for the surroundings monitor 3 are used not only for object detection but also for capturing images after notifying the user terminal 5 of the object detection. This makes it possible to obtain substantially the same effects as the second embodiment.

[0114] According to the sixth embodiment, the images captured by the cameras 371, 372, 381, and 382 for the surroundings monitor 3 are recorded in the drive recorder unit 2. The cameras 371, 372, 381, and 382 for the surroundings monitor 3 have a wider imaging range and fewer blind spots. This makes it easier to capture images of people approaching the vehicle 1. In the sixth embodiment, after transferring the instruction to the drive recorder unit 2 in step S76, the surroundings monitor 3 may transmit the video data output by the designated camera to the drive recorder unit 2 after receiving a startup notification from the drive recorder unit 2 notifying that the drive recorder unit 2 has started up. Alternatively, after transferring the instruction to the drive recorder unit 2 in step S76, the surroundings monitor 3 may transmit the video data output by the designated camera to the drive recorder unit 2 after a predetermined time has elapsed.

[0115] Next, a seventh embodiment will be described with reference to Fig. 16. Since the seventh embodiment is a modified example of the sixth embodiment, the description of the same parts as in the sixth embodiment will be omitted and the same reference numerals will be used. The notification system of the seventh embodiment performs processing in the following exemplary flow: Fig. 16 is a sequence diagram showing an example of the flow of processing performed by the notification system. Steps S1 to S5 are the same as those in the first embodiment.

[0116] The surroundings monitor 3 outputs a detection notification in step S3, and transmits an instruction to the drive recorder unit 2 to output a video (step S77). The surroundings monitor 3 starts capturing an image using the imaging unit (at least one of the cameras 371, 372, 381, and 382) (step S82). In other words, the control unit 30 of the surroundings monitor 3 executes capturing an image using at least one camera in response to the detection of an object by the object detection unit 301. The subsequent steps are the same as those in the sixth embodiment.

[0117] According to the seventh embodiment, it is possible to obtain substantially the same effects as those of the sixth embodiment. Furthermore, according to the seventh embodiment, the user can visually check the situation of the vehicle 1 at an earlier timing. In the seventh embodiment, steps S3 to S5 may be omitted. In other words, the notification system according to the seventh embodiment may be an imaging system.

[0118] Although some embodiments of the present disclosure have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as well as in the scope and spirit of the invention. It is also possible to implement a combination of multiple embodiments. [Explanation of symbols]

[0119] 1 …vehicle, 10 ... body, 11 ... windshield, 12 ... rear window, 13 ... side mirror, 15 ... electronic mirror, 16 ... display device, 2 ... Drive recorder unit, 20 ... control unit, 203: instruction receiving unit, 204: imaging control unit, 205: data output unit, 21 ...MCU, 22 ...Image processing unit, 23 ...CAN driver, 241...I / F, 251~253...Deserializer, 261~263...I / F, 271...front camera, 272...rear camera, 273...indoor camera, 281…deserializer, 29…Memory, 291…I / F, 3...Ambient monitor, 30 ... control unit, 301: object detection unit; 302: notification output unit; 303: instruction receiving unit, 304: imaging control unit, 305: data output unit, 31 ... ECU, 371...front camera, 372...rear camera, 381...right side camera, 382...left side camera, 4 ... communication unit (an example of a notification device); 40: control unit, 402: notification transfer unit, 403: instruction transfer unit, 405: data transfer unit, 41 ... ECU, 5...user terminal, 50 ... control unit, 501: communication control unit; 502: display control unit; 503: input reception unit; 504: instruction output unit; 51...CPU, 52...ROM, 53...RAM, 54 ... communication unit, 55 ... display unit, 56 ... operation unit, 59 ... memory unit, 591 ... program, 6. Wall

Claims

1. At least one camera that outputs an image capturing at least a portion of the surroundings of the vehicle; An object detection unit that detects objects around the vehicle; a notification unit that outputs a detection notification, which is information notifying an external device that an object has been detected by the object detection unit, to an external device; a control unit that, when receiving an instruction for capturing an image by the at least one camera from the external device after the output of the detection notification, executes capturing an image by the at least one camera in accordance with the instruction; A notification system comprising:

2. After outputting the detection notification, the control unit receives an instruction from the external device for a warning action to activate at least one of a light, a horn, and a display device provided in the vehicle, and executes the warning action in accordance with the instruction. The notification system of claim 1 .

3. A first device including the object detection unit and the control unit; a second device including the notification unit; The at least one camera includes a downward imaging camera that images an area below the mounting position. The notification system of claim 1 .

4. A receiving unit that receives an image captured by the at least one camera from the first device; A memory for recording the image; an output unit that outputs the image to the second device; a third device comprising: The notification system of claim 3.

5. The at least one camera includes at least one of a front camera that captures an image in front of the vehicle, a rear camera that captures an image in the rear of the vehicle, and an interior camera that captures an image in the interior of the vehicle. The notification system according to claim 1 or 2.

6. a notification unit that, when an object is detected around the vehicle, outputs a detection notification, which is information notifying an external device that the object has been detected; a reception unit that receives, after the detection notification is output to the external device, an instruction to start capturing an image of at least a part of the surroundings of the vehicle from the external device; A notification device comprising:

7. A detection step of detecting an object around the vehicle; a notification step of outputting a detection notification, which is information notifying that the object has been detected, to an external device; an imaging start step of, when receiving an instruction for imaging by at least one camera from the external device after the output of the detection notification, starting imaging by the at least one camera in accordance with the instruction; Notification methods, including:

8. At least one camera that outputs an image capturing at least a portion of the surroundings of the vehicle; An object detection unit that detects objects around the vehicle; an instruction output unit that outputs an image capturing instruction for instructing at least one camera to capture an image in response to the object being detected by the object detection unit; a control unit that executes imaging by the at least one camera in accordance with the imaging instruction; an image output unit that outputs an image output by the at least one camera; An imaging system comprising:

9. At least one camera that outputs an image capturing at least a portion of the surroundings of the vehicle; An object detection unit that detects objects around the vehicle; a control unit that performs imaging using the at least one camera in response to an object being detected by the object detection unit; an output unit that outputs an image output by the at least one camera; An imaging system comprising:

10. A first device including the object detection unit and the control unit; a second device including the output unit; The at least one camera includes a downward imaging camera that images an area below the mounting position. The imaging system according to claim 9 .

11. The second device further includes a receiving unit that receives an image captured by the at least one camera from the first device, and a recording unit that records the image, The output unit outputs the image to a third device. The imaging system according to claim 10.

Images