Vehicle surroundings monitoring device

Abstract

To secure and maintain sufficient evidence while suppressing power consumption by eliminating unnecessary imaging and recording.SOLUTION: An image acquisition unit configured to acquire an image of the periphery of the vehicle captured by the camera, a moving body detection unit configured to detect a moving body moving in the periphery of the vehicle based on the image and calculate a distance between the moving body and the vehicle, and an imaging control unit configured to control the camera according to the distance between the moving body and the vehicle, To provide a periphery monitoring device for a vehicle that estimates an arrival time at which a moving body arrives at the vehicle when a distance between the moving body and the vehicle is less than a predetermined distance, and controls a camera to start imaging a moving image at an imaging start time determined on the basis of the arrival time.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a vehicle periphery monitoring device that monitors the periphery of a vehicle, and more particularly to a vehicle periphery monitoring device that captures and records images of the periphery of a vehicle when the vehicle is parked or stopped. [Background technology]

[0002] In recent years, vehicles equipped with parking monitoring functions that capture and record images of the area around the vehicle while the vehicle is parked, i.e., while stopped or parked, are becoming more common as a measure against crimes such as hit-and-runs and car break-ins, and as a way to secure and preserve evidence in the event of any trouble or accident involving such crimes. As an example of a parking monitoring function, Patent Document 1 discloses that the recording function, which records video data captured while the vehicle is parked, can be set to a continuous storage mode, as well as a time lapse mode in which images are captured intermittently and recorded, and a moving object detection mode in which images are captured and recorded in response to the detection of a moving object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-20553 Summary of the Invention [Problem to be solved by the invention]

[0004] In order to maximize the effectiveness of the parking monitoring function described above in terms of crime prevention and securing and preserving evidence, it is preferable to constantly capture and record images of the area around the vehicle. However, because the parking monitoring function operates using power stored in the vehicle's battery, the longer the time spent capturing and recording images, the greater the battery power consumption, which could affect driving. On the other hand, when using the time lapse mode or motion detection mode as in Patent Document 1, power consumption can be reduced compared to when continuous imaging and recording is performed, but it may not be sufficient to secure and preserve evidence.

[0005] The present invention addresses this situation by eliminating unnecessary imaging and recording to reduce power consumption while still ensuring and preserving sufficient evidence. [Means for solving the problem]

[0006] In order to solve the above problems, a vehicle surroundings monitoring device according to the present invention has the following configuration. That is, one aspect of the present invention provides a vehicle periphery monitoring device that monitors the periphery of the vehicle based on images captured by a camera, comprising: an image acquisition unit that acquires images of the periphery of the vehicle captured by the camera; a moving object detection unit that detects a moving object moving around the vehicle based on the images and calculates the distance between the moving object and the vehicle; and an imaging control unit that controls the camera in accordance with the distance between the moving object and the vehicle, wherein the imaging control unit detects the moving object based on a plurality of images captured by the camera at a predetermined imaging period, and if the distance between the moving object and the vehicle is less than a predetermined distance, estimates the arrival time at which the moving object will arrive at the vehicle and controls the camera to start capturing video at an imaging start time determined based on the arrival time. [Effects of the Invention]

[0007] A vehicle surroundings monitoring device having such characteristics can secure and preserve sufficient evidence while reducing power consumption by eliminating unnecessary imaging and recording. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is an explanatory diagram showing a schematic configuration of a vehicle control system including a vehicle surroundings monitoring device according to an embodiment of the present invention; [Figure 2] 1 is an explanatory diagram showing a schematic configuration of a first ECU and a second ECU that function as a vehicle surroundings monitoring device according to an embodiment of the present invention; [Figure 3]FIG. 10 is an explanatory diagram illustrating an example in which a moving object is present around the vehicle. [Figure 4] FIG. 2 is a sequence diagram showing a process performed by the vehicle surroundings monitoring device according to the first embodiment of the present invention. [Figure 5] 4 is a flowchart showing processing based on a still image by a first ECU in the vehicle surroundings monitoring device according to the first embodiment of the present invention. [Figure 6] 5 is a flowchart showing a process based on a moving image by a second ECU in the vehicle surroundings monitoring device according to the first embodiment of the present invention. [Figure 7] FIG. 10 is a sequence diagram showing a process performed by a vehicle surroundings monitoring device according to a second embodiment of the present invention. [Figure 8] 10 is a flowchart showing a process based on a still image by a first ECU in a vehicle surroundings monitoring device according to a second embodiment of the present invention. [Figure 9] 10 is a flowchart showing a process based on a moving image by a first ECU in a vehicle surroundings monitoring device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same reference numerals in different drawings denote parts with the same functions, and duplicated descriptions in each drawing will be omitted as appropriate.

[0010] The vehicle surroundings monitoring device according to this embodiment functions as a part of a vehicle control system 1 mounted on a vehicle 100. As shown in Fig. 1, the vehicle control system 1 includes a plurality of sensors that acquire various information indicating the driving state of the vehicle 100 and the environment inside and outside the vehicle, various electronic devices required for driving the vehicle 100, and a plurality of ECUs (Electronic Control Units) that control these sensors and electronic devices.

[0011] 1 illustrates a first ECU 11, a second ECU 21, and a third ECU 31 among the multiple ECUs, while omitting the other ECUs. Also, among the multiple sensors and electronic devices, a camera 41 and a microphone 42 are illustrated, while the other sensors and electronic devices are omitted.

[0012] The sensors, electronic devices, and ECUs are interconnected so as to be able to communicate with each other via an in-vehicle network 3 such as a CAN (Controller Area Network) or a LIN (Local Interconnect Network) and a central gateway (CGW) 4 as a relay device.

[0013] In the vehicle control system 1, information acquired by each sensor is output to the in-vehicle network 3, and information indicating the operating state of an electronic device to be controlled (hereinafter referred to as a controlled device) is output from each ECU to the in-vehicle network 3. Furthermore, each ECU controls the operation of the controlled device based on information acquired from each sensor and other ECUs via the in-vehicle network 3.

[0014] In the vehicle control system 1, each ECU is classified into one of multiple functional systems, such as a powertrain system, a chassis system, a body system, an information system, or an ADAS system, depending on the function of the device to be controlled, and is connected to a bus line for the functional system to which it belongs.

[0015] The powertrain system is categorized into ECUs that control driving-related equipment such as the traction motor and transmission; the chassis system is categorized into ECUs that perform driving safety control such as brakes, suspension, and steering; the body system is categorized into ECUs that perform control to improve passenger convenience and comfort such as air conditioning and headlights; the information system is categorized into ECUs that control equipment that provides passengers with information visually or audibly such as audio and navigation systems; and the ADAS system is categorized into ECUs that assist in driving the vehicle 100, i.e., ECUs that control sensors and equipment necessary to realize each function of ADAS (Advanced driver-assistance systems).

[0016] Each ECU includes a processor, such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), which executes various processes. Each ECU also includes volatile storage elements, such as RAM (Random Access Memory) that temporarily processes data used by the processor, and non-volatile storage elements, such as ROM (Read Only Memory) that stores programs executed by the processor. Note that some or all of the operations executed by each ECU can also be implemented by hardware, such as an ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit).

[0017] Each ECU operates using power supplied from a battery (not shown) mounted on vehicle 100, and switches between a normal operating state (wake-up) in which each control target device is controlled and a power saving state (sleep) in which each control target device is not controlled. That is, in the normal operating state, each ECU is supplied with power from the battery necessary to control the operation of each control target device. On the other hand, in the power saving state, the power supplied from the battery to each ECU is minimized.

[0018] In normal operation, the processor in each ECU operates at an operating frequency that corresponds to the controlled device and the control content, and executes various processes by appropriately switching between multiple operating modes with different operating frequencies. At this time, each ECU is supplied with power according to the operating frequency.

[0019] In the vehicle surroundings monitoring device of this embodiment, one or more of the multiple ECUs of the vehicle control system 1 control the camera 41 and monitor the surroundings of the vehicle 100 based on images captured by the camera 41.

[0020] The camera 41 captures images of the periphery of the vehicle 100, that is, the outside of the vehicle 100 in all directions (360° around the vehicle 100). As the camera 41, a single camera capable of capturing images of the outside of the vehicle 100 in all directions, or, for example, a combination of a camera capturing images in a range of 180° in front of the vehicle 100 and a camera capturing images in a range of 180° behind the vehicle 100, can be used.

[0021] The camera 41 can change its imaging cycle and can capture still images or videos with different imaging cycles by switching between them as needed. Audio data collected by the microphone 42 can be added to the videos captured by the camera 41. The still images and videos captured by the camera 41 are output to the in-vehicle network 3. The microphone 42 collects various sounds generated around the vehicle 100 and outputs them to the in-vehicle network 3 as audio data.

[0022] Hereinafter, as a first embodiment, an example in which two ECUs, a first ECU 11 and a second ECU 21, function as a vehicle surroundings monitoring device will be described, and as a second embodiment, an example in which the first ECU 11 functions as a vehicle surroundings monitoring device will be described. In the following description, detailed description and illustration of ECUs, sensors and electronic devices that are not involved in the function of the vehicle surroundings monitoring device will be omitted even if they are included in the vehicle control system 1.

[0023] (First Example) In this embodiment, an example is described in which the first ECU 11 and the second ECU 21 function as a vehicle surroundings monitoring device, and the first ECU 11 and the second ECU 21 control the camera 41 and monitor the surroundings of the vehicle 100 based on images captured by the camera 41. Specifically, the camera 41 is controlled by the first ECU 11 when capturing a still image, and is controlled by the second ECU 21 when capturing a moving image. The first ECU 11 and the second ECU 21 will be described below.

[0024] As shown in FIG. 2, the first ECU 11 includes a CPU 111, a ROM 112, and a RAM 113, and the CPU 111 executes various processes based on programs stored in the ROM 112. The ROM 112, which is provided as a non-volatile memory element, stores a program for controlling the camera 41 and recording images captured by the camera 41 to monitor the surroundings of the vehicle 100, as well as various data required to execute this program. The RAM 113 provided as a volatile storage element is used as a work area when the CPU 111 executes various processes, and also temporarily stores images captured by the camera 41.

[0025] The CPU 111 loads a program stored in the ROM 112 into a memory such as the RAM 113 and executes it, thereby functioning as the image acquisition unit 121, the moving object detection unit 122, the imaging control unit 123, and the recording control unit 124 shown in Fig. 2. The image acquisition unit 121, the moving object detection unit 122, the imaging control unit 123, and the recording control unit 124 will be described below.

[0026] The image acquisition unit 121 acquires an image of the periphery of the vehicle 100 captured by the camera 41, particularly a still image, and outputs it to the RAM 113. The moving object detection unit 122 detects a moving object moving around the vehicle 100 based on the still images acquired by the image acquisition unit 121, and calculates the distance between the moving object and the vehicle 100. The moving object detection unit 122 detects a moving object moving around the vehicle 100 based on, for example, the difference between a plurality of still images acquired successively over time by the image acquisition unit 121. Alternatively, the moving object detection unit 122 may detect a moving object by analyzing, by machine learning, a plurality of still images acquired successively by the image acquisition unit 121. Furthermore, the moving object detection unit 122 may calculate the moving direction of the moving object and determine whether the moving object is moving in a direction approaching the vehicle 100.

[0027] The imaging control unit 123 controls the camera 41 in accordance with the distance between the moving object detected by the moving object detection unit 122 and the vehicle 100. In this case, the closer the moving object is to the vehicle 100, the shorter the imaging cycle of the camera 41 is changed to make it possible to grasp the movement of the moving object more clearly. When the distance between the moving object and the vehicle 100 is less than a predetermined distance, the imaging control unit 123 calculates the moving speed of the moving object and estimates the arrival time at which the moving object will arrive at the vehicle 100 based on the moving speed. Information about the moving object, including the estimated arrival time, the moving speed of the moving object, and the distance between the moving object and the vehicle 100, is notified to the second ECU 21.

[0028] Note that the imaging control unit 123 may not estimate the arrival time or calculate the movement speed when the distance between the moving object and the vehicle 100 is less than a predetermined distance but the moving direction of the moving object is not a direction approaching the vehicle 100. In other words, when the distance between the moving object and the vehicle 100 is less than a predetermined distance and the moving direction of the moving object is a direction approaching the vehicle 100, the imaging control unit 123 may calculate the movement speed of the moving object and estimate the arrival time at which the moving object will arrive at the vehicle 100 based on the movement speed. In this way, it is possible to reduce power consumption by not performing unnecessary processing on a moving object that simply passes by the periphery of the vehicle 100 or a moving object that moves in a direction away from the periphery of the vehicle 100, for example.

[0029] The recording control unit 124 converts the still image acquired by the image acquisition unit 121 and temporarily stored in the RAM 113 into a predetermined file format and records it in the ROM 112 or a removable memory card (not shown) or the like.

[0030] 2, like the first ECU 11, the second ECU 21 includes a CPU 211, a ROM 212, and a RAM 213, and the CPU 211 executes various processes based on programs stored in the ROM 212. The second ECU 21 differs from the first ECU 11, which performs processes such as capturing still images and detecting moving objects, in that the second ECU 21 causes the camera 41 to capture a moving image, determines the distance between a moving object and the vehicle 100 based on the moving image, and stores the moving image in the ROM 212, etc.

[0031] Therefore, the ROM 212, which is a non-volatile memory element in the second ECU 21, stores a program for monitoring the surroundings of the vehicle 100 by controlling the camera 41 to capture and record video, and various data required to execute this program. The RAM 213, which is a volatile storage element, is used as a work area when the CPU 211 executes various processes, and also temporarily stores moving images captured by the camera 41.

[0032] The second ECU 21 handles moving images in various processes, and therefore executes processes with a higher load than the first ECU 11 which handles still images. For this reason, the second ECU 21 is driven at a higher operating frequency than the first ECU 11.

[0033] In the second ECU 21, the CPU 211 reads a program stored in the ROM 212 into a memory such as the RAM 213 and executes it, thereby functioning as the image acquisition unit 221, the moving object detection unit 222, the imaging control unit 223, and the recording control unit 224 shown in Figure 2.

[0034] The image acquisition unit 221 acquires images of the surroundings of the vehicle captured by the camera 41, particularly video images, and outputs the images to the RAM 213. The moving object detection unit 222 detects a moving object moving around the vehicle 100 based on the video acquired by the image acquisition unit 221, and calculates the distance between the moving object and the vehicle 100. The moving object detection unit 222 detects a moving object moving around the vehicle 100 based on the difference between a predetermined number of frames of the video acquired by the image acquisition unit 221. Alternatively, the moving object detection unit 222 may detect a moving object by analyzing the video using machine learning.

[0035] The imaging control unit 223 acquires moving object information from the first ECU 11, determines the start time for the camera 41 to capture video based on the arrival time of the moving object contained in the moving object information at the vehicle 100, and controls the camera 41 to start capturing video at the determined start time for capturing video. The imaging control unit 223 can set the imaging start time to, for example, a time a predetermined time before the estimated arrival time of the moving object at the vehicle 100 or a time approximately the same as the estimated arrival time. The imaging control unit 223 can appropriately determine the imaging start time depending on the location and environment where the vehicle 100 is parked.

[0036] In addition, after the camera 41 starts capturing video, the imaging control unit 223 continues capturing video by the camera 41 until the distance between the moving object calculated by the moving object detection unit 222 and the vehicle 100 again becomes equal to or greater than the predetermined distance.

[0037] The recording control unit 224 converts the video acquired by the image acquisition unit 221 and temporarily stored in the RAM 223 into a predetermined file format, and records and saves the video in the ROM 212 or a removable memory card (not shown). The recording control unit 224 may output the saved video to a server (not shown) that is capable of communicating with the vehicle 100 and that manages various information related to the vehicle 100, and the video may be managed by the server.

[0038] As described above, the first ECU 11 mainly controls the camera 41 to capture still images and performs processing to detect moving objects based on the captured still images, while the second ECU 21 mainly controls the camera 41 to capture video and transmit the video to a server as needed. Due to these differences in processing, the CPU 111 of the first ECU 11 and the CPU 211 of the second ECU 21 require different operating frequencies and therefore consume different amounts of power to execute various processes. In other words, the CPU 211 is required to operate at a higher operating frequency than the CPU 111, which increases the amount of power required to execute the processes. Therefore, for example, the first ECU 11 may be an ECU belonging to a body system that consumes relatively little power, while the second ECU 21 may be an ECU belonging to an ADAS system or information system that consumes relatively more power.

[0039] Alternatively, the CPU 111 and the CPU 211 may be configured to be capable of switching between a plurality of operation modes with different operating frequencies. In this case, the first ECU 11 uses an operation mode with a relatively low operating frequency among the plurality of operation modes for processing based on a still image. The second ECU 21 uses an operation mode with a relatively high operating frequency among the plurality of operation modes for processing based on a moving image.

[0040] An example of processing in the vehicle surroundings monitoring device configured as above will be specifically described with reference to FIG. Fig. 3 is an explanatory diagram of an example in which a moving object M is present around the vehicle 100. In Fig. 3, the vehicle 100 is at the center, and an area that is less than a first distance RD1 from the vehicle 100 is indicated by a two-dot chain line, and a distance that is less than a second distance RD2 from the vehicle 100 is indicated by a one-dot chain line.

[0041] As shown in Figure 3, the moving object M moves from a position greater than the second distance RD2 along arrow A1 to an area greater than the first distance RD1 but less than the second distance RD2, then moves along arrow A2 to an area less than the first distance RD1, and then moves along arrow A3 in a direction approaching vehicle 100 in an area less than the first distance RD1.

[0042] For example, the imaging control unit 123 of the first ECU 11 sets the imaging cycle of the camera 41 to the second cycle in an area that is equal to or greater than the second distance RD2 from the vehicle 100. If a moving object M is detected based on second images captured at the second cycle and the position of the moving object M is in an area that is equal to or greater than the second distance RD2 from the vehicle 100, the imaging control unit 123 continues imaging at the second cycle by the camera 41. If it is detected based on second images acquired thereafter that the moving object M has moved to an area that is less than the second distance RD2 from the vehicle 100, the imaging control unit 123 sets the moving object M as a monitoring target and changes the imaging cycle by the camera 41 to the first cycle, which is shorter than the second cycle.

[0043] When the imaging control unit 123 detects that the moving object M, which is the monitoring target, has moved to an area less than the first distance RD1 from the vehicle 100 based on the first image captured in the first period, the imaging control unit 123 calculates the movement speed of the moving object M and estimates the arrival time at which the moving object M will arrive at the vehicle 100 based on the movement speed. The imaging control unit 123 notifies the second ECU 21 of information about the moving object M, including the estimated arrival time, the movement speed of the moving object M, and the distance between the moving object M and the vehicle 100.

[0044] Thereafter, the imaging control unit 223 of the second ECU 21 acquires moving object information regarding the moving object M from the first ECU 11, determines the start time for capturing the video based on the arrival time of the moving object M at the vehicle 100, and controls the camera 41 to start capturing the video at the determined time. The imaging control unit 223 causes the camera 41 to continue capturing the video until the moving object M moves in a direction away from the vehicle 100 and again reaches an area that is equal to or greater than the first distance RD1.

[0045] It is also possible to imagine a case where, for example, the moving object M temporarily moves to an area that is equal to or greater than the first distance RD1 and less than the second distance RD2 from the vehicle 100, but then moves along the arrow A4 and is detected to have moved back to an area that is equal to or greater than the second distance RD. In such a case, the imaging control unit 123 changes the imaging cycle of the camera 41 from the first cycle back to the second cycle.

[0046] Furthermore, for example, when a moving object M detected in an area less than the first distance RD1 from the vehicle 100 moves along the arrow A5, the movement direction of the detected moving object M is the direction along the arrow A5, that is, the direction away from the vehicle 100. In this case, since the movement direction of the moving object M is not a direction approaching the vehicle 100, the imaging control unit 123 may not estimate the arrival time of the moving object M at the vehicle 100 or calculate the movement speed, and may cause the camera 41 to continue capturing images at the first cycle without switching to capturing video.

[0047] The process of controlling the camera 41 and recording images captured by the camera 41 to monitor the periphery of the vehicle 100 by the vehicle periphery monitoring device configured as above will be described below with reference to the sequence diagram of Fig. 4 and the flowcharts of Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing the process by the first ECU 11, and Fig. 6 is a flowchart showing the process by the second ECU 21.

[0048] 5 and 6, a series of processes from "START" to "END" are repeatedly performed at a predetermined cycle (time interval) while the vehicle 100 is parked. In the following, a process of determining whether the moving object is moving in a direction approaching the vehicle 100 at the start of capturing a video will be described.

[0049] As shown in FIG. 5, when the vehicle 100 is parked (YES in step S111) and the monitoring mode is ON (YES in step S112), the first ECU 11 proceeds to the next step S113 and subsequent steps, and performs processing to record images captured by the camera 41 and monitor the surroundings of the vehicle 100.

[0050] The first ECU 11 causes the camera 41 to capture images at a second imaging cycle and acquire a second image (step S113). Subsequently, the first ECU 11 monitors the presence or absence of a moving object moving around the vehicle 100 based on the second image (step S114). If a moving object moving around the vehicle 100 is detected (YES in step S114), the first ECU 11 calculates a distance RD from the vehicle 100 to the detected moving object (step S115). If the calculated distance RD is equal to or greater than the second distance RD2, the first ECU 11 does not change the imaging cycle of the camera 41 and continues capturing the second image at the second cycle (NO in step S116). The processing from step S113 to step S116 corresponds to the loop processing at the top of the sequence diagram in FIG. 4.

[0051] On the other hand, if the calculated distance RD is less than the second distance RD2 (YES in step S116), that is, if the moving object is present in an area less than the second distance RD2 from the vehicle 100, the first ECU 11 monitors the moving object and changes the imaging period of the camera 41 to the first period (step S117).

[0052] Thereafter, the first ECU 11 acquires a first image captured by the camera 41 in a first cycle (step S118), and calculates the moving direction of the moving object being monitored and the distance RD from the vehicle 100 (step S119). If the distance RD is equal to or greater than the first distance RD1 (NO in step S120), or if the moving direction is not a direction approaching the vehicle 100 (NO in step S121), the first ECU 11 does not change the imaging cycle of the camera 41, and continues capturing the first image in the first cycle. The processing from step S118 to step S121 corresponds to the loop processing in the middle section in the sequence diagram of FIG. 4.

[0053] If the distance RD calculated in step S119 is less than the first distance RD1 (YES in step S120) and the moving direction of the moving object is a direction approaching the vehicle 100 (YES in step S121), preparations are made to switch to monitoring of the periphery of the vehicle 100 based on the video. That is, the first ECU 11 calculates the moving speed of the moving object based on the first image, and estimates the arrival time of the moving object at the vehicle 100 based on the moving speed and the distance RD (step S122). The first ECU 11 outputs moving object information including the estimated arrival time and the position of the moving object to the second ECU 21 (step S123), and causes the process to proceed to processing by the second ECU 21.

[0054] 6, the second ECU 21 receives moving object information from the first ECU 11 (step S211), and is activated by the reception of this moving object information, i.e., transitions from a sleep state to a normal operation state. The second ECU 21 then calculates the start time of capturing a video based on the moving object's speed and arrival time at the vehicle 100 from the received moving object information (step S212). Thereafter, the second ECU 21 controls the camera 41 to start capturing a video at the capture start time (step S213), and monitors the distance RD between the moving object and the vehicle 100 while continuing to capture the video (step S215).

[0055] The imaging control unit 223 of the second ECU 21 continues capturing the video until the moving object moves to an area that is at least the first distance RD1 from the vehicle 100 (NO in step S215). If the moving object moves to an area that is at least the first distance RD1 from the vehicle 100 (YES in step S215), the imaging control unit 223 ends capturing the video by the camera 41 and saves the captured video in a predetermined file format (step S216).

[0056] The video saved in step S216 may be transmitted to a server (not shown) and managed in the server, or in both the ROM 212 of vehicle 100 or the removable memory card and the server. The processing from step S214 to step S216 corresponds to the loop processing in the lower part of the sequence diagram in FIG. 4.

[0057] (Second Example) In this embodiment, the first ECU functions as a vehicle periphery monitoring device, and the first ECU 11 independently controls the camera 41 and monitors the periphery of the vehicle 100 based on images captured by the camera 41. The first ECU 11 in this embodiment differs from the first ECU 11 in the first embodiment, which performs processing based on still images, in that the first ECU 11 performs processing based on still images and moving images captured by the camera 41, but is otherwise the same as the first ECU 11 in the first embodiment. Therefore, while referring to FIG. 2, descriptions of the first ECU 11 in this embodiment that overlap with the first ECU 11 in the first embodiment will be simplified or omitted.

[0058] As shown in FIG. 2, the first ECU 11 includes a CPU 111, a ROM 112, and a RAM 113, and the CPU 111 executes various processes based on programs stored in the ROM 112. The ROM 112 provided as a non-volatile storage element stores a program for controlling the camera 41 and recording images (still images and video) captured by the camera 41 to monitor the periphery of the vehicle 100, as well as various data required to execute this program. In this embodiment, the first ECU 11 performs processing based on the images (still images and video) acquired from the camera 41, and therefore the ROM 112 stores the programs required for this processing. The RAM 113 provided as a volatile storage element is used as a work area when the CPU 111 executes various processes, and also temporarily stores images captured by the camera 41.

[0059] The CPU 111 then loads and executes a program stored in the ROM 112 into a memory such as the RAM 113, thereby functioning as an image acquisition unit 121, a moving object detection unit 122, an imaging control unit 123, and a recording control unit 124 shown in FIG. 2.

[0060] The CPU 111 can be driven in a plurality of operating modes with different operating frequencies, including an "L mode" that operates at a relatively low operating frequency and an "H mode" that operates at a relatively high operating frequency. In the following processes, the CPU 111 switches between the operating modes as appropriate to vary the operating frequency depending on the processing content, such that processing based on still images is driven in "L mode" and processing based on moving images is driven in "H mode."

[0061] The image acquisition unit 121 acquires still images and videos of the periphery of the vehicle 100 captured by the camera 41, and outputs them to the RAM 113. The moving object detection unit 122 detects a moving object moving around the vehicle 100 based on the still images or videos acquired by the image acquisition unit 121, and calculates the distance between the moving object and the vehicle 100.

[0062] The imaging control unit 123 controls the camera 41 in accordance with the distance between the moving object detected by the moving object detection unit 122 and the vehicle 100. In this case, the closer the moving object is to the vehicle 100, the shorter the imaging cycle of the camera 41 is changed, thereby enabling the movement of the moving object to be more clearly understood. When the distance between the moving object and the vehicle 100 is less than a predetermined distance, the imaging control unit 123 estimates the arrival time at which the moving object will arrive at the vehicle 100 and determines the start time for imaging the video. Then, the imaging control unit 123 controls the camera 41 to start imaging the video at the imaging start time. After the camera 41 starts capturing the video, the imaging control unit 123 causes the camera to continue capturing the video until the distance between the moving object detected based on the video and the vehicle 100 again becomes equal to or greater than the predetermined distance.

[0063] The recording control unit 124 converts the still images and videos temporarily stored in the RAM 113 into a predetermined file format and records and saves them in the ROM 112 or a removable memory card (not shown). When saving of the video is complete, the recording control unit 124 may output the video to a server (not shown) that can communicate with the vehicle 100 and manages various information related to the vehicle 100, and have the video managed by the server.

[0064] The process of controlling the camera 41 and recording images captured by the camera 41 to monitor the periphery of the vehicle 100 by the vehicle periphery monitoring device configured as above will be described below with reference to the sequence diagram of Fig. 7 and the flowcharts of Fig. 8 and Fig. 9. Fig. 8 is a flowchart showing the process driven by the CPU 111 in "L mode," and Fig. 9 is a flowchart showing the process driven by the CPU 111 in "H mode."

[0065] Fig. 9 is a flowchart showing the processing in "H mode" which is performed consecutively from the processing in "L mode" of CPU 111 shown in Fig. 8, and the series of processing from "START" in the flowchart shown in Fig. 8 to "END" in the flowchart shown in Fig. 9 is repeatedly performed at a predetermined cycle (time interval) while vehicle 100 is parked or stopped. In the following explanation, the processing for determining whether the moving object is moving in a direction approaching vehicle 100 at the start of capturing a video will be explained.

[0066] As shown in FIG. 8, when the vehicle 100 is parked (YES in step S311) and the monitoring mode is ON (YES in step S312), the first ECU 11 proceeds to the next step S313 and subsequent steps, and performs processing to record images captured by the camera 41 and monitor the surroundings of the vehicle 100.

[0067] The first ECU 11 first drives the CPU 111 in "L mode" with a relatively low operating frequency, causes the camera 41 to capture images at a second imaging cycle, and acquires a second image (step S313), and monitors the presence or absence of a moving object moving around the vehicle 100 based on the second image (step S314). If a moving object moving around the vehicle 100 is detected (YES in step S314), the first ECU 11 calculates the distance RD from the vehicle 100 of the detected moving object (step S315). If the calculated distance RD is equal to or greater than the second distance RD2, the first ECU 11 does not change the imaging cycle of the camera 41, and continues capturing the second image at the second cycle (NO in step S316). The processing from step S313 to step S316 corresponds to the topmost loop processing in the sequence diagram of FIG.

[0068] On the other hand, if the calculated distance RD is less than the second distance RD2 (YES in step S316), the first ECU 11 sets the moving object as a monitoring target and changes the image capturing cycle of the camera 41 to the first cycle (step S317). Thereafter, the first ECU 11 acquires a first image captured by the camera 41 in the first cycle (step S318), and calculates the moving direction of the moving object set as a monitoring target and the distance RD from the vehicle 100 (step S319).

[0069] If the distance RD is equal to or greater than the first distance RD1 (NO in step S320), or if the moving direction is not a direction approaching the vehicle 100 (NO in step S321), the first ECU 11 does not change the imaging cycle of the camera 41, and continues capturing the first image at the first cycle. The processing from step S318 to step S321 corresponds to the loop processing in the middle part of the sequence diagram in FIG. 7.

[0070] If the distance RD calculated in step S319 is less than the first distance RD1 (YES in step S320) and the moving direction of the moving object is a direction approaching the vehicle 100 (YES in step S321), preparations are made to switch to monitoring the periphery of the vehicle 100 based on the video. Then, the first ECU 11 calculates the moving speed of the moving object based on the first image, estimates the arrival time of the moving object at the vehicle 100 based on the moving speed and the distance RD (step S322), and calculates the start time of capturing the video based on the arrival time (step S323). Thereafter, the operation mode of the CPU 111 is switched from the "L mode" with a low operating frequency to the "H mode" with a high operating frequency (step S324), and the process proceeds to step S325 and subsequent steps shown in the flowchart of FIG. 9.

[0071] When the operating mode of CPU 111 is switched to "H mode" with a higher operating frequency, as shown in FIG. 9, CPU 111 controls camera 41 in "H mode" to start capturing video at the capture start time (step 325), and monitors the distance RD between the moving object and vehicle 100 while continuing to capture video (step S326) (step S327).

[0072] The first ECU 11 continues capturing the video until the moving object moves into an area that is at least the second distance RD2 from the vehicle 100 (NO in step S327). If the moving object moves into an area that is at least the first distance RD1 from the vehicle 100 (YES in step S327), the first ECU 11 ends capturing the video by the camera 41 and saves the captured video in a predetermined file format (step S328).

[0073] The video saved in step S328 may be transmitted to a server (not shown) and managed in the server, or in both ROM 112 of vehicle 100 or a removable memory card and the server. The processing from step S325 to step S328 corresponds to the loop processing in the lower part of the sequence diagram in FIG. 7.

[0074] As described above, the vehicle periphery monitoring device according to the first and second embodiments acquires a second image by causing the camera 41 to capture images at a second period, which is a relatively long imaging period, and determines the presence of a moving object moving around the vehicle 100 based on the second image. Thereafter, when the distance RD between the detected moving object and the vehicle 100 becomes shorter and the distance RD becomes shorter than the second distance RD2, the imaging period of the camera 41 is changed to the first period, which is shorter than the second period. In other words, the imaging period of the camera 41 for capturing still images is controlled to be shorter as the distance between the vehicle 100 and the moving object moving around the vehicle 100 becomes shorter. In this way, by detecting a moving object that should be monitored among the moving objects around the vehicle 100 and gradually shortening the imaging period as the monitored moving object approaches the vehicle, it is possible to accurately determine the movement of the monitored moving object while reducing the power consumed by the vehicle periphery monitoring device.

[0075] By continuing to monitor the distance RD between the moving object and the vehicle 100 based on the first images captured in the first cycle, which is a relatively short imaging cycle, it is possible to grasp the movement of the moving object that has moved to an area closer to the vehicle 100. Thereafter, when the moving object moves further and approaches the vehicle 100 and it is detected that the distance RD is equal to or shorter than the first distance, the camera 41 switches from capturing still images to capturing moving images.

[0076] When switching to video capture, the arrival time of the moving object at vehicle 100 is estimated and the optimal start time for video capture is determined based on the arrival time, so it is possible to avoid starting video capture earlier than necessary. Also, if the distance RD between the moving object and vehicle 100 is equal to or less than the first distance, video capture continues, and if it is equal to or greater than the first distance RD1, video capture ends. This allows the video capture period to be optimized, unnecessary video capture and recording to be eliminated, reducing power consumption while also ensuring the securement and preservation of sufficient evidence.

[0077] Furthermore, as described above, different CPUs, i.e., different processors are used, or operating modes with different operating frequencies are appropriately switched, depending on whether a still image is acquired from camera 41 and processing is performed based on the still image or a moving image is acquired and processing is performed based on the moving image. That is, the processor is made to perform processing based on a still image at a relatively low operating frequency, and the processor is made to perform processing based on the moving image at a relatively high operating frequency, thereby achieving both reduced power consumption and improved processing speed. In particular, in the first embodiment, the second ECU 21 can be put into a sleep state while the first ECU 11 is performing processing based on a still image, thereby reducing power consumption accordingly.

[0078] Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configurations are not limited to these embodiments, and the present invention also includes design changes within the scope of the present invention. Furthermore, the above-described embodiments can be combined by utilizing each other's technologies as long as there are no particular contradictions or problems in their purposes, configurations, etc.

[0079] From the above-described embodiments, a vehicle surroundings monitoring device according to the following supplementary clauses can be configured. (Additional note 1) A vehicle surroundings monitoring device that monitors the surroundings of a vehicle, a plurality of first processors and second processors that control cameras that capture images of the periphery of the vehicle and store the images captured by the cameras in a memory; The first processor Operated by the first frequency, detecting a moving object based on images captured by the camera at a predetermined cycle, and controlling the camera so that the image capturing cycle becomes shorter as the moving object approaches the vehicle; When the distance between the moving object and the vehicle is less than a predetermined distance, an arrival time of the moving object at the vehicle is estimated; The first processor or the second processor operating at a second frequency higher than the first frequency, The vehicle surroundings monitoring device controls the camera to start capturing video at a video capture start time that is determined based on the arrival time. [Explanation of symbols]

[0080] 1: Vehicle control system, 3: In-vehicle network, 4: CGW 11: 1st ECU, 21: 2nd ECU, 31: 3rd ECU, 41: Camera, 42: Microphone 100: Vehicle, 111: CPU, 112: ROM, 113: RAM 121: Image acquisition unit, 122: Moving object detection unit, 123: Imaging control unit, 124: Recording control unit 211: CPU, 212: ROM, 213: RAM 221: Image acquisition unit, 222: Moving object detection unit, 223: Imaging control unit, 224: Recording control unit

Claims

1. A vehicle surroundings monitoring device that monitors the surroundings of a vehicle based on an image captured by a camera, an image acquisition unit that acquires an image of the surroundings of the vehicle captured by the camera; a moving object detection unit that detects a moving object moving around the vehicle based on the image and calculates a distance between the moving object and the vehicle; an imaging control unit that controls the camera in accordance with the distance between the moving object and the vehicle, The imaging control unit A vehicle surroundings monitoring device that detects a moving object based on multiple images taken by the camera at a predetermined imaging period, and if the distance between the moving object and the vehicle is less than a predetermined distance, estimates the arrival time at which the moving object will arrive at the vehicle, and controls the camera to start capturing video at an imaging start time determined based on the arrival time.

2. The imaging control unit 2. The vehicle surroundings monitoring device according to claim 1, wherein the arrival time is estimated when the distance between the moving object and the vehicle is less than a predetermined distance and the moving object is moving in a direction approaching the vehicle.

3. The imaging control unit 2. The vehicle surroundings monitoring device according to claim 1, wherein when the moving object is detected based on a plurality of images taken by the camera at a predetermined imaging period, the camera is controlled so that the imaging period becomes shorter as the moving object approaches the vehicle.

4. The imaging control unit the moving object is detected based on second images captured by the camera at a second interval, and when the distance between the moving object and the vehicle is less than a second distance, the camera is controlled to change the imaging interval to a first interval that is shorter than the second interval; 4. A vehicle surroundings monitoring device as described in claim 3, wherein, when the distance between the moving object and the vehicle calculated based on a first image captured by the camera at a first period is less than a first distance that is shorter than a second distance, and the direction of movement of the moving object is a direction approaching the vehicle, an arrival time at which the moving object will arrive at the vehicle is estimated and the camera is controlled to start capturing video at a capture start time determined based on the arrival time.

5. The imaging control unit 2. The vehicle surroundings monitoring device according to claim 1, wherein the moving speed of the moving object is calculated, and the arrival time is estimated based on the distance and the moving speed.

6. The imaging control unit 2. The vehicle surroundings monitoring device according to claim 1, wherein the camera is controlled to start capturing the moving image at the capture start time, which is a predetermined time before the arrival time.

7. The imaging control unit 2. The vehicle surroundings monitoring device according to claim 1, wherein the camera continues capturing video until the distance between the moving object and the vehicle becomes equal to or greater than a predetermined distance.

8. the image acquisition unit, the moving object detection unit, and the imaging control unit are realized by one or more processors, the one or more processors are operable to operate at a first operating frequency or a second operating frequency higher than the first operating frequency; One of the processors, a camera driven by the first operating frequency, detecting the moving object based on a plurality of images captured by the camera at a predetermined imaging cycle, and controlling the camera so that the imaging cycle becomes shorter as the detected moving object approaches the vehicle; When the distance between the moving object and the vehicle is less than a predetermined distance, One or more processors may 2. The vehicle surroundings monitoring device according to claim 1, wherein the camera is driven at the second operating frequency, estimates the arrival time of the moving object at the vehicle, and controls the camera to start capturing video at an imaging start time determined based on the arrival time.

Images