Photographing image processing device

Abstract

To provide a photographing image processing device capable of realizing an effective display.SOLUTION: In a photographing image processing device, a specific part specifies an unnecessary image region including a region where a hood of a camera is reflected, in an image obtained by photographing a circumference region of a vehicle by the camera arranged in the vehicle. A display region setting part sets a display region displaying drive support information from an advanced drive support system to the unnecessary image region.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a photographic image processing device. [Background technology]

[0002] Patent Document 1 proposes a drive recorder having a recording unit that records camera images and support display information of a driving support system in association with each other. The in-vehicle multimedia device disclosed in Patent Document 1 displays the camera images and the support display information on a monitor (see FIG. 3 of Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-017117 [Patent Document 2] JP 2019-134210 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, an image captured by a camera may contain a reflected area of ​​the camera hood. The camera image of Patent Document 1 also contains a reflected area (see FIG. 3 of Patent Document 1). This reflected area does not provide any information about the situation around the vehicle and is considered to be an unnecessary image area. For this reason, the technology disclosed in Patent Document 1 may not be able to effectively utilize the display area of ​​the display device. In other words, the technology disclosed in Patent Document 1 may not be able to realize an efficient display.

[0005] An object of the present disclosure is to provide a captured image processing device capable of realizing efficient display. [Means for solving the problem]

[0006] In one aspect, the photographing image processing device includes: a specifying unit that specifies an unnecessary image area including a reflected area of ​​a hood of a camera in an image of a surrounding area of ​​the vehicle captured by the camera; a display area setting unit that sets a display area for displaying driving assistance information from an advanced driving assistance system within the unnecessary image area; Equipped with: Effect of the Invention

[0007] According to the present disclosure, it is possible to provide a captured image processing device capable of realizing efficient display. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of a captured image processing device in a first embodiment. [Diagram 2] FIG. 13 is a diagram showing an example of a reflection area of ​​a camera hood. [Diagram 3] FIG. 11 is a block diagram showing an example of a captured image processing device in a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described with reference to the drawings. Each embodiment can be combined with other embodiments as appropriate. In this disclosure, the drawings can be associated with one or more embodiments. Also, each element in the drawings can apply to one or more embodiments. Also, in the embodiments, the same or equivalent elements are given the same reference numerals, and duplicated explanations are omitted.

[0010] First Embodiment Fig. 1 is a block diagram showing an example of a captured image processing device in the first embodiment. In Fig. 1, the captured image processing device 10 has a specification unit 11 and a display area setting unit 12. The captured image processing device 10 may be, for example, a drive recorder attached to a vehicle. The captured image processing device 10 may have a CAN interface unit (not shown) connected to an advanced driving assistance system (ADAS: Advanced Driving Assistant System) (not shown) of the vehicle, an engine control ECU (Electronic Control Unit), a brake ECU, a camera ECU, and the like of the vehicle via a CAN (Controller Area Network).

[0011] The captured image processing device 10 acquires an image (i.e., a series of images) of the area around the vehicle captured by a camera (not shown) disposed in the vehicle. The camera may be, for example, a camera for capturing an image used for recognizing other vehicles in an advanced driving assistance system of the vehicle. The captured image processing device 10 also acquires driving assistance information from the advanced driving assistance system. Note that functions of the advanced driving assistance system include, for example, a forward collision warning function, a rearward vehicle detection warning function, a collision damage mitigation brake function, etc.

[0012] The identification unit 11 identifies an unnecessary image area including a reflection area of ​​the camera hood in the acquired image. The unnecessary image area is an image area that does not provide any information about the situation around the vehicle. For example, the reflection area of ​​the hood appears in the image in the form of an image area PA1 shown in FIG. 2. FIG. 2 is a diagram showing an example of the reflection area of ​​the camera hood. In addition to the reflection of the camera hood, a fixed object such as a vehicle bonnet may be reflected, or a portion that does not change over time may be determined as an unnecessary area.

[0013] Specifically, the identification unit 11 may identify the reflected area of ​​the hood and other areas based on the brightness level of the image data, as disclosed in Patent Document 2, for example. Alternatively, the reflected area of ​​the hood may be identified using a marker made of the material of the lens hood, as disclosed in Patent Document 2, for example. The tip of the lens hood may be painted with a paint that reflects infrared light, and the other inner diameter parts may be painted with a paint that absorbs infrared light. Alternatively, the identification unit 11 may identify an area that maintains a constant or a brightness value within a predetermined range in a series of multiple images as an unnecessary image area including the reflected area of ​​the hood.

[0014] The display area setting unit 12 sets a display area for displaying the driving support information within the unnecessary image area identified by the identification unit 11. For example, the display area setting unit 12 sets the display area DA1 shown in FIG. 2 within the unnecessary image area PA1, which is the reflected area of ​​the hood. By arranging the driving support information within this set display area, it is possible to display an image in which the driving support information is superimposed within the unnecessary image area. This allows the unnecessary image area to be effectively utilized, thereby realizing efficient display.

[0015] Note that a display control unit (not shown) of the captured image processing device 10 may perform control to display, on a display device (not shown), an image in which driving support information is superimposed on the unnecessary image area.

[0016] <Second embodiment> The second embodiment relates to image recording control.

[0017] 3 is a block diagram showing an example of a captured image processing device in the second embodiment. In FIG. 3, the captured image processing device 20 has a specification unit 11, a display area setting unit 12, a recording control unit 21, and a storage unit 22.

[0018] The recording control unit 21 executes control for storing, in the storage unit 22, an image in which the driving support information is superimposed on the display area, that is, recording control.

[0019] For example, when a predetermined condition is satisfied, the recording control unit 21 may start recording an image in which driving assistance information acquired from an advanced driving assistance system of the vehicle is superimposed on the display area. For example, the recording control unit 21 periodically receives a detection value from a sensor of the vehicle separately from the driving assistance information. The sensor of the vehicle may be, for example, a vehicle speed sensor. In this case, the detection value is a vehicle speed value. The predetermined condition may include that the detection value exceeds a first threshold value during a period from a first timing when the captured image processing device 20 receives the driving assistance information to a second timing a predetermined time after the first timing, and that a difference between the detection value at the second timing and the detection value at a third timing before the first timing is greater than a second threshold value. The detection value exceeding the first threshold value during a period from a first timing when the captured image processing device 20 receives the driving assistance information to a second timing a predetermined time after the first timing means, for example, that the running state of the vehicle becomes abnormal within a predetermined period from the timing when the captured image processing device 20 receives the driving assistance information. Moreover, the difference between the detection value at the second timing and the detection value at the third timing before the first timing being greater than the second threshold means, for example, that the vehicle does not return to the normal driving state before the image processing device 20 receives the driving support information even after a predetermined period has elapsed since the image processing device 20 received the driving support information. More specifically, after the image processing device 20 receives driving support information from the advanced driving support system of the vehicle, indicating that there is a possibility of a collision with a vehicle ahead, the driving support information such as "Caution: Collision with a vehicle ahead" is displayed in the unnecessary image area, and the driving support information is superimposed on the captured image of the front. After that, if there is a change in the vehicle speed value of the vehicle that exceeds a predetermined threshold (first threshold) compared to when the driving support information was obtained, and if the difference exceeds a predetermined threshold (second threshold) compared to the detection value before the driving support information was obtained, an image is recorded. (In cases where the driver receives the collision warning information, the driver rapidly decelerates, and maintains the deceleration state even after a predetermined time has elapsed, or stops, an image is recorded.)

[0020] On the other hand, the recording control unit 21 may not start recording an image with driving support information superimposed on the display area if a predetermined condition is not satisfied. That is, for example, if the detection value does not exceed the first threshold value during the period from the first timing when the captured image processing device 20 receives the driving support information to the second timing that is a predetermined time after the first timing, image recording is not started. That is, even if the captured image processing device 20 receives the driving support information, if the driving state of the vehicle does not become abnormal within a predetermined period from the receiving timing, image recording is not started. Also, for example, if the difference between the detection value at the second timing and the detection value at the third timing before the first timing is equal to or less than the second threshold value, image recording is not started. That is, even if the captured image processing device 20 receives the driving support information, if the driving state of the vehicle returns to the normal driving state before the captured image processing device 20 receives the driving support information within a predetermined period from the receiving timing, image recording is not started. More specifically, after the captured image processing device 20 acquires driving assistance information from the vehicle's advanced driving assistance system indicating the possibility of a collision with a vehicle ahead, the driving assistance information, such as "Watch out for a collision with a vehicle ahead," is displayed in the unnecessary image area, and the driving assistance information is superimposed on the captured image of the front. After that, if the vehicle speed value of the vehicle does not change beyond a predetermined threshold (first threshold) compared to when the driving assistance information was acquired for a predetermined time, and the difference with the detected value before the driving assistance information was acquired is equal to or less than a predetermined threshold (second threshold), the image may not be recorded. (In cases where the driver slightly decelerates upon receiving the collision warning information, but is able to quickly recover speed and resume normal driving, the image may not be recorded.) This makes it possible to avoid image recording in cases where there is little need to record the captured image, thereby preventing unnecessary use of resources of the captured image processing device 20. Note that each timing and the first and second thresholds may be set to changeable values.

[0021] In the above description, the recording control unit 21 is described as starting to record an image in which driving support information is superimposed on the display area when a predetermined condition is satisfied, but the present disclosure is not limited to this. For example, the recording control unit 21 may start controlling recording when the captured image processing device 20 receives driving support information as a trigger. The recording control unit 21 may be in a so-called standby state in which recording can be started.

[0022] The recording control unit 21 may also perform control to store in the storage unit 22 a voice collected by a microphone (not shown) disposed in the vehicle together with an image in which driving support information is superimposed on the display area.

[0023] <Third embodiment> In the third embodiment, a plurality of cameras are provided in a vehicle. Then, from among a plurality of images captured by the plurality of cameras, an image to be recorded is selected based on the type of driving support information. Note that the basic configuration of the captured image processing device in the third embodiment is the same as that of the captured image processing device 20 in the second embodiment, and therefore will be described with reference to FIG. 3.

[0024] In the captured image processing device 20 of the third embodiment, the identification unit 11 identifies the above-mentioned unnecessary image area in each of a plurality of images. Then, the display area setting unit 12 sets a display area for displaying the above-mentioned driving support information within the unnecessary image area identified by the identification unit 11.

[0025] The recording control unit 21 selects an image to be recorded from a plurality of images based on the type of driving support information received by the captured image processing device 20. For example, a vehicle is provided with a first camera for capturing an area in front of the vehicle and a second camera for capturing an area behind the vehicle. The image captured by the first camera is called the first image, and the image captured by the second camera is called the second image. For example, when the captured image processing device 20 receives driving support information from a forward collision warning function, the recording control unit 21 may select the first image as the image to be recorded. Also, for example, when the captured image processing device 20 receives driving support information from a reverse vehicle detection warning function, the recording control unit 21 may select the second image as the image to be recorded. This allows an image that matches the type of driving support information to be selected as the image to be recorded.

[0026] Each of the photographic image processing devices 10 and 20 described in the first to third embodiments may have a processor and a memory. The processor may include a plurality of processors. The memory may be configured by a combination of a volatile memory and a non-volatile memory.

[0027] The identification unit 11, the display area setting unit 12, and the recording control unit 21 of the captured image processing device 10, 20 of the first and second embodiments may be realized by a processor reading and executing a program stored in a memory. The storage unit 22 may be realized by a memory. [Explanation of symbols]

[0028] 10. Image capture device 11 Specific section 12 Display area setting section 20. Image capture device 21 Recording control section 22 Memory section

Claims

1. a specifying unit that specifies an unnecessary image area including a reflected area of ​​a fixed object of a camera in an image of a surrounding area of ​​the vehicle captured by the camera; a display area setting unit that sets a display area for displaying driving assistance information from an advanced driving assistance system within the unnecessary image area; A photographic image processing device comprising:

2. An image recording control unit that controls recording of the image on which the driving support information is superimposed on the display area.

2. The photographed image processing device according to claim 1.

3. The image recording control unit starts controlling the recording when the driving assistance information is received as a trigger.

3. The photographed image processing device according to claim 2.

4. The image recording control unit includes: Periodically receiving detection values ​​from sensors of the vehicle; starting the recording when the detection value exceeds a first threshold value during a period from a first timing at which the driving assistance information is received to a second timing that is a predetermined time after the first timing, and a difference between the detection value at the second timing and a detection value at a third timing prior to the first timing is greater than a second threshold value; 4. The photographed image processing device according to claim 2.

5. The identification unit identifies the unnecessary image area in each of a plurality of images captured by a plurality of cameras disposed in the vehicle, the display area setting unit sets the display area within the unnecessary image area for each of the plurality of images; The image recording control unit selects an image to be recorded from the plurality of images based on a type of the driving support information.

4. The photographed image processing device according to claim 2.

Images