Vehicle cameras and camera systems

Abstract

Disclosed are a vehicle-mounted camera and a camera system. In the vehicle-mounted camera, an image generation unit generates at least one image of a directional view from a vehicle, and an object recognition unit recognizes a target object in the at least one image. An area recognition unit recognizes a specific area in the target object recognized by the object recognition unit in the at least one image. A vehicle control unit performs a vehicle control task based on the recognized target object. A mosaic unit recognizes a specific area in the at least one image corresponding to a specific area in the at least one image, and performs a mosaic task, thereby blurring a mosaic area in the at least one image. The mosaic area includes at least a specific area in the at least one image.

Description

Technical Field

[0001] The present disclosure relates to a vehicle-mounted camera and a camera system each including the vehicle-mounted camera. Background Art

[0002] The driving recorders are all installed in vehicles. Such driving recorders installed in vehicles allow a camera to capture images of various views from the vehicle and record the captured images therein. An example of these driving recorders is disclosed in Japanese Patent Application Publication No. 2011-232858.

[0003] A vehicle-mounted camera may also be installed in the vehicle. Such a vehicle-mounted camera installed in the vehicle also captures images of various views from the vehicle. The images captured by the vehicle-mounted camera may be used to control and / or assist the driving operation of the vehicle. Summary of the invention

[0004] Directly using the images recorded in such a driving recorder may cause privacy issues. Installing the camera of such a driving recorder and such a vehicle-mounted camera in a vehicle may reduce the installable space of the vehicle.

[0005] From these viewpoints, the present disclosure attempts to provide vehicle-mounted cameras, each of which is capable of maintaining user privacy while reducing the required installation space.

[0006] According to an exemplary aspect of the present disclosure, a vehicle-mounted camera includes: an image generation unit configured to generate at least one image of a directional view from a vehicle; and an object recognition unit configured to recognize a target object in at least one image. The vehicle-mounted camera includes a region recognition unit configured to recognize, in at least one image, a specific region in a target object recognized by the object recognition unit. The vehicle-mounted camera includes a vehicle control unit configured to perform a vehicle control task based on the recognized target object. The vehicle-mounted camera includes a mosaic unit configured to: recognize a specific region in at least one image corresponding to a specific region in at least one image, and to perform a mosaic task, thereby blurring a mosaic region in at least one image, the mosaic region including at least the specific region in at least one image.

[0007] The vehicle-mounted camera according to the exemplary aspect is configured to generate at least one image, and the vehicle-mounted camera is configured to perform a vehicle control task based on the at least one image. This configuration of the vehicle-mounted camera eliminates the need to install a dashcam camera in the vehicle in addition to the vehicle-mounted camera 1, so that the installable space in the vehicle can be saved.

[0008] The vehicle-mounted camera is also configured to perform a mosaic task, thereby blurring a specific area in at least one image in at least one image. This configuration protects the privacy of information represented by the specific area in at least one image. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Other aspects of the present disclosure will become apparent from the following description of embodiments with reference to the accompanying drawings, in which:

[0010] Figure 1 is a block diagram schematically showing an example of a structure of a vehicle-mounted camera according to an exemplary embodiment of the present disclosure;

[0011] Figure 2 It is schematically shown Figure 1 A block diagram showing an example of a functional structure of a controller;

[0012] Figure 3 is a flowchart schematically showing an example of an image processing routine executed by the controller;

[0013] Figure 4 is a flowchart schematically showing an example of a recognition image processing subroutine included in the image processing routine;

[0014] Figure 5 is a flowchart schematically showing an example of a drive recorder image processing subroutine included in the image processing routine;

[0015] Figure 6 is a view schematically showing an example of a recognition image including a pedestrian according to an exemplary embodiment;

[0016] Figure 7 is a view schematically showing an example of a drive recorder image according to an exemplary embodiment; and

[0017] Figure 8 is a view schematically illustrating another example of a recognition image including a preceding vehicle according to an exemplary embodiment. DETAILED DESCRIPTION

[0018] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. In the exemplary embodiments and their modifications, similar reference numerals are assigned to similar or identical parts between the exemplary embodiments and their modifications, so that the description of one of the similar or identical parts is applied to another of the similar or identical parts.

[0019] Refer to the following Figure 1 and Figure 2The structure of the vehicle-mounted camera 1 according to the exemplary embodiment of the present disclosure is described. The vehicle-mounted camera 1 designed as a monocular camera or a stereo camera is mounted to a vehicle 2 .

[0020] Reference Figure 1 For example, the vehicle-mounted camera 1 includes an imaging unit 3 , a controller 5 , a wireless communication device 7 , a storage medium 9 , an input / output (I / O) interface 11 , and a housing 12 .

[0021] The imaging unit 3 , the wireless communication device 7 , the storage medium 9 , and the I / O interface 11 are communicably connected to the controller 5 .

[0022] The imaging unit 3 includes an imager 3 a and a lens system 3 b (not shown).

[0023] The imager 3a includes a plurality of light receiving elements respectively corresponding to a plurality of pixels, two-dimensionally arranged in both the vertical direction and the horizontal direction corresponding to the respective height direction and width direction of the vehicle 2. The two-dimensionally arranged pixels constitute a light receiving surface of the imager 3a.

[0024] The lens system 3 b is configured to focus light incident from, for example, an area in front of the vehicle 2 , onto the light receiving surface of the imager 3 a ; the area in front of the vehicle 2 corresponds to, for example, a view from the vehicle 2 .

[0025] The imager 3 a is configured to receive light focused on its light receiving surface during a controllable shutter time (ie, exposure duration) so that each of the two-dimensionally arranged photosensitive elements (pixels) receives a corresponding light component under the control of the controller 5 .

[0026] Then, the imager 3a converts the corresponding received light component into an electric charge (i.e., an electric signal) corresponding to the intensity of the received light component using each light receiving element, thereby outputting the electric signal as received light data, i.e., an image of one frame showing the front image of the vehicle 2. That is, the imager 3a is configured to cyclically perform the light receiving operation, thereby obtaining an image of one frame including the intensity of the corresponding received light component for each pixel.

[0027] The controller 5 is mainly constituted by a computer including a central processing unit (CPU) 13 and a memory 15 including at least one of storage media including a RAM and a ROM. The CPU 13 is communicably connected to the memory 15.

[0028] At least a portion of all functions provided by the controller 5 may be implemented by at least one processor; the at least one processor may include

[0029] (1) A combination of at least one programmable processing unit (i.e., at least one programmable logic circuit) and at least one memory

[0030] (2) At least one hard-wired logic circuit

[0031] (3) At least one mixed circuit of hardwired logic and programmable logic

[0032] Specifically, the controller 5 is configured so that the CPU 13 executes instructions of routines stored in the memory 15, thereby performing predetermined software tasks; at least some of the software tasks implement the following functional components of the controller 5. The controller 5 may also be configured so that at least one dedicated electronic circuit performs predetermined hardware tasks; at least some of the hardware tasks implement the following functional components of the controller 5. The controller 5 may be configured to perform both software tasks and hardware tasks; at least some of the software and hardware tasks implement the following functional components of the controller 5.

[0033] Reference Figure 2 The controller 5 (ie, the CPU 13 ) functionally includes an image generating unit 17 , a vehicle speed estimating unit 19 , a pitch amount calculating unit 21 , an image storing unit 23 , an image correcting unit 25 , an object identifying unit 27 , and a region identifying unit 29 .

[0034] The controller 5 (i.e., CPU 13) functionally also includes a pixelation unit (i.e., mosaic unit 31), a vehicle control unit 33, an image reading unit 35, a vehicle position obtaining unit 37, a vehicle position determining unit 39, an image storage unit 41, an output unit 43 and an image compression unit 45.

[0035] The wireless communication device 7 can communicate wirelessly with a storage device 8 located outside the vehicle 2. The storage device 8 can store various information items (such as images). The wireless communication device 7 and the storage device 8 can use, for example, network, A network or other similar wireless network performs wireless communication therebetween. A storage terminal that can be worn by a user may be used as the storage device 8.

[0036] The output port 11 enables a communicative connection between the controller 5 of the vehicle-mounted camera 1 and the vehicle network 47 established in the vehicle 2 according to a predetermined interface. A universal serial bus (USB) interface, a low voltage differential signal (LVDS) interface, An HDMI interface or other similar interface may be used as the interface of the output port 11 .

[0037] The housing 12 is configured to house the imaging unit 3, the controller 5, the wireless communication device 7, and the storage medium 9. The output port 11 is attached to the housing 12. The housing 12 is arranged in the vehicle 2 so that the imaging unit 3 can capture an image from a front view of the vehicle 2.

[0038] In addition to the vehicle-mounted camera 1, the vehicle 2 includes a vehicle network 47, a recorder 49, a G sensor 51, a vehicle speed sensor 53, a global positioning system (GPS) receiver 55, and a vehicle controller 57; these components 49, 51, 53, 55, and 57 are communicably connected to the vehicle network 47. For example, the recorder 49 serves as an external device of the vehicle-mounted camera 1.

[0039] That is, the vehicle network 47 provides communicable connections between the vehicle-mounted camera 1 and the components 49 , 51 , 53 , 55 , and 57 .

[0040] The recorder 49 is configured to store drive recorder images as described in detail below.

[0041] The G sensor 51 used as an accelerometer sensor is configured to measure the values ​​of acceleration about the corresponding pitch axis, roll axis, and yaw axis of the vehicle 2. These pitch, roll, and yaw axes pass through the center of gravity of the vehicle 2. The pitch axis represents a horizontal axis parallel to the width direction of the vehicle 2, so that the pitch angle of the vehicle 2 represents the rotation angle of the vehicle 2 around the pitch axis. The yaw axis represents a vertical axis parallel to the height direction of the vehicle 2, so that the yaw angle of the vehicle 2 represents the rotation angle of the vehicle 2 around the yaw axis. The roll axis represents a longitudinal axis parallel to the longitudinal direction of the vehicle 2, so that the roll angle of the vehicle 2 represents the rotation angle of the vehicle 2 around the roll axis.

[0042] Then, the G sensor 51 transmits measurement signals respectively indicating the values ​​of acceleration about the corresponding pitch axis, roll axis, and yaw axis of the vehicle 2 to the controller 5 of the onboard camera 1 .

[0043] The vehicle speed sensor 53 is configured to measure the speed of the vehicle 2 and transmit a measurement signal indicating the speed of the vehicle 2, which will be referred to as the vehicle speed V, to the controller 5. The GPS receiver 55 is configured to receive a global positioning system (GPS) signal transmitted from a GPS satellite, thereby calculating the latitude and longitude of the current position of the vehicle 2 based on the received GPS signal. Then, the GPS receiver 55 is configured to transmit a measurement signal indicating the latitude and longitude of the current position of the vehicle 2 to the controller 5.

[0044] The vehicle controller 57 is configured to perform vehicle control tasks, including, for example

[0045] 1. Task of decelerating vehicle 2 using the deceleration device installed in vehicle 2

[0046] 2. The task of stopping vehicle 2 using the deceleration device

[0047] 3. The task of accelerating the vehicle 2 using the acceleration device installed in the vehicle 2

[0048] 4. The task of steering the vehicle 2 using the steering mechanism installed in the vehicle 2

[0049] 5. Task of outputting a visible and / or audible warning message from the warning device to the driver of the vehicle 2

[0050] Next, refer to Figures 3 to 7 The image processing routine executed every predetermined period by the controller 5 is described. One image processing routine periodically executed by the controller 5 will be referred to as a cycle.

[0051] When the current cycle of the image processing routine is started, the controller 5 functions as the image generation unit 17 to cause the imaging unit 3 to capture (i.e., generate) a frame image of, for example, a front view of the self-vehicle 2 at a corresponding date and time in step S1. Note that the controller 5 according to the exemplary embodiment is configured to cause the imaging unit 3 to continuously capture frame images according to a predetermined value of the exposure duration (i.e., shutter time) and a predetermined value of the frame rate. For example, the controller 5 sets the exposure duration to a value selected from a range of 1 to 100 μs, and sets the frame rate to a value selected from a range of 10 to 60 frames / second (fps).

[0052] For example, the controller 5 is programmed to execute a loop of the image processing routine each time the imaging unit 3 captures a frame image according to the frame rate.

[0053] In step S2 , the controller 5 functions as the vehicle speed estimation unit 19 to estimate the vehicle speed V at the time when the frame image is captured in step S1 based on the measurement signal sent from the vehicle speed sensor 53 .

[0054] Next, the controller 5 functions as the pitch calculation unit 21 to calculate the pitch amount of the vehicle 2 when the frame image is captured in step S1 based on the measurement signal sent from the G sensor 51 .

[0055] Then, the controller 5 functions as the image storage unit 23 to determine in step S4 whether it is time to store the recognition image in the memory 15. Note that the recognition image is a frame image generated in step S1 of the current cycle of the image processing routine and used for the recognition image processing subroutine described later.

[0056] For example, the controller 5 functions as the image storage unit 23 to determine whether a predetermined time interval T1 has elapsed since the recognition image was stored in the memory 15 for the last time.

[0057] When it is determined that the predetermined time interval T1 has elapsed since the last time the recognition image was stored in the memory 15 ("Yes" in step S4), the image processing routine proceeds to step S5. Otherwise, when it is determined that the predetermined time interval T1 has not elapsed since the last time the recognition image was stored in the memory 15 ("No" in step S4), the image processing routine proceeds to step S8.

[0058] In step S5, the controller 5 functions as the image correction unit 25 to perform correction on the frame image generated in step S1 of the current cycle of the image processing routine, thereby reducing the distortion of the frame image due to the lens distortion contained in the frame image. Note that the lens distortion represents the distortion of the lens system 3b. The memory 15 of the controller 5 according to the exemplary embodiment stores distortion correction information about the lens distortion, and the controller 5 is configured to perform correction on the frame image based on the distortion correction information to reduce the distortion of the frame image accordingly.

[0059] Then, the controller 5 functions as the image storage unit 23 to

[0060] 1. Determine that the frame image corrected in step S5 of the current cycle of the image processing routine is a recognition image,

[0061] 2. In step S6, the recognition image, the vehicle speed estimated in step S2, the pitch amount calculated in step S3, and the corresponding date and time are stored in the memory 15 so that the vehicle speed, the pitch amount, and the corresponding date and time are associated with the recognition image.

[0062] Next, the controller 5 executes a recognition image processing subroutine in step S7.

[0063] Refer to the following Figure 4 The recognition image processing subroutine in step S7 is described.

[0064] When the recognition image processing subroutine is started, the controller 5 functions as the image readout unit 35 to read out the recognition image stored in the immediately preceding step S6 of the current cycle of the image processing routine from the memory 15 in step S11 .

[0065] Next, the controller 5 functions as the object recognition unit 27 to recognize at least one pedestrian as a target object to be recognized in the read recognition image according to a known image recognition technique in step S12 when at least one pedestrian is included in the read recognition image. Figure 6An example of a recognition image is shown with reference numeral 59 and a pedestrian included in the recognition image is shown with reference numeral 61 . The controller 5 functions as the object recognition unit 27 to recognize the pedestrian 61 in the recognition image 59 .

[0066] Following the operation in step S12, in step S13, the controller 5 functions as the region recognition unit 29 to recognize (i.e., identify) a region of the face of the pedestrian recognized in the recognition image; the region of the face of the recognized pedestrian contains the face of the recognized pedestrian. The region of the face of the recognized pedestrian, which will be referred to as the face region, represents a specific region in the recognized target object. For example, the controller 5 functions as the region recognition unit 29 to recognize the face region 63 of the pedestrian 61 recognized in the recognition image (see Figure 6 ).

[0067] Next, in step S14 , the controller 5 functions as the mosaic unit 31 to read out the pitch amount associated with the identification image read out in step S11 from the memory 15 .

[0068] Then, in step S15 , the controller 5 functions as the mosaic unit 31 to correct the position of the face region of the recognized pedestrian in the recognition image based on the pitch amount read out in step S14 .

[0069] Specifically, the mosaic unit 31 corrects the position of the facial area of ​​the recognized pedestrian to its position obtained when the pitch amount is 0. The mosaic unit 31 also defines the corrected facial area as a mosaic area. For example, the mosaic unit 31 corrects the position of the facial area 63 to the position 65 as the mosaic area 65.

[0070] Next, in step S16, the controller 5 functions as the mosaic unit 31 to obtain information indicating the position, size, and shape of the mosaic area obtained in step S15 in the recognition image as mosaic information. Then, the controller 5 functions as the mosaic unit 31 to store the mosaic information in the memory 15 so as to be associated with the recognition image used in step S15. For example, an address is assigned to the position of each pixel of the recognition image, and the position of the mosaic area includes the address of the selected pixel in the mosaic area, and the size of the mosaic area indicates the area of ​​the mosaic area or the number of pixels contained in the mosaic area.

[0071] Subsequently, in step S17 , the controller 5 functions as the vehicle control unit 33 to determine whether the controller 5 needs to perform at least one of the vehicle control tasks based on the at least one pedestrian identified in step S12 .

[0072] For example, the vehicle control unit 33 calculates the minimum distance between at least one pedestrian located in front of the vehicle 2 and the vehicle 2 based on, for example, the positional relationship between the vehicle 2 and at least one pedestrian in the recognition image, wherein the recognition image is obtained in the current cycle of the image processing routine, the stored recognition image and / or in another known distance measurement process in step S17.

[0073] Then, in step S17 , the vehicle control unit 33 determines whether the measured minimum distance is equal to or smaller than a predetermined threshold distance.

[0074] When it is determined that the measured minimum distance is equal to or less than the predetermined threshold distance, the vehicle control unit 33 determines that the vehicle control unit 33 needs to perform at least one of the vehicle control tasks ("Yes" in step S17). Otherwise, when it is determined that the measured minimum distance is greater than the predetermined threshold distance, the vehicle control unit 33 determines that the vehicle control unit 33 does not need to perform at least one of the vehicle control tasks ("No" in step S17).

[0075] When it is determined that the vehicle control unit 33 needs to perform at least one of the vehicle control tasks ("Yes" in step S17), the image processing subroutine proceeds to step S18. Otherwise, when it is determined that the vehicle control unit 33 does not need to perform at least one of the vehicle control tasks, the controller 5 returns to the operation in step S8 of the main image processing routine.

[0076] In step S18, the controller 5 functions as the vehicle control unit 33 to output one or more vehicle control signals to the vehicle controller 57; for example, the vehicle control signals include:

[0077] (1) a vehicle control signal instructing the vehicle controller 57 to use the deceleration device to decelerate the vehicle 2;

[0078] (2) a vehicle control signal instructing the vehicle controller 57 to use the deceleration device to stop the vehicle 2;

[0079] (3) a vehicle control signal instructing the vehicle controller 57 to use the steering mechanism to turn the vehicle 2 in a selected direction to avoid at least one pedestrian;

[0080] (4) a vehicle control signal instructing the vehicle controller 57 to cause the warning device to visually and / or audibly output a warning message,

[0081] Following the operation in step S18 , in step S19 , the controller 5 functions as the vehicle control unit 33 to store the output history of the vehicle control signal in the memory 15 , and returns to the operation in step S8 of the main image processing routine.

[0082] In step S8 of the main image processing routine, the controller 5 functions as the image storage unit 23 to determine whether it is time to store the drive recorder image in the memory 15. Note that the drive recorder image is a frame image generated in step S1 of the current cycle of the image processing routine for recording the travel of the vehicle 2 and used for the drive recorder image processing subroutine described later.

[0083] For example, the controller 5 functions as the image storage unit 23 to determine whether a predetermined time interval T2 has elapsed since the last time the drive recorder image was stored in the memory 15. The predetermined time interval T2 is set to be greater than the predetermined time interval T1.

[0084] When it is determined that the predetermined time interval T2 has elapsed since the last time the dashcam image was stored in the memory 15 ("Yes" in step S8), the image processing routine proceeds to step S9. Otherwise, when it is determined that the predetermined time interval T2 has not elapsed since the last time the dashcam image was stored in the memory 15 ("No" in step S8), the controller 5 terminates the current cycle of the image processing routine.

[0085] In step S9, the controller 5 operates as the image storage unit 23 to

[0086] 1. Determine that the frame image generated in step S1 of the current cycle of the image processing routine is used as the driving recorder image

[0087] 2. In step S9 , the drive recorder image and the corresponding date and time are stored in the memory 15 , so that the corresponding date and time are associated with the drive recorder image.

[0088] It should be noted that at least a portion of the frame image generated in step S1 can be used as both the identification image and the drive recorder image. In addition, any frame image generated in step S1 can be used as one of the identification image and the drive recorder image.

[0089] Next, the controller 5 executes the drive recorder image processing subroutine in step S7 .

[0090] Refer to the following Figure 5 The drive recorder image processing subroutine in step S10 is described.

[0091] When the drive recorder image processing subroutine is started, in step S21 , the controller 5 functions as the image readout unit 35 to read out from the memory 15 the drive recorder image stored in the immediately preceding step S9 of the current cycle of the image processing routine.

[0092] Next, in step S22, the controller 5 functions as the vehicle position obtaining unit 37 to obtain the current position, that is, the current location, of the vehicle 2 using the measurement signal transmitted from the GPS 55. Then, in step S23, the controller 5 functions as the vehicle position determining unit 39 to determine whether the current position of the vehicle 2 is located in one of the previously prepared specific sections whose position data items are stored in the memory 15.

[0093] For example, a non-imaging area or a non-photographing area can be used as an example of a previously prepared specific section.

[0094] When it is determined that the current position of the vehicle 2 is located in one of the previously prepared specific sections ("Yes" in step S23), the driving recorder image processing subroutine proceeds to step S24. Otherwise, when it is determined that the current position of the vehicle 2 is not located in any of the previously prepared specific sections ("No" in step S23), the driving recorder image processing subroutine proceeds to step S25.

[0095] In step S24 , the controller 5 functions as the mosaic unit 31 to perform a mosaic task or a pixelation task to blur the entire drive recorder image, thereby generating a full mosaic drive recorder image with a lower resolution.

[0096] Note that, as described above, the mosaic task to be applied to the target area includes a task of blurring (ie, pixelating (mosaicing)) the target area, thereby, for example, masking or covering the target area.

[0097] Note that the predetermined first encryption key is a unique or non-unique data item having a predetermined bit length previously defined for the mosaic task performed by the mosaic unit 31 in step S24. Specifically, the first encryption key is input to the controller 5 using an unillustrated device so that the mosaic unit 31 or a general computer can decode the full mosaic drive recorder image into the original drive recorder image. The first encryption key for the mosaic task performed by the mosaic unit 31 in step S24 or information from which the first encryption key can be obtained is stored in the memory 15. After the operation in step S24 is completed, the drive recorder image processing subroutine proceeds to step S31.

[0098] Otherwise, in step S25, the controller 5 functions as the mosaic unit 31 to extract (i.e., read out) one of the identification images as a reference image; the identification image has been stored in the memory 15, and the reference image is the most recently generated identification image among the identification images stored in the memory 15.

[0099] That is, the mosaic unit 31 recognizes the reference image among the recognition images stored in the memory 15 .

[0100] Then, in step S26, the controller 5 functions as the mosaic unit 31 to read out mosaic information related to the reference image from the memory 15. Thereafter, in step S27, the controller 5 functions as the mosaic unit 31 to correct the position and size of the mosaic area defined by the mosaic information read out in step S26.

[0101] The following describes how the mosaic unit 31 corrects the position and size of the mosaic area.

[0102] The position and size of the mosaic area defined by the mosaic information are calculated based on the recognition image in which the distortion caused by the lens distortion has been corrected. In contrast, the distortion contained in the dashcam image caused by the lens distortion is not corrected. Figure 7 As shown, if the position and size of the mosaic area (see reference numeral 65 ) is applied to the dashcam image (see reference numeral 60 ), there will be a large deviation between the facial area 63 of the recognized pedestrian and the mosaic area 65 .

[0103] In contrast, Figure 7 As shown, the mosaic unit 31 according to the exemplary embodiment corrects the mosaic area 65 in the drive recorder image 60 in step S27 according to the distortion correction information, which represents the information about the lens distortion of the lens system 3b stored in the memory 15. This enables the position and size of the mosaic area 65 to be corrected to substantially match the facial area 63 of the recognized pedestrian. In other words, this enables the mosaic area 65 to mask or cover the facial area 63 of the recognized pedestrian.

[0104] Following the operation in step S27 , in step S28 , the controller 5 functions as the mosaic unit 31 to further correct the position and size of the mosaic area according to the movement of the vehicle 2 .

[0105] If the vehicle 2 is traveling, the position of the vehicle 2 at which the dashcam image is generated in step S9 is different from the position of the vehicle 2 at which the reference image is identified in step S25 in terms of the travel distance (see reference mark D). If the position and size of the mosaic area corrected in step S27 are applied to the dashcam image, there will be a large deviation between the facial area of ​​the recognized pedestrian and the mosaic area in the dashcam image.

[0106] From this point of view, in step S28, the mosaic unit 31 reads out the date and time when the drive recorder image is generated and the date and time when the reference image is generated from the memory 15. Then, the mosaic unit 31 calculates the time difference ΔT between the date and time when the drive recorder image is generated and the date and time when the reference image is recognized in step S28.

[0107] Next, in step S28, the mosaic unit 31 reads out the value of the vehicle speed V of the recognized reference image, and multiplies the time difference ΔT by the read value of the vehicle speed V, thereby calculating the travel distance D. Then, the mosaic unit 31 further corrects the mosaic area corrected in step S27 in the drive recorder image according to the calculated travel distance D, so that the further corrected mosaic area substantially matches the recognized pedestrian's facial area.

[0108] In addition, if the identified pedestrian is moving, the position of the pedestrian at the dashcam image generated in step S9 is a certain moving distance away from the position of the pedestrian at the reference image identified in step S25. If the position and size of the mosaic area corrected in step S27 are applied to the dashcam image, there will be a large deviation between the facial area of ​​the identified pedestrian and the mosaic area in the dashcam image.

[0109] From this point of view, the mosaic unit 31 can read out the date and time when the drive recorder image is generated, and read out the date and time when the reference image is generated in step S28 from the memory 15. Then, the mosaic unit 31 can calculate the time difference ΔT1 between the date and time when the drive recorder image is generated and the date and time when the reference image is recognized in step S28.

[0110] Next, in step S28, the mosaic unit 31 can calculate the moving speed value of the pedestrian at the identified reference image based on, for example, the position difference of the pedestrian between the captured images, and can multiply the time difference ΔT1 by the calculated moving speed value of the pedestrian, thereby calculating the moving distance. Then, the mosaic unit 31 can also correct the mosaic area corrected in step S27 in the drive recorder image according to the calculated travel distance, so that the further corrected mosaic area actually matches the facial area of ​​the identified pedestrian.

[0111] Note that the controller 5 can be programmed to perform one of the operations in steps S27 and S28.

[0112] Next, the controller 5 functions as a mosaic unit 31 to perform a mosaic task or a pixelation task, thereby blurring the mosaic area of ​​the dashcam image corrected in each operation in steps S27 and S28, thereby generating a partial mosaic dashcam image having a lower resolution per pixel of its mosaic area in step S29. Note that the predetermined second encryption key is a unique or non-unique data item having a predetermined bit length previously defined for the mosaic task performed by the mosaic unit 31 in step S29. Specifically, the second encryption key is input to the controller 5 using an unillustrated device so that the mosaic unit 31 or an ordinary computer can decode the partial mosaic dashcam image into the original dashcam image. The second encryption key of the mosaic unit 31 for the mosaic task performed in step S29 or information that can obtain the second encryption key is stored in the memory 15. After the operation in step S29 is completed, the dashcam image processing subroutine proceeds to step S30.

[0113] In step S30, the controller 5 functions as the image storage unit 41 to determine whether the history of the output of the vehicle control signal has been stored in the memory 15 to be associated with the reference image identified in step S25. When it is determined that the history of the output of the vehicle control signal has been stored in the memory 15 to be associated with the reference image identified in step S25 ("Yes" in step S30), the drive recorder image processing subroutine proceeds to step S31. Otherwise, when it is determined that the history of the output of the vehicle control signal has not been stored in the memory 15 to be associated with the reference image identified in step S25 ("No" in step S30), the drive recorder image processing subroutine proceeds to step S32.

[0114] In step S31 , the controller 5 functions as the image storage unit 41 to store the partial mosaic drive recorder image to which the mosaic task is applied in step S29 in the memory 15 ; the partial mosaic drive recorder image is generated immediately after identifying the reference image for controlling the vehicle 2 .

[0115] Next, in step S32, the controller 5 functions as the output unit 43 to determine whether a wireless output request has been input into the controller by radio. For example, the storage device 8 can output a wireless output request to the vehicle-mounted camera 1 by radio.

[0116] When it is determined that the wireless output request has been input to the controller 5 ("Yes" in step S32), the drive recorder image processing subroutine proceeds to step S33. Otherwise, when it is determined that the wireless output request has not been input to the controller 5 ("No" in step S32), the drive recorder image processing subroutine proceeds to step S33.

[0117] In step S33 , the controller 5 functions as the image compression unit 45 to compress at least one of the full-mosaic drive recorder image and the partial-mosaic drive recorder image by a predetermined first compression condition including a predetermined first compression ratio.

[0118] Then, in step S34, the controller 5 is used as the output unit 43 to output at least one of the compressed full-mosaic dash cam image and the partial-mosaic dash cam image to the recorder 49 via the output port 11, thereby storing at least one of the compressed full-mosaic dash cam image and the partial-mosaic dash cam image in the recorder 49.

[0119] After the operation in step S34 or in parallel with the operation, in step S35, the controller 5 is used as the output unit 43 to output at least one of the compressed full-mosaic dash cam image and the partial mosaic dash cam image to the storage device 8 via the wireless communication device 7, thereby storing at least one of the compressed full-mosaic dash cam image and the partial mosaic dash cam image in the storage device 8.

[0120] Otherwise, in step S36, the controller 5 functions as the image compression unit 45 to compress at least one of the full-mosaic drive recorder image and the partial-mosaic drive recorder image by a predetermined second compression condition including a predetermined second compression ratio.

[0121] Then, in step S37, the controller 5 is used as the output unit 43 to output at least one of the compressed full-mosaic dash cam image and the partial-mosaic dash cam image to the recorder 49 via the output port 11, thereby storing at least one of the compressed full-mosaic dash cam image and the partial-mosaic dash cam image in the recorder 49.

[0122] The vehicle-mounted camera 1 of the exemplary embodiment is configured to generate a dashcam image and output the dashcam image, and is also configured to perform at least one of the vehicle control tasks based on the recognition image. This configuration of the vehicle-mounted camera 1 eliminates the need to install a dashcam camera in the vehicle 2 in addition to the vehicle-mounted camera 1, so that the installable space in the vehicle 2 can be saved.

[0123] The vehicle-mounted camera 1 is also configured to perform a mosaic task, thereby blurring a predetermined area of ​​the face of at least one identified pedestrian in the dashcam image. This configuration masks the predetermined area, thereby protecting the privacy of at least one identified pedestrian.

[0124] The vehicle-mounted camera 1 is also configured as follows:

[0125] 1. Correct the distortion of the recognition image caused by lens distortion;

[0126] 2. Identify a facial region in the corrected recognition image that contains the face of at least one recognized pedestrian

[0127] 3. Obtaining dashcam images for applying correction for lens distortion

[0128] 4. Correct the position and size of the mosaic area in the drive recorder image according to the distortion correction information, which represents information about the lens distortion of the lens system 3b.

[0129] Thus, this configuration enables the mosaic region to be matched with the facial region of at least one identified pedestrian with greater accuracy.

[0130] The first encryption key is input to the controller 5 so that the mosaic unit 31 can decode the full mosaic dashcam image to be decoded into the original dashcam image. Similarly, the second encryption key is input to the controller 5 so that the mosaic unit 31 can decode the partial mosaic dashcam image to be decoded into the original dashcam image. This enables a user with at least one of the first encryption key and the second encryption key to obtain the original dashcam image without the mosaic area. In contrast, this makes it impossible for a user without the first encryption key and the second encryption key to obtain the original dashcam image.

[0131] In addition, the vehicle-mounted camera 1 is configured to calculate the pitch amount of the vehicle 2, and determine the position of the mosaic area in the recognition image based on the pitch amount of the vehicle 2. This configuration enables the mosaic area to substantially match the facial area of ​​at least one recognized pedestrian even if the vehicle 2 is pitched up or down about the pitch axis by the pitch amount.

[0132] The vehicle-mounted camera 1 is configured to obtain the current position of the vehicle 2 and determine whether the current position of the vehicle 2 is located within one of the previously prepared specific sections. The vehicle-mounted camera 1 is configured to perform a mosaic task or a pixelation task when determining that the current position of the vehicle 2 is located within one of the previously prepared specific sections. This configuration enables the privacy of at least one object located within one of the specific areas to be protected.

[0133] The vehicle-mounted camera 1 mounted to the vehicle 2 is configured to store a drive recorder image generated immediately after recognizing a reference image in the storage unit 9. This configuration enables a drive recorder image showing a front view to be stored in the storage medium 9.

[0134] The vehicle-mounted camera 1 can wirelessly output the generated drive recorder image to the outside of the vehicle 2. This enables the drive recorder image to be easily stored.

[0135] The vehicle camera 1 is configured so that the output port 11 for outputting the driving recorder image therefrom uses any one of a USB interface, an LVDS interface, or an Ethernet interface. Therefore, this configuration enables the vehicle camera 1 to be easily connected to other devices installed in the vehicle 2 via the vehicle network.

[0136] Modification

[0137] The first embodiment of the present disclosure has been described above, but the present disclosure is not limited thereto. Specifically, in step S12, the object recognition unit 27 may recognize at least one target object other than the pedestrian to be recognized in the acquired recognition image according to known image recognition techniques. For example, the object recognition unit 27 may be configured to recognize the front vehicle 73 (see FIG. 1 ) in addition to the vehicle 2 in the recognition image 59. Figure 8 ).

[0138] When the object recognition unit 27 recognizes the front vehicle 71 in addition to the vehicle 2, for example, the area recognition unit 29 may recognize the area 73 of the license plate of the front vehicle 71 in step S13; the area 73 of the license plate represents an example of a specific area in the recognition image. In addition, the mosaic unit 31 may correct the position of the area 73 of the license plate in the recognition image based on the pitch amount read out in step S14 in step S15. Thereafter, the mosaic unit 31 may perform a mosaic task or a pixelation task, thereby blurring the area 73 of the license plate in the drive recorder image, thereby generating a partially mosaic drive recorder image.

[0139] The recorder 49 may be installed in the vehicle-mounted camera 1 .

[0140] The vehicle-mounted camera 1 may be configured to store the full-mosaic driving recorder image or the partial-mosaic driving recorder image in the storage medium 9 .

[0141] The image storage unit 23 may store the recognition image in one of the memory 15 and a memory device other than the memory 15 , and may also store the drive recorder image in the other of the memory 15 and the memory device.

[0142] like Figure 1 As shown by the dotted line in , the vehicle-mounted camera 1 may include a card slot CS into which a memory card MC may be inserted. When the memory card MC is inserted into the card slot CS, the memory card MC may be electrically connected to the controller 5. This enables the controller 5 to record the driving recorder image in the memory card MC.

[0143] The vehicle-mounted camera 1 may be configured to capture an image from another direction view of the vehicle 2 , such as an image from a rear view, a right view, or a left view of the vehicle 2 .

[0144] The function of an element in the exemplary embodiment can be distributed as multiple elements, and the function of an element in the exemplary embodiment can be implemented by multiple elements. The function that multiple elements have can be implemented by one element, and a function performed by multiple elements can be implemented by one element. At least a part of the structure of the embodiment can be replaced with a known structure having the same function as at least a part of the structure of the corresponding embodiment. A part of the structure of the above-mentioned embodiment can be removed. At least a part of the structure of the above-mentioned embodiment can be added to or replaced with one of the structures of the modification. All aspects included in the technical idea specified by the language adopted by the claims constitute the embodiment of the present disclosure.

[0145] The present disclosure may be implemented through various embodiments, including: a system each including a vehicle-mounted camera 1, a program for a computer used as a controller 5 of the vehicle-mounted camera 1, and an involuntary storage medium (such as a semiconductor memory) storing the program. The present disclosure may be implemented through an image generation method, a cruise control method, and / or a mosaic processing method.

Claims

1. A vehicle-mounted camera, comprising: an image generation unit configured to generate a first image and a second image of a directional view from the vehicle such that the first image and the second image are temporally different from each other; an object recognition unit configured to recognize a target object in the first image; a region recognition unit configured to recognize, in the first image, a specific region in the target object recognized by the object recognition unit; a vehicle control unit configured to perform a vehicle control task based on the identified target object; as well as Mosaic unit, which is configured to: identifying a specific area in the second image that corresponds to a specific area in the first image; setting a mosaic area in the first image according to information about a specific area in the first image, and setting a mosaic area in the second image according to information about the mosaic area in the first image; When at least one of a position and a size of a mosaic area in the second image is different from at least one of a position and a size of a mosaic area in the first image, correcting the at least one of a position and a size of the mosaic area in the second image so that the corrected mosaic area in the second image matches the mosaic area in the first image, thereby masking a specific area in the second image; as well as A mosaic task is performed to blur a corrected mosaic area in the second image, the corrected mosaic area including at least a specific area in the second image, wherein: The image generation unit is configured to generate one of the first image and the second image based on incident light via an optical system, The vehicle-mounted camera also includes: an image correction unit configured to perform correction on the first image so as to reduce distortion of the first image caused by distortion of the lens system, And among them: The object recognition unit is configured to recognize a target object in the first image that has been corrected by the image correction unit; and The mosaic unit is configured to correct at least one of a position and a size of a mosaic area in the second image according to information about the correction performed by the image correction unit.

2. The vehicle-mounted camera according to claim 1, wherein: The target object identified by the object identification unit is a pedestrian; and The specific area in each of the first image and the second image represents an area of ​​the pedestrian's face.

3. The vehicle-mounted camera according to claim 1, wherein: the target object recognized by the object recognition unit is a target vehicle other than the vehicle; and The specific area in each of the first image and the second image represents an area of ​​the license plate of the target vehicle.

4. The vehicle-mounted camera according to claim 1, wherein: The mosaic unit is configured to perform a mosaic task, thereby generating at least one mosaic image, a mosaic area of ​​the at least one mosaic image being blurred; and The at least one mosaic image is configured to be decodable based on key information previously defined for the mosaic task.

5. The vehicle-mounted camera according to claim 1, wherein: The target object is used by the vehicle control unit to perform the vehicle control task; and The second image in which the mosaic area has been blurred is generated by the image generation unit immediately after the first image is used by the vehicle control unit for executing the vehicle control task.

6. The vehicle-mounted camera according to claim 1, wherein: The mosaic unit is configured to store the second image with the blurred mosaic area in a storage medium.

7. The vehicle-mounted camera according to claim 1, further comprising: An output unit is configured to output the second image having the blurred mosaic area to the outside.

8. The vehicle-mounted camera according to claim 7, wherein: The output unit is configured to wirelessly output the second image having the blurred mosaic area to an external device.

9. The vehicle-mounted camera according to claim 7, wherein: The output unit is configured to output the second image having the blurred mosaic area to an external device through an output port, and the output port has one of a universal serial bus interface or a low voltage differential signaling interface.

10. A vehicle-mounted camera, comprising: an image generation unit configured to generate a first image and a second image of a directional view from the vehicle such that the first image and the second image are temporally different from each other; an object recognition unit configured to recognize a target object in the first image; a region recognition unit configured to recognize, in the first image, a specific region in the target object recognized by the object recognition unit; a vehicle control unit configured to perform a vehicle control task based on the identified target object; a pitch calculation unit configured to calculate a pitch amount of the vehicle at a time when the image generation unit generates the first image; as well as Mosaic unit, which is configured to: identifying a specific area in the second image that corresponds to a specific area in the first image; setting a mosaic area in the first image according to information about a specific area in the first image, and setting a mosaic area in the second image according to information about the mosaic area in the first image; When at least one of a position and a size of a mosaic area in the second image is different from at least one of a position and a size of a mosaic area in the first image, correcting the at least one of a position and a size of the mosaic area in the second image so that the corrected mosaic area in the second image matches the mosaic area in the first image, thereby masking a specific area in the second image; as well as performing a mosaic task so as to blur a corrected mosaic area in the second image, the corrected mosaic area including at least a specific area in the second image, in: The mosaic unit is configured to correct at least one of a position and a size of a mosaic area in the second image according to the pitch amount.

11. A vehicle-mounted camera, comprising: an image generation unit configured to generate a first image and a second image of a directional view from the vehicle such that the first image and the second image are temporally different from each other; an object recognition unit configured to recognize a target object in the first image; a region recognition unit configured to recognize, in the first image, a specific region in the target object recognized by the object recognition unit; a vehicle control unit configured to perform a vehicle control task based on the identified target object; a vehicle position obtaining unit configured to obtain a position of the vehicle; a vehicle position determination unit configured to determine whether the position of the vehicle is located within a predetermined specific section; as well as Mosaic unit, which is configured to: identifying a specific area in the second image that corresponds to a specific area in the first image; setting a mosaic area in the first image according to information about a specific area in the first image, and setting a mosaic area in the second image according to information about the mosaic area in the first image; When at least one of a position and a size of a mosaic area in the second image is different from at least one of a position and a size of a mosaic area in the first image, correcting the at least one of a position and a size of the mosaic area in the second image so that the corrected mosaic area in the second image matches the mosaic area in the first image, thereby masking a specific area in the second image; as well as performing a mosaic task so as to blur a corrected mosaic area in the second image, the corrected mosaic area including at least a specific area in the second image, in: The mosaic unit is configured to, when it is determined that the position of the vehicle is located within a predetermined specific section, set the entire second image as a mosaic area and perform a mosaic task to blur the entire second image without correcting the mosaic area.

12. A vehicle-mounted camera, comprising: an imaging unit installed in a vehicle; a housing configured to house the imaging unit so that the imaging unit can capture a first image and a second image of a directional view from the vehicle, wherein the first image and the second image are temporally distinct from each other; an output port mounted on the housing and communicatively connectable to an external device; and A processor installed in the housing and capable of communicating with a memory storing a program, the processor being configured to execute the following instructions according to the program: a first instruction for identifying a target object in the first image; a second instruction for identifying, in the first image, a specific area in the target object identified by the object identification unit; a third instruction for identifying a specific area in the second image that corresponds to a specific area in the first image; a fourth instruction, which sets a mosaic area in the first image according to information about a specific area in the first image, and sets a mosaic area in the second image according to information about the mosaic area in the first image; a fifth instruction, which, when at least one of a position and a size of the mosaic area in the second image is different from at least one of a position and a size of the mosaic area in the first image, corrects the at least one of a position and a size of the mosaic area in the second image so that the corrected mosaic area in the second image matches the mosaic area in the first image, thereby masking a specific area in the second image; a sixth instruction, which performs a mosaic task to blur a corrected mosaic area in the second image, wherein the corrected mosaic area includes at least a specific area in the second image; a seventh instruction to output the second image having the blurred mosaic area to the outside via the output port mounted to the housing; and an eighth instruction for performing correction on the first image so as to reduce distortion of the first image caused by distortion of the lens system, wherein the first instruction identifies a target object in the corrected first image, The fifth instruction corrects at least one of a position and a size of a mosaic area in the second image according to information about the performed correction.

Images