Imaging device and program

Abstract

To provide an imaging apparatus and a program that can restrict a reproduction of an imaging prohibition object while maintaining a function as a dashboard camera.SOLUTION: An imaging apparatus 3 comprises a control unit 6 capable of restricting a display of a target included in a captured image imaging the surrounding of a vehicle 2, wherein the control unit determines whether or not the captured image includes an imaging restriction target in which imaging is restricted, based on an imaging condition under which the captured image has been imaged, and in the case that the captured image includes the imaging restriction target, extracts a predetermined area including the imaging restriction target in the captured image, and performs display restriction processing for restricting a display of the predetermined area when the captured image is used.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an imaging device and a program capable of storing images around a vehicle. [Background Art]

[0002] In recent years, drive recorders that capture images while a vehicle is traveling have become widespread. Since drive recorders constantly store captured images while the vehicle is in operation, there was a possibility that objects for which imaging is prohibited may be stored and then reproduced. For example, Patent Document 1 describes an imaging device that restricts the storage of objects for which imaging is prohibited. The imaging device described in Patent Document 1 includes a control unit that, when it is determined that an imaging-prohibited object exists based on position information, stores the captured image with restrictions such as blurring applied to the entire captured image. [Prior Art Documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2020-160915 [Summary of the Invention] [Problems to be Solved by the Invention]

[0004] In the imaging device described in Patent Document 1, even if the captured image includes traffic objects related to traffic such as traffic signals or vehicles, if the captured image contains an imaging-prohibited object, restrictions are applied to the entire captured image before storage. Therefore, there is a risk that the imaging device described in Patent Document 1 cannot store images necessary for a drive recorder to analyze the cause of an accident.

[0005] An object of the present invention is to provide an imaging device and a program capable of restricting the reproduction of imaging-prohibited objects while maintaining the function as a drive recorder. [Means for Solving the Problems]

[0006] One aspect of the present invention is an imaging device comprising a control unit capable of restricting the display of objects included in an image captured of the area around a vehicle, wherein the control unit determines, based on the imaging conditions under which the image was captured, whether or not the image contains an object whose image capture is restricted, and if the image contains the object whose image capture is restricted, it extracts a predetermined region in the image containing the object whose image capture is restricted, and performs display restriction processing to restrict the display of the predetermined region when the image is used. [Effects of the Invention]

[0007] According to the present invention, it is possible to restrict the playback of images of objects that should not be captured while maintaining the function of a dashcam. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the configuration of the imaging system according to the embodiment. [Figure 2] This figure shows an example of an captured image. [Figure 3] This figure shows an example of an image captured that includes the subject of the image capture restriction. [Figure 4] This figure shows an example of an image captured where the object subject to imaging restrictions and the traffic object overlap. [Figure 5] This is a flowchart showing the processing flow of the imaging method. [Modes for carrying out the invention]

[0009] As shown in Figure 1, the imaging system 1 comprises a vehicle 2, an imaging device 3 mounted on the vehicle 2, and a server device 20 that is communicatively connected to a network W. The vehicle 2 is, for example, a four-wheeled automobile. The vehicle 2 may be a vehicle with two or more wheels, not just a four-wheeled automobile. The imaging device 3 is a drive recorder that captures images of the external environment of the vehicle 2 and stores the captured images. The server device 20 communicates with the imaging device 3 via the network W and backs up and stores the captured images taken by the imaging device 3. The server device 20 may also be used as a search database.

[0010] The imaging device 3 includes, for example, a detection unit 4 that detects information related to the movement of the vehicle 2. The imaging device 3 includes an operation unit 5 that accepts user input related to imaging. The imaging device 3 includes a control unit 6 that executes control related to imaging. The imaging device 3 includes a storage unit 7 that stores data and programs necessary for imaging and stores data of the captured images. The imaging device 3 includes a display unit 8 that displays the captured images. The imaging device 3 includes a communication unit 9 that can be connected to the network W.

[0011] The detection unit 4 includes, for example, a camera 4A that captures images of the external environment of the vehicle 2. The camera 4A is mounted in a position on the vehicle 2 that allows it to capture images of the environment in front of the vehicle. The camera 4A may also be mounted in a position that allows it to capture images of the environment behind or to the sides of the vehicle 2, in addition to the position that allows it to capture images of the environment in front of the vehicle 2. The imaging device 3 may be a panoramic camera that can capture wide-angle images of the environment around the vehicle 2. The camera 4A starts capturing images simultaneously with the start of operation of the vehicle 2 and stores the captured images in the storage unit 7, which will be described later.

[0012] The detection unit 4 includes a position detection unit 4B that detects the position, attitude, and acceleration of the vehicle 2. The position detection unit 4B may include, for example, a GPS (Global Positioning System) that detects the latitude and longitude coordinates of the vehicle 2. The detection unit 4 may also include sensors such as an angular velocity sensor and a gyrocompass to detect the direction of the vehicle 2. The detection unit 4 outputs the detected values ​​to the control unit 6.

[0013] The control unit 6 performs control related to the operation and imaging of the imaging device 3. Based on the detection values ​​detected by the detection unit 4, the control unit 6 performs processing to restrict the display of objects included in the captured image of the area around the vehicle 2. The control details of the control unit 6 will be described later. The control unit 6 is composed of at least one hardware processor such as a CPU (Central Processing Unit). The control unit 6 performs control based on data and programs stored in the storage unit 7. The control unit 6 stores unprocessed data of the captured image in the storage unit 7. As described later, if the captured image contains an object that should not be captured, the control unit 6 generates restricted data by adding restrictions to the unprocessed data of the captured image stored in the storage unit 7 and stores it in the storage unit 7.

[0014] The storage unit 7 is composed of non-temporary storage media such as a hard disk drive (HDD) or solid-state disk (SSD). The storage unit 7 stores computer programs and data necessary for control. The storage unit 7 also stores images captured by the camera 4A.

[0015] The control unit 5 is comprised of an interface that receives operations related to the camera 4A. The control unit 5 consists of physical operation buttons and operation images displayed on a touch panel. The control unit 5 receives user operations based on the operations entered into the physical operation buttons and the operations entered into the operation images displayed on the touch panel. Based on the operations received by the control unit 5, the control unit 6 adjusts the degree of processing applied to the captured image.

[0016] The display unit 8 displays a captured image captured by the camera 4A. The display unit 8 is configured by a display device capable of displaying a display image, such as a liquid crystal display or an organic EL (Electro-Luminescence) display, for example. The display unit 8 may be configured by a touch panel, or may be integrated with the operation unit 5.

[0017] As shown in Figure 2, the captured image M captured by the camera 4A may include an imaging restriction target R for which imaging is restricted. The imaging restriction target R is set based on, for example, laws and regulations, customs, etc. of the area where the vehicle 2 travels, such as military facilities and religious facilities. The imaging restriction targets R are buildings, structures, standing statues, portraits, and the like.

[0018] When the captured image M includes an imaging restriction target, the control unit 6 generates restriction data obtained by imposing restrictions on unprocessed data of the captured image stored in the storage unit 7. The control unit 6 executes processing to restrict display of an object included in the captured image M obtained by imaging the surroundings of the vehicle 2. The control unit 6 determines whether or not the captured image M includes the imaging restriction target R for which imaging is restricted, based on imaging conditions under which the captured image M was captured. The imaging conditions include various conditions such as laws and regulations and customs at the current position of the vehicle 2, and settings by a user.

[0019] The control unit 6 determines whether or not the imaging restriction target R is included, based on data related to imaging conditions stored in the storage unit 7. The control unit 6 may connect to the server device 20 via the network W, and determine whether or not the imaging restriction target R is included based on data stored in the storage unit 22 of the server device 20. For example, the control unit 6 determines whether or not an imaging restriction target R for which imaging is prohibited exists around the position of the vehicle 2. For example, the control unit 6 determines whether or not the captured image includes the imaging restriction target R based on position information of the vehicle 2 detected by the position detection unit 4B and data related to the imaging restriction target R stored in the storage unit 7.

[0020] The control unit 6 calculates the position and orientation of the vehicle 2 based on the position information of the vehicle 2. The control unit 6 determines whether an imaging restriction target R is included in the captured image M based on the position information of the vehicle 2 and the calculated value of the orientation of the vehicle 2. If the detection unit 4 is provided with an object detection unit such as a lidar device capable of detecting objects around the vehicle 2, the control unit 6 may use the detection value of the object detection unit for the determination.

[0021] The control unit 6 is adjusted in advance to execute image processing capable of recognizing an object included in the captured image M based on, for example, machine learning using deep learning. The control unit 6 is configured to improve the recognition accuracy of the target object by, for example, using an image obtained by capturing the target object as teacher data and repeating machine learning to extract the target object from the image. The control unit 6 recognizes objects such as buildings, roads, vehicles, pedestrians, structures, and signs included in the captured image M based on image processing, for example, and extracts a region in the image corresponding to the target object.

[0022] When the imaging restriction target R is included in the captured image M, the control unit 6 extracts a predetermined region R1 including the imaging restriction target R from the captured image M. For example, based on analysis of pixels of the captured image M, the control unit 6 detects edges included in the image of the imaging restriction target R, and extracts the predetermined region R1 along the contour of the imaging restriction target R. The control unit 6 generates unprocessed data of the captured image M stored in the storage unit 7 and restricted data obtained by imposing restrictions on the unprocessed data. The storage unit 7 stores the unprocessed data and the restricted data. The unprocessed data is used as evidence data for a drive recorder. The restricted data is used for the user's ordinary purposes.

[0023] The control unit 6 performs display restriction processing for restricting display of the predetermined region when the captured image M is used. Cases where the captured image M is used include, for example, situations such as reproduction of the captured image M, extraction of data of the captured image M, and communication for accessing data of the captured image M. As shown in FIG. 3, the control unit 6 causes the display unit 8 to display a processed captured image M1 including the display restriction process RA based on the restricted data obtained by performing the display restriction process RA on the predetermined region R1.

[0024] The control unit 6 performs display restriction processing RA based on image processing such as blurring or mosaic on a predetermined region R1, for example, to make the area R subject to imaging restriction unidentifiable. The control unit 6 may also perform image processing to insert a colored image into the predetermined region R1, or to insert text such as "Imageing Prohibited". The control unit 6 may perform any image processing as long as it makes the predetermined region R1 unidentifiable.

[0025] As shown in Figure 4, if the captured image includes an area R subject to image restriction, the area R subject to image restriction and traffic objects J related to traffic, such as other vehicles V, may overlap in the captured image. In this state, it is desirable that the control unit 6 does not perform display restriction processing on the traffic objects J present in the captured image so as not to affect the evidence preservation function as a drive recorder or driving assistance. The control unit 6 determines whether or not there is an overlapping area RC in the captured image M in which a predetermined area R1 and traffic objects J overlap.

[0026] If the control unit 6 finds an overlapping region RC in the captured image where a predetermined region R1 and the object of traffic J overlap, it does not perform display restriction processing on the overlapping region, but instead performs an adjusted display restriction processing RB that excludes the overlapping region RC from the predetermined region R1, and displays the processed captured image M2 including the processed predetermined region R1 on the display unit. If the control unit 6 performs an adjusted display restriction processing RB that makes it impossible to identify the predetermined region R1, it stores restricted data in the storage unit 7 by adding restrictions to the unprocessed data of the captured image M.

[0027] The captured image M may be processed based on user operations. When using the captured image M, the user may set imaging restrictions based on privacy protection considerations. The control unit 6 may extract imaging restrictions that require privacy protection, such as people, vehicle license plates, signs, nameplates, address displays, buildings, structures, and landscapes, that are present in the captured image M, based on the setting level input to the operation unit 5. The control unit 6 may perform display restriction processing on the extracted imaging restrictions, display the processed captured image on the display unit 8, and store the processing data in the storage unit 7.

[0028] The data of the captured image captured by the imaging device 3 may be stored in the storage unit 22 of the server device 20 via the network W. In this case, the processing related to display restriction processing of the captured image M by the control unit 6 described above may be performed by the control unit 21 of the server device 20. The control unit 21 of the server device 20 may cooperate with the control unit 6 of the imaging device 3 to perform some or all of the above processing.

[0029] Figure 5 shows the processing flow of the imaging method performed in the imaging device 3. The imaging method is performed by a program installed on the computer mounted in the imaging device 3. The program causes the computer to perform the following processes: The control unit 6 acquires an image of the area around the vehicle 2 (step S100). Based on the imaging conditions under which the image was taken, the control unit 6 determines whether or not the image contains an object that is restricted from being captured (step S102).

[0030] If the captured image contains an object subject to image capture restrictions (step S102: Yes), the control unit 6 extracts a predetermined region in the captured image that contains the object subject to image capture restrictions (step S104). The control unit 6 performs display restriction processing to restrict the display of the predetermined region when the captured image is used (step S106). The control unit 6 displays the processed captured image with the display restriction processing on the display unit 8 (step S108). If the captured image does not contain an object subject to image capture restrictions in step S102 (step S102: No), the control unit 6 repeatedly performs the determination process in step S102.

[0031] As described above, the imaging device 3 can restrict the playback of captured images that include objects that are prohibited from being captured, while maintaining its function as a drive recorder. The imaging device 3 can extract a predetermined area containing objects that are restricted from being captured and apply display restriction processing to that predetermined area if such objects are included. The imaging device 3 can restrict the use of objects that are prohibited from being captured while maintaining its function as a drive recorder by applying an adjusted display restriction processing to areas in which objects that are prohibited from being captured and traffic objects overlap, thereby removing the overlapping area from the objects that are prohibited from being captured.

[0032] In the embodiments described above, the computer programs executed in each configuration of the imaging system 1 may be provided in the form of being stored in a computer-readable, portable storage medium such as a semiconductor memory, a magnetic storage medium, or an optical storage medium. [Explanation of Symbols]

[0033] 1 Imaging system, 2 Vehicle, 3 Imaging device, 4 Detection unit, 5 Operation unit, 6 Control unit, 7 Storage unit, 8 Display unit, 9 Communication unit, 20 Server device, 21 Control unit, 22 Storage unit, J Traffic object, M Imaging image, M1, M2 Processed imaging image, R Imaging restriction target, R1 Determined area, RA Display restriction processing, RB Adjusted display restriction processing, RC Overlap area, V Vehicle, W Network

Claims

1. It includes a control unit that can restrict the display of objects included in the captured image of the area around the vehicle, The control unit, Based on the imaging conditions under which the image was captured, it is determined whether or not the image contains an object that is restricted from being captured. If the captured image includes the object subject to image capture restrictions, a predetermined region in the captured image that includes the object subject to image capture restrictions is extracted based on machine learning using deep learning, or by detecting edges included in the image of the object subject to image capture restrictions. When the captured image is used, display restriction processing is performed to restrict the display of the predetermined area. If, in the captured image, there is an overlapping region where the predetermined region and a traffic object related to traffic overlap, an adjustment display restriction process is performed to remove the overlapping region from the predetermined region. Imaging device.

2. Acquiring images of the area around the vehicle, Based on the imaging conditions under which the image was captured, it is determined whether or not the image contains an object that is restricted from being captured. If the captured image includes the object subject to image capture restrictions, a predetermined region in the captured image that includes the object subject to image capture restrictions is extracted based on machine learning using deep learning, or by detecting edges included in the image of the object subject to image capture restrictions. When the captured image is used, display restriction processing is performed to restrict the display of the predetermined area. A program that causes a computer to perform an adjustment display restriction process, which removes the overlapping area from the predetermined area if an overlapping area exists in the captured image where the predetermined area and a traffic object related to traffic are present.

Images