Image processing device, image processing system, and image processing program

Abstract

To provide an image processing system, an image processing method, and an image processing program for concealing subtlety information controllable whether or not restoration is possible with high security.SOLUTION: An image processing system 10 includes a recognition unit 153, a processing unit 154, an encryption unit 156 and an incorporation unit 157. The recognition unit 153 recognizes a portion to be concealed from a captured image. The processing unit 154 processes the captured image so that the portion recognized by the recognition unit 153 is concealed. The encryption unit 156 encrypts data related to the portion recognized by the recognition unit 153. The incorporation unit 157 incorporates the image data processed by the processing unit 154 and the data encrypted by the encryption unit 156.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing system and image processing programs. [Background technology]

[0002] Conventionally, there are known techniques for concealing sensitive information, such as the faces of people and the license plates of other vehicles, that appear in images captured from a vehicle. For example, there are known methods for painting over or mosaic-processing parts of an image that correspond to sensitive information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-164275 [Patent Document 2] Japanese Patent Application Laid-Open No. 2010-278650 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional techniques have a problem in that it is difficult to conceal sensitive information in an image in a way that is both highly secure and allows control over whether or not it can be restored. For example, conventional techniques use irreversible concealment methods such as painting.

[0005] The present invention has been made in consideration of the above, and aims to provide an image processing device and an image processing method that are highly secure and can conceal sensitive information in an image in a way that allows control over whether it can be restored. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the image processing device according to the present invention has a recognition unit, a processing unit, an encryption unit, and a combining unit. The recognition unit recognizes a portion to be concealed from a captured image. The processing unit processes the captured image so that the portion recognized by the recognition unit is hidden. The encryption unit encrypts data related to the portion recognized by the recognition unit. The combining unit combines the data of the image processed by the processing unit with the data encrypted by the encryption unit. [Effects of the Invention]

[0007] According to the present invention, it is possible to conceal sensitive information in an image with high security and in a manner that allows control over whether or not it can be restored. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an image processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of an image. [Figure 3] FIG. 3 is a diagram illustrating recognition of the secret portion. [Figure 4] FIG. 4 is a diagram illustrating image processing. [Figure 5] FIG. 5 is a diagram illustrating encryption of secret information and merging of data. [Figure 6] FIG. 6 is a diagram showing an example of the data structure of data obtained by merging. [Figure 7] FIG. 7 is a diagram showing an example of a processed image. [Figure 8] FIG. 8 is a diagram showing an example of an image selection screen. [Figure 9] FIG. 9 is a diagram showing an example of the setting screen. [Figure 10] FIG. 10 is a diagram showing an example of an image display mode. [Figure 11] FIG. 11 is a diagram illustrating an example of the configuration of an image processing device. [Figure 12] FIG. 12 is a diagram illustrating an example of the configuration of a terminal. [Figure 13] FIG. 13 is a diagram illustrating an example of the configuration of a terminal. [Figure 14] FIG. 14 is a flowchart showing the flow of processing by the image processing device. [Figure 15] FIG. 15 is a flowchart showing the flow of processing by the terminal. [Figure 16] FIG. 16 is a diagram illustrating image processing. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of an image processing device, an image processing method, and an image processing program disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

[0010] First, an image processing system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an image processing system according to an embodiment.

[0011] As shown in FIG. 1, the image processing system 1 includes a vehicle V, an image processing device 10 mounted on the vehicle V, a terminal 20, and a terminal 30.

[0012] The image processing device 10 is connected to a terminal 20 and a terminal 30 so as to be able to communicate data with each other via a network N. For example, the network N is the Internet.

[0013] The following describes the flow of processing in the image processing system 1. As shown in Fig. 1, the image processing device 10 captures an image using a camera (step S11).

[0014] For example, the image processing device 10 captures an image of the area ahead of the vehicle V. At this time, the captured image may include so-called sensitive information, such as a license plate displayed on the rear of a vehicle ahead of the vehicle V and the face of a person passing by.

[0015] Fig. 2 is a diagram showing an example of an image. In the example of Fig. 2, an image 500 captured by the image processing device 10 includes a vehicle license plate and a person's face.

[0016] One of the purposes of the image processing system 1 of this embodiment is to appropriately handle sensitive information captured in an image. Specifically, the image processing system 1 can control whether or not to display sensitive information depending on whether or not the terminal that outputs the image has authority.

[0017] The image processing device 10 encodes the image that has been processed to hide the secret portion, and further encrypts the data of the secret portion (step S12).

[0018] The confidential portion is a portion of an image from which predetermined sensitive information can be identified, such as a vehicle license plate or a person's face.

[0019] First, the image processing device 10 recognizes a portion to be concealed from a captured image. Fig. 3 is a diagram illustrating the recognition of a concealed portion. In the example of Fig. 3, the image processing device 10 recognizes a rectangular area including a license plate of a vehicle or a person's face as a concealed portion.

[0020] For example, the image processing device 10 can perform pattern recognition of the secret portion using a predetermined image recognition model generated by a machine learning method such as deep learning.

[0021] At this time, the portion recognized by the image processing device 10 is called a secret portion.

[0022] Fig. 4 is a diagram for explaining image processing. As shown in Fig. 4, the image processing device 10 processes a captured image so that the secret portion is hidden.

[0023] For example, the image processing device 10 processes the image by replacing the secret portion with a predetermined pattern such as mosaic or blackout. The image 501 is an image that has been processed. This makes it possible to easily hide the secret portion.

[0024] The image processing device 10 encrypts data relating to the secret portion. For example, the image processing device 10 encrypts an image of an area including the secret portion. By decrypting the encrypted data, the image of the secret portion can be restored without changing it.

[0025] 4, the image processing device 10 encrypts the license plate image 502. The image processing device 10 may perform encryption by converting (compressing) the image into a password-protected ZIP file.

[0026] When there are multiple secret parts, the image processing device 10 can encrypt each secret part individually.

[0027] The image processing device 10 may encrypt not only images but also data such as character strings that can identify confidential parts.

[0028] Then, the image processing device 10 transmits data obtained by combining the encoded data and the encrypted data (hereinafter, referred to as combined data) (step S13).

[0029] This provides combined data with high security and allows control over the decryption of the secret portion.

[0030] The image processing device 10 encodes the image processed by the image processing device 10 into data in a predetermined format. The image processing device 10 generates a single file that combines the data encoded by the image processing device 10 and the data encrypted by the image processing device 10.

[0031] In this way, the image processing device 10 can combine the encoded data and the encrypted data into one file, making it easier to handle the combined data.

[0032] In this embodiment, "encoding" means converting an image into a predetermined format such as H.264, and "encrypting" means concealing data using a password or the like.

[0033] For example, as shown in Fig. 5, the image processing device 10 converts a processed image 501 into video data 501a in H.264 format. Fig. 5 is a diagram illustrating encryption of confidential information and combining of data.

[0034] Furthermore, the image processing device 10 combines the video data 501 with encrypted data 502a obtained by encrypting the image 502 of the secret portion.

[0035] Fig. 6 is a diagram showing an example of the data structure of the data obtained by combining. As shown in Fig. 6, video data 501a and encrypted data 502a are stored in an mp4 format container.

[0036] The encrypted data 502a may be video data as shown in Fig. 6, or may be still image data. The encrypted data 502a may be stored in an area for predetermined extended data in mp4 format, or may be stored in an area that exceeds the size of the video portion.

[0037] The encrypted data 502a may also include coordinate information indicating the position of the image of the secret portion in each frame of the image output by the video data 501a.

[0038] Here, it is assumed that the terminal 20 does not have the authority to display sensitive information, whereas the terminal 30 has the authority to display sensitive information.

[0039] For example, the presence or absence of authorization is realized by whether or not a key for decrypting the encrypted portion or an application having the function for decrypting the encrypted portion is provided.

[0040] Since the terminal 20 does not have the authority, it does not decrypt the encrypted data included in the combined data, and displays the decoded image (step S14).

[0041] The terminal 20 outputs a processed image as shown in Fig. 7. Fig. 7 is a diagram showing an example of a processed image. In the example of Fig. 7, the secret part remains hidden.

[0042] The combined data received by the terminal 20 includes encrypted data 503, but the terminal 20 does not have the authority to decrypt the encrypted data 503.

[0043] Therefore, the terminal 20 outputs the video data 501a as is, and as a result, the terminal 20 displays a processed image as shown in FIG.

[0044] On the other hand, since the terminal 30 has the authority, it combines the secret portion obtained by decrypting the encrypted data included in the combined data with the decoded image and displays the combined image (step S15).

[0045] The terminal 30 can restore, for example, the license plate and the person's face and combine them into the processed image. As a result, the terminal 30 displays an image equivalent to the captured image shown in FIG. 2.

[0046] The terminal 30 may accept the selection of an image to be output from a selection screen such as that shown in Fig. 8. Fig. 8 is a diagram showing an example of the image selection screen. The terminal 30 combines the selected image with a secret portion and outputs the image.

[0047] Furthermore, the terminal 30 may accept function settings from a setting screen such as that shown in Fig. 9. Fig. 9 is a diagram showing an example of the setting screen.

[0048] On the setting screen, the terminal 30 first accepts a setting as to whether or not to decipher. Then, when deciphering, the terminal 30 accepts a setting as to whether or not to display the concealed portion in a composite manner, whether or not to display the concealed portion independently, and the setting as to the object to be deciphered.

[0049] For example, if both a license plate and a person's face are present as concealed portions, the terminal 30 can display only the selected concealed portion.

[0050] Furthermore, the terminal 30 may display the image together with a map and parameters such as the G value, as shown in Fig. 10. Fig. 10 is a diagram showing an example of a display mode of the image.

[0051] The following describes the configuration of each device in the image processing system 1. The image processing device 10 may be a drive recorder mounted on a vehicle V.

[0052] The image processing device 10 may also be realized by an ECU (Electronic Control Unit) mounted on the vehicle V, or a server capable of data communication with the vehicle V.

[0053] 11 is a functional block diagram showing the configuration of an image processing device according to an embodiment. As shown in FIG. 11, an image processing device 10 includes a communication unit 11, a camera 12, a microphone 13, a storage unit 14, and a control unit 15.

[0054] The communication unit 11 is an interface for communicating data with other devices. The image processing device 10 communicates with the terminal 20 and the terminal 30 via the communication unit 11.

[0055] The control unit 15 and memory unit 14 of the image processing device 10 are realized by a computer having, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), flash memory, input / output ports, etc., or various circuits.

[0056] The storage unit 14 corresponds to a RAM or a flash memory. The RAM or the flash memory stores model information 141 and image data 142.

[0057] The model information 141 is information for constructing a model for recognizing the concealed part. The image data 142 is data combining an image captured by the camera 12 with a sound collected by the microphone 13.

[0058] The computer's CPU functions as the judgment unit 151, recording unit 152, recognition unit 153, processing unit 154, encoding unit 155, encryption unit 156 and combination unit 157 of the control unit 15, for example, by reading and executing a program stored in ROM.

[0059] The image processing device 10 may acquire the above-mentioned programs and various information via another computer or portable recording medium connected via a wired or wireless network.

[0060] The determination unit 151 determines whether a predetermined event has occurred. For example, if the G-value being measured in real time satisfies a predetermined condition, it determines that an impact has been applied to the vehicle V.

[0061] The recording unit 152 records the images captured by the camera 12 and the audio collected by the microphone 13. The recording unit 152 stores the images and audio as image data 142 in the storage unit .

[0062] Furthermore, the recording unit 152 may start recording when the determining unit 151 determines that an event has occurred.

[0063] The recognition unit 153 recognizes the part to be concealed from the captured image.

[0064] The processing unit 154 processes the captured image so as to hide the portion recognized by the recognition unit 153. For example, the processing unit 154 replaces the portion recognized by the recognition unit 153 with a predetermined pattern.

[0065] The encoding unit 155 encodes the image processed by the processing unit 154 into data in a predetermined format.

[0066] The encryption unit 156 encrypts data relating to the portion recognized by the recognition unit 153. For example, the encryption unit 156 encrypts an image of an area including the portion recognized by the recognition unit 153.

[0067] The combining unit 157 combines the image data processed by the processing unit 154 with the data encrypted by the encryption unit 156. For example, the combining unit 157 generates one file by combining the data encoded by the encoding unit 155 and the data encrypted by the encryption unit 156.

[0068] The configuration of a terminal that does not have the authority to display sensitive information (for example, terminal 20 in FIG. 1) will be described with reference to FIG. 12. FIG. 12 is a diagram showing an example of the configuration of a terminal.

[0069] As shown in FIG. 12, the terminal 20 includes a communication unit 21, a display unit 22, an operation unit 23, a storage unit 24, and a control unit 25.

[0070] The communication unit 21 is an interface for communicating data with other devices. The terminal 20 communicates with the image processing device 10 via the communication unit 21.

[0071] The display unit 22 is a display device such as a display, and the operation unit 23 is an input device such as a button. The display unit 22 and the operation unit 23 may be touch panel displays.

[0072] The control unit 25 and storage unit 24 of the terminal 20 are realized by a computer having, for example, a CPU, a ROM, a RAM, a flash memory, an input / output port, etc., or various circuits.

[0073] The CPU of the computer functions as the decoding unit 251 and the output control unit 252 of the control unit 25 by reading and executing a program stored in, for example, the ROM.

[0074] The image processing device 10 may acquire the above-mentioned programs and various information via another computer or portable recording medium connected via a wired or wireless network.

[0075] The decoding unit 251 decodes the image data included in the combined data received from the image processing device 10. The output control unit 252 causes the display unit 22 to output the image obtained by the decoding unit 251.

[0076] In this way, the terminal 20 does not decrypt the encrypted data included in the combined data.

[0077] The configuration of a terminal (for example, terminal 30 in FIG. 1) that has the authority to display sensitive information will be described with reference to Fig. 13. Fig. 13 is a diagram showing an example of the configuration of a terminal.

[0078] As shown in FIG. 13, the terminal 30 includes a communication unit 31, a display unit 32, an operation unit 33, a storage unit , and a control unit .

[0079] The communication unit 31 is an interface for communicating data with other devices. The terminal 30 communicates with the image processing device 10 via the communication unit 31.

[0080] The display unit 32 is a display device such as a display, and the operation unit 33 is an input device such as a button. The display unit 32 and the operation unit 33 may be touch panel displays.

[0081] The control unit 35 and storage unit 34 of the terminal 30 are realized by a computer having, for example, a CPU, a ROM, a RAM, a flash memory, an input / output port, etc., or various circuits.

[0082] The CPU of the computer functions as a decoder 351, a decoding unit 352, a synthesis unit 353, and an output control unit 354 of the control unit 35 by reading and executing a program stored in, for example, the ROM.

[0083] The image processing device 10 may acquire the above-mentioned programs and various information via another computer or portable recording medium connected via a wired or wireless network.

[0084] The decoding unit 351 decodes the image data included in the combined data received from the image processing device 10 .

[0085] The decryption unit 352 decrypts the encrypted data included in the combined data to obtain an image of the secret part.

[0086] The synthesis unit 353 synthesizes the image obtained by encoding with the secret part obtained by decrypting the encrypted data.

[0087] The output control unit 354 causes the image synthesized by the synthesis unit 353 to be output to the display unit 32.

[0088] In this way, the terminal 30 decrypts the encrypted data included in the combined data and displays the sensitive information.

[0089] 14 is a flowchart showing the flow of processing by the image processing device. As shown in Fig. 14, the image processing device 10 waits until an event occurs (step S101, No). For example, the event is an impact to the vehicle V.

[0090] When an event occurs (step S101, Yes), the image processing device 10 records images and audio (step S102).

[0091] Next, the secret portion is recognized from the captured image (hereinafter, the entire image) (step S103).

[0092] If there is no secret portion (No at step S104), the image processing device 10 proceeds to step S108.

[0093] On the other hand, if there is a hidden portion (step S104, Yes), the image processing device 10 processes the region of the hidden portion in the entire image (step S105).

[0094] Then, the image processing device 10 encrypts the image of the secret area (step S106), and encodes the processed whole image (step S107).

[0095] The image processing device 10 combines the encrypted data and the encoded data (step S108), and transmits the combined data to each terminal (step S109).

[0096] 15 is a flowchart showing the flow of processing by the terminal. The terminal waits until an instruction to play an image is received (step S201, No). The processing in FIG.

[0097] First, the terminal decodes the encoded data (step S202).

[0098] If the confidential portion is not to be displayed (step S203, No), the terminal proceeds to step S206.

[0099] On the other hand, if the confidential part is to be displayed (step S203, Yes), the terminal decrypts the encrypted data (step S204). Note that the terminal displays the confidential part if it has the authority to display sensitive information.

[0100] Then, the terminal combines the entire image with the secret portion (step S205).

[0101] Thereafter, the terminal displays the designated image (step S206). If the secret portion is to be displayed, the terminal displays the image synthesized in step S205.

[0102] As described above, the image processing device 10 according to the embodiment includes the recognition unit 153, the processing unit 154, the encryption unit 156, and the combining unit 157. The recognition unit 153 recognizes a portion to be concealed from a captured image. The processing unit 154 processes the captured image so that the portion recognized by the recognition unit 153 is hidden. The encryption unit 156 encrypts data related to the portion recognized by the recognition unit 153. The combining unit 157 combines the data of the image processed by the processing unit 154 with the data encrypted by the encryption unit 156.

[0103] In this way, the image processing device 10 encrypts the secret portion separately from the image. As a result, according to this embodiment, it is possible to conceal sensitive information in an image with high security and with control over whether it can be restored.

[0104] For example, the encrypted hidden portion cannot be viewed from a terminal that does not have the key to decrypt the encrypted hidden portion data. In this way, whether or not the hidden portion can be restored can be controlled by assigning a key.

[0105] For example, when images are used for scenic footage in the tourism industry or for accident footage in driving schools, there is no need to keep the confidential portion, so a key is given to businesses that use such images.

[0106] On the other hand, if the images are to be used for vehicle insurance assessments, police investigations, marketing analysis, etc., confidential information is required, so businesses that use such images are given a key.

[0107] The combining unit 157 combines the data encoded by the encoding unit 155 and the data encrypted by the encryption unit 156 to generate one file.

[0108] The terminal decodes the processed image data included in the combined data acquired from the image processing device 10, and when the terminal is given information for decrypting the data encrypted by the image processing device 10, it combines the image obtained by decrypting the encrypted data with the decoded image.

[0109] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

[0110] For example, the image processing device 10 may regard an image 511 of a strip-shaped (band-shaped) area as data of a secret portion as shown in Fig. 16. Fig. 16 is a diagram for explaining image processing.

[0111] In this case, the encryption unit 156 encrypts the image of a rectangular area that includes the portion recognized by the recognition unit 153 and has a width equal to the width of the captured image.

[0112] In this way, by processing a rectangular area along the scanning direction of a general image, the load of image processing (memory usage, etc.) can be reduced. [Explanation of symbols]

[0113] 1. Image processing system 10 Image processing device 11, 21, 31 Communications Department 12 Camera 13. Mike 14, 24, 34 storage section 15, 25, 35 Control section 20, 30 terminals 22, 32 Display section 23, 33 Operation section 141 Model Information 142 Image data 151 Judgment section 152 Recording Department 153 Recognition part 154 Processing Department 155 Encoding section 156 Encryption section 157 Combined part 251, 351 Decoder 252, 354 Output control section 352 Decoding Unit 353 Synthesis Section 500, 501, 502 images N Network V vehicle

Claims

1. Recognizing a confidential object from an image taken by an in-vehicle camera, Encoding the processed image that has been processed to hide the secret target; Separating the area to be concealed from the image and encrypting the separated image data to be concealed; a control unit that generates a file that combines the encoded data of the processed image and the encrypted data in which the confidentiality target is encrypted; Image processing device.

2. The object to be concealed is a license plate, The control unit Encoding the processed image in which a rectangular area including the license plate, the width of which is the width of the image, and the height of which is the height of the license plate, has been concealed; generating a file that combines the encoded data of the processed image and the encrypted data in which the rectangular area is encrypted; The image processing device according to claim 1 .

3. The control unit If there are multiple confidentiality targets, the image data of each of the multiple confidentiality targets is individually encrypted; generating a file that combines the data of the processed image and a plurality of encrypted data in which the plurality of confidentiality targets are respectively encrypted; The image processing device according to claim 1 .

4. The plurality of concealment objects include a license plate and a person's face, The image processing device according to claim 3 .

5. The control unit transmitting the file to a terminal device; The image processing device according to claim 1 .

6. An image processing system comprising an image processing device and a terminal device capable of communicating with the image processing device, The image processing device includes: Recognizes confidential information from images captured by an in-vehicle camera, Encoding the processed image that has been processed to hide the secret target; Separating the area to be concealed from the image and encrypting the separated image data to be concealed; generating a file that combines the encoded data of the processed image and the encrypted data in which the confidentiality target is encrypted, and transmitting the file to the terminal device; The terminal device If the key for decrypting the encrypted data is available, the decrypted image to be concealed and the decoded processed image are combined and displayed; If the user does not have a key for decrypting the encrypted data, the decoded processed image is displayed. Image processing system.

7. The image processing device If there are multiple confidentiality targets, the image data of each of the multiple confidentiality targets is individually encrypted; generating a file that combines the data of the processed image and a plurality of encrypted data in which the plurality of confidentiality targets are respectively encrypted; The terminal device If the user has a key for decrypting the plurality of encrypted data, decrypting the encrypted data to be concealed selected by a user from the plurality of encrypted data, and combining the decrypted image to be concealed with the decoded processed image for display; The image processing system according to claim 6 .

8. Recognizing a confidential object from an image taken by an in-vehicle camera, Encoding the processed image that has been processed to hide the secret target; Separating the area to be concealed from the image and encrypting the separated image data to be concealed; generating a file that combines the encoded data of the processed image and the encrypted data in which the confidentiality target is encrypted; An image processing program that causes a computer to perform the processing.

Images