Image processing apparatus, image processing method, and image processing program

Abstract

To provide an in-vehicle device capable of reliably concealing a confidential information area in an image.SOLUTION: An in-vehicle device in an embodiment includes a detection unit and an anonymization unit. The detection unit detects a detection area, which is an area in which a predetermined object is captured from the image taken by the camera mounted on the vehicle. The anonymization unit anonymizes the detection area expanded by a predetermined expansion rate.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to Image processing an apparatus, an image processing method, and an image processing program.

Background Art

[0002] Conventionally, for the purpose of personal information protection, a method has been proposed in which a region corresponding to confidential information such as a face and a vehicle number plate is detected from an image taken by a drive recorder or the like by an object detection method, and the region is further anonymized by mask processing.

[0003] Also, a method of changing the position and size of a region where an object is detected is known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the conventional technology, it may be difficult to surely anonymize the region of confidential information in an image.

[0006] For example, the region detected by the object detection method does not always match the region to be anonymized. Therefore, for example, simply performing mask processing on the detected region of the object may not sufficiently protect personal information.

[0007] The present invention has been made in view of the above, and an object thereof is to surely anonymize the region of confidential information in an image.

Means for Solving the Problems

[0008] In order to solve the above-described problems and achieve the object, the Image processing device according to the present invention , the camera conceals an area expanded by an expansion rate, which is a detection area where a predetermined object appears, from an image taken by a camera. is specified, the expansion rate corresponding to the detection area is calculated, and the detection area is

Advantages of the Invention

[0009] According to the present invention, the area of confidential information in an image can be surely concealed.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Embodiments for Carrying Out the Invention

[0011] Hereinafter, with reference to the accompanying drawings, embodiments of the in-vehicle device, image processing method, and image processing program disclosed in the present application will be described in detail. Note that the present invention is not limited by the embodiments shown below.

[0012] First, the in-vehicle device according to the embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram showing a configuration example of the in-vehicle device according to the embodiment.

[0013] The in-vehicle device 10 detects a predetermined object from an image captured by a camera mounted on the vehicle and anonymizes the detected object.

[0014] The in-vehicle device 10 can anonymize an object by mask processing. The mask processing is, for example, filling in and mosaic processing of areas in the image.

[0015] FIG. 2 is a diagram showing an example of an image on which mask processing has been performed. Region 201 in FIG. 2 is a region on which mask processing has been performed by the in-vehicle device 10. Region 201 is a region where a person's face was shown before the mask processing.

[0016] The in-vehicle device 10 outputs, for example, an image on which mask processing has been performed.

[0017] Note that the in-vehicle device 10 may be a drive recorder mounted on the vehicle. Also, the processing by the in-vehicle device 10 may be executed by a personal computer, a server device, or the like having a function equivalent to that of the in-vehicle device 10.

[0018] As shown in FIG. 1, the in-vehicle device 10 includes an interface unit 11, a storage unit 12, and a control unit 13.

[0019] The interface unit 11 is an interface for communicating data with other devices. The in-vehicle device 10 receives an image input via the interface unit 11. Further, the in-vehicle device 10 outputs an image on which mask processing has been performed via the interface unit 11.

[0020] The control unit 13 and the storage unit 12 of the in-vehicle device 10 are realized by, for example, a computer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), flash memory, input / output ports, etc., and various circuits.

[0021] The storage unit 12 corresponds to a RAM or a flash memory. The RAM or the flash memory stores the model information 121.

[0022] The model information 121 is information for constructing a model for detecting an object from an image.

[0023] Here, the model constructed by the model information 121 is, for example, a learned neural network. In this case, the model information 121 is parameters such as weights and biases for constructing the neural network.

[0024] The CPU of the computer functions as the detection unit 131 and the determination unit 132 of the control unit 13 by, for example, reading and executing a program stored in the ROM.

[0025] Note that the in-vehicle device 10 may acquire the above-described program and various information via other computers connected by a wired or wireless network or a portable recording medium.

[0026] The detection unit 131 detects a detection area, which is an area in which a predetermined object appears, from an image captured by a camera mounted on the vehicle. Further, the detection unit 131 detects the detection area together with the degree of certainty (hereinafter, confidence).

[0027] The detection unit 131 detects an object by a predetermined method using a model such as a neural network constructed from the model information 121. For example, the in-vehicle device 10 may detect an object by a known method using deep learning such as YOLO.

[0028] For example, the detection unit 131 detects a rectangular area called a bounding box.

[0029] Also, the confidence level may be the probability that an object exists in the area, or a value that increases as the probability increases.

[0030] The determination unit 132 determines whether the detection of the object is effective based on the detection result by the detection unit 131.

[0031] For example, the determination unit 132 determines that the detection of the object is effective when the confidence level included in the detection result is equal to or higher than a threshold value.

[0032] The anonymization unit 133 anonymizes an area obtained by expanding the detection area by a predetermined expansion rate. The anonymization unit 133 may perform anonymization when it is determined by the determination unit 132 that the detection is effective.

[0033] Here, the detection area may not match the area where anonymization is expected. Therefore, in the present embodiment, an area obtained by expanding the detection area is anonymized. Thereby, the area actually anonymized can be made closer to the area where anonymization is expected.

[0034] FIG. 3 is a diagram showing an example of a method for expanding a detection area. An area 211 in FIG. 3 is an area where anonymization is expected for a human face. Also, an area 212 is a detection area. Also, an area 213 is an area expanded by the anonymization unit 133.

[0035] In this case, the anonymization unit 133 anonymizes the area 213. The area 213 is closer to the area 211 where anonymization is expected than the area 212 which is the detection area. Therefore, it becomes possible to more reliably anonymize the area of the confidential information in the image.

[0036] FIG. 4 is a diagram for explaining the expansion ratio. For example, α and β correspond to the expansion ratio. α is the expansion ratio in the horizontal direction (left - right direction). Also, β is the expansion ratio in the vertical direction (up - down direction).

[0037] For example, when α = 0.1 and β = 0.2, it can be said that the area 213 is an area obtained by expanding the area 212 by 1.2 times horizontally and 1.4 times vertically.

[0038] A method for determining the expansion ratio will be described. The in - vehicle device 10 can expand the detection area according to any one or a combination of the following embodiments and anonymize the expanded area.

[0039] (Embodiment 1) As shown in FIG. 5, the anonymization unit 133 anonymizes the detection area by expanding it at an expansion ratio such that the larger the size of the detection area, the smaller the expansion ratio.

[0040] When the size of the detection area is large, if the expansion ratio becomes too large, unnecessary areas may be anonymized. According to Embodiment 1, it is possible to prevent unnecessary areas from being anonymized.

[0041] FIG. 5 is a diagram showing an example of the expansion ratio according to the size of the detection area. The anonymization unit 133 determines the expansion ratio according to the curve 301 in FIG. 5. The curve 301 shows that the expansion ratio decreases as the size (W or H) of the detection area increases.

[0042] The anonymization unit 133 calculates the horizontal expansion ratio as α = a / W + b. Also, the anonymization unit 133 calculates the vertical expansion ratio as β = a / H + b (for example, a = 3, b = 0.05).

[0043] Note that a and b are predefined constants. For example, a = 3 and b = 0.05.

[0044] (Example 2) When a plurality of overlapping regions are detected as detection regions by the detection unit 131, the anonymization unit 133 anonymizes one of the detection regions by an enlarged region enlarged at an enlargement rate that becomes smaller as the number of detection regions increases.

[0045] FIG. 6 is a diagram showing an example of a method for enlarging a detection region. In the example of FIG. 6, the detection unit 131 detects human faces in the regions 221, 222, and 223. The regions 221, 222, and 223 are a plurality of overlapping regions.

[0046] At this time, the anonymization unit 133 reduces the enlargement rate of the regions 221, 222, and 223 as compared with the case where only one region is detected.

[0047] This is because anonymization based on a plurality of detection regions enables sufficient anonymization even with a small enlargement rate. If all of the regions 221, 222, and 223 are enlarged at a large enlargement rate, unnecessary regions may be anonymized.

[0048] (Example 3) When a plurality of overlapping regions are detected as detection regions by the detection unit 131, the anonymization unit 133 anonymizes one of the detection regions by an enlarged region enlarged until it has a size that covers all of the detection regions.

[0049] As shown on the right side of FIG. 6, the anonymization unit 133 anonymizes all of the regions 221, 222, and 223 into the region 224 with a large number. The region 224 is, for example, a region obtained by enlarging the region 221.

[0050] The anonymization unit 133 matches the coordinates of each vertex of the rectangular region 224 with the minimum coordinates and the maximum coordinates of the vertices of the rectangular regions 221, 222, and 223.

[0051] That is, the anonymization unit 133 aligns the coordinates of the upper right vertex of the area 224 with the coordinates of the upper right vertex of the area 222. Also, the anonymization unit 133 aligns the horizontal coordinate of the lower left vertex of the area 224 with the coordinates of the left vertex of the area 221. Further, the anonymization unit 133 aligns the vertical coordinate of the lower left vertex of the area 224 with the coordinates of the lower vertex of the area 223.

[0052] According to the third embodiment, it is possible to prevent unnecessary areas from being anonymized.

[0053] (Embodiment 4) The anonymization unit 133 anonymizes the detection area by expanding the detection area by an expansion rate determined according to the position of the detection area within the image.

[0054] For example, as shown in FIG. 7, when the detection area is inside the area 231 or when the overlapping ratio between the detection area and the area 231 is equal to or greater than a certain value, the anonymization unit 133 increases the expansion rate as compared with the case where the detection area is outside the area 231. FIG. 7 is a diagram showing an example of a method for expanding the detection area.

[0055] This is because the resolution is better near the center of the image than at the edges, and the accuracy of the detection area is high.

[0056] The high accuracy of the detection area means that an object can be detected in a smaller area. For this reason, it is considered that the degree of separation between the detection area and the area where anonymization is expected is greater near the center of the image.

[0057] According to the fourth embodiment, it is possible to more reliably anonymize necessary areas while preventing unnecessary areas from being anonymized.

[0058] (Embodiment 5) The anonymization unit 133 anonymizes the detection area by expanding the detection area by an expansion rate that increases as the confidence level decreases.

[0059] FIG. 8 is a diagram showing an example of the expansion rate according to the size of the detection area. As shown in FIG. 8, the anonymization unit 133 changes the position and shape of the curve 301 according to the confidence level.

[0060] Specifically, the anonymization unit 133 changes a and b according to the confidence level.

[0061] For example, the higher the confidence level, the larger the anonymization unit 133 makes a or the smaller it makes b. In this case, the curve 301 approaches the curve 302.

[0062] For example, the lower the confidence level, the smaller the anonymization unit 133 makes a or the larger it makes b. In this case, the curve 301 approaches the curve 303.

[0063] This is because the lower the confidence level, the higher the possibility that the detection result is inaccurate, and the higher the confidence level, the higher the possibility that the detection result is accurate. It is considered that the closer the detection result is to accurate, the closer the detection area and the area where anonymization is expected will be.

[0064] (Example 6) The anonymization unit 133 anonymizes the detection area to an area expanded in a direction according to the shape of the object.

[0065] According to Example 6, more effective anonymization can be performed according to the detected object.

[0066] For example, it is considered that the area where anonymization of a human face is expected is vertically long compared to a square. Therefore, as shown in FIG. 9, the anonymization unit 133 anonymizes the area 252 obtained by expanding the area 251, which is the detection area, in the vertical direction. FIG. 9 is a diagram showing an example of the method of expanding the detection area.

[0067] Also, for example, it is considered that the area where anonymization of a license plate is expected is horizontally long compared to a square. Therefore, as shown in FIG. 10, the anonymization unit 133 anonymizes the area 257 obtained by expanding the area 256, which is the detection area, in the horizontal direction. FIG. 9 is a diagram showing an example of the method of expanding the detection area.

[0068] In addition, when the detected object is a product with a determined standard such as a license plate, the anonymization unit 133 may make the aspect ratio of the expanded area a predetermined aspect ratio.

[0069] (Example 7) The anonymization unit 133 anonymizes the expanded area in the direction matching the moving direction of the object in the detection area.

[0070] For example, the anonymization unit 133 acquires a vector (for example, optical flow) indicating the moving direction of the detected object. In the example of FIG. 11, the anonymization unit 133 acquires a vector with a rightward component of u and an upward component of v as the vector indicating the moving direction of the object in the area 261.

[0071] At this time, the anonymization unit 133 anonymizes the area 262 expanded in the direction of the acquired vector for the area 261.

[0072] FIG. 12 is a diagram for explaining the expansion rate. As shown in FIG. 12, the anonymization unit 133 calculates the size of the expanded area based on the vector (u, v) and the constant γ.

[0073] In the example of FIG. 12, the anonymization unit 133 expands the detection area by γ×u / (u + v) in the horizontal direction and by γ×v / (u + v) in the vertical direction.

[0074] According to Example 7, even when the object shown in the moving image moves, since the area of the moving destination is also anonymized, more reliable anonymization can be performed.

[0075] FIG. 13 is a flowchart showing the processing flow of the in-vehicle device. As shown in FIG. 13, first, the in-vehicle device 10 detects a detection area in which an object appears from an image (step S101).

[0076] Next, the in-vehicle device 10 determines whether detection is enabled based on the detection result (step S102).

[0077] Here, when the detection is valid (step S103, Yes), the in-vehicle device 10 anonymizes the area obtained by expanding the detection area at a predetermined expansion rate (step S104). The anonymization unit 133 determines the expansion rate by the method described in each embodiment.

[0078] On the other hand, when the detection is not valid (step S103, No), the in-vehicle device 10 ends the process.

[0079] As described above, the in-vehicle device 10 according to the embodiment includes a detection unit 131 and an anonymization unit 133. The detection unit 131 detects a detection area, which is an area in which a predetermined object appears, from an image captured by a camera mounted on the vehicle. The anonymization unit 133 anonymizes the area obtained by expanding the detection area at a predetermined expansion rate.

[0080] In this way, since the in-vehicle device 10 anonymizes the detection area after expanding it at a predetermined expansion rate, the area of the confidential information in the image can be surely anonymized.

[0081] Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the present invention are not limited to the specific details and representative embodiments presented and described above. Accordingly, various changes are possible without departing from the spirit or scope of the general inventive concept defined by the appended claims and their equivalents.

[0082] For example, in the first embodiment, the anonymization unit 133 may determine different expansion rates for each of the left, right, up, and down directions.

[0083] In this case, the anonymization unit 133 calculates the expansion rate in the left direction as α 1 =a / W+b 1 as shown. Also, the anonymization unit 133 calculates the expansion rate in the right direction as α 2 =a / W+b 2 as shown. Also, the anonymization unit 133 calculates the expansion rate in the upward direction as β 1 =a / W+b3 Calculate as follows. Also, the anonymization unit 133 has a downward expansion rate of β 2 =a / W + b 4 Calculate as follows. Note that b 1 b 2 b 3 b 4 are all constants and may be different from each other.

Explanation of Signs

[0084] 10 Vehicle-mounted device 11 Interface unit 12 Memory unit 13 Control unit 121 Model information 131 Detection unit 132 Judgment unit 201, 211, 212, 213, 221, 222, 223, 231, 251, 252, 256, 257, 261, 262 Regions 301, 302, 303 Curves

Claims

1. Identify a detection area, which is an area in the image captured by the camera and where a predetermined object appears, Obfuscate the area obtained by expanding the size of the detection area at a decreasing expansion rate as the size of the detection area increases, An image processing apparatus.

2. When a plurality of the detection areas overlap, decrease the expansion rate as the number of the detection areas increases, The image processing apparatus according to Claim 1.

3. When a plurality of the detection areas overlap, increase the expansion rate of any one of the detection areas until the size covers all of the detection areas, The image processing apparatus according to Claim 1.

4. Determine the expansion rate according to the position of the detection area in the image, The image processing apparatus according to Claim 1.

5. Calculate a confidence level, which is a degree of certainty of the detection area, Increase the expansion rate as the confidence level decreases, The image processing apparatus according to Claim 1.

6. Expand the detection area in a direction corresponding to the shape of the object, The image processing apparatus according to Claim 1.

7. Expand the detection area in a direction matching the moving direction of the object, The image processing apparatus according to Claim 1.

8. An image processing method executed by an image processing apparatus, the method including: Identify a detection area, which is an area in the image captured by the camera and where a predetermined object appears; Obfuscate the area obtained by expanding the size of the detection area at a decreasing expansion rate as the size of the detection area increases. An image processing method.

9. Cause a computer to Identify a detection area, which is an area in the image captured by the camera and where a predetermined object appears; and Obfuscate the area obtained by expanding the size of the detection area at a decreasing expansion rate as the size of the detection area increases. An image processing program.

10. The detection area is rectangular, The expansion rate is different in the horizontal and vertical directions of the detection area, The image processing apparatus according to Claim 1.

11. The relationship between the size of the detection area and the expansion rate is represented by a curve, The image processing apparatus according to Claim 1.

12. The camera is an in-vehicle camera mounted on a vehicle, The image processing apparatus according to Claim 1.

13. The predetermined object includes a human face or a vehicle number plate, The image processing apparatus according to Claim 12. ​ ​

Images