Information processing apparatus, information processing method, information processing program, and information processing system

Abstract

To efficiently collect training data in which personal information is protected.SOLUTION: An information processing apparatus comprises a concealing unit and a data control unit. The concealing unit performs specific image information concealing processing on the entirety of an image including specific image information of a subject. The data control unit stores a concealed image after the concealing processing is performed, as training data used for training of a machine learning model that receives input of an input image including the subject and then outputs position information of the subject.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, an information processing program, and an information processing system. [Background technology]

[0002] There is a technology that detects an object or a specific part including a specific part that needs to be concealed to protect personal information from a captured image, and conceals the detected object or specific part. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2022-64718 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, images captured by a camera may be used as training data for an object detection model that detects specific subjects from images. When analyzing captured images for commercial purposes, it is necessary to conceal some of the image information contained in the captured images (e.g., people's faces, car license plates, etc., hereinafter referred to as "specific image information") from the perspective of personal information and privacy.

[0005] Here, the task of the object detection model is to detect a specific subject from an input image, specifically, to predict the position information (coordinate information) of the specific subject in the input image.

[0006] For this reason, in the photographed images used for training the object detection model, the important information is the location information of where a specific subject is located within the photographed image, and specific image information does not need to be included. In other words, even if a photographed image that has been anonymized so as to leave only the minimum information about the specific subject is used, as long as the correct location information is included, the object detection model may be able to learn to properly detect a specific subject from such an image even when an anonymized image is input.

[0007] Based on the above, in captured images used for training an object detection model, there is no need to detect specific image information and then conceal the detected specific image information; simply concealing the entire captured image can make the collection of training data more efficient.

[0008] However, in the above-mentioned conventional technology, only specific parts related to personal information and privacy, such as a person's face and a vehicle license plate number included in an image captured by a camera, are concealed. While the idea of ​​concealing only specific image information is thus reached from the perspective of protecting personal information and privacy, the present invention focuses on collecting learning data containing only the minimum information of a specific subject so that a specific subject can be detected even from a concealed image, and thus has a different problem from the conventional technology.

[0009] The present invention provides an information processing device, an information processing method, and an information processing program that can efficiently collect learning data in which specific image information is protected. [Means for solving the problem]

[0010] In order to solve the above problem, an information processing device according to one embodiment of the present invention includes: a concealment unit that performs a process of concealing specific image information of a subject for an entire image including the specific image information; and a data control unit that stores the concealed image after the concealment process as training data to be used for training a machine learning model that outputs position information of the subject when an input image including the subject is input. [Effects of the Invention]

[0011] According to the present invention, it is possible to efficiently collect training data in which personal information and privacy are protected. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram for explaining and comparing a conventional method relating to an object detection model with the method proposed by the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of an information processing system according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of an imaging device according to an embodiment. [Figure 4] FIG. 4 is a diagram illustrating an overview of the concealment process according to the embodiment. [Figure 5] FIG. 5 is a diagram showing a specific example of the concealment method 1. In FIG. [Figure 6] FIG. 6 is a diagram showing a specific example of the concealment method 2. In FIG. [Figure 7] FIG. 7 is a flowchart showing an example of the operation of the imaging device. [Figure 8] FIG. 8 is a diagram showing the results of verifying the accuracy of an existing object detection model according to a conventional method and an object detection model according to the method proposed by the present invention. [Figure 9] FIG. 9 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the imaging device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0014] One or more embodiments (including examples, modifications, and application examples) described below can be implemented independently. However, at least a portion of the embodiments described below may be implemented in appropriate combination with at least a portion of another embodiment. These embodiments may include novel features that are different from each other. Therefore, these embodiments may contribute to solving different purposes or problems and may produce different effects from each other.

[0015] Furthermore, in the following embodiments, the image (sometimes referred to as "image data") is assumed to be an image captured by a camera (e.g., a surveillance camera) installed in a location where personal information or privacy-related parts may be captured. Furthermore, the object to be detected by the object detection model is a person, and an example is given of a scene in which the subject related to personal information or privacy is a person. On the other hand, the object to be detected by the object detection model is not limited to a person. Furthermore, as described above, of the image information (captured image), image information of parts of the subject related to personal information or privacy (e.g., a person's face, a car license plate number, etc.) is referred to as "specific image information."

[0016] (Embodiment) 1. Introduction An object detection model that detects objects from images is known. Images captured by a camera may be used for training the object detection model. However, such captured images may contain personal information or privacy-related information. Therefore, when collecting such captured images as training data for the object detection model, it is necessary to conceal specific image information.

[0017] As described above, in the captured images used for training the object detection model, location information indicating where a specific subject is located within the captured image is important information, and specific image information does not need to be included. Therefore, the inventors of the present invention have noticed that even if a captured image that has been anonymized so as to leave only the minimum information regarding the specific subject is used, as long as the image contains correct location information, the object detection model may be able to properly detect the specific subject even from the anonymized image.

[0018] Here, when concealing a specific subject included in an image as in the above-mentioned conventional technology, a process of concealing only specific image information related to personal information or privacy is considered. However, concealing only the specific image information not only requires steps of detecting specific parts related to personal information or privacy from the captured image and concealing the specific image information that is the detected specific parts, but also poses a risk of overlooking the specific parts. For this reason, it is difficult to say that learning data can be collected efficiently.

[0019] In light of the above, the inventors of the present invention have realized that it is efficient to conceal specific image information for the entire photographed image itself, that is, the entire photographed image, rather than concealing specific parts contained in the photographed image, since photographed images serving as training data only need to contain the minimum information about a specific subject. This point will be explained using FIG. 1. FIG. 1 is a diagram illustrating a comparison between a conventional method relating to an object detection model and the method proposed by the present invention. FIG. 1(a) shows the conventional method relating to an object detection model, and FIG. 1(b) shows the method proposed by the present invention relating to an object detection model.

[0020] In conventional methods, image data that has not undergone anonymization processing to conceal specific image information is used in the learning phase. At this time, annotation is performed to add information to the image data about the location of the object to be identified. In the annotation, the position and range of the object are specified using a rectangular area (bounding box), and the object's category is assigned. The annotated image data is then used as training data for the object detection model.

[0021] In the conventional method, in the inference phase, image data that has not been subjected to an anonymization process is input to an object detection model, and the object detection model detects an object from the input image data. Specifically, the object detection model estimates position information indicating the position of the object in the input image data. The estimated position information is depicted as a rectangular area.

[0022] Next, in the training phase of the proposed method of the present invention, image data that has been subjected to a concealment process to conceal specific image information is used. The proposed method also involves annotating the image data to provide information about the location of an object to be identified in the image data, and the annotated image data is used as training data for the object detection model. It is preferable that the object detection model used for training in the proposed method of the present invention is an object detection model that has been pre-trained using a conventional method.

[0023] In the inference phase of the proposed method of the present invention, image data that has been anonymized is input to an object detection model, and the object detection model estimates location information indicating the location of an object in the input image data. The estimated location information is depicted as a rectangular area. As described above, the object detection model according to the proposed method of the present invention has been pre-trained using a conventional method. Therefore, even if image data that has not been anonymized is input, it can still detect an object, and its accuracy may be improved compared to existing object detection models according to conventional methods. Accuracy comparison will be described later.

[0024] [2. System Configuration Overview] The configuration of the information processing system 1 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the information processing system 1 according to the embodiment. As shown in Fig. 2, the information processing system 1 includes an imaging device 10 and a cloud-side system (cloud system) 20. The imaging device 10 and the cloud system 20 are connected to each other via a predetermined communication network (network N) so as to be able to communicate with each other via wired or wireless communication.

[0025] As shown in FIG. 2, an imaging device 10 corresponding to an information processing device according to the embodiment may be, for example, an edge computer (edge ​​device) belonging to a user that provides a model learning service. The imaging device 10 includes a camera (imaging unit) 13 as an imaging function. The camera 13 may be built into the imaging device 10 or may be an externally attached type to the imaging device 10. In this embodiment, the imaging device 10 is assumed to be fixedly installed in a specific location for the purpose of surveillance or crime prevention.

[0026] Furthermore, the imaging device 10 is assumed to be an AI camera having a function of detecting an object from a captured image and automatically annotating the detected object. For example, the imaging device 10 annotates the captured image each time a captured image is successively acquired by continuous shooting using the camera 13, and uploads the annotated captured image to the cloud system 20.

[0027] The cloud system 20 includes a server device 110 as a cloud computer and a storage unit 120 as cloud storage, and the server device 110 uses captured images stored in the storage unit 120 as training data to train an object detection model M. The server device 110 corresponds to a training device according to the embodiment.

[0028] Consider a situation in which the imaging device 10 annotates an original captured image (hereinafter referred to as an "original image") that has not been subjected to a concealment process for concealing specific image information, and uploads the annotated original image as learning data to the cloud-side storage unit 120. Original images that have not been concealed may contain personal information or parts related to privacy, and uploading them to the cloud-side storage unit 120 is not appropriate as there is a risk of personal information or information that one does not want others to know leaking.

[0029] Therefore, the imaging device 10 acquires a concealed image by performing a concealment process on the annotated original image to conceal specific image information. The imaging device 10 then uploads the concealed image to the storage unit 120, thereby storing the uploaded concealed image as learning data. As a result, the server device 110 can generate an object detection model M that can appropriately detect a specific subject (e.g., a person) even from a concealed image.

[0030] 3. Configuration of the imaging device The imaging device 10 according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the imaging device 10 according to the embodiment. As shown in Fig. 3, the imaging device 10 includes a communication unit 111, a storage unit 12, an imaging unit 13, an input unit 14, an output unit 15, and a control unit 16.

[0031] <Communications Department 11> The communication unit 11 is realized by, for example, a network interface card (NIC), etc. For example, the communication unit 11 transmits and receives information to and from the cloud system 20.

[0032] <Storage section 12> The storage unit 12 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 12 may store, for example, data and programs related to the information processing according to the embodiment. The storage unit 12 may also store captured images.

[0033] <Imaging unit 13> The imaging unit 13 corresponds to a camera function for capturing an image of a target. Although the example in Fig. 3 shows an example in which the imaging unit 13 is built into the imaging device 10, the imaging unit 13 may be external to the imaging device 10.

[0034] <Input unit 14, output unit 15> The input unit 14 is an input device that accepts various operations from a user. For example, the input unit 14 is realized by a keyboard, a mouse, operation keys, etc. The output unit 15 is a display device that displays various information. For example, the output unit 15 may be a display screen realized by a liquid crystal display, etc. Note that if a touch panel is adopted in the imaging device 10, the input unit 14 and the output unit 15 may be integrated.

[0035] <Control unit 16> The control unit 16 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like using RAM as a work area to execute various programs (for example, the information processing program according to the embodiment) stored in a storage device inside the imaging device 10. The control unit 16 is also realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0036] As shown in Fig. 3, the control unit 16 has an acquisition unit 16a, an annotation unit 16b, a concealment unit 16c, and a data control unit 16d, and realizes or executes the functions and actions of information processing described below. Note that the internal configuration of the control unit 16 is not limited to the configuration shown in Fig. 3, and may be other configurations as long as they perform the information processing described below. Furthermore, the connection relationship between the processing units included in the control unit 16 is not limited to the connection relationship shown in Fig. 3, and may be other connection relationships.

[0037] <Acquisition part 16a> The acquisition unit 16a acquires various information used in the information processing according to the embodiment. For example, the acquisition unit 16a acquires an image captured by the imaging unit 13 and transmits the acquired image to the annotation unit 16b.

[0038] <Annotation section 16b> The annotation unit 16b performs annotation on the acquired captured image. The annotation unit 16b assigns information to the captured image indicating the position of the object to be identified in the captured image. For example, the annotation unit 16b assigns a rectangular area indicating the position and range of the object to be identified and the category of the object as a correct label. For example, the annotation unit 16b may perform annotation in response to a user's instruction, or may perform annotation automatically. When performing annotation automatically, the annotation unit 16b may use an AI model to detect objects in the captured image and assign correct labels to the detected objects. Note that the annotation may be performed on the server device 110 side. In other words, the server device 110 may have a processing unit equivalent to the annotation unit 16b.

[0039] <Concealment unit 16c> The anonymization unit 16c performs an anonymization process on the specific image information for the entire captured image. For example, the anonymization unit 16c may determine (adjust) the degree of anonymization used in the anonymization process based on the size of a subject that, among subjects included in the captured image, is reflected in the largest area of ​​the specific image information.

[0040] The concealment unit 16c may perform the following process for each captured image: identify the subject that is reflected in the largest area of ​​specific image information among the subjects contained in the captured image; and determine the degree of concealment according to the size of the subject.

[0041] On the other hand, the anonymization unit 16c may determine the subject that has the largest area of ​​specific image information reflected among the subjects included in each of the multiple different captured images, and determine a uniform degree of anonymization among the multiple different captured images depending on the size of the determined subject.

[0042] The anonymization unit 16c may dynamically change the degree of anonymization according to the size of the subject. The anonymization unit 16c may also determine the degree of anonymization according to the estimated maximum size of the subject included in the captured image.

[0043] <Data control unit 16d> The data control unit 16d stores the anonymized image after the anonymization process as learning data to be used for learning a machine learning model that outputs position information (position coordinates) of a subject when an input image including the subject is input. Specifically, the data control unit 16d uploads the anonymized image after the anonymization process to the storage unit 120 on the cloud side, thereby causing the server device 110 to perform learning of the object detection model M using the anonymized image as learning data.

[0044] [4. Overview of the concealment process] Next, an overview of the concealment process will be described. Fig. 4 is a diagram showing an overview of the concealment process according to an embodiment. The concealment process here refers to applying mosaic processing or blurring to a captured image to reduce the resolution (image quality) of the captured image, thereby making specific image information unidentifiable. Therefore, the concealment degree refers to the strength of the mosaic or blurring. Furthermore, the strength of the mosaic or blurring essentially corresponds to the compression rate and reduction rate, which will be described later.

[0045] Here, even if the entire captured image is concealed at a certain concealment level, depending on the size of the subject included in the captured image, it may not be possible to completely conceal the specific image information of the subject, and concealment processing at a higher concealment level may be required. As such, there is an appropriate correspondence between the size of the subject included in the captured image and the concealment level that allows the specific image information to be completely concealed, and the concealment unit 16c determines the concealment level to be used in the concealment processing depending on the size of the subject. This point will be specifically described using the examples of Figures 4(a) to 4(c).

[0046] 4(a) to 4(c) show an example in which the target of specific image information concealment is a person included in a captured image. Furthermore, the size of a person may be the area of ​​a rectangular area when the person is surrounded by the rectangular area, or the number of vertical or horizontal pixels of the rectangular area. Of course, the index for determining the size of a person is not limited to the above example.

[0047] 4(a) shows an example in which the size of a person included in an original image exceeds a reference value and is therefore determined to be in the size category of "Large." Also shown are appearance patterns when the anonymization unit 16c performs anonymization processing on the original image at anonymization levels of "20%, " "40%, and "60%."

[0048] In a concealed image in which an original image is concealed at a concealment level of "20%," it is possible to capture the facial expression of a person, but it can be said that the specific image information (an image of the person's face) is not fully concealed. In a concealed image in which an original image is concealed at a concealment level of "40%," it is possible to capture the facial expression of a person, but it can be said that the specific image information is not fully concealed. On the other hand, in a concealed image in which an original image is concealed at a concealment level of "60%," it is difficult to capture the facial expression of a person, but the specific image information is fully concealed. According to this example, the concealment unit 16c may be controlled to perform concealment processing at a concealment level of "60%" or higher when the size of a person included in the original image is classified as "large."

[0049] 4(b) shows an example in which the size of a person included in an original image is determined to be approximately the reference value and the person size category is determined to be "Middle." Also shown are appearance patterns when the anonymization unit 16c performs an anonymization process on the original image at anonymization levels of "20%, " "40%, and "60%."

[0050] In a concealed image in which the original image is concealed at a concealment level of "20%," it is possible to capture the facial expressions of people, but the specific image information is not completely concealed. On the other hand, in a concealed image in which the original image is concealed at a concealment level of "40%," it is difficult to capture the facial expressions of people, but the specific image information is concealed. In a concealed image in which the original image is concealed at a concealment level of "60%,", it is also difficult to capture the facial expressions of people, but the specific image information is concealed. According to this example, the concealment unit 16c may be controlled to perform the concealment process at a concealment level of "40%" or higher when the size of the person included in the original image is classified as "Middle."

[0051] 4(c) shows an example in which the size of a person included in an original image is determined to be in the "Small" size category because the size of the person is below a reference value. Also shown are appearance patterns when the anonymization unit 16c performs anonymization processing on the original image at anonymization levels of "20%, " "40%, and "60%".

[0052] In a concealed image in which the original image is concealed at a concealment level of "20%," it is difficult to capture a person's facial expression, and specific image information can be concealed. In a concealed image in which the original image is concealed at a concealment level of "40%," it is also difficult to capture a person's facial expression, and specific image information can be concealed. In a concealed image in which the original image is concealed at a concealment level of "60%,", it is also difficult to capture a person's facial expression, and specific image information can be concealed. According to this example, the concealment unit 16c may be controlled to perform the concealment process at a concealment level of "20%" or higher when the size of the person included in the original image is classified as "Small".

[0053] Note that, when the imaging unit 13 is a fixed camera, the maximum size at which a person can appear in a captured image is roughly determined. For example, in a scene where the imaging unit 13 is fixed to a utility pole so as to capture a road, there is a limit to the range and size at which a person can appear in the captured image. This is because a person cannot get closer than a certain amount to the imaging unit 13. In light of this, the concealment unit 16c may determine the concealment degree according to the estimated maximum size at which a person included in the captured image will appear. For example, when it can be estimated that the size of the person appearing in the captured image is classified as "large," the concealment unit 16c may conceal the entire captured image at a concealment degree of "60%" or more.

[0054] [5. Specific examples of confidentiality processing] Next, a specific example of the concealment process will be described with reference to Fig. 5 and Fig. 6. The concealment process according to the embodiment can be divided into concealment method 1, in which an original image is irreversibly compressed and the compressed original image is restored to obtain a concealed image with a lower resolution (mosaicing, blurring), and concealment method 2, in which an original image is resized (reduced or enlarged) to obtain a concealed image with a lower resolution.

[0055] [5-1. Specific example of confidentiality processing (1)] Fig. 5 is a diagram showing a specific example of concealment method 1. Fig. 5 shows a scene in which an original image OG is irreversibly compressed and then the compressed original image OG is restored to generate a low-resolution concealed image EG. In addition, in the original image OG, a person U1 is detected by a rectangular area AR1, and a person U2 is detected by a rectangular area AR2.

[0056] In this way, when multiple people are included in one original image OG, the anonymization unit 16c identifies the person among the multiple people who has the largest area of ​​specific image information, and determines the compression rate (degree of anonymization) based on the size of the identified person in the original image OG. For example, the anonymization unit 16c may compare the areas of rectangular areas AR1 and AR2, and identify the person detected in the larger rectangular area as the person who has the largest area of ​​specific image information. Then, the anonymization unit 16c classifies the size of the identified person into one of "Large," "Middle," or "Small" based on the area of ​​the rectangular area surrounding the identified person. Furthermore, the anonymization unit 16c determines the compression rate for lossy compression according to the size classification.

[0057] In the example of FIG. 5, the anonymization unit 16c compares the areas of rectangular area AR1 and rectangular area AR2 and determines that person U1, who is surrounded by the larger rectangular area AR1, is the person who is most prominent in the specific image information. Based on the area of ​​rectangular area AR1, the anonymization unit 16c then classifies the size of person U1 into one of "Large," "Middle," and "Small." In the example of FIG. 5, the anonymization unit 16c classifies the size of person U1 in the original image OG as "Middle," and performs lossy compression on the original image OG at a compression rate of 40% or higher, which corresponds to "Middle." The anonymization unit 16c then restores the compressed original image OG to generate an anonymized image EG.

[0058] In this way, in lossy compression, some data (pixels) of the original image OG are discarded, and therefore the data capacity of the concealed image EG is smaller than that of the original image OG. Therefore, in concealment method 1, by reducing the resolution of the original image OG using lossy compression, it is possible to obtain a concealed image EG with a reduced data capacity, and therefore it is possible to reduce the load on the network when uploading the concealed image EG to the storage unit 120 on the cloud side.

[0059] 5, when multiple people are included in one original image OG, the anonymization unit 16c performs lossy compression using a compression rate that can reduce the resolution to a level that sufficiently conceals the specific image information of the largest person in the original image OG. In other words, the anonymization unit 16c determines the compression rate based on the size of the largest person in the original image OG. This configuration makes it possible to anonymize the specific image information of all of the multiple people at once.

[0060] 5, the concealment unit 16c determines a compression rate for each original image OG obtained by continuous shooting by the imaging unit 13, and performs lossy compression on each original image OG at the determined compression rate. However, the concealment unit 16c may determine, when a certain number of original images OG have been accumulated, a person in which the area of ​​specific image information is largest among the people included in each of the multiple different original images OG, determine a uniform compression rate for the multiple different original images OG according to the size of the determined person, and perform lossy compression on each of the multiple different original images OG at the determined compression rate.

[0061] [5-2. Specific example of confidentiality processing (2)] Fig. 6 is a diagram showing a specific example of concealment method 2. Fig. 6 shows a scene in which an original image OG is resized (reduced or enlarged) to generate a concealed image EG with a lower resolution. In the original image OG, a person U1 is detected by a rectangular area AR1, and a person U2 is detected by a rectangular area AR2.

[0062] Note that concealment method 2 is a method of mosaic processing by resizing the original image OG to reduce the size of the original image OG, and then resizing the reduced original image OG back to the size of the original image OG. For this reason, in concealment method 2, a small image size is specified when reducing, and the original image size is specified when enlarging.

[0063] Furthermore, when reducing the original image OG, the smaller the original image OG is made, the stronger the mosaic becomes, making it more difficult to guess the contents of the original image OG. For this reason, in concealment method 2, the reduction rate corresponds to the degree of concealment.

[0064] In addition, in concealment method 2, when the original image OG is reduced, a process of thinning out unnecessary pixels is performed using nearest neighbor interpolation, resulting in a small original image OG with reduced resolution due to mosaic, and the image is then enlarged in this reduced resolution state, resulting in a concealed image EG.

[0065] As shown in FIG. 6, when multiple people are included in one original image OG, the anonymization unit 16c identifies the person among the multiple people who has the largest area of ​​specific image information, and determines the reduction rate (degree of anonymization) based on the size of the identified person in the original image OG. For example, the anonymization unit 16c may compare the areas of rectangular areas AR1 and AR2, and identify the person detected in the larger rectangular area as the person who has the largest area of ​​specific image information. Then, the anonymization unit 16c classifies the size of the identified person into one of "Large," "Middle," or "Small" based on the area of ​​the rectangular area surrounding the identified person. Furthermore, the anonymization unit 16c determines the reduction rate for resizing based on the size classification.

[0066] In the example of FIG. 6, the anonymization unit 16c compares the areas of the rectangular areas AR1 and AR2 and determines that the person U1 surrounded by the larger rectangular area AR1 is the person whose area of ​​specific image information is the largest. Then, the anonymization unit 16c classifies the size of the person U1 into one of "Large," "Middle," and "Small" based on the area of ​​the rectangular area AR1. In the example of FIG. 6, the anonymization unit 16c classifies the size of the person U1 in the original image OG as "Middle," and reduces the original image OG by a reduction rate of "40%" or more, which corresponds to "Middle." The anonymization unit 16c then enlarges the reduced original image OG to its original size before reduction, thereby generating an anonymized image EG.

[0067] As described above, in size reduction using nearest neighbor interpolation, some data (pixels) of the original image OG are discarded, and therefore the data capacity of the concealed image EG is smaller than that of the original image OG. Therefore, in concealment method 2, by reducing the resolution of the original image OG through size reduction using nearest neighbor interpolation, it is possible to obtain a concealed image EG with a reduced data capacity, and therefore it is possible to reduce the load on the network when uploading the concealed image EG to the storage unit 120 on the cloud side.

[0068] 6, when multiple people are included in one original image OG, the anonymization unit 16c reduces the image using a reduction ratio that can reduce the resolution to a level that sufficiently conceals the specific image information of the largest person in the original image OG. In other words, the anonymization unit 16c determines the reduction ratio based on the size of the largest person in the original image OG. This configuration makes it possible to anonymize the specific image information of all of the multiple people at once.

[0069] 6, the concealment unit 16c determines a reduction rate for each original image OG obtained by continuous shooting by the imaging unit 13, and resizes each original image OG at the determined compression rate. However, the concealment unit 16c may determine, when a certain number of original images OG have been accumulated, a person in which the area of ​​specific image information is largest among the people included in each of the multiple different original images OG, and determine a uniform reduction rate for the multiple different original images OG according to the size of the determined person.

[0070] [6. Operational example of imaging device] Fig. 7 is a flowchart showing an example of the operation of the imaging device 10. Fig. 7 shows an example of the operation in which concealment processing is performed on each captured image obtained by continuous shooting by the imaging unit 13.

[0071] The acquiring unit 16a determines whether or not a captured image has been acquired (step S701). If a captured image has not been acquired (step S701; No), the acquiring unit 16a waits until the captured image can be acquired.

[0072] On the other hand, if a photographed image is acquired (step S701; Yes), the annotation unit 16b detects a person from the acquired photographed image (step S702).

[0073] Next, the concealment unit 16c determines the person whose specific image information is most prominently reflected among the people detected from the photographed image (step S703).

[0074] Furthermore, the anonymization unit 16c determines a degree of anonymization based on the size of the identified person in the photographed image (step S704). Then, the anonymization unit 16c executes an anonymization process to anonymize specific image information of the person detected from the photographed image using the determined degree of anonymization (step S705). Specifically, the anonymization unit 16c executes an anonymization process on the entire photographed image using the determined degree of anonymization.

[0075] The data control unit 16d generates learning data by associating the anonymized image obtained by the anonymization process with position information of people in the anonymized image (step S706). The position information used here may be annotation information added to the captured image by the annotation unit 16b.

[0076] Then, the data control unit 16d uploads the learning data to the storage unit 120 (step S707).

[0077] 7, when a sufficient number of anonymized images serving as training data have been accumulated in the storage unit 120, the server device 110 acquires a group of anonymized images from the storage unit 120. Then, the server device 110 provides the group of anonymized images to an object detection model that has been pre-trained using a conventional method (a machine learning model that has trained using only photographed images that have not been anonymized as training data), and causes the model to learn the features of the anonymized images. As a result, the server device 110 can generate an object detection model M that can detect people with high accuracy from an input image, regardless of whether an original image that has not been anonymized or an anonymized image is input.

[0078] [7. Accuracy Comparison] 8 shows the results of verifying the accuracy of an existing object detection model according to a conventional method and the object detection model according to the method proposed by the present invention. The verification was conducted to verify whether anonymized images can be used as learning data for object detection.

[0079] 8(a) shows the relationship between an existing object detection model X according to a conventional method and an object detection model M according to the method proposed by the present invention. Specifically, the object detection model X is a machine learning model trained using only original captured images (original images) that have not been subjected to anonymization processing as training data. The object detection model M is a machine learning model obtained by using the object detection model X, which has been trained using only original images as training data, as a pre-training model, and further training the object detection model X using anonymized images as training data.

[0080] Furthermore, three types of object detection models X and X were prepared: one trained on images of people categorized as "large" (Large Group), one trained on images of people categorized as "middle" (Middle Group), and one trained on images of people categorized as "small" (Small Group). Figure 8(b) shows a table summarizing the detection accuracy for each group. AP (Average Precision) was used as an index value to evaluate the detection accuracy.

[0081] Referring to FIG. 8(b), the accuracy of the object detection model X trained only on original images that have not been subjected to concealment processing will be described.

[0082] In the example of Figure 8(b), when an original verification image in which the person's size is classified as "Large" is input to object detection model X, which has been trained only with original images in which the person's size is classified as "Large," the accuracy of person detection for the original verification image is "88%." On the other hand, when an anonymized verification image in which the person's size is classified as "Large" is input to object detection model X, which has been trained only with original images in which the person's size is classified as "Large," the accuracy of person detection for the anonymized verification image is "10%."

[0083] When object detection model X, which was trained only with original images in which people were categorized as "Medium" in size, was input with original verification images in which people were categorized as "Medium" in size, the accuracy of person detection for the original verification images was 88%. On the other hand, when anonymized verification images in which people were categorized as "Medium" in size were input to object detection model X, which was trained only with original images in which people were categorized as "Medium" in size, the accuracy of person detection for the anonymized verification images was 17%.

[0084] When object detection model X, which was trained only with original images in which people's size was classified as "Small," was input with original verification images in which people's size was classified as "Small," the accuracy of person detection for the original verification images was "88%." On the other hand, when anonymized verification images in which people's size was classified as "Small" were input with object detection model X, which was trained only with original images in which people's size was classified as "Small," the accuracy of person detection for the anonymized verification images was "18%."

[0085] From the above, it was found that object detection model X, which was trained only on original images that had not been anonymized, could make accurate inferences on original images, but its inference accuracy significantly decreased on anonymized images.

[0086] Next, in FIG. 8(b), the accuracy of an object detection model M obtained by further training an object detection model X that has been trained in advance using only original images and anonymized images as training data will be described.

[0087] In the example of FIG. 8(b), when an original verification image in which the person's size is classified as "Large" is input to object detection model M trained using an original image and an anonymized image in which the person's size is classified as "Large," the accuracy of person detection for the original verification image is "80%." On the other hand, when an anonymized verification image in which the person's size is classified as "Large" is input to object detection model M trained using an original image and an anonymized image in which the person's size is classified as "Large," the accuracy of person detection for the anonymized verification image is "75%."

[0088] When an original verification image in which the person's size is classified as "Medium" was input to object detection model M, which was trained using an original image and an anonymized image in which the person's size is classified as "Medium," the accuracy of person detection for the original verification image was "81%." On the other hand, when an anonymized verification image in which the person's size is classified as "Medium" was input to object detection model M, which was trained using an original image and an anonymized image in which the person's size is classified as "Medium," the accuracy of person detection for the anonymized verification image was "81%."

[0089] When an original verification image in which the person's size is classified as "Small" was input to object detection model M, which was trained using an original image and an anonymized image in which the person's size is classified as "Small," the accuracy of person detection for the original verification image was "65%." On the other hand, when an anonymized verification image in which the person's size is classified as "Small" was input to object detection model M, which was trained using an original image and an anonymized image in which the person's size is classified as "Small," the accuracy of person detection for the anonymized verification image was "65%."

[0090] From the above, it was found that the object detection model M, which was trained using both original and anonymized images, can maintain a high inference accuracy of above 1 for both original and anonymized images. In other words, the verification results shown in Figure 8(b) suggest that anonymized images can be used as training data for object detection.

[0091] [8. Hardware Configuration] The imaging device 10 according to the embodiment may be realized, for example, by a computer 1000 configured as shown in Fig. 9. Fig. 9 is a hardware configuration diagram showing an example of a computer that realizes the functions of the imaging device 10 according to the embodiment. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0092] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, programs that depend on the hardware of the computer 1000, and the like.

[0093] The HDD 1400 stores programs executed by the CPU 1100, data used by these programs, etc. The communication interface 1500 receives data from other devices via a predetermined communication network and sends the data to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network.

[0094] The CPU 1100 controls an output device such as a display and an input device such as a keyboard via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. The CPU 1100 also outputs generated data to the output device via the input / output interface 1600.

[0095] Media interface 1700 reads a program or data stored in recording medium 1800 and provides it to CPU 1100 via RAM 1200. CPU 1100 loads the program or data from recording medium 1800 onto RAM 1200 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0096] For example, when the computer 1000 functions as the imaging device 10 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200, thereby realizing the functions of the control unit 16. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a predetermined communication network.

[0097] [9. Other] Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0098] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0099] Furthermore, the above-described embodiments can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0100] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that include the aspects described in the "present invention" section and that have been modified and improved in various ways based on the knowledge of those skilled in the art. [Explanation of symbols]

[0101] 1. Information Processing Systems 10. Imaging device 11 Communications Department 12 Storage section 13 Imaging unit 14 Input section 15 Output section 16 Control Unit 16a Acquisition part 16b Annotation section 16c Concealment Unit 16d Data control section 20 Cloud System 110 Server device 120 Storage section

Claims

1. a concealment unit that performs a concealment process on specific image information of a subject for the entire image including the specific image information; a data control unit that stores the anonymized image after the anonymization process as learning data to be used for learning a machine learning model that outputs position information of a subject when an input image including the subject is input; and An information processing device comprising:

2. The concealment unit determines the degree of concealment to be used in the concealment process based on the size of the subject that is reflected in the largest area of ​​the specific image information among the subjects included in the image. The information processing device according to claim 1 .

3. The concealment unit determines, for each image, a subject that is reflected in the largest area of ​​the specific image information among the subjects included in the image, and determines the degree of concealment for each image according to the size of the determined subject. The information processing device according to claim 2 .

4. The concealment unit determines, among the subjects included in each of the plurality of different images, a subject in which the area of ​​the specific image information is reflected most largest, and determines a uniform degree of concealment among the plurality of different images according to the size of the determined subject. The information processing device according to claim 2 .

5. The anonymization unit dynamically changes the degree of anonymization depending on the size of the subject.

5. The information processing device according to claim 3.

6. The concealment unit determines the degree of concealment according to a maximum estimated reflection size of a subject included in the image.

5. The information processing device according to claim 3.

7. The concealment unit irreversibly compresses the entire image including the specific image information at a compression rate corresponding to the degree of concealment, and restores the compressed image to conceal the entire image including the specific image information. The information processing device according to claim 2 .

8. An information processing method executed by an information processing device, a concealment unit that performs a concealment process on specific image information of a subject for the entire image including the specific image information; a data control unit that stores the anonymized image after the anonymization process as learning data to be used for learning a machine learning model that outputs position information of a subject when an input image including the subject is input; and An information processing method including:

9. An information processing program executed by an information processing device, a concealment procedure for concealing specific image information of a subject for an entire image including the specific image information; a data control procedure for storing the anonymized image after the anonymization process as learning data to be used for training a machine learning model that outputs position information of a subject when an input image including the subject is input; An information processing program that causes the information processing device to execute the above.

10. 10. An information processing system including the information processing device according to claim 1 , and a learning device that learns a machine learning model so as to output position information of a subject when an input image including specific image information of the subject is input based on the learning data, The information processing device includes: a concealment unit that performs a concealment process on specific image information of a subject for the entire image including the specific image information; a data control unit that transmits information associating the anonymized image after the anonymization process with positional information of a subject included in the image to the learning device so that the information is stored as the learning data; and Equipped with Information processing system.

Images