Image processing apparatus, image processing system, image processing method, and program

Abstract

To create an image for learning with a high learning effect, while protecting information to be protected.SOLUTION: An image processing apparatus has: area setting means that sets one or more areas in a first image as protected areas; image information generation means that generates image explanation information as information representing at least part of the first image; image processing means that creates a second image on the basis of irreversible processing on the protected areas in the first image; and control means that performs control so that an image for learning used for training of a learning model for image analysis processing is created on the basis of the second image and the image explanation information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a process for generating an image. [Background technology]

[0002] In recent years, image analysis has been performed in a variety of situations, using images captured by imaging devices such as surveillance cameras and machine learning technology to detect, track, and estimate attributes of objects. To improve the accuracy of image analysis, additional training is sometimes performed using data from actual scenes. In such cases, the image may contain areas that must be protected, such as people's faces or confidential information. If such areas are simply subjected to irreversible processing (also known as irreversible transformation), such as blurring, the feature values ​​of the image will change significantly as the deviation from the image before the irreversible processing increases, making the image unsuitable for training data.

[0003] To address this issue, Patent Document 1 discloses a technique in which feature amounts in a protected region and an image obtained by irreversibly transforming the protected region are output, and the feature amounts in the protected region are used during learning. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-126597 Summary of the Invention [Problem to be solved by the invention]

[0005] Patent Document 1 has a problem in that the feature amount of the protection region is used for learning.

[0006] The present invention has been made in view of the above circumstances, and has an object to make it possible to generate learning images with high learning effectiveness while protecting information that should be protected. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, an image processing device as one aspect of the present invention comprises an area setting means for setting one or more areas in a first image as a protected area, an image information generation means for generating image description information as information representing at least a portion of the first image, an image processing means for generating a second image based on irreversible processing of the protected area in the first image, and a control means for controlling so that a learning image used for training a learning model for image analysis processing is generated based on the second image and the image description information. [Effects of the Invention]

[0008] According to the present invention, it is possible to generate learning images that are highly effective for learning while protecting information that should be protected. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of the configuration of an image processing device and a learning device according to a first embodiment. [Figure 2] 1 is a diagram illustrating an example of the functional configuration of an image processing device and a learning device according to a first embodiment. [Figure 3] 4 is a flowchart illustrating an additional learning process according to the first embodiment. [Figure 4] 4 is a flowchart illustrating an additional learning process according to the first embodiment. [Figure 5] FIG. 3 is a diagram illustrating an additional learning process according to the first embodiment. [Figure 6] FIG. 3 is a diagram illustrating an additional learning process according to the first embodiment. [Figure 7] FIG. 10 is a diagram showing an example of a screen for correcting image description information according to the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of the functional configuration of an image processing device and a learning device according to a second embodiment. [Figure 9] 10 is a flowchart illustrating an additional learning process according to the second embodiment. [Figure 10] 10 is a flowchart illustrating an additional learning process according to the second embodiment. [Figure 11] FIG. 10 is a diagram illustrating an additional learning process according to the second embodiment. [Figure 12] FIG. 10 is a diagram illustrating an additional learning process according to the second embodiment. [Figure 13] FIG. 10 is a diagram illustrating an additional learning process according to the third embodiment. [Figure 14] FIG. 10 is a diagram illustrating an additional learning process according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[0011] <Embodiment 1> 1 is a block diagram showing an example of the configuration of an image processing system according to this embodiment, which includes an image processing device 100 and a learning device 110. The image processing device 100 and the learning device 110 are connected via a network 120 so as to be able to communicate with each other.

[0012] The image processing device 100 in this embodiment has a data generation function that allows a user to select an image that they want to learn for an object detection device using machine learning (not shown) and generate data necessary for additional learning, and a transmission function that transmits data to the learning device 110.

[0013] The learning device 110 has an image generation function that generates images from data required for additional learning, a learning function that performs additional learning using the generated images, and a transmission function that transmits the learning results to the image processing device 100. The following describes the case of additional learning of a machine learning model that detects vehicles as an example, but the present invention is not limited to this and can be applied to a system that learns any machine learning model.

[0014] The image processing device 100 according to this embodiment includes a CPU 101, a memory 102, a communication interface (I / F) unit 103, a storage unit 104, an input unit 105, and a display unit 106. The CPU 101, the memory 102, the communication I / F unit 103, the storage unit 104, the input unit 105, and the display unit 106 are communicably connected via a system bus. Note that the image processing device 100 according to this embodiment may further include other components.

[0015] The CPU (Central Processing Unit) 101 is a central processing unit that controls the entire image processing device 100. The CPU 101 controls the operation of each functional unit of the image processing device 100 that is connected via, for example, a system bus. The memory 102 stores data, programs, etc. that the CPU 101 uses for processing. The memory 102 also functions as the main memory, work area, etc. of the CPU 101. The CPU 101 executes processing based on the programs stored in the memory 102, thereby realizing the functional configuration of the image processing device 100 shown in FIG. 2 (described later) and the processing of the flowchart shown in FIG. 3 (described later).

[0016] The communication I / F unit 103 is an interface that connects the image processing device 100 to a network. The storage unit 104 stores, for example, various data and the like necessary when the CPU 101 performs processing related to a program. The storage unit 104 also stores, for example, various data and the like obtained when the CPU 101 performs processing related to a program. Note that data, programs, and the like used by the CPU 101 for processing may also be stored in the storage unit 104. The input unit 105 has operation members such as a mouse or buttons, and inputs user operations to the image processing device 100. The display unit 106 has a display member such as a liquid crystal display, and displays the results of processing by the CPU 101, etc.

[0017] The learning device 110 includes a CPU 111, a memory 112, a communication I / F unit 113, and a storage unit 114. The CPU 111, the memory 112, the communication I / F unit 113, and the storage unit 114 are communicatively connected via a system bus. The CPU 111, the memory 112, the communication I / F unit 113, and the storage unit 114 of the learning device 110 have functions equivalent to those of the CPU 101, the memory 102, the communication I / F unit 103, and the storage unit 104 of the image processing device 100. Therefore, a description of the CPU 111, the memory 112, the communication I / F unit 113, and the storage unit 114 of the learning device 110 will be omitted. The CPU 111 executes processing based on a program stored in the memory 112, thereby realizing the functional configuration of the learning device 110 shown in FIG. 2 (described later) and the processing of the flowchart shown in FIG. 4 (described later).

[0018] 2 is a block diagram showing an example of the functional configuration of the image processing device 100 and the learning device 110. The image processing device 100 includes an image acquisition unit 201, a correct answer assignment unit 202, a protection area setting unit 203, an image information generation unit 204, a protected image generation unit 205, a transmission unit 206, a reception unit 207, and a storage unit 208.

[0019] The image acquisition unit 201 acquires one or more specified images. In this embodiment, the image acquisition unit 201 acquires one or more images specified by the user via the input unit 105. At this time, the image acquisition unit 201 acquires the image(s) specified by the user from the images stored in the storage unit 104.

[0020] The correct answer assigning unit 202 assigns correct answer data to the image. In this embodiment, for the image acquired by the image acquiring unit 201, the user specifies (determines) as correct answer data the position (x, y) and size (w, h) in the image of a frame (rectangular frame) surrounding a vehicle (object) to be detected by the input unit 105.

[0021] The protection area setting unit 203 sets an area of ​​an image that is to be protected as a protection area. In this embodiment, for an image acquired by the image acquisition unit 201, the user uses the input unit 105 to specify one or more rectangular areas in the image that are to be protected. The protection area setting unit 203 then sets the rectangular area specified by this user operation as a protection area. Furthermore, the protection area setting unit 203 detects (or calculates) and sets the position (x, y) of the rectangle in the image and the size (w, h) in the image of the protection area.

[0022] The image information generation unit 204 analyzes the designated area in the image set by the correct answer assignment unit 202 and generates semantic information (image description information) that is information that represents the designated area (at least a part of the image). In this embodiment, information that represents the designated area is, for example, information such as "the face of a person wearing a hat" or "a van with a Japanese license plate parked in a parking lot and Japanese writing on the hood." In other words, information that represents the designated area is information that represents subjects such as people, living things, and objects that appear in the designated area of ​​the image, or scenes in the image. In this embodiment, the entire image is treated as the designated area, and text that describes the image (describes the designated area of ​​the image) is generated and output as image description information (text information). Such a function of analyzing an image and generating predetermined information can be realized by applying known technology, so a description thereof will be omitted.

[0023] The protected image generation unit 205 performs irreversible processing on the protected area of ​​the image. In this embodiment, for the image acquired by the image acquisition unit 201, a new image is generated in which the protected area, which is an area designated by the user, is protected by filling it with gray, for example. In other words, the protected image generation unit 205 functions as an image processing unit. Note that the irreversible processing is not limited to filling, and various techniques such as irreversible mosaic processing and blurring processing can be used. Furthermore, the color used to fill the protected area is not limited to gray, and can be specified as desired.

[0024] The transmission unit 206 transmits data required for additional learning to an external device, etc. The image processing device 100 of this embodiment transmits an image, correct answer data corresponding to the image, and image description information, which is information representing a specified area, to the learning device 110. The reception unit 207 receives a trained machine learning model from an external device, etc. The image processing device 100 of this embodiment receives a trained machine learning model from the learning device 110. The storage unit 208 stores data used for processing in the image acquisition unit 201, correct answer assignment unit 202, protection area setting unit 203, image information generation unit 204, protected image generation unit 205, transmission unit 206, and reception unit 207 of the image processing device 100, as well as data obtained as a processing result. The machine learning model received from the learning device 110 and stored in the storage unit 208 is used as appropriate for various image analysis processes such as object detection, tracking, and attribute estimation. In other words, the CPU 101 of the image processing device 100 can perform image analysis processes using the machine learning model received from the learning device 110.

[0025] The learning device 110 includes a receiving unit 211 , an image generating unit 212 , a learning data generating unit 213 , a learning unit 214 , a transmitting unit 215 , and a storage unit 216 .

[0026] The receiving unit 211 receives predetermined data required for additional learning from an external device, etc. In this embodiment, the receiving unit 211 receives an image, correct answer data corresponding to the image, and image description information that is information expressing the image from the image processing device 100.

[0027] The image generation unit 212 generates training images (learning images) from data required for additional learning. In this embodiment, for an image that has image description information, which is information that expresses the image, a training image is generated as an image for learning from the image and the image description information, which is information that expresses (explains) the image. The function of generating an image from image description information (prompt), which is information that expresses such an image, can be realized by applying known technology, so a description thereof will be omitted.

[0028] The training data generation unit 213 saves images and the correct answer data corresponding to the images as training data. The training unit 214 trains a machine learning model using the training data. In this embodiment, a combination of training images and the corresponding correct answer data is used as training data, and a machine learning model for detecting vehicles is additionally trained using the training data. As there are known techniques for the machine learning model to be additionally trained, a description thereof will be omitted.

[0029] The transmission unit 215 transmits the trained machine learning model to the outside. In this embodiment, the trained machine learning model for detecting vehicles is transmitted to the image processing device 100. The storage unit 216 stores data used for processing by the reception unit 211, image generation unit 212, training data generation unit 213, and training unit 214 of the training device 110, data obtained as a processing result, and the like.

[0030] Next, the processing performed by the image processing device 100 and the learning device 110 will be described with reference to Fig. 3 to Fig. 6. Fig. 3 and Fig. 4 are flowcharts illustrating the additional learning processing according to the first embodiment. Specifically, Fig. 3 is a flowchart of the processing on the image processing device 100 side in the additional learning processing, and Fig. 4 is a flowchart of the processing on the learning device 110 side in the additional learning processing. Fig. 5 and Fig. 6 are diagrams illustrating the additional learning processing according to the first embodiment.

[0031] In the following, the additional learning process of this embodiment will be described, firstly, with reference to Fig. 3, with respect to the process performed by the image processing device 100. Each of the following processes shown in Fig. 3 is realized by the CPU 101 of the image processing device 100 executing a program stored in the memory 102. Each process (step) is represented by adding an S to the beginning, thereby abbreviating the representation of the process (step).

[0032] In S301, the image acquisition unit 201 acquires one image specified by the user (hereinafter referred to as a specified image). That is, when the user specifies an image to be acquired using the input unit 105, the image acquisition unit 201 acquires the specified image (specified image) from the storage unit 104. In this embodiment, the image acquisition unit 201 acquires the image shown in FIG. 5(A) as an example. 400 in the figure is the image number of the acquired image.

[0033] In S302, the correct answer assigning unit 202 designates (determines) the position of a frame 410 surrounding a vehicle that the user wants to detect and the size of the frame 410 for the specified image acquired in S301 as correct answer data. The correct answer data is stored in the storage unit 104 together with the image number 400. That is, when the user sets a frame 410 surrounding a vehicle for the specified image using the input unit 105, the correct answer assigning unit 202 designates the frame 410 surrounding the vehicle and the size of the frame 410 (width 200, height 300) as shown in FIG. 5(B). Then, the correct answer data is stored in the storage unit 104 in association with the image number 400.

[0034] In S303, the protection area setting unit 203 determines whether or not to set a protection area, which is an area that the user wishes to protect, for the specified image. If it is determined that a protection area should be set, the process proceeds to S304. On the other hand, if it is determined that a protection area should not be set, the process proceeds to S307. Specifically, the protection area setting unit 203 notifies (inquires of) the user whether or not to set a protection area, and the user transmits a response corresponding to the notification to the protection area setting unit 203 by operating the input unit 105. That is, if the control command transmitted to the protection area setting unit 203 by the user's operation is a command to set a protection area, the protection area setting unit 203 determines that a protection area will be set and proceeds to S304. On the other hand, if the control command transmitted to the protection area setting unit 203 by the user's operation is a command to not set a protection area, the protection area setting unit 203 determines that a protection area will not be set and proceeds to S307.

[0035] In S304, when the user specifies an area to be protected (hereinafter referred to as a protection area) in the designated image using a rectangle, the protection area setting unit 203 sets the position and size of the specified rectangle. The protection area setting unit 203 then stores the specified protection area together with the image number 400. That is, when the user uses the input unit 105 to specify one or more areas in the designated image to be protected using a rectangle, the protection area setting unit 203 sets the specified area as a protection area. Furthermore, the protection area setting unit 203 detects the position (x, y) and size (w, h) of the rectangle in the designated image that is the protection area, and assigns a number to the protection area. For example, as shown in FIG. 5(C), when three areas are specified using a rectangle by the user's operation, each specified area is set as a protection area. Furthermore, the protection area setting unit 203 detects the position (x, y) and size (w, h) of the rectangle in each protection area, and assigns a number to each protection area 420, 421, and 422. The protection area setting unit 203 then stores the protection areas 420, 421, and 423 in association with the image number 400 in the storage unit 104.

[0036] In S305, the image information generation unit 204 analyzes the designated area of ​​the designated image and outputs text describing the designated image as image description information. In this example, it is assumed that the text obtained is "A van with a Japanese license plate and Japanese writing on the hood, parked in a parking lot. A person wearing a hat is in the driver's seat." The image description information is stored together with the image number. As described above, in this embodiment, the entire designated image is the designated area.

[0037] In S306, the protected image generation unit 205 performs irreversible conversion on the protected area of ​​the designated image to generate a new image. As an example of irreversible conversion, the protected image generation unit 205 of this embodiment generates an image in which the protected area set in the designated image is filled in gray (hereinafter referred to as a protected image). Figure 5(D) shows an example of a protected image (second image) in which the protected area is filled in gray by the protected image generation unit 205. In addition, the protected image generation unit 205 updates the image number 400 of the correct answer data, the protected area, and the image description information to image number 430.

[0038] In S307, the image acquisition unit 201 determines whether or not there are other images that the user wants to use in additional learning. If it is determined that there are other images, the process proceeds to S301, and the same process as described above is performed. On the other hand, if it is determined that there are no other images, the process proceeds to S308. Specifically, the image acquisition unit 201 notifies (inquires of) the user whether or not there are other images that the user wants to use in additional learning, and the user transmits a response corresponding to the notification to the image acquisition unit 201 by operating the input unit 105. That is, if the control command transmitted to the image acquisition unit 201 by the user's operation indicates that there are other images that the user wants to use in additional learning, the process proceeds to S301. On the other hand, if the control command transmitted to the image acquisition unit 201 by the user's operation indicates that there are no other images that the user wants to use in additional learning, the process proceeds to S308.

[0039] In S308, if a protection area is set, the transmission unit 206 transmits the protected image, the correct answer data corresponding to the protected image, and image description information to the learning device 110. On the other hand, if a protection area is not set, the transmission unit 206 transmits the designated image and the correct answer data corresponding to the designated image to the learning device 110. Note that Figure 6(A) shows an example of the protected image, the correct answer data corresponding to the protected image, and the image description information that are transmitted to the learning device 110 when a protection area is set. The above is the processing on the image processing device 100 side.

[0040] Next, the additional learning process of this embodiment, which is performed by the learning device 110, will be described with reference to Fig. 4. Each of the following processes shown in Fig. 4 is realized by the CPU 111 of the learning device 110 executing a program stored in the memory 112. Each process (step) is denoted by adding an S to the beginning, thereby abbreviating the notation of the process (step).

[0041] In S401, if a protection area has been set, the receiving unit 211 receives the protected image, the correct answer data corresponding to the protected image, and the image description information from the image processing device 100. On the other hand, if a protection area has not been set, the receiving unit 211 receives the designated image and the correct answer data corresponding to the designated image.

[0042] In S402, when the image description information is received in S401, the image generation unit 212 generates an image for learning (hereinafter referred to as a learning image) based on the protected image and the image description information. An example of the learning image generated by the image generation unit 212 is shown in Fig. 6(B). After that, the image generation unit 212 replaces image number 430 of the correct answer data corresponding to the protected image with image number 450 of the learning image.

[0043] In S403, the training data generation unit 213 stores the training images and the corresponding supervised data as training data in the storage unit 114. An example of the training data generated by the training data generation unit 213, the training images and the corresponding supervised data, is shown in FIG. 6(C).

[0044] In S404, the learning unit 214 additionally learns a machine learning model for detecting vehicles using the learning data saved in S403 (by reading the learning data from the storage unit 114). In S405, the transmission unit 215 transmits the trained machine learning model for detecting vehicles to the image processing device 100. The above is the processing on the learning device 110 side.

[0045] Note that, instead of the image generation unit 212 of the learning device 110, for example, the image information generation unit 204 or the protected image generation unit 205 of the image processing device 100 may generate the training images. Specifically, the image information generation unit 204 or the protected image generation unit 205 generates the training images based on the protected images and image description information. In addition, the image number of the correct answer data corresponding to the protected image is replaced with the number of the training image. Thereafter, the transmission unit 206 may transmit the generated training images and the correct answer data corresponding to the training images to the learning device 110. Then, the learning data generation unit 213 of the learning device 110 stores the training images and the correct answer data corresponding to the training images received via the reception unit 211 in the storage unit 114. Then, the processing of S404 and S405 is performed in the same manner as described above.

[0046] (Variation) In this embodiment, the image processing device 100 and the learning device 110 are configured to be separated via a network. However, they may be integrated into a single device (image processing device). That is, each component and functional unit of the learning device 110 can be configured within the image processing device 100. In this manner, the processes shown in FIGS. 3 and 4 can be performed by a single CPU. In this embodiment, the CPU 101 or CPU 111 executes a program stored in either memory 102 or memory 112 to control the operation of each functional unit, thereby realizing the processes shown in FIGS. 3 and 4. Furthermore, when integrated into a single device, the CPU, memory, communication I / F unit, and storage unit that the image processing device 100 and the learning device 110 each have may be unified. In this case, the CPU within the single device executes a program stored in memory to control the operation of each functional unit of each device, thereby realizing the processes shown in FIGS. 3 and 4. In this case, for example, the image processing device 100 can perform the training image generation process of the image generation unit 212, the training data generation process of the training data generation unit 213, and the machine learning process of the training unit 214.

[0047] Furthermore, the above-described learning device 110 has an image generation function for generating images from data required for additional learning, a learning function for performing additional learning using the generated images, and a transmission function for transmitting the learning results to the image processing device 100, but is not limited to this. For example, the learning device 110 may send the original image before privacy protection to the image processing device 100, and receive the image generated by the image processing device 100 to perform learning. In other words, the learning device 110 may have an acquisition means, a designation means, a reception means, and a learning means.

[0048] The acquisition means, like the image acquisition unit 201, acquires one or more images (the original images) specified by a user. The designation means, like the protection area setting unit 203, designates an area to be protected in the image acquired by the acquisition means as a protection area. The transmission means transmits the image acquired by the acquisition means and information (e.g., coordinates) of the protection area designated by the designation means to the image processing device 100. The learning means receives predetermined data required for additional learning transmitted from the image processing device, such as an image, correct answer data corresponding to the image, and image description information that is information expressing the image, generates learning data, and uses the learning data to train a machine learning model.

[0049] In addition, in this embodiment, additional learning processing is performed as a post-processing using the generated image, but new learning can also be performed in the same way. Furthermore, this embodiment is not limited to learning processing, and can be applied to various post-processing using images, such as analysis processing of false detections and non-detections.

[0050] Furthermore, the image information generation unit 204 may output text describing the input image as image description information, taking into account the area (protected area) set by the protected area setting unit 203. For example, the image information generation unit 204 may input each image cut out from each protected area shown in FIG. 5(C) and add the resulting text to the text describing the entire image. Using FIG. 5(C) as an example, text such as "black-haired man wearing a hat" for protected area 420, "four Japanese characters" for protected area 421, and "white Japanese license plate" for protected area 422 may be added to the text describing the entire image. Similarly, text may be generated using the correct answer data (frame position and size) specified by the correct answer assignment unit 202 as the designated area, and added to the text describing the entire image. This allows the image description information (semantic information) of the designated area to be more detailed.

[0051] Additionally, the obtained text may be given position information for each region (for example, "a black-haired man wearing a hat in the center of the image," "four Japanese characters in the bottom center of the image," "a white Japanese license plate at the bottom of the image," etc.). In other words, image description information may be generated based on the position of the protected region in the specified image. This allows for more detailed image description information (semantic information) for the specified region.

[0052] Furthermore, the image processing device 100 may further include an image information correction unit that corrects image description information, which is information representing the specified area. For this correction, the user checks the text generated by the image information generation unit 204 in S305 through a display device such as the display unit 106 and corrects it through the input unit 105. That is, when the image information correction unit receives a control command related to the correction of the image description information through a user operation, the image information correction unit corrects the image description information. The image description information correction process is preferably executed immediately after the completion of the process of S305, for example, but may be executed between the completion of the process of S305 and the start of S308. For example, the image information generation unit 204 may function as the image information correction unit.

[0053] FIG. 7 is a diagram showing an example of a screen for correcting image description information. That is, it shows an example of a UI (user interface) for correcting image description information. In FIG. 7, a protected image and text as image description information are entered in an editable text box. The user can edit and save the text in the text box using the input unit 105 or the like. For example, a UI such as that shown in FIG. 7 is displayed on the display unit 106, and the user is prompted to select whether or not to correct the image description information. To correct the image description information, the user corrects the text in the text box displayed on the UI using the input unit 105 and presses the "Save" button to save the corrected image description information. On the other hand, the user can choose not to correct the image description information by pressing the "Cancel" button displayed on the UI. This allows the user to control the information they want to protect.

[0054] Furthermore, a detection means may be newly provided to detect an area containing any content in an image, and the protection area setting unit 203 may use the detection result to set the protection area. For example, a face area detected by a face detector may be set as the protection area, or a license plate area detected by a license plate detector may be set as the protection area.

[0055] As described above, according to the configuration shown in the first embodiment, it is possible to generate data that has little impact on processes such as learning (learning images with high learning effectiveness) while protecting information that needs to be protected in processes such as learning.

[0056] <Embodiment 2> In the first embodiment, training images are generated using information that represents the entire image, but in the second embodiment, a case will be described in which the fineness of the representation is adjusted for each protection area. In the following description of the second embodiment, explanations of points that are the same as those in the first embodiment will be omitted.

[0057] 8 is a block diagram showing an example of the functional configuration of an image processing device 600 and a learning device 610 according to the second embodiment. The image processing device 600 includes an image acquisition unit 201, a correct answer assignment unit 202, a protection area setting unit 203, an image information adjustment unit 601, an image information generation unit 602, a protection image generation unit 205, a transmission unit 603, a reception unit 207, and a storage unit 208.

[0058] The image information adjustment unit 601 sets the fineness of the information (hereinafter referred to as information granularity) generated by the image information generation unit 602. In this embodiment, the information granularity is set for each protection area based on the size of the protection area. Specifically, it is defined as in the following equation (1). Information granularity G=(w×h) / (wi×hi) (1) Here, w is the width of the protection area, h is the height of the protection area, wi is the width of the image, and hi is the height of the image.

[0059] The image information generation unit 602 analyzes the image and generates image description information (semantic information) that is information that expresses the designated area, as in embodiment 1. In this embodiment, the protected area is set as the designated area, and text that explains the inside of the protected area of ​​the image for each protected area is generated as image description information, and output based on the information granularity.

[0060] Specifically, first, an image is cut out for each protection area. In conventional technology, the greater the number of attention areas in an image, the finer the information granularity. For this reason, in this embodiment, when the information granularity is 0 to 0.25, a protection area is specified as the attention area. Furthermore, when the information granularity is 0.25 to 0.5, four areas obtained by equally dividing the protection area into four are specified as the attention areas. Furthermore, when the information granularity is 0.5 to 1, nine areas obtained by equally dividing the protection area into nine are specified as the attention areas. Thereafter, the image information generation unit 602 generates and outputs text describing the cut-out image as image description information based on the cut-out image (i.e., each protection area) and the specified areas.

[0061] The transmission unit 603 transmits data required for additional learning to an external device. In this embodiment, the transmission unit 603 transmits to the learning device 610 an image, correct answer data corresponding to the image, a protection area, and image description information representing the protection area.

[0062] The learning device 610 includes a receiving unit 611 , an image generating unit 612 , a learning data generating unit 213 , a learning unit 214 , a transmitting unit 215 , and a storage unit 216 .

[0063] The receiving unit 611 receives data required for additional learning from the outside. In this embodiment, the receiving unit 611 receives from the image processing device 600 an image, correct answer data corresponding to the image, a protection area, and image description information that is information expressing the protection area.

[0064] The image generation unit 612 generates learning data from data required for additional learning. In this embodiment, for an image for which image description information exists, a learning image is generated from the image, a protection area, and image description information representing the protection area. Specifically, an image of the protection area is generated for each protection area from the image description information representing the protection area, and the image is superimposed on the image based on the position of each protection area to generate the learning image.

[0065] Next, the processing performed by the image processing device 600 and the learning device 610 will be described with reference to Figs. 9 to 12. Figs. 9 and 10 are flowcharts illustrating the additional learning processing according to the second embodiment. Specifically, Fig. 9 is a flowchart of the processing on the image processing device 600 side in the additional learning processing, and Fig. 10 is a flowchart of the processing on the learning device 610 side in the additional learning processing. Figs. 11 and 12 are diagrams illustrating the additional learning processing according to the second embodiment.

[0066] In the following, the additional learning process of this embodiment will be described, firstly, with reference to Fig. 9, with respect to the process performed by the image processing device 600. Each of the following processes shown in Fig. 9 is realized by the CPU 101 of the image processing device 600 executing a program stored in the memory 102. Each process (step) is represented by adding an S to the beginning, thereby abbreviating the representation of the process (step).

[0067] 9 are the same as S301 to S303 in Fig. 3 of the second embodiment, and therefore their explanations will be omitted. Furthermore, S907 to S908 are the same as S306 to S307, and therefore their explanations will be omitted.

[0068] In S904, the protection area setting unit 203 receives a request from the user to specify a protection area for the specified image using a rectangle, and sets the position and size of the rectangle along with a protection area number. That is, when the user uses the input unit 105 to specify one or more rectangles in the specified image that the user wants to protect, the protection area setting unit 203 sets the specified areas as protection areas. Furthermore, the protection area setting unit 203 detects the position (x, y) and size (w, h) of the rectangle in the specified image that is the protection area, and assigns a number to the protection area. For example, as shown in FIG. 11A, if two areas are specified using rectangles by the user's operation, the protection area setting unit 203 sets each specified area as a protection area. Furthermore, the protection area setting unit 203 detects the position (x, y) and size (w, h) of the rectangle for each protection area, and assigns a number to each protection area 801, 802. The protection area setting unit 203 then associates the protection areas 801 and 802 with the image number 800 and stores them in the storage unit 104.

[0069] In S905, the image information adjustment unit 601 sets the information granularity based on the protection area set in S904. In the example of FIG. 11A, the image size is 320×320 and the protection area size is 160×180, so the information granularity G of the protection area 801 is The information granularity G801 = (160 × 180) / (320 × 320) ≒ 0.28.

[0070] Similarly, the information granularity G of the protection area 802 is 40×20, so The information granularity G802 = (40 × 20) / (320 × 320) ≒ 0.008. In this way, the image information adjustment unit 601 adjusts the amount of information in the image description information generated by the image information generation unit 602 based on the size information of the protection area.

[0071] In S906, the image information generation unit 602 outputs text describing each protected area of ​​the designated image based on the information granularity as image description information. Specifically, as shown in FIG. 11B, since the information granularity of protected area 801 is 0.28, protected area 801 is first equally divided into four parts. Next, an image (hereinafter referred to as a cut-out image) is created by cutting out protected area 801 from the designated image, and text describing the cut-out image is obtained using information from the four divided protected areas 811, 812, 813, and 814. In this example, it is assumed that the text obtained for protected area 801 is "Front of a vehicle with Japanese writing on the hood. A person wearing a hat is in the driver's seat." This process is then repeated for each protected area. Similarly, since the information granularity of protected area 802 is 0.008, protected area 802 is not divided, but is cut out from the designated image to create a cut-out image, and text describing the cut-out image is obtained using information from protected area 802. Here, it is assumed that the text "Japanese license plate" is obtained for protected area 802. Finally, image information generation unit 602 generates and outputs image description information by associating description text for all protected areas with the protected area numbers.

[0072] In S909, if a protection area is set, the transmission unit 603 transmits the protected image, the correct answer data corresponding to the protected image, the protection area, and image description information to the learning device 610. If a protection area is not set, the transmission unit 603 transmits the designated image and the correct answer data corresponding to the designated image to the learning device 610. Note that Figure 11(C) shows an example of the protected image, the correct answer data corresponding to the protected image, and the image description information that are transmitted to the learning device 610 when a protection area is set.

[0073] Next, the additional learning process of this embodiment, performed by the learning device 610, will be described with reference to Fig. 10. Each of the following processes shown in Fig. 10 is realized by the CPU 111 of the learning device 610 executing a program stored in the memory 112. Each process (step) is represented by adding an S to the beginning to omit the description of the process (step). Note that S1003 to S1005 are the same as S403 to S405, and therefore description thereof will be omitted.

[0074] In S1001, if a protection area has been set, the receiving unit 611 receives data of a protected image, correct answer data corresponding to the protected image, the protection area, and image description information corresponding to the protected image from the image processing device 600. On the other hand, if a protection area has not been set, the receiving unit 611 receives data of a designated image and correct answer data corresponding to the designated image.

[0075] In S1002, if the image description information is received in S1001, the image generation unit 612 generates training images based on the protected image, protected area, and image description information. Specifically, taking FIG. 12(A) as an example, in the case of the protected image with image number 820, an image is generated for protected area 801 based on the text "Front of a vehicle with Japanese writing on the hood. A person wearing a hat is in the driver's seat." In addition, an image is generated for protected area 802 based on the text "Japanese license plate." Then, the image generation unit 612 generates a training image as shown in FIG. 12(B) by superimposing these generated images on the protected image based on the coordinates of each protected area. Thereafter, image number 820 of the correct answer data corresponding to the protected image is replaced with image number 850 of the training image.

[0076] (Variation) The image information generation unit 602 may set the designated area as an area obtained by enlarging the protected area vertically and horizontally at a certain magnification. This makes it possible to obtain information about the periphery of the protected area by enlarging the area of ​​the image used to generate image description information. In other words, the image information generation unit 602 can refine the image description information (semantic information) of the designated area by adjusting the amount of information in the protected area.

[0077] In addition to adjusting the amount of image description information based on the size of the protected area, the image information adjustment unit 601 may also obtain the complexity within the protected area, for example, the variance of pixel values ​​within the protected area, and set the information granularity to a larger value as the variance increases. The user may also directly specify the information granularity via the input unit 105. In this way, the image description information (semantic information) of the specified area can be appropriately expressed.

[0078] As in the first embodiment, in the second embodiment, the image processing device 600 may generate the training images instead of the image generation unit 612 of the training device 610. For example, the image information generation unit 602 or the protected image generation unit 205 of the image processing device 600 may generate the training images. As in the first embodiment, in the second embodiment, the image processing device 600 and the training device 610 may be integrated into a single device (image processing device).

[0079] As described above, in the second embodiment, as in the first embodiment, it is possible to protect information that needs to be protected in processes such as learning, while generating data that has little impact on processes such as learning (learning images with high learning effectiveness).

[0080] <Embodiment 3> In the first embodiment, text is used as image description information, but in the third embodiment, a case will be described in which human body posture information is also used as image description information. In the following description of the third embodiment, explanations of points similar to those in the first embodiment will be omitted.

[0081] The block diagram of this embodiment is the same as that of embodiment 1. The image processing device 100 in this embodiment has a data generation function for generating data necessary for additional learning by selecting an image that the user wants to learn from a human body posture estimation device using machine learning (not shown), and a transmission function for transmitting data to the learning device 110. Below, we will explain the relevant parts of the block diagram in Figure 2 in order to also use the above-mentioned human body posture information.

[0082] The correct answer assigning unit 202 assigns correct answer data to the image. In this embodiment, for the image acquired by the image acquiring unit 201, the user specifies, as correct answer data, the position (x, y) of a frame (rectangular frame) surrounding the human body to be detected and its size (w, h) in the image via the input unit 105. Furthermore, the posture of the human body is analyzed from the image, and posture information is specified as correct answer data. Note that in this embodiment, a human body posture estimation function for generating correct answer data, different from the human body posture estimation device, is used to analyze the posture of the human body. In this embodiment, the positions (x, y) of specified human body feature points (e.g., nose, neck, left and right eyes, ears, etc.) in the image are used as posture information. The function of analyzing the posture of the human body from such an image and extracting the human body feature points as posture information can be realized by applying known technology, and therefore a description thereof will be omitted.

[0083] The image information generating unit 204 analyzes the designated area in the image set by the correct answer assigning unit 202 and generates semantic information (image description information) that is information expressing the designated area. Furthermore, the positions of the feature points of the human body in the correct answer data are added to the image description information.

[0084] The image generation unit 212 generates training images from data required for additional learning. In this embodiment, for an image that has information expressing the image and image description information indicating the positions of feature points on the human body, a training image is generated from the image and the image description information as an image for learning. The function of generating an image by inputting other information such as the positions of feature points on the human body in addition to information expressing such an image (prompt) can be realized by applying known technology, and therefore a description thereof will be omitted.

[0085] The training data generation unit 213 saves images and corresponding supervised data as training data. The training unit 214 uses the training data to train a machine learning model. In this embodiment, a combination of training images and corresponding supervised data is used as training data, and the training data is used to additionally train a machine learning model that estimates human body posture.

[0086] Next, the processing performed by the image processing device 100 and the learning device 110 will be described with reference to Figures 13 and 14. Furthermore, since the processing in this embodiment is basically the same as in embodiment 1, the flowcharts will be described with reference to Figures 3 and 4. Below, with regard to the additional learning processing in this embodiment, first, the processing performed by the image processing device 100 will be described with reference to Figure 3.

[0087] In S301, as in the first embodiment, the image acquisition unit 201 acquires one image designated by the user (hereinafter referred to as the designated image). In this embodiment, the image acquisition unit 201 acquires the image shown in Fig. 13(A) as an example. 1300 in the figure is the image number of the acquired image.

[0088] In S302, the correct answer assigning unit 202 assigns, as correct answer data, the position and size of a frame 1310 surrounding a human body whose detection and posture estimation the user wishes to perform for the specified image acquired in S301. Furthermore, the correct answer assigning unit 202 analyzes the posture of the human body from the image and assigns the positions (x, y) of the specified feature points in the image as correct answer data. The correct answer data is stored in the storage unit 104 together with the image number 1300. That is, when the user sets a frame 1310 surrounding a human body for the specified image using the input unit 105, the correct answer assigning unit 202 assigns, as correct answer data, the position and size (width 200, height 300) of the frame 1310 surrounding the human body, as shown in FIG. 13(B). Furthermore, the posture of the human body is analyzed for the region of the frame 1310 in the specified image, and feature points for the nose, neck, left and right eyes, ears, shoulders, elbows, wrists, and waist are obtained, as shown in FIG. 13(C). The coordinates of each feature point in the image (here, the nose is (120,160), the neck is (160,210), etc.) are designated as correct data, and the coordinates of the other obtained feature points in the image are also designated as correct data. The correct data is then stored in memory unit 104 in association with image number 1300.

[0089] In S303, similarly to the first embodiment, the protection area setting unit 203 determines whether or not to set a protection area, which is an area that the user wishes to protect, for the designated image.

[0090] In S304, as in the first embodiment, when a user specifies a protection area for a specified image using a rectangle, the protection area setting unit 203 sets the position and size of the specified rectangle. The protection area setting unit 203 then stores the specified protection area together with the image number 1300. For example, as shown in FIG. 14(D), when a user specifies a rectangle by operation, the specified area is set as a protection area. The protection area setting unit 203 then detects the position (x, y) and size (w, h) of the rectangle of the protection area and assigns a number to each protection area 1320. The protection area setting unit 203 then stores the protection area 1320 in the storage unit 104 in association with the image number 1300.

[0091] In S305, the image information generation unit 204 analyzes the specified area of ​​the specified image and outputs text describing the specified image as image description information. In this example, it is assumed that the text obtained is "a face looking to the right and smiling." The image description information is stored together with the image number. Furthermore, the position information of the feature points of the human body in the correct answer data is added to the image description information.

[0092] In S306, as in the first embodiment, the protected image generation unit 205 performs irreversible processing on the protected area of ​​the designated image to generate a new image. An example of a protected image (second image) in which the protected area is filled in gray by the protected image generation unit 205 is shown in Fig. 13(E). The protected image generation unit 205 also updates the image number 1300 of the correct answer data, the protected area, and the image description information to image number 1330.

[0093] In S307, as in the first embodiment, the image acquisition unit 201 determines whether there are other images to be used in additional learning. If it is determined that there are, the process proceeds to S301, and the same process as above is performed. On the other hand, if it is determined that there are no other images, the process proceeds to S308.

[0094] In S308, as in the first embodiment, if a protection area is set, the transmission unit 206 transmits the protected image, the correct answer data corresponding to the protected image, and image description information to the learning device 110. On the other hand, if a protection area is not set, the transmission unit 206 transmits the designated image and the correct answer data corresponding to the designated image to the learning device 110.

[0095] Next, the additional learning process of this embodiment, which is performed by the learning device 110, will be described with reference to FIG.

[0096] In S401, as in the first embodiment, if a protection area has been set, the receiving unit 211 receives from the image processing device 100 the data of the protected image, the correct answer data corresponding to the protected image, and the image description information. On the other hand, if a protection area has not been set, the receiving unit 211 receives the data of the designated image and the correct answer data corresponding to the designated image. Note that FIG. 14(A) shows an example of the protected image, the correct answer data corresponding to the protected image, and the image description information that are transmitted to the learning device 110 when a protection area has been set. The above is the processing on the image processing device 100 side.

[0097] In S402, when the image generation unit 212 receives image description information in S401, the image generation unit 212 generates an image for learning (hereinafter referred to as learning image) based on the protected image and the image description information. An example of the learning image generated by the image generation unit 212 is shown in Fig. 14(B). Thereafter, the image generation unit 212 replaces image number 1330 of the correct answer data corresponding to the protected image with image number 1350 of the learning image.

[0098] In S403, the training data generation unit 213 stores the training images and the corresponding supervised data as training data in the storage unit 114. An example of the training data generated by the training data generation unit 213, the training images and the corresponding supervised data, is shown in FIG. 14(C).

[0099] In S404, the learning unit 214 additionally learns a machine learning model for estimating a human body posture using the learning data saved in S403 (by reading the learning data from the storage unit 114). In S405, the transmission unit 215 transmits the learned machine learning model for estimating a human body posture to the image processing device 100. The above is the processing on the learning device 110 side.

[0100] (Variation) In this embodiment, the specified feature points of the human body that become the correct answer data in the correct answer assigning unit 202 are estimated using a human body posture estimation function for generating correct answer data, but this is not limiting. For example, the user may directly input the positions of the feature points, or the image information generating unit 204 may be provided with a posture estimation function. Furthermore, as long as the posture information represents the posture of the human body, any feature can be used, such as the neck, nose, etc. specified in this embodiment, for example, the mouth, eyebrows, etc., specifically for the face direction. Furthermore, information representing the connection of each joint point can also be used as the posture information. Furthermore, the outline of the human body can also be used as the posture information.

[0101] In the above-described embodiment, the image description information includes text and posture information, but it may include posture information only. In other words, if there is information that characterizes the posture of the human body, it is possible to generate a training image such as that shown in Figure 14(B).

[0102] Furthermore, in this embodiment, posture information is used in addition to text as image description information, but three-dimensional information representing depth data of the image or image region segmentation information may also be used. Region segmentation information is a well-known process that groups similar features, such as color or pattern, within an image into regions and segments them. In other words, any information that represents the contents of a protected region can be used as image description information.

[0103] As described above, in the third embodiment, similarly to the first embodiment, it is possible to protect information that is desired to be protected in processing such as learning, while generating data that has little effect on processing such as learning.

[0104] Furthermore, in embodiments 2 and 3, it is of course possible to apply a configuration in which the image processing device 100 and the learning device 110 are integrated, as described in the modified example of embodiment 1, or a configuration in which the learning device 110 sends the original image before privacy protection to the image processing device 100, receives the image generated on the image processing device 100 side, and performs learning.

[0105] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0106] The disclosure of this embodiment includes the following configuration, method, system, and program.

[0107] (Configuration 1) an area setting means for setting one or more areas in the first image as a protection area; an image information generating means for generating image description information as information representing at least a part of the first image; an image processing means for generating a second image based on irreversible processing of the protected area in the first image; and a control means for controlling the generation of learning images used in learning a learning model for image analysis processing based on the second image and the image description information. 1. An image processing device comprising:

[0108] (Configuration 2) The image processing device described in configuration 1, characterized in that the control means transmits the second image and the image description information to a device that generates the learning image based on the second image and the image description information.

[0109] (Configuration 3) 3. The image processing device according to configuration 1 or 2, wherein the control means generates the learning image based on the second image and the image description information.

[0110] (Configuration 4) 4. The image processing device according to any one of configurations 1 to 3, wherein the irreversible processing performed by the image processing means includes at least one of filling, mosaic processing, and blurring processing for the protected area.

[0111] (Configuration 5) The image processing device described in any one of configurations 1 to 4, characterized in that the image information generation means generates text describing at least one of the subject of the first image or the scene in the first image as the image description information.

[0112] (Configuration 6) 6. The image processing device according to any one of configurations 1 to 5, wherein the image information generating means generates the image description information as information representing the protection area.

[0113] (Configuration 7) The image processing device described in any one of configurations 1 to 6, characterized in that the image information generation means generates at least one of posture information of the subject of the first image, three-dimensional information of the first image, and area division information of the first image as the image description information.

[0114] (Configuration 8) 8. The image processing device according to any one of configurations 1 to 7, wherein the image information generating means generates the image description information based on the position of the protection area in the first image.

[0115] (Configuration 9) 9. The image processing device according to any one of configurations 1 to 8, wherein the image information generating means generates the image description information for each of the protection areas.

[0116] (Configuration 10) 10. The image processing device according to any one of configurations 1 to 9, further comprising image information adjusting means for adjusting the amount of information in the image description information.

[0117] (Configuration 11) 11. The image processing device according to configuration 10, wherein the image information adjustment means adjusts the amount of information based on the size of the protection area.

[0118] (Configuration 12) the image information adjusting means divides the protection area according to the size of the protection area, 12. The image processing device according to claim 10, wherein the image information generating means generates the image description information for each area divided by the image information adjusting means.

[0119] (Configuration 13) 13. The image processing device according to any one of configurations 1 to 12, further comprising image information correcting means for correcting the image description information generated by the image information generating means.

[0120] (Configuration 14) further comprising a detection means for detecting an area containing any content in the first image; 14. The image processing device according to any one of configurations 1 to 13, wherein the area setting means sets the area detected by the detection means as the protection area.

[0121] (Configuration 15) 15. The image processing device according to any one of configurations 1 to 14, further comprising: learning means for performing machine learning using the learning images.

[0122] (Configuration 16) A learning device communicably connected to the image processing device according to any one of configurations 1 to 14, an acquisition means for acquiring the first image; a designation means for designating the protection area in the first image; a transmitting means for transmitting the image acquired by the acquiring means and information on the protection area designated by the designating means to the image processing device; a receiving means for receiving the learning image generated by the image processing device; a learning means for executing machine learning of a learning model for image analysis processing using the learning images received by the receiving means; A learning device comprising:

[0123] (Configuration 17) An image processing system including an image processing device and a learning device communicably connected to the image processing device, an area setting means for setting one or more areas in the first image as a protection area; an image information generating means for generating image description information as information representing at least a part of the first image; an image processing means for generating a second image based on irreversible processing of the protected area in the first image; a generation means for generating a learning image to be used for learning a learning model for image analysis processing based on the second image and the image description information; and a learning means for performing machine learning using the learning images. An image processing system comprising:

[0124] (Configuration 18) An image processing method performed by an image processing device, a region setting step of setting one or more regions in the first image as a protection region; an image information generating step of generating image description information as information representing at least a part of the first image; an image processing step of generating a second image based on a lossy process applied to the protected area in the first image; An image processing method characterized by comprising a control step of controlling the generation of a learning image to be used for learning a learning model for image analysis processing based on the second image and the image description information.

[0125] (Configuration 19) A program for causing a computer to function as each means of the image processing device according to any one of configurations 1 to 15. [Explanation of symbols]

[0126] 100 Image processing device 110 Learning Device 203 Protection area setting section 204 Image information generation unit 205 Protected image generation unit

Claims

1. an area setting means for setting one or more areas in the first image as a protection area; an image information generating means for generating image description information as information representing at least a part of the first image; an image processing means for generating a second image based on a lossy process performed on the protected area in the first image; and a control means for controlling the generation of learning images used in learning a learning model for image analysis processing based on the second image and the image description information.

1. An image processing device comprising:

2. The image processing device according to claim 1, characterized in that the control means transmits the second image and the image description information to a device that generates the learning image based on the second image and the image description information.

3. 2. The image processing apparatus according to claim 1, wherein the control means generates the learning image based on the second image and the image description information.

4. 2. The image processing device according to claim 1, wherein the irreversible processing executed by the image processing means includes at least one of painting, mosaic processing, and blurring of the protected area.

5. 2. The image processing apparatus according to claim 1, wherein the image information generating means generates, as the image description information, text that describes at least one of a subject of the first image and a scene in the first image.

6. 2. The image processing apparatus according to claim 1, wherein said image information generating means generates said image description information as information representing said protection area.

7. The image processing device described in claim 1, characterized in that the image information generation means generates at least one of posture information of the subject of the first image, three-dimensional information of the first image, and area division information of the first image as the image description information.

8. 2. The image processing apparatus according to claim 1, wherein the image information generating means generates the image description information based on the position of the protection area in the first image.

9. 2. The image processing apparatus according to claim 1, wherein said image information generating means generates said image description information for each of said protection areas.

10. 2. The image processing apparatus according to claim 1, further comprising image information adjusting means for adjusting the amount of said image description information.

11. 11. The image processing apparatus according to claim 10, wherein the image information adjusting means adjusts the amount of information based on the size of the protection area.

12. the image information adjusting means divides the protection area according to the size of the protection area, 11. The image processing apparatus according to claim 10, wherein the image information generating means generates the image description information for each area divided by the image information adjusting means.

13. 2. The image processing apparatus according to claim 1, further comprising image information correcting means for correcting the image description information generated by the image information generating means.

14. further comprising a detection means for detecting an area containing any content in the first image; 2. The image processing apparatus according to claim 1, wherein the area setting means sets the area detected by the detection means as the protection area.

15. The image processing device according to claim 1 , further comprising: a learning unit that performs machine learning using the learning images.

16. A learning device communicably connected to the image processing device according to any one of claims 1 to 14, an acquisition means for acquiring the first image; a designation means for designating the protection area in the first image; a transmitting means for transmitting the image acquired by the acquiring means and information on the protection area designated by the designating means to the image processing device; a receiving means for receiving the learning image generated by the image processing device; a learning means for executing machine learning of a learning model for image analysis processing using the learning images received by the receiving means; A learning device comprising:

17. An image processing system including an image processing device and a learning device communicably connected to the image processing device, an area setting means for setting one or more areas in the first image as a protection area; an image information generating means for generating image description information as information representing at least a part of the first image; an image processing means for generating a second image based on a lossy process performed on the protected area in the first image; a generation means for generating a learning image to be used for training a learning model for image analysis processing based on the second image and the image description information; A learning means for performing machine learning using the learning images. An image processing system comprising:

18. An image processing method performed by an image processing device, a region setting step of setting one or more regions in the first image as a protection region; an image information generating step of generating image description information as information representing at least a part of the first image; an image processing step of generating a second image based on a lossy process applied to the protected area in the first image; An image processing method characterized by comprising a control step of controlling the generation of a learning image to be used for learning a learning model for image analysis processing based on the second image and the image description information.

19. A program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 15.

Images