Information processing device, program, information processing system, and information processing method

Abstract

This information processing device (100) is characterized by comprising: an estimation unit (131) that detects, using an estimation model, an object from an image shown by image data and estimates object-related information that is related to the detected object; and a scene information selection unit (133) that selects object-related information satisfying a selection condition, from the estimated object-related information, as at least some of scene information.

Description

Information processing device, program, information processing system, and information processing method

[0001] The present disclosure relates to an information processing device, a program, an information processing system, and an information processing method.

[0002] In recent years, the application of artificial intelligence (AI) to surveillance systems using surveillance cameras has been progressing. In general, to improve the accuracy of an AI model (e.g., an object detection AI model) installed in a surveillance system, it is effective to perform training using image data acquired at the location where the surveillance camera is installed.

[0003] For example, Patent Document 1 discloses a method for easily generating training data by generating a composite image by pasting an image of an object onto a background image.

[0004] Japanese Patent Application Laid-Open No. 2020-149086

[0005] However, due to issues of privacy and portrait rights of the subjects captured on camera, it is difficult to use images for AI training without the subjects' consent. One possible method is to remove private information from the image by masking the subject's entire body or facial area, but this may result in important information being omitted from the AI ​​training data.

[0006] Therefore, one or more aspects of the present disclosure aim to make it possible to identify features desired as learning data without using the subject's private information.

[0007] An information processing device according to one aspect of the present disclosure is characterized by comprising an estimation unit that uses an estimation model to detect an object from an image represented by image data and estimates object-related information, which is information related to the object, and a scene information selection unit that selects object-related information from the object-related information that satisfies a selection condition as at least a part of scene information.

[0008] A program according to one aspect of the present disclosure is characterized in that it causes a computer to function as an estimation unit that uses an estimation model to detect an object from an image represented by image data and estimate object-related information, which is information related to the object, and a scene information selection unit that selects object-related information from the object-related information that satisfies a selection condition as at least part of the scene information.

[0009] An information processing system according to one aspect of the present disclosure is characterized by comprising an estimation unit that uses an estimation model to detect an object from an image represented by image data and estimates object-related information, which is information related to the object, and a scene information selection unit that selects object-related information from the object-related information that satisfies a selection condition as at least a part of the scene information.

[0010] An information processing method according to one aspect of the present disclosure is characterized in that it uses an estimation model to detect an object from an image represented by image data, and estimates object-related information, which is information related to the object, and selects object-related information from the object-related information that satisfies a selection condition as at least part of the scene information.

[0011] According to one or more aspects of the present disclosure, it is possible to identify features desired as training data without using the subject's private information.

[0012] 1 is a block diagram showing a simplified configuration of an information processing device according to a first embodiment; FIG. 2 is a schematic diagram showing an example of an image captured by an imaging unit; FIG. 3 is a schematic diagram showing an example of an estimation result; FIG. 4 is a schematic diagram showing an example of scene information; (A) and (B) are schematic diagrams showing an example of the scene information shown in FIG. 4 superimposed on a background image; (A) and (B) are schematic diagrams showing examples of first learning data as shown in (A) and (B); (A) and (B) are schematic diagrams showing an example of an image to be displayed; and (B) are schematic diagrams showing an example of confirmation scene information. FIG. 1 is a block diagram showing a simplified configuration of an information processing device according to a second and third embodiment; FIG. 2 is a block diagram showing a schematic configuration of a learning unit; and FIG. 3 is a block diagram showing a schematic configuration of a computer.

[0013] 1 is a block diagram showing a simplified configuration of an information processing device 100 according to embodiment 1. The information processing device 100 includes an imaging unit 110, a learning scene collection unit 130, and a learning data generation and storage unit 150.

[0014] The image capturing unit 110 captures an image including a target to be studied, and generates image data representing the image. The generated image data is provided to the learning scene collection unit 130.

[0015] The training scene collection unit 130 receives image data from the image capture unit 110 as input data, performs estimation using an estimation model, and generates scene information that is the basis for generating training data.

[0016] The learning scene collection unit 130 includes an estimation unit 131 , a selection condition specification unit 132 , a scene information selection unit 133 , and a scene information storage unit 134 .

[0017] The estimation unit 131 detects an object from an image represented by image data from the imaging unit 110 using an estimation model, and estimates object-related information, which is information related to the object. The estimated object-related information is provided to the scene information selection unit 133 as an estimation result. Here, the object-related information obtained includes position information, a class name, and a certainty factor of the estimated object. The class name is, for example, type identification information indicating the type of object. Here, the class name is an object name, which is the name of the object. The certainty factor is an index indicating the likelihood of the estimation result, and is, for example, greater than or equal to 0 and less than or equal to 1.

[0018] Fig. 2 is a schematic diagram showing an example of an image captured by the imaging unit 110. As shown in Fig. 2, image IM1 shows a person PE1 and a person PE2. From the image data of such an image, the estimation unit 131 estimates the position information, class name, and certainty factor of each of person PE1 and person PE2, as shown in Fig. 3.

[0019] When the position information is expressed as (top left x-coordinate, top left y-coordinate, width, height) of a rectangle surrounding an object, for example, the position information, class name, and confidence level of person PE1 can be expressed as (700, 100, 200, 900), "Person," and "0.3," respectively. Also, the position information, class name, and confidence level of person PE2 can be expressed as (1500, 800, 300, 1000), "Person," and "0.9," respectively.

[0020] 1 , the selection condition specification unit 132 specifies a selection condition to be selected as scene information from the estimation result of the estimation unit 131. The selection condition may be, for example, a condition for selection based on at least one of a position, a class name, and a certainty factor.

[0021] For example, if it is desired to collect scenes with a low degree of certainty in the estimation by the estimation unit 131, it is sufficient to select, as scene information, estimation results whose certainty is equal to or less than a predetermined threshold (e.g., 0.6). In this case, the selection condition is that the certainty is equal to or less than a predetermined threshold. Also, for example, if it is desired to collect scenes of places where objects frequently pass by as learning data, it is sufficient to select, as scene information, estimation results whose position information indicates the place to be collected. In this case, the selection condition is that the position information is within a predetermined range.

[0022] The scene information selection unit 133 selects, from the estimation results of the estimation unit 131, estimation results that satisfy the selection conditions specified by the selection condition specification unit 132 as at least part of the scene information. Then, the scene information selection unit 133 provides the selected scene information to the scene information storage unit 134.

[0023] The scene information storage unit 134 stores the received scene information. Here, the scene information does not include privacy information and indicates information that serves as a basis for generating training data. Fig. 4 is a schematic diagram showing an example of scene information.

[0024] Examples of information that serves as a basis for generating training data include location information, class names, and confidence levels, which are estimation results from the estimation unit 131. In addition, the scene information may include other information that does not fall under privacy information, such as the color of an object, the time when an image represented by the image data was captured, or the illuminance of an image represented by the image data.

[0025] Furthermore, even when the estimation unit 131 uses an estimation model other than object detection, the scene information may include information from the estimation results that does not fall under privacy information.

[0026] Note that, as information that does not fall under the category of privacy information, for example, the color of an object may be determined from position information. Specifically, by obtaining a histogram of pixel values ​​near the center of a detection frame that identifies a person as an object and its position information, the general color of the person's clothing, in other words, the color of the object, can be determined. In other words, the object's position information can be associated with color information. Note that the detection frame here is one means for identifying the aforementioned detection position. In addition, if an estimation model for estimating age, gender, posture, etc. can be used, these can be estimated and used as scene information within a range that does not fall under privacy information.

[0027] In this way, scene information showing characteristics desired as learning data can be acquired without using the subject's private information.

[0028] 1 , the training data generation and storage unit 150 generates training data in accordance with the scene information collected by the training scene collection unit 130. The training data generation and storage unit 150 stores the generated training data. The training data generation and storage unit 150 includes a training data generation unit 151, a first training data storage unit 152, a training data confirmation unit 153, and a second training data storage unit 154.

[0029] The learning data generation unit 151 acquires scene information from the scene information storage unit 134 and generates image data representing a new image as learning data according to the scene information. The learning data generated here is called first learning data. This first learning data is stored in the first learning data storage unit 152. The new image here is an image different from the image represented by the image data used when the learning scene collection unit 130 collected scene information.

[0030] The learning data generation unit 151 executes the process when a certain amount of scene information has accumulated in the scene information storage unit 134 or when a certain period of time has passed. Alternatively, the learning data generation unit 151 may execute the process when instructed to start from outside. The learning data may be updated every time scene information is generated.

[0031] For example, the learning data generation unit 151 acquires scene information stored in the scene information storage unit 134, and generates first learning data based on the scene information using a subject that does not pose a privacy issue. Note that the first learning data is assumed to be identifiably associated with the scene information that was the basis for generating the first learning data.

[0032] An object with no privacy issues refers to an object such as a person who has consented to being the subject of the training data, or an image of an object such as a person that does not contain privacy information. Hereinafter, the generation of the first training data by the training data generation unit 151 will be described with reference to the drawings.

[0033] 5A and 5B are schematic diagrams showing an example in which the scene information shown in FIG. 4 is superimposed on a background image. As shown in FIG. 5A, the first scene information of the scene information shown in FIG. 4 has position information (700, 100, 200, 900), a class name "Person", and a confidence level of "0.3". In FIG. 5A, a dashed rectangle, the class name, and the confidence level are shown at the position indicated by the position information.

[0034] As shown in Fig. 5B, the second scene information of the scene information shown in Fig. 4 has position information (900, 50, 200, 800), a class name "Person", and a confidence level of 0.2. In Fig. 5B, a dashed rectangle, the class name, and the confidence level are shown at the position indicated by the position information.

[0035] The training data generation unit 151 then places objects that do not pose privacy issues at the positions indicated by the dashed rectangles in the examples shown in Figures 5(A) and (B), thereby generating first training data such as those shown in Figures 6(A) and (B). While Figure 6 shows one piece of first training data generated for one piece of scene information, multiple pieces of first training data may be generated for one piece of scene information. Conversely, one piece of first training data may include multiple pieces of scene information.

[0036] Here, the background image is assumed to be captured by, for example, the image capturing unit 110. The privacy-friendly object may be generated using, for example, a known technique using CG (Computer Graphics) or AI. Note that if the scene information includes date and time or illuminance, the influence of these may be reflected in the privacy-friendly object.

[0037] When CG is used, the user may manually synthesize an object that does not pose a privacy problem using CG. In this case, the learning data generation unit 151 may instruct the user on which object to synthesize and at what position in the background image, based on the scene information.

[0038] Furthermore, the learning data generation unit 151 may use, as the background image, an image similar to the image captured at the position where the image from which the scene information was acquired, rather than an image captured at the position where the image from which the scene information was acquired. For example, the learning data generation unit 151 may use, as the background image, an image that the user has generated by CG to imitate the background.

[0039] In the above example, the learning data generation unit 151 generates the first learning data by placing an object that does not pose a privacy issue in a background image in accordance with scene information. However, the first learning data may also be generated by capturing an image of a person who has agreed to the use of their image as learning data using the imaging unit 110.

[0040] For example, the learning data generation unit 151 may display an example for capturing a new image in accordance with the scene information on a display unit (not shown), and generate the first learning data from an image of a subject whose privacy is not an issue in accordance with the example. Here, a person who has agreed to the use of their photograph as learning data will be described as the subject.

[0041] The learning data generation unit 151 acquires scene information from the scene information storage unit 134 and, based on the scene information, displays on a display unit (not shown) what kind of scene image is required for the subject. Specifically, the learning data generation unit 151 may display the subject using an image, or may display the subject using a sentence such as "Person at coordinates (700, 100, 200, 900)."

[0042] Furthermore, when the learning data generation unit 151 acquires scene information such as that shown in FIG. 4, it may indicate what kind of scene image is required for the subject by displaying an image such as that shown in FIG. 7(A) or 7(B) on a display unit not shown.

[0043] 7A, the desired position of the person's image is displayed as a circle. Based on this image, the subject moves around the specified position and its surroundings, and the imaging unit 110 captures this state, thereby generating the first learning data.

[0044] FIG. 7B also shows the route the subject should take. Information about the orientation of the person when moving along the route, such as front, back, or side, or the angle or direction, may be added. The route is created so as to pass through the positions indicated by the position information in the acquired scene information. The subject moves along the specified route, and the imaging unit 110 captures the state, thereby generating first learning data. By specifying the shortest route, it is possible to shorten the imaging time.

[0045] If the scene information also includes color tone, posture, or the like, the learning data generating unit 151 may display such information on a display unit (not shown).

[0046] Returning to FIG. 1 , the first learning data storage unit 152 stores the first learning data generated by the learning data generation unit 151 .

[0047] The learning data confirmation unit 153 confirms whether the first learning data stored in the first learning data storage unit 152 is desirable as learning data. For example, if the first learning data satisfies a predetermined condition, the learning data confirmation unit 153 determines that such first learning data is desirable as learning data and stores such first learning data in the second learning data storage unit 154 as second learning data. On the other hand, if the first learning data does not satisfy the predetermined condition, the learning data confirmation unit 153 transmits the scene information from which such first learning data was generated to the learning data generation unit 151 and instructs the learning data generation unit 151 to generate the first learning data again. Here, the predetermined condition is, for example, satisfying a selection condition identified by the selection condition identification unit 132, as described below.

[0048] Specifically, the training data confirmation unit 153 acquires the first training data and provides the first training data to the estimation unit 131 of the training scene collection unit 130 .

[0049] The estimation unit 131 performs estimation on the first training data and provides the estimation result to the training data confirmation unit 153. The estimation result obtained in this way is called confirmation scene information.

[0050] Upon receiving the confirmation scene information, the learning data confirmation unit 153 confirms whether the confirmation scene information satisfies the selection conditions identified by the selection condition identification unit 132. Furthermore, the learning data confirmation unit 153 may confirm whether the confirmation scene information that satisfies the selection conditions identified by the selection condition identification unit 132 is similar to the scene information that was the source of generating the first learning data.

[0051] In this embodiment, the similarity of scene information is determined by comparing position information, class name, and confidence level. The similarity of position information can be determined by whether or not the Intersection over Union (IoU), which indicates the degree of overlap of the position information, is equal to or greater than a predetermined threshold. Alternatively, the similarity may be determined by whether or not the difference between the center coordinates or the coordinates of the four corners of the position information is equal to or less than a predetermined threshold. The above thresholds are set in advance for each device.

[0052] The closeness of the certainty levels may be determined by whether the difference between the certainty level of the confirmation scene information and the certainty level set as the selection condition by the selection condition specifying unit 132 is equal to or less than a predetermined threshold value.

[0053] Here, an example will be described in which confirmation scene information such as that shown in Fig. 8 is obtained for the first learning data shown in Fig. 6(A) and (B). In this embodiment, an example will be described in which it is confirmed whether the selection conditions identified by the selection condition identification unit 132 are satisfied, and in addition, whether the position information and class name of the confirmation scene information are similar to those of the original scene information.

[0054] Here, the selection condition specification unit 132 sets the selection condition as "certainty level of 0.6 or less," so that a certainty level of "0.2" for the first confirmation scene information satisfies the selection condition, but a certainty level of "0.7" for the second confirmation scene information does not satisfy the selection condition.

[0055] The learning data confirmation unit 153 compares the scene information shown in Fig. 4 with the confirmation scene information in Fig. 8. The location information is determined to be similar because the IoU value is equal to or greater than a threshold. In addition, the class names are both "Person," so they are the same.

[0056] From the above, the first learning data shown in FIG. 6(A) is desirable as learning data, but the first learning data shown in FIG. 6(B) is not desirable as learning data.

[0057] 6A corresponding to the first scene information, the learning data confirmation unit 153 stores the first learning data as second learning data in the second learning data storage unit 154. The second data is data that is used as input data for training the learning model.

[0058] As another example, a case will be described in which the selection conditions are defined as a class name of "Person" and a color of "black." Here, the learning data confirmation unit 153 only checks whether the selection conditions are met. In this case, the learning data confirmation unit 153 checks whether the class name and color of the confirmation scene information meet the selection conditions. If they are met, the learning data confirmation unit 153 determines that the target first learning data is desirable data as learning data, without checking whether the confirmation scene information is similar to the original scene information.

[0059] Next, the learning data confirmation unit 153 notifies the learning data generation unit 151 of identification information for identifying the second scene information corresponding to the first learning data shown in Figure 6 (B), thereby instructing the learning data generation unit 151 to re-generate the first learning data corresponding to this second scene information.

[0060] Upon receiving an instruction from the learning data confirmation unit 153, the learning data generation unit 151 regenerates first learning data corresponding to the specified second scene information. The regenerated first learning data is provided to the learning data confirmation unit 153. The regenerated first learning data is also referred to as regenerated learning data or first regenerated learning data.

[0061] The learning data confirmation unit 153 again checks the first regenerated learning data that has been regenerated by the learning data generation unit 151. If the first regenerated learning data is determined to be a scene that satisfies the conditions, it is stored as second learning data in the second learning data storage unit 154. If it is again determined that it does not satisfy the conditions, the learning data confirmation unit 153 either issues an instruction to the learning data generation unit 151 again or terminates the processing.

[0062] The number of times that the learning data confirmation unit 153 instructs regeneration may be set in advance. Alternatively, the user may instruct the cancellation of regeneration via an input unit (not shown).

[0063] An example will be described in which the training data generation unit 151 prepares N types of person images as subjects that do not pose privacy issues. In this case, it is possible to generate first training data by randomly selecting one and combining it with a background image based on scene information.

[0064] If the learning data confirmation unit 153 determines that the first learning data generated in this manner does not satisfy the conditions, the learning data generation unit 151 randomly selects another one of the N types of person images and generates the first learning data again. When performing such processing, the learning data confirmation unit 153 can issue a regeneration instruction up to N-1 times. Note that a maximum number of times to issue a regeneration instruction may be set in advance to prevent the processing time from becoming too long.

[0065] In the above example, the learning data generation unit 151 may first use N types of person images to generate N types of first learning data from scene information. Then, of the N types of first learning data, all of the data that the learning data confirmation unit 153 determines to satisfy the conditions may be stored as second learning data in the second learning data storage unit 154. In such a case, the learning data confirmation unit 153 does not need to issue an instruction for regeneration.

[0066] In addition to the above example, when instructed to regenerate, the learning data generation unit 151 may regenerate N types of first learning data by randomly shifting the position or size indicated in the scene information and combining an object that does not pose a privacy problem with the background image. In such a case, the learning data confirmation unit 153 will continue to instruct regeneration until N types of first learning data are generated.

[0067] As described above, when it is determined that the training data is not desirable, the training data generation unit 151 generates regenerated training data that shows an image different from the image shown in the training data in accordance with the scene information, and the training data confirmation unit 153 checks whether the regenerated training data is desirable or not, and sets the desirable training data as the second training data, thereby reducing the possibility that second training data does not exist corresponding to each piece of scene information.

[0068] The second learning data storage unit 154 stores the first learning data that is determined by the learning data checking unit 153 to satisfy the conditions as second learning data.

[0069] As a result, second training data, which are images desired as training data, can be obtained without using privacy information.

[0070] 9 is a block diagram showing a simplified configuration of an information processing device 200 according to embodiment 2. The information processing device 200 according to embodiment 2 trains a training model using second training data generated in the same manner as the information processing device 100 according to embodiment 1.

[0071] The information processing device 200 includes an imaging unit 110 , a learning scene collection unit 130 , a learning data generation and storage unit 150 , and a learning unit 270 .

[0072] The imaging unit 110, the learning scene collection unit 130, and the learning data generation and storage unit 150 of the information processing device 200 according to embodiment 2 are similar to the imaging unit 110, the learning scene collection unit 130, and the learning data generation and storage unit 150 of the information processing device 100 according to embodiment 1.

[0073] The learning unit 270 acquires the second learning data from the second learning data storage unit 154 of the learning data generation and storage unit 150. Then, the learning unit 270 generates teacher data using the second learning data as input data, and uses the teacher data to train a learning model.

[0074] Generally, image data and annotation data as correct answer labels indicating the corresponding correct answers are required as training data for training an AI learning model. Since the second learning data is image data, annotation data must be created. The annotation data is usually added manually. Therefore, the learning unit 270 can acquire annotation data indicating the correct answers from the user using a GUI (Graphical User Interface) via a display unit and an input unit (not shown).

[0075] Then, the learning data generation unit 151 uses the teacher data to learn the learning model.

[0076] As described above, according to the second embodiment, the collected second training data can be used to train the training model.

[0077] Embodiment 3. As shown in FIG. 9 , an information processing device 300 according to embodiment 3 includes an imaging unit 110, a learning scene collection unit 130, a learning data generation and storage unit 150, and a learning unit 370. While embodiment 2 described an example in which annotation data was acquired from a user, embodiment 3 describes an example in which annotation data is automatically assigned using a high-precision AI model. A high-precision AI model refers to a trained AI model that can be used as a means for enabling a learning model trained in the present disclosure to reach a target accuracy, such as an AI model that runs on a computer capable of higher-performance computational processing than an edge device such as a camera, or an AI model that runs on an edge device with similar processing performance but has a desired standard of accuracy when estimating a detection target in an expected imaging situation.

[0078] The imaging unit 110, the learning scene collection unit 130, and the learning data generation and storage unit 150 of the information processing device 300 according to embodiment 3 are similar to the imaging unit 110, the learning scene collection unit 130, and the learning data generation and storage unit 150 of the information processing device 100 according to embodiment 1.

[0079] The learning unit 370 acquires second training data from the second training data storage unit 154 of the training data generation and storage unit 150. Then, the learning unit 370 generates training data using the second training data as input data, and uses the training data to train a training model. For example, the learning unit 370 trains the training model using training data generated by attaching correct answer labels to input data.

[0080] 10 is a block diagram showing a schematic configuration of the learning unit 370. The learning unit 370 includes a learning data acquisition unit 371, a model estimation unit 372, a teacher data generation unit 373, and a model generation unit 374.

[0081] The learning data acquisition unit 371 acquires second learning data from the second learning data storage unit 154 of the learning data generation and storage unit 150. The acquired second learning data is provided to the model estimation unit 372.

[0082] The model estimation unit 372 estimates information required as annotation data for the second learning data using a highly accurate AI estimation model. Then, the model estimation unit 372 provides the annotation data indicating the estimated information and the second learning data to the teacher data generation unit 373.

[0083] The teacher data generation unit 373 generates teacher data by assigning the annotation data from the model estimation unit 372 as correct labels to the second training data. The generated teacher data is provided to the model generation unit 374.

[0084] The model generation unit 374 uses the training data from the training data generation unit 373 to train a learning model.

[0085] As described above, according to the third embodiment, by automatically estimating annotation data using a highly accurate AI model, it is possible to automatically generate teacher data from the second learning data and learn a learning model.

[0086] The model generation unit 374 may learn the learning model using "knowledge distillation," which is a well-known technique for transferring knowledge of a highly accurate AI estimation model to another model. The highly accurate AI estimation model here may be the estimation model used by the model estimation unit 372.

[0087] In addition, the weights of the learning model learned by the model generation unit 374 can be sent to the estimation unit 131 of the learning scene collection unit 130 to update the estimation model used by the estimation unit 131.

[0088] The information processing devices 100, 200, and 300 described above can be realized by, for example, a computer 10 as shown in Fig. 11. The computer 10 includes a camera 11, an auxiliary storage device 12 such as a hard disk drive (HDD) or a solid state drive (SSD), a main storage device 13 such as a volatile or non-volatile memory, a display device 14, an input device 15 such as a keyboard or a mouse, and a processor 16 such as a central processing unit (CPU).

[0089] For example, the imaging unit 110 can be realized by the camera 11. The scene information storage unit 134, the first learning data storage unit 152, and the second learning data storage unit 154 can be realized by the auxiliary storage device 12 or the main storage device 13. The estimation unit 131, the selection condition specification unit 132, the scene information selection unit 133, the learning data generation unit 151, the learning data confirmation unit 153, and the learning units 270 and 370 can be realized by the processor 16 reading a program stored in the auxiliary storage device 12 into the main storage device 13 and executing the program. When performing processing, the processor 16 can appropriately use the display device 14 functioning as a display unit and the input device 15 functioning as an input unit.

[0090] The functions executed by the information processing devices 100, 200, and 300 do not necessarily have to be realized by a single computer 10, but may be realized by multiple computers connected to a network such as the Internet, thereby constituting an information processing system. In this case, the computer has a communication interface such as a NIC (Network Interface Card) that functions as a communication unit in addition to the configuration shown in Fig. 11. In this case, the multiple computers may transmit and receive information and programs required for processing as appropriate.

[0091] Specifically, each of the imaging unit 110, the learning scene collection unit 130, the learning data generation and storage unit 150, and the learning units 270 and 370 may be realized by a single computer, or any combination of the imaging unit 110, the learning scene collection unit 130, the learning data generation and storage unit 150, and the learning units 270 and 370 may be realized by a single computer.

[0092] In the above-described second and third embodiments, the training data is generated by the learning units 270 and 370, but the second and third embodiments are not limited to such examples. For example, when the training data generation unit 151 creates the first training data, it may assign information necessary for the training data, such as location information where a privacy-friendly object is combined with a background image and the type of privacy-friendly object, to the first training data as a correct label, thereby enabling the first training data stored as the second training data to be used as training data as is.

[0093] 100, 200, 300 Information processing device, 110 Imaging unit, 130 Learning scene collection unit, 131 Estimation unit, 132 Selection condition identification unit, 133 Scene information selection unit, 134 Scene information storage unit, 150 Learning data generation storage unit, 151 Learning data generation unit, 152 First learning data storage unit, 153 Learning data confirmation unit, 154 Second learning data storage unit, 270, 370 Learning unit, 371 Learning data acquisition unit, 372 Model estimation unit, 373 Teacher data generation unit, 374 Model generation unit.

Claims

1. an estimation unit that detects an object from an image represented by image data using an estimation model and estimates object-related information that is information related to the object; a scene information selection unit that selects, from the object-related information, object-related information that satisfies a selection condition as at least a part of the scene information. An information processing device comprising:

2. The selection condition is at least one of the position, the class name, and the confidence level.

2. The information processing apparatus according to claim 1,

3. The scene information includes at least one of the color of the object, the time when the image was captured, and the illuminance of the image, in addition to the object-related information.

2. The information processing apparatus according to claim 1,

4. The scene information includes, in addition to the object-related information, at least one of a color tone of the object, a time when the image was captured, and an illuminance of the image.

3. The information processing apparatus according to claim 2,

5. a learning data generation unit that generates learning data representing new images according to the scene information; and a learning data confirmation unit that confirms whether the learning data is desirable or not, and uses the learning data that is desirable as input data for training a learning model.

2. The information processing apparatus according to claim 1,

6. A learning data generation unit that generates learning data representing new images according to the scene information; and a learning data confirmation unit that confirms whether the learning data is desirable or not, and uses the learning data that is desirable as input data for training a learning model.

3. The information processing apparatus according to claim 2,

7. A learning data generation unit that generates learning data representing new images according to the scene information; and a learning data confirmation unit that confirms whether the learning data is desirable or not, and uses the learning data that is desirable as input data for training a learning model.

4. The information processing apparatus according to claim 3,

8. A learning data generation unit that generates learning data representing new images according to the scene information; and a learning data confirmation unit that confirms whether the learning data is desirable or not, and uses the learning data that is desirable as input data for training a learning model.

5. The information processing apparatus according to claim 4,

9. The learning data confirmation unit confirms whether or not an estimation result obtained by performing estimation from the new image using the estimation model satisfies the selection condition, and sets the learning data that satisfies the selection condition as the input data.

6. The information processing device according to claim 5,

10. The learning data confirmation unit confirms whether or not an estimation result obtained by performing estimation using the estimation model from the new image satisfies the selection condition, and sets the learning data that satisfies the selection condition as the input data. The information processing device according to claim 6 .

11. The learning data confirmation unit confirms whether or not an estimation result obtained by performing estimation using the estimation model from the new image satisfies the selection condition, and sets the learning data that satisfies the selection condition as the input data. The information processing device according to claim 7 .

12. The learning data confirmation unit confirms whether or not an estimation result obtained by performing estimation using the estimation model from the new image satisfies the selection condition, and sets the learning data that satisfies the selection condition as the input data. The information processing device according to claim 8 .

13. the learning data confirmation unit confirms whether or not an estimation result obtained by performing estimation from the new image using the estimation model satisfies the selection condition, and if the selection condition is satisfied, confirms whether or not the estimation result obtained by performing estimation from the new image using the estimation model is close to the scene information used when generating the learning data, and sets the learning data close to the scene information used when generating the learning data as the input data.

6. The information processing device according to claim 5,

14. The learning data confirmation unit confirms whether or not an estimation result obtained by performing estimation using the estimation model from the new image satisfies the selection condition, and if the selection condition is satisfied, confirms whether or not the estimation result obtained by performing estimation using the estimation model from the new image is close to the scene information used in generating the learning data, and sets the learning data close to the scene information used in generating the learning data as the input data. The information processing device according to claim 6 .

15. The learning data confirmation unit confirms whether or not an estimation result obtained by performing estimation using the estimation model from the new image satisfies the selection condition, and if the selection condition is satisfied, confirms whether or not the estimation result obtained by performing estimation using the estimation model from the new image is close to the scene information used in generating the learning data, and sets the learning data close to the scene information used in generating the learning data as the input data. The information processing device according to claim 7 .

16. The learning data confirmation unit confirms whether or not an estimation result obtained by performing estimation using the estimation model from the new image satisfies the selection condition, and if the selection condition is satisfied, confirms whether or not the estimation result obtained by performing estimation using the estimation model from the new image is close to the scene information used in generating the learning data, and sets the learning data close to the scene information used in generating the learning data as the input data. The information processing device according to claim 8 .

17. The learning data generation unit generates the learning data by arranging an object that does not pose a privacy problem in a background image according to the scene information. The information processing device according to claim 5 ,

18. A display unit that displays an example for capturing a new image according to the scene information, The learning data generating unit generates the learning data from an image of a subject that does not have a privacy problem according to the example. The information processing device according to claim 5 ,

19. the learning data generation unit generates regenerated learning data indicating an image different from the new image according to the scene information when it is determined that the learning data is not the desired data; The learning data confirmation unit confirms whether the regenerated learning data is the desired data, and sets the regenerated learning data that is the desired data as the input data. The information processing device according to claim 5 ,

20. when it is determined that the learning data generation unit does not become the desired data, the learning data generation unit generates regenerated learning data indicating an image different from the new image according to the scene information, The learning data confirmation unit confirms whether the regenerated learning data is the desired data, and sets the regenerated learning data that is the desired data as the input data. The information processing device according to claim 17 .

21. when it is determined that the learning data generation unit does not become the desired data, the learning data generation unit generates regenerated learning data indicating an image different from the new image according to the scene information, The learning data confirmation unit confirms whether the regenerated learning data is the desired data, and sets the regenerated learning data that is the desired data as the input data. The information processing device according to claim 18 .

22. Further comprising a learning unit that learns the learning model using the teacher data generated by attaching a correct answer label to the input data. The information processing device according to claim 5 ,

23. Further comprising a learning unit that learns the learning model using the teacher data generated by attaching a correct answer label to the input data. The information processing device according to claim 17 .

24. Further comprising a learning unit that learns the learning model using the teacher data generated by attaching a correct answer label to the input data. The information processing device according to claim 18 .

25. Further comprising a learning unit that learns the learning model using the teacher data generated by attaching a correct answer label to the input data.

20. The information processing device according to claim 19,

26. The method further comprises a learning unit that learns the learning model using the teacher data generated by attaching a correct answer label to the input data.

21. The information processing apparatus according to claim 20,

27. ​​Further comprising a learning unit that learns the learning model using the teacher data generated by attaching a correct answer label to the input data.

22. The information processing device according to claim 21,

28. Computer, an estimation unit that detects an object from an image represented by image data using an estimation model and estimates object-related information that is information related to the object; and a scene information selection unit that selects, from the object-related information, object-related information that satisfies a selection condition as at least a part of the scene information. A program characterized by.

29. an estimation unit that detects an object from an image represented by image data using an estimation model and estimates object-related information that is information related to the object; a scene information selection unit that selects, from the object-related information, object-related information that satisfies a selection condition as at least a part of the scene information. An information processing system comprising:

30. Detecting an object from an image represented by the image data using an estimation model and estimating object-related information that is information related to the object; selecting, from the object-related information, object-related information that satisfies a selection condition as at least a part of the scene information; An information processing method comprising: