Information processing device, information processing method, and recording medium

Abstract

An information processing device according to the present disclosure comprises: an acquisition unit that acquires a processing image; a generation unit that generates a work image including a part of the processing image; a work image processing unit that executes at least one of first processing and second processing on the work image; and an output unit that outputs an inference result of at least one of the first processing and the second processing. The first processing is processing for inferring suitability of an image with respect to at least one analyzer. The second processing is processing for inferring an influence, of at least one image editing on the image, on suitability of the at least one analyzer.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to an information processing device, an information processing method, a program, and a recording medium.

[0002] Various image analyses are being performed, such as face recognition and object detection. However, when analyzing processed images, it is difficult for people without specialized knowledge to determine what kind of processing is appropriate for the processed image. The processing of processed images that needs to be determined varies widely, including what kind of analysis to apply and what kind of image editing to perform.

[0003] A related technique is disclosed in Patent Literature 1. Patent Literature 1 describes evaluating pass / fail judgment results of an inspected object using a plurality of trained machine learning models and selecting an optimal machine learning model. This optimal machine learning model is selected through a trial judgment before being applied to actual inspections before shipment.

[0004] Japanese Patent Application Laid-Open No. 2022-43134

[0005] The technology described in Patent Literature 1 selects an appropriate machine learning model for each image. While the technology described in Patent Literature 1 can determine the machine learning model appropriate for each image, it cannot determine which part of each image is appropriate for that machine learning model.

[0006] One example of the objective of the present disclosure is to provide, in consideration of the above-mentioned problems, an information processing device, an information processing method, and a program that enable a determination of which part of a processed image is suitable for processing in a technology for estimating appropriate processing for the processed image.

[0007] According to the present disclosure, there is provided an information processing device having: an acquisition means for acquiring a processed image; a generation means for generating a processed image including a portion of the processed image; a processed image processing means for executing at least one of a first process for inputting an image into a first estimation model trained using first training data and estimating the suitability of the image for at least one analyzer, and a second process for inputting the image into a second estimation model trained using second training data and estimating the effect of at least one image edit on the image on the suitability for at least one of the analyzers, on the processed image; and an output means for outputting an estimation result of at least one of the first process and the second process.

[0008] The present disclosure also provides an information processing method in which one or more computers acquire a processed image, generate an edited image including a portion of the processed image, and perform at least one of a first process, in which the image is input to a first estimation model trained using first training data, and the suitability of the image for at least one analyzer, and a second process, in which the image is input to a second estimation model trained using second training data, and the suitability of the image for at least one analyzer is estimated, and output an estimation result of at least one of the first process and the second process.

[0009] Furthermore, according to the present disclosure, there is provided a program that causes a computer to function as: an acquisition means that acquires a processed image; a generation means that generates a processed image including a portion of the processed image; a processed image processing means that executes at least one of a first process that inputs an image into a first estimation model trained using first training data and estimates the suitability of the image for at least one analyzer, and a second process that inputs the image into a second estimation model trained using second training data and estimates the effect that at least one image edit on the image has on the suitability for at least one analyzer; and an output means that outputs an estimation result of at least one of the first process and the second process.

[0010] According to one aspect of the present disclosure, an information processing device, an information processing method, and a program are realized that, in a technology for estimating appropriate processing for a processed image, enable determination of which part of a processed image is suitable for that processing.

[0011] FIG. 1 is a diagram showing an example of a functional block diagram of an information processing device according to the present disclosure. FIG. 2 is a flowchart showing an example of a processing flow of an information processing device according to the present disclosure. FIG. 3 is a diagram showing an example of a hardware configuration of an information processing device according to the present disclosure. FIG. 4 is a diagram showing an example of a processed image according to the present disclosure. FIG. 5 is a diagram showing an example of a processed image according to the present disclosure. FIG. 6 is a diagram showing an example of a method for generating a processed image according to the present disclosure. FIG. 7 is a diagram showing an example of information output by an information processing device according to the present disclosure. FIG. 8 is a diagram showing another example of information output by an information processing device according to the present disclosure. FIG. 9 is a diagram showing an example of a functional block diagram of an information processing device according to the present disclosure. FIG. 10 is a flowchart showing another example of a processing flow of an information processing device according to the present disclosure.

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In this disclosure, the drawings relate to one or more embodiments. In all drawings, similar components are designated by similar reference numerals, and descriptions thereof will be omitted as appropriate.

[0013] <<First embodiment>> Fig. 1 is a functional block diagram showing an overview of an information processing device 10. Fig. 2 is a flowchart showing an example of the flow of processing executed by the processing device 10.

[0014] 1, the processing device 10 includes an acquisition unit 11, a generation unit 12, a processed image processing unit 13, and an output unit 14. These functional units execute the processing of the flowchart in FIG.

[0015] In S10, the acquisition unit 11 acquires a processed image.

[0016] In S11, the generation unit 12 generates a processed image that includes a part of the processed image.

[0017] In S12, the processed image processing unit 13 performs at least one of a first process and a second process on the processed image. The first process is a process of inputting the image into a first estimation model trained using first training data and estimating the suitability of the image for at least one analyzer. The second process is a process of inputting the image into a second estimation model trained using second training data and estimating the effect of at least one image edit on the image on the suitability of at least one analyzer.

[0018] In S13, the output unit 14 outputs the estimation result of at least one of the first process and the second process.

[0019] In this way, the information processing device 10 estimates an appropriate process for the processed image by executing at least one of the first process and the second process. The estimation result indicates at least one of which analyzer is appropriate for analyzing the processed image and which image editing is appropriate for the processed image.

[0020] The information processing device 10 then performs at least one of the first process and the second process on the "processed image including a part of the processed image" rather than on the "processed image." As will be described in the following embodiment, the information processing device 10 can also perform at least one of the first process and the second process on both the "processed image" and the "processed image including a part of the processed image."

[0021] When at least one of the first process and the second process is performed on the processed image, an estimated result is obtained for the entire processed image. In this case, it is not possible to determine which part of the processed image the estimated result applies to. In this case, it is possible to determine "appropriate processing for the processed image" based on the estimated result, but it is not possible to determine which part of the processed image is suitable for that processing.

[0022] In contrast, when at least one of the first process and the second process is performed on a processed image that includes a portion of the processed image, an estimated result for that portion of the processed image is obtained. In this case, it is clear that the obtained estimated result is the result for that portion of the processed image. In this case, it is possible to determine "appropriate processing for the processed image" based on the estimated result, and to determine which portion of the processed image is suitable for that processing.

[0023] According to the information processing device 10, in a technique for estimating appropriate processing for a processing image, it becomes possible to determine which part of the processing image is suitable for that processing.

[0024] <<Second Embodiment>> <Overview> An information processing apparatus 10 according to a second embodiment is a specific implementation of the configuration of the information processing apparatus 10 according to the first embodiment. A detailed description will be given below.

[0025] <Hardware Configuration> Next, an example of the hardware configuration of the information processing device 10 will be described. Each functional unit of the information processing device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the realization method and device. The software includes programs that are pre-stored in the device before shipping, and programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet.

[0026] FIG. 3 is a block diagram illustrating an example of the hardware configuration of an information processing device 10. As shown in FIG. 3, the information processing device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The information processing device 10 does not necessarily have to have the peripheral circuit 4A. Note that the information processing device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices may have the above hardware configuration.

[0027] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to mutually transmit and receive data. The processor 1A is, for example, a central processing unit (CPU) or a graphics processing unit (GPU). The memory 2A is, for example, a random access memory (RAM) or a read-only memory (ROM). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes an interface for connecting to a communication network such as the Internet. Examples of input devices include a keyboard, mouse, microphone, physical buttons, and touch panel. Examples of output devices include a display, speaker, printer, and mailer. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0028] <Functional Configuration> Next, a detailed description will be given of the functional configuration of the information processing device 10. Fig. 1 shows an example of a functional block diagram of the information processing device 10. As shown in the figure, the information processing device 10 has an acquisition unit 11, a generation unit 12, a processed image processing unit 13, and an output unit 14.

[0029] The acquisition unit 11 acquires the processed image.

[0030] The "processed image" is an image that is the target of analysis processing by the analyzer. The processed image may be a still image. The processed image may also be a frame image that constitutes a moving image.

[0031] The "analyzer" performs image analysis. The analyzer is not particularly limited, but for example, the analysis performed by the analyzer includes at least one of detection and recognition of an analysis target included in the processed image. The analysis target is, for example, at least one of a person, a part of a person, an animal, a part of an animal, a plant, a part of a plant, an object, a part of an object, a color, text, and weather. The analysis performed by the analyzer aims, for example, at least one of detection, identification, segmentation, authentication, and state recognition of the analysis target.

[0032] Specific examples of analyses performed by the analyzer include, but are not limited to, the following: Object detection Person detection Pose estimation Product recognition Speed ​​estimation Crowd behavior analysis Social distance recognition Abandoned object detection Appearance attribute recognition Object detection Object color feature recognition Object gradient feature recognition Human form recognition Face recognition Face recognition Iris recognition Facial expression recognition Handwritten character recognition Printed character recognition Vehicle recognition License plate recognition Image segmentation Instance segmentation Animal recognition Fruit recognition Flower recognition Fish recognition Insect recognition Animal skeleton recognition Bird recognition Food recognition Weather recognition

[0033] Humanoid authentication is authentication based on the physical characteristics of the person to be authenticated (for example, height, body width, limb length, facial contour, or a combination thereof).

[0034] The analyzer may include, for example, a trained neural network. The input data input to the analyzer includes an image. This image may be a single still image, or one of multiple images constituting a video, i.e., a frame image. The analyzer also outputs at least an analysis result. The form of the analysis result is not particularly limited, but may be, for example, an image with annotations, a map, a numerical value, a vector, or the like. The analyzer may further output the reliability (likelihood) of the analysis result. Furthermore, if the analysis is not successful, for example, if the analysis target is not detected, the analyzer may output information indicating that the analysis is not possible.

[0035] In cases where the analyzer performs an analysis that recognizes movement or change, the input data to the analyzer may include a moving image. In this case, the analyzer may output an analysis result for the input moving image rather than for each individual image. The following mainly describes an example in which a single image is input to the analyzer, but each embodiment is not limited to this example.

[0036] "Acquisition" includes at least one of the following: a device going to retrieve data or information stored in another device or storage medium (active acquisition), and a device inputting data or information output from another device (passive acquisition). Examples of active acquisition include making a request to another device and receiving a response, and accessing and reading information from another device or storage medium. An example of passive acquisition is receiving information that is distributed (or transmitted, push notification, etc.). Furthermore, "acquisition" may also mean selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.

[0037] An example of a processed image is shown in Figure 4. As shown in Figure 4, the processed image may include multiple analysis objects. Alternatively, although not shown, the processed image may include only one analysis object. The analysis object shown in Figure 4 is a person.

[0038] Returning to FIG. 1, the generation unit 12 generates a processed image.

[0039] The "processed image" is an image that includes a portion of the processed image (hereinafter, sometimes referred to as the "target portion").

[0040] It is preferable to ensure the identity of the target portion included in the processed image and the target portion included in the processed image. That is, it is preferable to generate a processed image that includes the target portion included in the processed image as is, without performing image editing such as enlarging, reducing, rotating, or color correction on the target portion included in the processed image. If this identity can be achieved, it becomes possible to determine the suitability of the target portion included in the processed image for various analyzers based on the suitability of the target portion included in the processed image for various analyzers.

[0041] The size of the processed image is a size that can be applied to the first estimation model and the second estimation model described below. As will be described in the following embodiment, there are cases where both the processed image and the processed image are applied to the first estimation model and the second estimation model. In this case, it is preferable that the processed image and the processed image have the same size.

[0042] The generation unit 12 can generate a processed image by, for example, applying a predetermined process to the processed image. The predetermined process is a process that makes it impossible to recognize the analysis target that exists in other parts of the processed image (parts other than the target part). For example, as shown in FIG. 5, the process may be a process of filling in other parts of the processed image with a single color. Note that the process may also be other processes such as mosaic processing, blurring, or superimposing a mark of a predetermined shape. Furthermore, the process may be a process of filling in other parts of the processed image with an image of the target part of the processed image, as shown in FIG. 6.

[0043] Note that the generation unit 12 may generate a new image (processed image) using the processed image, rather than processing the processed image to generate the processed image. That is, the generation unit 12 may cut out a target portion from the processed image and use the image of the cut-out target portion to generate a new processed image as described above, such as that shown in FIG. 5 or FIG. 6. In this case, the position of the target portion in the processed image may be the same as or different from the position of the target portion in the processed image. As described above, it is preferable to ensure that the target portion included in the processed image is identical to the target portion included in the processed image, but differences in position within the image do not significantly affect compatibility with various analyzers.

[0044] The processed image thus generated cannot recognize the analysis target that exists in other parts of the processed image, i.e., parts other than the target part. The recognition can be by computer or by human. Furthermore, the processed image thus generated does not include other parts of the processed image, i.e., parts other than the target part.

[0045] The generation unit 12 may randomly select a portion from the processed image and set the selected portion as the target portion. Alternatively, the generation unit 12 may determine the target portion from the target image using a method described in the following embodiment.

[0046] Returning to FIG. 1, the processed image processing unit 13 executes at least one of a first process and a second process on the processed image.

[0047] The "first process" is a process of inputting an image into a first estimation model that has been trained using first training data and estimating the suitability of the image for at least one analyzer. For example, the processed image processing unit 13 can estimate the suitability of the image for one analyzer as the first process. Alternatively, the processed image processing unit 13 can estimate the suitability of the image for each of multiple analyzers as the first process.

[0048] The "first estimation model" includes a neural network. The first estimation model is a trained model based on machine learning. The input data input to the first estimation model includes at least an image. This image may be a single still image, or may be one of multiple images constituting a video, i.e., a frame image. In cases where the input data to the analyzer includes a video, the input data to the first estimation model may also include the video. Furthermore, the output data from the first estimation model includes compatibility information. The configuration of the first estimation model is not particularly limited, but the first estimation model may include, for example, a feature extractor and a classifier.

[0049] The "suitability information" is information indicating the suitability of an image input to the first estimation model (hereinafter referred to as the "input image") for an analyzer. The first estimation model may be configured to output suitability information indicating suitability for one analyzer. Alternatively, the first estimation model may be configured to output suitability information indicating suitability for each of multiple analyzers.

[0050] The suitability information is, for example, information indicating whether the input image can be analyzed by the analyzer. For example, the suitability information can be at least one of a value indicating the likelihood that the analyzer will successfully analyze the input image and a value indicating the likelihood that the analyzer will not successfully analyze the input image. Success in analyzing the input image by the analyzer means, for example, that the objective of the analysis by the analyzer is achieved in the input image. In other words, successful analysis of the input image by the analyzer means, for example, successful detection of an object in the input image.

[0051] Alternatively, the suitability information may be information indicating the suitability of the input image for the analyzer. The suitability indicates, for example, the reliability of the results obtained by analyzing the input image with the analyzer.

[0052] Alternatively, the suitability information may be information indicating whether the input image can be analyzed by the analyzer. For example, the analyzer may determine that the input image is analyzable if the value or suitability indicating the likelihood of the analyzer successfully analyzing the input image is equal to or greater than a reference value, and may determine that the input image is unanalyzable if these values ​​are less than the reference value. Alternatively, the analyzer may determine that the input image is analyzable if the value indicating the likelihood of the analyzer not successfully analyzing the input image is equal to or less than a reference value, and may determine that the input image is unanalyzable if the value is greater than the reference value.

[0053] A method for generating such a first estimation model will be described in the following embodiment.

[0054] The "second process" is a process of inputting an image into a second estimation model trained using second training data and estimating the effect of at least one image edit on the image on the suitability of at least one analyzer. For example, as the second process, the processed image processing unit 13 can estimate the effect of one image edit on the suitability of one analyzer. Alternatively, as the second process, the processed image processing unit 13 can estimate the effect of one image edit on the suitability of each of multiple analyzers. Alternatively, as the second process, the processed image processing unit 13 can estimate the effect of each of multiple image edits on the suitability of one analyzer. Alternatively, as the second process, the processed image processing unit 13 can estimate the effect of each of multiple image edits on the suitability of each of multiple analyzers.

[0055] The "second estimation model" includes a neural network. The second estimation model is a trained model based on machine learning. The input data input to the second estimation model includes one or more images. The input data input to the second estimation model includes a pre-edited image and a post-edited image. The pre-edited image is an image that has not been subjected to a predetermined image editing process, and the post-edited image is an image that has been subjected to image editing on the pre-edited image. In addition, the output data from the second estimation model includes influence information. The configuration of the second estimation model is not particularly limited, but the second estimation model may include, for example, a feature extractor and a classifier.

[0056] In cases where the input data to the analyzer includes a moving image, the input data to the second estimation model may include a moving image. That is, the second estimation model may be input with an unprocessed moving image (a collection of unedited images) and a processed moving image (a collection of edited images).

[0057] "Impact information" is information indicating how image editing performed on a pre-edited image affects the suitability of an analyzer. The impact information indicates the impact of at least one image edit on the suitability of at least one analyzer. For example, the impact information may indicate the impact of one image edit on the suitability of one analyzer. Alternatively, the impact information may indicate the impact of one image edit on the suitability of each of multiple analyzers. Alternatively, the impact information may indicate the impact of each of multiple image edits on the suitability of one analyzer. Alternatively, the impact information may indicate the impact of each of multiple image edits on the suitability of each of multiple analyzers. The impact information can also be information obtained by comparing the results of analyzing a pre-edited image by an analyzer with the results of analyzing a post-edited image by an analyzer. The impact information may be information indicating, for example, how the likelihood of successful analysis by an analyzer differs between a pre-edited image and a post-edited image. The impact information may also be information indicating, for example, how the suitability of a pre-edited image and a post-edited image for an analyzer differs.

[0058] For example, the impact information may be at least one of a value indicating the likelihood that image editing is effective and a value indicating the likelihood that image editing is ineffective. Image editing being "effective" means that performing image editing improves the suitability of the image for the analyzer. As another example, the impact information may be the probability that the input image belongs to each of the classes of "effective," "unchanged," and "adverse effect." An "adverse effect" means that performing image editing reduces the suitability of the image for the analyzer. Furthermore, the impact information may be a value indicating the degree of effect.

[0059] Alternatively, the impact information may be information indicating whether the image editing is effective or not. For example, the image editing may be determined to be effective if the value indicating the likelihood that the image editing is effective is equal to or greater than a reference value, and may be determined to be ineffective if the value is less than the reference value. Alternatively, the image editing may be determined to be effective if the value indicating the likelihood that the image editing is ineffective is equal to or less than a reference value, and may be determined to be ineffective if the value is greater than the reference value.

[0060] Alternatively, the impact information may be information indicating whether the image editing is “effective,” “unchanged,” or “opposite effect.” For example, the impact information may indicate the class with the highest probability of the input image belonging to each of the classes “effective,” “unchanged,” and “opposite effect.”

[0061] The content of image editing is not particularly limited. Examples of image editing include enlargement, reduction, rotation, brightening, darkening, and removal of obstructions. Enlargement may simply enlarge an image to change its size (enlargement), or it may enlarge an image without changing its size. In the latter case, an image of the same size as the original image is generated by cropping a portion of the enlarged image. Similarly, reduction may simply reduce an image to change its size (reduction), or it may reduce an image without changing its size. In the latter case, when an image is first reduced, a margin that is insufficient compared to the original image size is generated. By filling this margin in some way, an image of the same size as the original image is generated. Methods for filling the margin include filling the margin with white, black, or another solid color, and arranging the reduced image in the margin. However, the method for filling the margin is not limited to these examples. Although the following description mainly assumes enlargement and reduction without changing the image size, the enlargement and reduction may also be processes that change the image size.

[0062] A method for generating such a second estimation model will be described in the following embodiment.

[0063] The output unit 14 outputs an estimation result of at least one of the first process and the second process. For example, the output unit 14 can output an estimation result of at least one of the first process and the second process executed on the processed image. The estimation result of the first process is the compatibility information described above. The estimation result of the second process is the impact information described above.

[0064] The output unit 14 can output the estimation result via an output device such as a display or a projection device. Furthermore, when the information processing device 10 is a server, the output unit 14 can transmit the estimation result to a client terminal and cause the client terminal to output the estimation result. Examples of the client terminal include, but are not limited to, a personal computer, a smartphone, a tablet terminal, etc.

[0065] The output unit 14 can create and output a UI (User Interface) screen that displays the estimation results as described above, for example. Examples of the UI screen will be described in the following embodiments.

[0066] Next, an example of the flow of processing by the information processing device 10 will be described with reference to the flowchart of Fig. 2. Details of each process have been described above, so a description thereof will be omitted here.

[0067] In S10, the information processing device 10 acquires a processed image.

[0068] In S11, the information processing device 10 generates a processed image that includes a part of the processed image.

[0069] In S12, the information processing device 10 performs at least one of a first process and a second process on the edited image. The first process is a process of inputting the image into a first estimation model trained using first training data and estimating the suitability of the image for at least one analyzer. The second process is a process of inputting the image into a second estimation model trained using second training data and estimating the effect of at least one image edit on the image on the suitability of at least one analyzer.

[0070] In S13, the information processing device 10 outputs an estimation result of at least one of the first process and the second process.

[0071] <Effects> According to the information processing device 10 of this embodiment, the same effects as those of the information processing device 10 of the first embodiment are achieved.

[0072] Furthermore, the information processing device 10 of this embodiment can generate an image by applying a predetermined process to a part of the processed image as a processed image that includes the part of the processed image. With this information processing device 10, it is possible to easily create a processed image that includes the part of the processed image but does not include / is not recognizable as the other part of the processed image.

[0073] <<Third Embodiment>> In this embodiment, the information processing device 10 generates, from a single processed image, multiple processed images each including a different portion of the processed image. The information processing device 10 then performs at least one of the first process and the second process described above on each of the multiple processed images. This will be described in detail below.

[0074] The generating unit 12 generates, from a single processed image, a plurality of processed images each including a different portion (target portion) of the processed image. The target portions included in each of the plurality of processed images may or may not overlap with each other.

[0075] The generating unit 12 may randomly select multiple portions from the processing image and generate multiple processed images that include each of the selected portions as a target portion. Alternatively, the generating unit 12 may select multiple portions from the processing image according to a predetermined rule and generate multiple processed images that include each of the selected portions as a target portion. Alternatively, the generating unit 12 may determine multiple target portions from the target image using a method described in the following embodiment.

[0076] The processed image processing unit 13 performs at least one of the first process and the second process on each of a plurality of processed images generated from one processed image.

[0077] The output unit 14 outputs an estimated result of at least one of the first processing and the second processing executed on each of a plurality of processed images generated from one processed image.

[0078] Next, an example of the flow of processing by the information processing device 10 will be described with reference to the flowchart of Fig. 7. Details of each process have been described above, so a description thereof will be omitted here.

[0079] In S20, the information processing device 10 acquires a processed image.

[0080] In S21, the information processing device 10 generates, from one processed image acquired in S20, a plurality of processed images each including a different portion (target portion) of the processed image.

[0081] In S22, the information processing device 10 performs at least one of a first process and a second process on each of a plurality of processed images generated from one processed image acquired in S20. The first process is a process of inputting an image into a first estimation model trained using first training data and estimating the suitability of the image for at least one analyzer. The second process is a process of inputting an image into a second estimation model trained using second training data and estimating the effect of at least one image edit on the image on the suitability of at least one analyzer.

[0082] In S23, the information processing device 10 outputs an estimation result of at least one of the first processing and the second processing executed on each of the multiple processed images.

[0083] Other configurations of the information processing device 10 of this embodiment are similar to those of the information processing device 10 of the first and second embodiments.

[0084] According to the information processing device 10 of this embodiment, the same effects as those of the information processing device 10 of the first and second embodiments are achieved.

[0085] Furthermore, the information processing device 10 of this embodiment generates multiple processed images from a single processed image, each containing a different portion of the processed image, and performs at least one of the first process and the second process on each of the multiple processed images. This information processing device 10 can determine "appropriate processing for the processed image" based on the estimation results, and can determine which of multiple portions of the processed image are suitable for that processing. This information processing device 10, in a technology for estimating appropriate processing for a processed image, can determine which portion of the processed image is suitable for that processing.

[0086] <<Fourth Embodiment>> The information processing apparatus 10 of this embodiment determines a part (target part) of a processing image to be included in a processed image using a distinctive method, which will be described in detail below.

[0087] The generating unit 12 can execute at least one of the following first to third determination methods.

[0088] First Determination Method The generating unit 12 divides the processing image into a plurality of regions according to a predetermined division rule, and generates a plurality of processed images including each of the plurality of regions.

[0089] The division rule may have various contents. For example, the division rule may be a rule that divides the processed image equally into a plurality of regions. In this case, the sizes of the plurality of regions will be equal. Alternatively, the division rule may be a rule that divides the processed image unequally into a plurality of regions. In this case, the sizes of the plurality of regions will vary.

[0090] The multiple regions may or may not overlap each other. Preferably, there are no gaps between the multiple regions in the processed image. The number of multiple regions is a design factor and is not particularly limited. The shape of each region is also a design factor and is not particularly limited. The shape of each region may be, for example, a square, a rectangle with a predetermined aspect ratio, a triangle, a circle, etc., but is not limited to these.

[0091] For example, the division rule may be a rule that divides the processing image into M × N regions by dividing the processing image vertically equally into M regions and horizontally equally into N regions. Alternatively, the division rule may be a rule that divides the processing image into multiple regions by arranging regions of a predetermined shape and a predetermined size on the processing image at predetermined intervals without any gaps. Note that examples of division rules are not limited to those exemplified here.

[0092] "Second Determination Method" The generating unit 12 generates a processed image that includes a part of the processed image that is specified by a user input.

[0093] The generating unit 12 may receive a user input specifying one portion of the processed image. Then, the generating unit 12 may generate one processed image including that portion. Alternatively, the generating unit 12 may receive a user input specifying multiple different portions of the processed image. Then, the generating unit 12 may generate multiple processed images including each of the multiple portions.

[0094] There are various methods for accepting the user input. For example, the generation unit 12 outputs a UI screen including a processed image. Then, the generation unit 12 accepts a user input specifying a portion of the processed image on the processed image displayed on the UI screen. For example, the generation unit 12 displays a frame on the UI screen. The user performs an input to change the position, size, shape, etc. of this frame. Then, the generation unit 12 generates a processed image including a portion enclosed by the frame. Note that this method is merely an example and is not limited to this.

[0095] "Third Determination Method" The generation unit 12 selects a part of the processed image based on attention information obtained by inputting the processed image to the attention mechanism, and generates a processed image including the selected part.

[0096] The generating unit 12 may select one portion of the processed image based on the attention information. Then, the generating unit 12 may generate one processed image that includes that portion. Alternatively, the generating unit 12 may select multiple different portions of the processed image based on the attention information. Then, the generating unit 12 may generate multiple processed images that each include one of the multiple portions.

[0097] Attention information indicates which parts of the processed image input to the attention mechanism should be focused on. There are no particular limitations on the configuration of the attention mechanism, and any widely known technology can be used. For example, the attention mechanism applies the input processed image to a convolutional neural network (CNN) to generate a feature map. The attention mechanism then applies a convolution layer (Conv layer) to this feature map and passes it through an activation function to create an attention mask (attention information) with one channel but with the same width and height. Sigmoid, ReLU, Spatial Softmax, and other activation functions can be used. Note that the configuration of the attention mechanism illustrated here is merely an example and is not limited to this.

[0098] An example of attention information is shown in Fig. 8A. The attention information shown indicates positions that require attention by differences in brightness. The more attention a position requires, the higher the brightness. In the case of Fig. 8A, the upper left area of ​​the processed image is indicated as the position that requires attention.

[0099] The generation unit 12 selects a portion of the processed image so as to include a position to be noted (for example, a position where the value is equal to or greater than a threshold) indicated by the attention information.

[0100] Other configurations of the information processing apparatus 10 of this embodiment are similar to those of the information processing apparatus 10 of the first to third embodiments.

[0101] According to the information processing device 10 of this embodiment, the same effects as those of the information processing devices 10 of the first to third embodiments are realized.

[0102] Furthermore, the information processing device 10 of the present embodiment can determine a portion (target portion) of the processed image to be included in the processed image using a characteristic method. Specifically, the information processing device 10 can determine a portion (target portion) of the processed image to be included in the processed image using at least one of the first to third determination methods described above.

[0103] In the first determination method, the information processing device 10 divides the processing image into a plurality of regions and generates a plurality of processed images each including each of the plurality of regions. By performing at least one of the first process and the second process on each of the plurality of processed images thus generated, it is possible to determine "appropriate processing for the processing image" and to determine which of the plurality of parts of the processing image is suitable for that processing.

[0104] In the second determination method, the information processing device 10 generates a processed image that includes a portion of the processed image specified by user input. By performing at least one of the first process and the second process on the processed image generated in this manner, it is possible to determine "appropriate processing for the processed image" and to determine whether the portion of the processed image specified by the user is suitable for that processing.

[0105] In addition, in the third determination method, the information processing device 10 generates a processed image that includes a portion of the processed image selected based on the Attention information. By performing at least one of the first process and the second process on the processed image generated in this manner, it is possible to determine "appropriate processing for the processed image" and to determine whether a notable portion of the processed image indicated by the Attention information is suitable for that processing.

[0106] <<Fifth Embodiment>> The information processing apparatus 10 of this embodiment outputs the estimation result of at least one of the first processing and the second processing executed on the processed image using a distinctive method, which will be described in detail below.

[0107] The output unit 14 outputs a UI screen showing an estimated result of at least one of the first processing and the second processing executed on the processed image.

[0108] The output unit 14 can execute at least one of the following output processes: A first output process in which at least one of the first process and the second process is executed in advance on at least one processed image to obtain an estimation result, and then a UI screen is displayed using the estimation result; A second output process in which, in response to a user input specifying at least one target portion on the UI screen, at least one of the first process and the second process is executed in real time on a processed image including a target portion specified by the user input, and a UI screen including the estimation result is displayed in real time.

[0109] "First Output Process" FIG. 9 shows an example of a UI screen.

[0110] The processed image is displayed in the area (A), and a frame W indicating at least one target portion is displayed on the processed image.

[0111] Here, an example of a method for displaying the frame W will be described. In one example, when the generation unit 12 generates at least one processed image, it passes information indicating the target portion included in each processed image to the output unit 14. This information indicates a partial area within the processed image and is expressed, for example, by coordinates. Based on this information, the output unit 14 displays the frame W as shown in area (A).

[0112] In addition, as described in the first determination method of the fourth embodiment, when the generation unit 12 determines multiple target parts according to a predetermined division rule, the output unit 14 may divide the processed image into multiple regions based on the division rule and display a frame W surrounding each region.

[0113] The user can perform an operation to designate one of at least one frame W. This operation can be realized using any technology. This operation may be, for example, an operation of aligning the position of the illustrated pointer P with the position of one frame W and performing a predetermined input. Alternatively, if the UI screen is displayed on a touch panel display, this operation may be a predetermined touch operation on the position of one frame. Note that the examples given here are merely examples, and the present invention is not limited to these examples.

[0114] In area (A), a UI button is displayed that accepts an operation to switch the processed image to be displayed. For example, one frame image from a moving image is displayed in area (A) as the processed image. Then, by operating the UI button, the processed image displayed in area (A) can be switched to a processed image that is earlier or later in chronological order than the processed image currently being displayed.

[0115] In the area (B), the estimated result of the first processing executed on the processed image including the target portion surrounded by the frame W specified by the user is displayed.

[0116] The "analysis method" column indicates the content analyzed by at least one analyzer.

[0117] The "Recommendation Level" column shows the suitability described in the second embodiment. That is, the suitability of the target portion enclosed by the frame W specified by the user for each of the multiple analyzers is shown. In addition to or instead of the suitability, a value indicating the likelihood of successful analysis or a value indicating the likelihood of unsuccessful analysis may be shown.

[0118] The "Decision" column shows the result of the determination as to whether or not each of the analyzers can analyze the target portion enclosed by the frame W specified by the user.

[0119] The output unit 14 can realize the display of the area (B) based on the compatibility information described in the second embodiment.

[0120] In the area (C), the estimated result of the second processing executed on the processed image including the target portion surrounded by the frame W specified by the user is displayed.

[0121] The "editing method" column indicates at least one image editing method.

[0122] The "Recommendation Level" column shows the effect that various image edits on the target portion enclosed by the frame W specified by the user have on the suitability of a specified analyzer. Specifically, at least one of a value indicating the likelihood that the image edits will be effective and a value indicating the likelihood that the image edits will be ineffective is shown. Note that the figure shows the effect that various image edits on the target portion enclosed by the frame W specified by the user have on the suitability of the "analyzer that performs facial recognition." The user can switch the type of analyzer for which the recommendation level is displayed by operating the UI button displayed in area (C).

[0123] The "Judgment" column shows the result of the judgment as to whether the image editing is effective. For example, if the impact information described in the second embodiment indicates "effective," the result is "OK," and if it indicates "no change" or "opposite effect," the result is "NG."

[0124] The output unit 14 can realize the display of the area (C) based on the influence information described in the second embodiment.

[0125] 9, multiple processed images are generated that include different portions (target portions) of the processed image, and frames W corresponding to each of the multiple target portions are displayed in area (A). Areas (B) and (C) display the estimated results of the first and second processes performed on the processed image that includes the target portion surrounded by the frame W specified by the user.

[0126] However, there are cases where a single processed image is generated that includes a portion (target portion) of a single processed image. In this case, a single frame W is displayed in area (A). Then, without receiving a user specification, areas (B) and (C) display the estimated results of the first and second processes performed on the processed image that includes the target portion enclosed by that single frame.

[0127] FIG. 10 shows another example of the UI screen.

[0128] In area (A), a processed image is displayed. A frame W indicating at least one target portion is displayed on the processed image. Then, the result of the determination as to whether each target portion can be analyzed by a predetermined analyzer is displayed, linked to each target portion (linked to each frame W). The predetermined analyzer is the analyzer designated by a user operation in area (B).

[0129] The output unit 14 can display the frame W in the same manner as in the example of Fig. 9. The output unit 14 can also display the determination result based on the compatibility information described in the second embodiment.

[0130] The area (B) shows the content to be analyzed by at least one analyzer. The output unit 14 accepts a user input specifying one analyzer in the area (B). The user input can be realized by an operation via the pointer P or a touch operation, but is not limited to these examples.

[0131] "Second Output Process" FIG. 9 shows an example of a UI screen.

[0132] The area (A) displays the processed image. A frame W indicating at least one target portion is displayed on the processed image. The target portion is a part of the processed image that is included in the processed image.

[0133] Here, we will explain an example of a method for displaying the frame W. As an example, as described in the first determination method of the fourth embodiment, when the generation unit 12 determines multiple target portions according to a predetermined division rule, the output unit 14 may divide the processed image into multiple regions based on the division rule and display a frame W surrounding each region.

[0134] The user can then perform an operation to designate one of the displayed frames W. This operation can be achieved using the same method as the first output process. In response to this designation by the user, the following process is executed in real time. First, the generation unit 12 generates a processed image including a target portion surrounded by the designated frame W. Then, the processed image processing unit 13 performs at least one of the first process and the second process on the generated processed image. Then, the output unit 14 displays the estimated result of the first process on the generated processed image in area (B). Furthermore, the output unit 14 displays the estimated result of the second process on the generated processed image in area (C).

[0135] Alternatively, only the processed image may be initially displayed in area (A), and the frame W may not be displayed. Then, the output unit 14 may display a frame W of a shape and size determined by a user operation in area (A) at a position determined by the user operation. The output unit 14 may display one frame, or may display the number of frames W determined by the user operation. The following processing is performed in real time in response to the user operation. First, the generation unit 12 generates a processed image including a target portion surrounded by the frame W specified in this manner. Then, the processed image processing unit 13 performs at least one of the first processing and the second processing on the generated processed image. Then, the output unit 14 displays the estimated result of the first processing on the generated processed image in area (B). Furthermore, the output unit 14 displays the estimated result of the second processing on the generated processed image in area (C).

[0136] The display contents in the areas (B) and (C) are the same as those in the first output process.

[0137] FIG. 10 shows another example of the UI screen.

[0138] The area (A) displays the processed image. A frame W indicating at least one target portion is displayed on the processed image. The target portion is a part of the processed image that is included in the processed image.

[0139] Here, we will explain an example of a method for displaying the frame W. As an example, as described in the first determination method of the fourth embodiment, when the generation unit 12 determines multiple target portions according to a predetermined division rule, the output unit 14 may divide the processed image into multiple regions based on the division rule and display a frame W surrounding each region.

[0140] The user may be able to select the division rule. The user may also be able to customize the content of the division rule. In response to the user operation, the generation unit 12 displays a frame W on the processed image. In response to the user operation, the following processing is executed in real time. First, the generation unit 12 generates a processed image including a target portion surrounded by the frame W specified in this manner. Then, the processed image processing unit 13 executes at least one of a first processing and a second processing on the generated processed image. Then, the output unit 14 displays an estimated result of at least one of the first processing and the second processing on the generated processed image in the area (A), linked to each target portion (each frame W).

[0141] Alternatively, the output unit 14 may display a frame W having a shape and size determined by a user operation in the area (A) at a position determined by the user operation. The output unit 14 may display one frame, or may display the number of frames W determined by the user operation. In response to the user operation, the generation unit 12 displays the frame W on the processed image. Furthermore, in response to the user operation, the following processing is performed in real time. First, the generation unit 12 generates a processed image including a target portion surrounded by the frame W specified in this manner. Then, the processed image processing unit 13 performs at least one of a first processing and a second processing on the generated processed image. Then, the output unit 14 displays an estimated result of at least one of the first processing and the second processing on the generated processed image in the area (A) by linking it to each target portion (each frame W).

[0142] The display content in the area (B) is the same as in the first output process.

[0143] As described above, the output unit 14 can output a UI screen that displays a processed image.

[0144] 9 , when a user input is made on the UI screen specifying a portion of the processed image, the output unit 14 can display, on the UI screen, an estimated result of the processed image including the specified portion. For example, the output unit 14 can display, on the UI screen, a plurality of portions (target portions) of the processed image in a selectable manner, and accept, as a user input specifying a portion of the processed image, a user input specifying one of the plurality of selectably displayed portions. In addition, the output unit 14 can accept, as a user input specifying a portion of the processed image, an input specifying the position and size of a partial area on the processed image.

[0145] Furthermore, when a user input is made on the UI screen specifying one of at least one analyzer, as in the example of Figure 10, the output unit 14 can link each of the multiple portions of the processed image and display the estimated results of the processed image including each of the multiple portions of the processed image on the UI screen.

[0146] Other configurations of the information processing apparatus 10 of this embodiment are similar to those of the information processing apparatuses 10 of the first to fourth embodiments.

[0147] According to the information processing device 10 of this embodiment, the same effects as those of the information processing devices 10 of the first to fourth embodiments are realized.

[0148] Furthermore, the information processing device 10 of this embodiment can output the estimation results of at least one of the first processing and the second processing executed on the processed image using a characteristic method as shown in Fig. 9 and Fig. 10. As a result, the user can intuitively and easily grasp the estimation results by distinguishing them by item.

[0149] <<Sixth Embodiment>> An information processing apparatus 10 of this embodiment executes a first process on a processed image, and generates an edited image and executes the first process on the edited image based on the result of the process. This will be described in detail below.

[0150] 11 shows an example of functional blocks of the information processing device 10. As shown in the figure, the information processing device 10 includes an acquisition unit 11, a generation unit 12, a processed image processing unit 13, an output unit 14, and a processed image processing unit 15.

[0151] The processed image processing unit 15 executes a first process on the processed image. That is, the processed image processing unit 15 inputs the processed image to a first estimation model and obtains an output of the first estimation model. Details of the first process and the first estimation model are as described in the second embodiment. As a result of this process, the suitability of the entire processed image for at least one analyzer is estimated.

[0152] When the first process is performed on a processed image that includes a portion of the processed image (target portion), the suitability of the portion of the processed image (target portion) for at least one analyzer is estimated. Thus, the difference between when the first process is performed on a processed image and when the first process is performed on a processed image is whether the suitability of the entire processed image is estimated or the suitability of a portion of the processed image (target portion) is estimated.

[0153] If the first processing executed by the processed image processing unit 15 results in a result indicating that the processed image is suitable for at least one analyzer, the generation unit 12 generates a processed image. Then, the processed image processing unit 13 performs the first processing on the generated processed image. Then, the output unit 14 outputs the estimated result of the first processing performed on the generated processed image. The details of the generation unit 12, the processed image processing unit 13, and the output unit 14 are the same as those of the first to fifth embodiments.

[0154] "The processed image is suitable for at least one analyzer" means that the compatibility information described above satisfies any of the following: - For at least one analyzer, it is shown that a value indicating the likelihood of successful analysis of the input image is equal to or greater than a reference value. - For at least one analyzer, it is shown that a value indicating the likelihood of unsuccessful analysis of the input image is equal to or less than a reference value. - It is shown that the compatibility for at least one analyzer is equal to or greater than a reference value. - It is shown that analysis is possible for at least one analyzer.

[0155] Next, an example of the flow of processing by the information processing device 10 will be described with reference to the flowchart of Fig. 12. Note that details of each process have been described above, and therefore a description thereof will be omitted here.

[0156] In S30, the information processing device 10 acquires a processed image.

[0157] In S31, the information processing device 10 executes a first process on the processed image acquired in S30. The first process is a process of inputting the image into a first estimation model trained using first training data, and estimating the suitability of the image for at least one analyzer.

[0158] In S32, the information processing device 10 determines whether or not a result indicating that the processed image is suitable for at least one analyzer has been obtained in the first process executed in S31.

[0159] If the first process performed in S31 results in a result indicating that the processed image is suitable for at least one analyzer (Yes in S32), in S33, the information processing device 10 generates at least one processed image that includes a portion of the processed image obtained in S30.

[0160] Then, in S34, the information processing device 10 executes the first process on at least one processed image generated in S33.

[0161] In S35, the information processing device 10 outputs the estimation result of the first processing executed on each processed image.

[0162] On the other hand, if the first process executed in S31 does not produce a result indicating that the processed image is suitable for at least one analyzer (No in S32), the information processing device 10 does not execute the processes in S33 and S34. Then, in S35, the information processing device 10 outputs the result of the first process executed in S31 for the processed image.

[0163] Other configurations of the information processing apparatus 10 of this embodiment are similar to those of the information processing apparatuses 10 of the first to fifth embodiments.

[0164] According to the information processing device 10 of this embodiment, the same effects as those of the information processing devices 10 of the first to fifth embodiments are achieved.

[0165] Furthermore, the information processing device 10 of this embodiment performs a first process on the processed image, and generates a processed image and performs the first process on the processed image according to the result. Specifically, when the first process on the processed image results in a result indicating that the processed image is suitable for at least one analyzer, the information processing device 10 generates a processed image and performs the first process on the processed image.

[0166] If the first processing of the processed image results in a result indicating that the processed image is unsuitable for any analyzer, the result for the processed image that includes a part of the processed image will naturally be the same. In such a case, there is no point in generating at least one processed image from one processed image and performing the first processing on at least one processed image, and it would only increase the processing load on the computer.

[0167] On the other hand, if a first processing of a processed image produces results indicating that the processed image is suitable for at least one analyzer, it is natural that the results for a processed image containing any part of the processed image will also produce the same results. In such a case, it makes sense to generate at least one processed image from one processed image and search for a part that produces the same results as the results for the processed image.

[0168] As in this embodiment, by performing the first processing on the processed image, and then generating an edited image and performing the first processing on the edited image based on the results, the processing burden on the computer can be reduced while obtaining the desired information.

[0169] Seventh Embodiment An information processing apparatus 10 of this embodiment executes the second processing on a processed image, and generates an edited image and executes the second processing on the edited image based on the result of the processing. This will be described in detail below.

[0170] 11 shows an example of functional blocks of the information processing device 10. As shown in the figure, the information processing device 10 includes an acquisition unit 11, a generation unit 12, a processed image processing unit 13, an output unit 14, and a processed image processing unit 15.

[0171] The processed image processing unit 15 performs a second process on the processed image. That is, the processed image processing unit 15 inputs the processed image to a second estimation model and obtains an output of the second estimation model. Details of the second process and the second estimation model are as described in the second embodiment. As a result of this process, the effect of at least one image edit on the entire processed image on the suitability of at least one analyzer is estimated.

[0172] When the second process is performed on a processed image that includes a portion of the processed image (target portion), the effect of at least one image edit on the portion of the processed image (target portion) on the suitability of at least one analyzer is estimated. Thus, the difference between when the second process is performed on a processed image and when the second process is performed on a processed image is whether the effect of the image edit on the entire processed image is estimated or the effect of the image edit on the portion of the processed image (target portion) is estimated.

[0173] If the second processing executed by the processed image processing unit 15 results in a result indicating that at least one image edit on the processed image improves compatibility with at least one analyzer, the generation unit 12 generates a processed image. Then, the processed image processing unit 13 performs a second processing on the generated processed image. Then, the output unit 14 outputs an estimated result of the second processing performed on the generated processed image. The details of the generation unit 12, the processed image processing unit 13, and the output unit 14 are the same as those of the first to fifth embodiments.

[0174] "At least one image edit on the processed image improves compatibility with at least one analyzer" means that the impact information described above satisfies any of the following conditions: - It indicates that the value indicating the likelihood that the image edit is effective is equal to or greater than a reference value - It indicates that the value indicating the likelihood that the image edit is effective is equal to or less than a reference value - It indicates that, among the probabilities of belonging to the classes "effective," "unchanged," and "opposite effect," the probability of belonging to the "effective" class is the highest - It indicates that "effective" is true

[0175] Next, an example of the flow of processing by the information processing device 10 will be described with reference to the flowchart of Fig. 13. Note that details of each process have been described above, and therefore a description thereof will be omitted here.

[0176] In S40, the information processing device 10 acquires a processed image.

[0177] In S41, the information processing device 10 executes a second process on the processed image acquired in S40. The second process is a process of inputting the image to a second estimation model trained using second training data, and estimating the effect of at least one image edit on the image on the suitability of at least one analyzer.

[0178] In S42, the information processing device 10 determines whether the second process executed in S41 has produced a result indicating that at least one image edit on the processed image improves compatibility with at least one analyzer.

[0179] If the second process performed in S41 yields results indicating that at least one image edit on the processed image improves compatibility with at least one analyzer (Yes in S42), in S43 the information processing device 10 generates at least one processed image that includes a portion of the processed image obtained in S40.

[0180] Then, in S44, the information processing device 10 executes the second process on at least one processed image generated in S43.

[0181] In S45, the information processing device 10 outputs the estimated results of the second processing executed on each of the processed images.

[0182] On the other hand, if the second processing executed in S41 does not produce a result indicating that at least one image edit on the processed image improves compatibility with at least one analyzer (No in S42), the information processing device 10 does not execute the processing in S43 and S44. Then, in S45, the information processing device 10 outputs the result of the second processing on the processed image executed in S41.

[0183] Other configurations of the information processing apparatus 10 of this embodiment are similar to those of the information processing apparatuses 10 of the first to sixth embodiments.

[0184] According to the information processing device 10 of this embodiment, the same effects as those of the information processing devices 10 of the first to sixth embodiments are achieved.

[0185] Furthermore, the information processing device 10 of this embodiment performs a second process on the processed image, and generates an edited image and performs the second process on the edited image according to the result. Specifically, when the second process on the processed image results in a result indicating that at least one image edit on the processed image improves compatibility with at least one analyzer, the information processing device 10 generates an edited image and performs the second process on the edited image.

[0186] If the second processing of the processed image shows that none of the image edits on the processed image improves the compatibility with any of the analyzers, then the results for the processed image that includes part of the processed image will naturally be the same. In such a case, there is no point in generating at least one processed image from one processed image and performing the second processing on at least one processed image, and this would only increase the processing load on the computer.

[0187] On the other hand, if the second processing of the processed image produces results indicating that at least one image edit on the processed image improves compatibility with at least one analyzer, then naturally the results for an edited image that includes any part of the processed image will also produce the same results. In such cases, it makes sense to generate at least one edited image from one processed image and search for a part that produces the same results as the results for the processed image.

[0188] As in this embodiment, by performing the second processing on the processed image, and then generating an edited image and performing the second processing on the edited image based on the results in two steps, it is possible to obtain the desired information while reducing the processing burden on the computer.

[0189] Eighth Embodiment In this embodiment, a method for generating a first estimation model is embodied. The method will be described in detail below.

[0190] A model generation device generates a first estimation model. The model generation device generates the first estimation model by performing learning using first training data.

[0191] The first training data includes learning images and information (relevance information) about the results of analyzing the learning images with an analyzer.

[0192] The learning image may be a single still image, or may be one of a plurality of images that make up a moving image, that is, a frame image.

[0193] The model generation device inputs training images to an analyzer, generates information about the results of analyzing the training images with the analyzer, and generates first training data that combines the training images with the information about the results of analyzing the training images with the analyzer.

[0194] For example, when an analyzer to which a training image is input outputs an analysis result for the training image, the model generation device generates first training data that combines the training image with information (correct answer data) indicating that the analyzer can analyze the training image. On the other hand, when an analyzer to which a training image is input outputs information indicating that the training image cannot be analyzed, the model generation device generates first training data that combines the training image with information (correct answer data) indicating that the analyzer cannot analyze the training image. The "information indicating that the analyzer can analyze the training image" may be, for example, the maximum value of the "value indicating the likelihood that the analyzer will successfully analyze the input image" exemplified as the compatibility information. Furthermore, the "information indicating that the analyzer cannot analyze the training image" may be, for example, the minimum value of the "value indicating the likelihood that the analyzer will successfully analyze the input image" exemplified as the compatibility information.

[0195] As another example, when the analyzer outputs the reliability of the analysis result, the model generation device may generate first training data by combining the learning images with the reliability (ground truth data). The reliability corresponds to the relevance given as an example of the relevance information.

[0196] The model generation device generates a first estimation model by performing machine learning using the first training data, i.e., the model generation device updates parameters of the first estimation model by performing machine learning using the first training data.

[0197] Machine learning can be performed using existing technology. For example, the model generation device updates the parameters of the first estimation model based on the difference between the output of the first estimation model when a learning image is input and the ground truth data included in the same first training data as the learning image.

[0198] The model generation device acquires multiple training images and generates multiple first training data for each training image. The model generation device then repeatedly updates parameters of the first estimation model using the multiple first training data. Each parameter update may be performed using only one piece of first training data, or may be performed using multiple pieces of first training data, i.e., in a batch format. The parameter update is repeated until, for example, a termination condition is satisfied. The termination condition may be, for example, at least one of: the parameter update has been performed a predetermined number of times; and the difference between the output of the first estimation model and the correct answer data is equal to or less than a predetermined standard.

[0199] The model generation device can generate a first estimation model by, for example, the above-described process. The information processing device 10 can then use the first estimation model thus generated. The other configurations of the information processing device 10 are the same as those of the first to seventh embodiments.

[0200] Ninth Embodiment In this embodiment, a method for generating a second estimation model is embodied. Hereinafter, the method will be described in detail.

[0201] The model generation device generates a second estimation model by performing learning using the second training data.

[0202] The second training data includes learning images and influence information.

[0203] The learning image may be a single still image, or may be one of a plurality of images that make up a moving image, that is, a frame image.

[0204] The model generation device generates influence information using the training images, and generates second training data that combines the training images and the influence information. This will be described in detail below.

[0205] First, the model generation device performs predetermined image editing on the training image to generate an edited training image. Then, the model generation device generates influence information based on an analysis result obtained by inputting the training image to an analyzer and an analysis result obtained by inputting the edited training image to the analyzer.

[0206] For example, if an analyzer that has received a learning image outputs information indicating that analysis is not possible, and an analyzer that has received an edited learning image outputs a normal analysis result, the model generation device can generate impact information indicating a class of “effective.” For example, if the impact information is a value indicating the likelihood that image editing is effective, the maximum value of the impact information is used as the impact information.

[0207] On the other hand, if the difference between the output from the analyzer that inputs the training image and the output from the analyzer that inputs the edited training image is within a predetermined range, i.e., is almost the same, the model generation device can generate impact information indicating a class of "no effect" or "no change."

[0208] In addition, when the analyzer outputs the reliability of the analysis result, the model generation device may generate impact information based on the reliability. For example, the model generation device may use a value obtained by subtracting the reliability output when the learning image is input from the reliability output when the edited learning image is input as impact information. In this case, the impact information is a value indicating the degree of effect.

[0209] The model generation device generates a second estimation model by performing machine learning using the second training data, i.e., the model generation device updates parameters of the second estimation model by performing machine learning using the second training data.

[0210] Machine learning can be performed using existing technology. Specifically, the model generation device updates the parameters of the second estimation model based on the difference between the output of the second estimation model when a training image and an edited training image are input and the ground truth data (influence information) corresponding to the training image.

[0211] The model generation device acquires multiple training images, generates multiple edited training images, and generates influence information for each pair, thereby generating multiple second training data. The model generation device then repeatedly updates parameters of the second estimation model using the multiple second training data. Each parameter update may be performed using only one piece of second training data, or may be performed using multiple pieces of second training data, i.e., in a batch format. The parameter update is repeated until, for example, a termination condition is satisfied. The termination condition is, for example, at least one of: the parameter update has been performed a predetermined number of times; and the difference between the output of the second estimation model and the correct answer data is equal to or less than a predetermined standard.

[0212] The model generation device can generate a second estimation model by, for example, the above-described process. The information processing device 10 can then use the second estimation model thus generated. The other configurations of the information processing device 10 are the same as those of the first to eighth embodiments.

[0213] <<Modifications>> Next, a description will be given of modifications of the information processing device 10. In these modifications, the same effects as those of the above embodiment can be achieved.

[0214] <Modification 1> Modification 1 is applicable to the information processing device 10 having the configurations of the sixth and seventh embodiments.

[0215] The information processing apparatus 10 of the sixth and seventh embodiments executes the first and / or second processing on the processed image, and generates a processed image and executes the first and / or second processing on the processed image according to the result of the processing.

[0216] In the first modification, a method for generating a processed image is specified before the first and / or second processing is performed on the processed image. That is, before the first and / or second processing is performed on the processed image, a target portion to be included in each processed image is specified. In one example, the user specifies this. That is, the user specifies one of the first to third determination methods described in the fourth embodiment. The user also specifies the items that need to be specified in each determination method. The items that need to be specified in each determination method are as described in the fourth embodiment.

[0217] Furthermore, when a moving image is the target of processing, the user may further specify a section of the frame image in the moving image that is to be used as the processed image.

[0218] <Modification 2> Modification 2 is applicable to the information processing device 10 having the configurations of the sixth and seventh embodiments.

[0219] The information processing apparatus 10 of the sixth and seventh embodiments executes the first and / or second processing on the processed image, and generates a processed image and executes the first and / or second processing on the processed image according to the result of the processing.

[0220] In Modification 2, a method for generating a processed image is specified after the first and / or second processing is performed on the processed image. That is, after the first and / or second processing is performed on the processed image, the target portion to be included in each processed image is specified. In one example, the user specifies this. That is, the user specifies one of the first to third determination methods described in the fourth embodiment based on the estimated results of the first and / or second processing on the processed image. The user also specifies the items that need to be specified in each determination method based on the estimated results of the first and / or second processing on the processed image. The items that need to be specified in each determination method are as described in the fourth embodiment.

[0221] Furthermore, when a moving image is the target of processing, the user may further specify a section of the frame image in the moving image that is to be used as the processed image.

[0222] <Modification 3> The information processing device 10 can be applied to the information processing device 10 of all the embodiments.

[0223] When performing the second process on the processed image or the manipulated image, the information processing device 10 may perform at least one image edit on the processed image and display the edited processed image on the UI screen. The information processing device 10 may also input the edited processed image to at least one analyzer and obtain an analysis result. The information processing device 10 may then further display the obtained analysis result on the UI screen.

[0224] The information processing device 10 may display the processed image after the image editing and the analysis results together with the information on a UI screen that displays the estimated results obtained by performing at least one of the first process and the second process on the processed image.

[0225] In addition, the information processing device 10 may display the processed image after the image editing and the analysis results together with the information on a UI screen that displays the estimation results obtained by performing at least one of the first process and the second process on the processed image.

[0226] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0227] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order. However, the order of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of the content.

[0228] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes: 1. An information processing device comprising: acquisition means for acquiring a processed image; generation means for generating a processed image including a portion of the processed image; processed image processing means for executing at least one of a first process for inputting the image into a first estimation model trained using first training data and estimating the suitability of the image for at least one analyzer, and a second process for inputting the image into a second estimation model trained using second training data and estimating the effect of at least one image edit on the image on the suitability for at least one of the analyzers; and output means for outputting an estimation result of at least one of the first process and the second process. 2. The information processing device described in 1, wherein the output means outputs a screen displaying the processed image, and, when a user inputs a portion of the processed image on the screen, displays the estimation result of the processed image including the specified portion on the screen. 3. 3. The information processing apparatus according to claim 2, wherein the output means selectably displays a plurality of portions of the processed image on the screen, and accepts the user input specifying one of the plurality of selectably displayed portions as the user input specifying the portion of the processed image. 4. The information processing apparatus according to claim 2, wherein the output means accepts an input specifying a position and size of a partial area on the processed image as the user input specifying the portion of the processed image. 5. The information processing apparatus according to claim 1, wherein the output means outputs a screen displaying the processed image, and when a user input specifying one of at least one of the analyzers is made on the screen, displays the estimation results of the processed image including each of the plurality of portions of the processed image on the screen, linked to each of the plurality of portions of the processed image. 6. The information processing apparatus according to any of claims 1 to 5, wherein the generation means generates a plurality of the processed images including mutually different portions of the processed image, and the processed image processing means executes at least one of the first process and the second process on each of the plurality of processed images generated from one of the processed images.7. An information processing device according to any one of 1 to 6, wherein the generating means divides the processed image into a plurality of regions and generates a plurality of the processed images including each of the plurality of regions. 8. An information processing device according to any one of 1 to 7, wherein the generating means generates the processed image including a portion of the processed image specified by user input. 9. An information processing device according to any one of 1 to 8, wherein the generating means selects a portion of the processed image based on attention information obtained by inputting the processed image to an attention mechanism, and generates the processed image including the selected portion. 10. An information processing device according to any one of 1 to 9, further comprising processed image processing means that executes the first processing on the processed image, wherein if a result indicating that the processed image is suitable for at least one of the analyzers is obtained in the first processing, the generating means generates the processed image, the processed image processing means performs the first processing on the processed image, and the output means outputs the estimation result of the first processing. 11. 11. The information processing device according to any one of 1 to 10, further comprising a processed image processing means that executes the second processing on the processed image, and when a result is obtained in the second processing that indicates that at least one image edit on the processed image improves compatibility with at least one of the analyzers, the generating means generates the processed image, the processed image processing means performs the second processing on the processed image, and the output means outputs the estimation result of the second processing. 12. The information processing device according to any one of 1 to 11, wherein the generating means generates, as the processed image, an image in which a predetermined processing has been applied to another part of the processed image.13. An information processing method in which one or more computers acquire a processed image, generate an edited image including a portion of the processed image, and perform at least one of a first process for inputting the image to a first estimation model trained using first training data and estimating suitability of the image for at least one analyzer, and a second process for inputting the image to a second estimation model trained using second training data and estimating an effect of at least one image edit on the image on the suitability for at least one of the analyzers, and output an estimation result of at least one of the first process and the second process. A program that causes a computer to function as: an acquisition means that acquires a processed image; a generation means that generates a processed image that includes a portion of the processed image; a processed image processing means that executes at least one of a first process that inputs an image to a first estimation model that has been trained using first training data and estimates the suitability of the image for at least one analyzer, and a second process that inputs the image to a second estimation model that has been trained using second training data and estimates the effect that at least one image edit on the image has on the suitability for at least one analyzer; and an output means that outputs an estimation result of at least one of the first process and the second process.

[0229] Some or all of Supplements 2 to 12 that are dependent on the information processing device of Supplement 1 described above may also be dependent on the information processing method of Supplement 13 and the program of Supplement 14 in the same dependent relationship as Supplement 1 and Supplements 2 to 12. Furthermore, within the scope of each of the above-mentioned embodiments, some or all of the configurations described as Supplements can be realized in various hardware, software, various recording means for recording software, or systems.

[0230] This application claims priority based on Japanese Patent Application No. 2023-204550, filed December 4, 2023, the disclosure of which is incorporated herein by reference in its entirety.

[0231] REFERENCE SIGNS LIST 10 Information processing device 11 Acquisition unit 12 Generation unit 13 Processed image processing unit 14 Output unit 15 Processed image processing unit 1A Processor 2A Memory 3A Input / output I / F 4A Peripheral circuit 5A Bus

Claims

1. An information processing device having: an acquisition means for acquiring a processed image; a generation means for generating a processed image including a portion of the processed image; a processed image processing means for executing at least one of a first process for inputting an image into a first estimation model trained using first training data and estimating the suitability of the image for at least one analyzer, and a second process for inputting the image into a second estimation model trained using second training data and estimating the effect that at least one image edit made to the image has on the suitability of at least one of the analyzers, on the processed image; and an output means for outputting an estimation result of at least one of the first process and the second process.

2. An information processing device as described in claim 1, wherein the output means outputs a screen displaying the processed image, and when a user input is made on the screen specifying a portion of the processed image, the estimated result of the processed image including the specified portion is displayed on the screen.

3. An information processing device as described in claim 2, wherein the output means displays a plurality of selectable portions of the processed image on the screen, and accepts the user input specifying one of the plurality of selectably displayed portions as the user input specifying a portion of the processed image.

4. An information processing device according to claim 2, wherein said output means receives an input specifying a position and a size of a partial area on said processed image as said user input specifying a portion of said processed image.

5. An information processing device as claimed in any one of claims 1 to 4, wherein the output means outputs a screen displaying the processed image, and when a user input is made on the screen to designate one of at least one of the analyzers, the output means links the estimation results of the processed image including each of the multiple portions of the processed image to each of the multiple portions of the processed image and displays them on the screen.

6. An information processing device as claimed in any one of claims 1 to 5, wherein the generation means generates a plurality of the processed images each including a different portion of the processed image, and the processed image processing means performs at least one of the first processing and the second processing on each of the plurality of the processed images generated from one of the processed images.

7. An information processing device as described in any one of claims 1 to 6, further comprising a processed image processing means for executing the first processing on the processed image, and when a result is obtained in the first processing indicating that the processed image is suitable for at least one of the analyzers, the generation means generates the processed image, the processed image processing means performs the first processing on the processed image, and the output means outputs the estimated result of the first processing.

8. An information processing device as described in any one of claims 1 to 7, further comprising a processed image processing means for executing the second processing on the processed image, and when a result is obtained in the second processing indicating that at least one image edit on the processed image increases compatibility with at least one of the analyzers, the generation means generates the processed image, the processed image processing means performs the second processing on the processed image, and the output means outputs the estimated result of the second processing.

9. An information processing device according to any one of claims 1 to 8, wherein the generating means divides the processed image into a plurality of regions and generates a plurality of the processed images each including a plurality of the regions.

10. An information processing device according to any one of claims 1 to 9, wherein the generating means generates the processed image including a portion of the processed image specified by user input.

11. An information processing device as described in any one of claims 1 to 10, wherein the generation means selects a portion of the processed image based on attention information obtained by inputting the processed image into an attention mechanism, and generates the processed image including the selected portion.

12. An information processing method in which one or more computers acquire a processed image, generate a manipulated image including a portion of the processed image, and perform at least one of a first process on the manipulated image, inputting the image into a first estimation model trained using first training data and estimating the suitability of the image for at least one analyzer, and a second process inputting the image into a second estimation model trained using second training data and estimating the effect of at least one image edit on the image on the suitability for at least one of the analyzers, and outputting an estimation result of at least one of the first process and the second process.

13. An information processing method as described in claim 12, wherein in outputting the estimation result, a screen displaying the processed image is output, and when a user input specifying a portion of the processed image is made on the screen, the estimation result of the processed image including the specified portion is displayed on the screen.

14. An information processing method as described in claim 13, wherein in outputting the estimation result, a plurality of portions of the processed image are displayed on the screen in a selectable manner, and the user input specifying one of the plurality of selectably displayed portions of the processed image is accepted as the user input specifying the portion of the processed image.

15. An information processing method according to claim 13, wherein, in outputting the estimation result, an input specifying a position and size of a partial area on the processed image is accepted as the user input specifying a portion of the processed image.

16. An information processing method according to any one of claims 12 to 15, wherein the output of the estimation result comprises outputting a screen displaying the processed image, and when a user input is made on the screen to designate one of at least one of the analyzers, the estimation result of the processed image including each of the multiple portions of the processed image is displayed on the screen, linked to each of the multiple portions of the processed image.

17. A recording medium having recorded thereon a program that causes a computer to function as: an acquisition means for acquiring a processed image; a generation means for generating a processed image including a portion of the processed image; a processed image processing means for executing at least one of the following on the processed image: a first process for inputting an image into a first estimation model trained using first training data and estimating the suitability of the image for at least one analyzer; and a second process for inputting an image into a second estimation model trained using second training data and estimating the effect of at least one image edit on the image on the suitability for at least one of the analyzers; and an output means for outputting an estimation result of at least one of the first process and the second process.

18. The recording medium according to claim 17, wherein the output means outputs a screen displaying the processed image, and when a user input is made on the screen specifying a portion of the processed image, the estimation result of the processed image including the specified portion is displayed on the screen.

19. The recording medium according to claim 18, wherein the output means displays a plurality of selectable portions of the processed image on the screen, and accepts the user input specifying one of the plurality of selectably displayed portions as the user input specifying a portion of the processed image.

20. A recording medium according to claim 18, wherein said output means receives an input designating a position and a size of a partial area on said processed image as said user input designating a portion of said processed image.

Images