Information processing device, program, and information processing method

Abstract

An information processing device (100) comprises: a data acquisition unit (103) that acquires a first data set including a plurality of pieces of first image information indicating mutually different images, and a second data set including a plurality of pieces of second image information indicating mutually different images; a display object detection unit (107) that detects a specific display object having a display size equal to or larger than a preset second threshold value from within a specific image indicated by specific image information among the pieces of first image information; an image information acquisition unit (111) that acquires new image information in which a display range of the specific display object in the specific image is replaced with another image; and a data determination unit (112) that determines whether to exclude the specific image information from the first data set.

Description

Information Processing Apparatus, Program, and Information Processing Method

[0001] The present disclosure relates to an information processing apparatus, a program, and an information processing method.

[0002] Conventionally, an action recognition system for identifying an action type corresponding to sensor data included in a target domain has been disclosed (see, for example, Patent Document 1). The action recognition system described in Patent Document 1 is configured to perform identification using a discriminator trained using data of a domain having the highest similarity to the target domain among discriminators for each domain trained to identify an action type corresponding to sensor data using data for each domain of a plurality of domains.

[0003] Japanese Unexamined Patent Application Publication No. 2020-101948

[0004] For example, when generating a learned model using a learning dataset including a plurality of pieces of image information each showing a different image as a source domain, it is desirable to exclude in advance image information having a low correlation with the target domain. However, depending on the content of the image, there is a problem that image information that should not be excluded may be determined to have a low correlation with the target domain and thus be excluded from the dataset.

[0005] The present disclosure has been made in view of recognition of the above problems, and an object thereof is to provide an information processing apparatus, a program, and an information processing method capable of suppressing exclusion of image information that should not be excluded from a dataset.

[0006] The information processing device relating to this disclosure includes: a data acquisition unit that acquires a first dataset containing a plurality of first image information, each showing a different image; a second dataset containing a plurality of second image information, each showing a different image; a correlation value calculation unit that calculates a correlation value indicating the correlation between each first image information and the plurality of second image information based on the feature quantities of each first image information and the feature quantities of each second image information; a correlation determination unit that determines for each first image information whether or not it is a specific image information for which the correlation value calculated by the correlation value calculation unit is less than a preset first threshold; and a second threshold unit that determines whether or not the display size of the displayed objects within the specific image shown by the specific image information is a preset second threshold. The system comprises: a display object detection unit that detects a specific display object; an image information acquisition unit that acquires new image information in which the display range of a specific image of the specific display object is replaced by another image, based on the detection of the specific display object by the display object detection unit; and a data determination unit that determines whether or not to exclude the specific image information from the first dataset based on the determination result by the correlation determination unit. The system is configured such that the correlation value calculation unit calculates a correlation value between the new image information and a plurality of second image information, and the data determination unit determines whether or not to exclude the specific image information from the first dataset based on the correlation value between the new image information and the plurality of second image information.

[0007] According to this disclosure, it is possible to suppress the exclusion of image information that should not be excluded from the dataset.

[0008] This is a block diagram showing the schematic configuration of an information processing system according to Embodiment 1. This is a block diagram showing an example of the hardware configuration of an information processing device according to Embodiment 1. This is a block diagram showing an example of the hardware configuration of an information processing device according to Embodiment 1. This is a flowchart showing an example of processing performed by an information processing device according to Embodiment 1. This is a scatter plot in the feature space showing a plurality of first features and a plurality of second features acquired by the information processing device according to Embodiment 1. Figure 6A is a diagram showing an example of a specific image shown by specific image information acquired by an information processing device according to Embodiment 1, Figure 6B is a diagram showing a specific image in a state where masking processing has been performed by an information processing device according to Embodiment 1, and Figure 6C is a diagram showing an image shown by new image information acquired by an information processing device according to Embodiment 1. This is a scatter plot in the feature space showing the features of new image information acquired by an information processing device according to Embodiment 1. This is a block diagram showing the schematic configuration of an information processing system according to Embodiment 2. This is a flowchart showing an example of processing performed by an information processing device according to Embodiment 2. This is a block diagram showing the schematic configuration of an information processing system according to Embodiment 3. This is a flowchart showing an example of processing performed by an information processing device according to Embodiment 3. This is a diagram showing location information acquired by an information processing device according to Embodiment 3.

[0009] The embodiments of this disclosure will now be described in detail with reference to the drawings. Embodiment 1. First, an information processing system 1 according to Embodiment 1 will be described with reference to Figure 1. Figure 1 is a block diagram showing the schematic configuration of the information processing system 1 according to Embodiment 1. The information processing system 1 according to Embodiment 1 is a system for identifying image information to be excluded from a dataset containing multiple image information. As shown in Figure 1, the information processing system 1 according to Embodiment 1 includes a database DB 1 and an information processing device 100, which are connected wirelessly or by wire to enable communication between them. The database DB 1 and the information processing device 100 may be connected to each other via devices or communication lines not shown, enabling them to communicate information with each other.

[0010] Database DB1 stores various types of information used in the processing performed by the information processing device 100. For example, database DB1 stores a dataset containing multiple image data used in the processing performed by the information processing device 100. Database DB1 may also be configured to store information indicating various threshold values ​​used in the processing performed by the information processing device 100.

[0011] The information processing device 100 includes an input unit 101, an output unit 102, a data acquisition unit 103, a feature quantity conversion unit 104, a correlation value calculation unit 105, a correlation determination unit 106, a display object detection unit 107, an image information acquisition unit 111, and a data determination unit 112.

[0012] The input unit 101 receives various types of information input to the information processing device 100. For example, the input unit 101 receives various types of information input to the information processing device 100 from the database DB1. In addition to information from the database DB1, the input unit 101 may also be configured to receive information input to the information processing device 100 from other devices (not shown) that are connected to the information processing device 100 wirelessly or by wire so as to be able to communicate with each other.

[0013] The output unit 102 outputs various types of information from the information processing device 100 to the database DB1. For example, the output unit 102 outputs information indicating the results of processing performed by the information processing device 100 to the database DB1 in order to store the results of processing performed by the information processing device 100 in the database DB1. In addition to the database DB1, the output unit 102 may be configured to output information to other devices (not shown) that are wirelessly or wired and can communicate with the information processing device 100. For example, the output unit 102 may be configured to output a dataset obtained as a result of processing performed by the information processing device 100 to other devices (not shown) in order to cause those devices to generate a trained model. Alternatively, for example, the output unit 102 may be configured to output the results of processing performed by the information processing device 100 to a display device (not shown) that displays images.

[0014] The data acquisition unit 103 acquires a first dataset containing multiple first image information sets, each representing a different image, and a second dataset containing multiple second image information sets, each representing a different image, from the database DB1 via the input unit 101. For example, the data acquisition unit 103 acquires a first dataset containing multiple first image information sets, each with a label, as a source domain for generating a trained model. Specifically, the data acquisition unit 103 acquires a first dataset containing multiple first image information sets, each with a segment set for each object in the image, as a source domain for generating a trained model. Alternatively, for example, the data acquisition unit 103 acquires a second dataset containing multiple second image information sets, different from the first image information, as a target domain for allowing the trained model to perform inference.

[0015] The feature transformation unit 104 transforms each of the multiple first image information contained in the first dataset and the multiple second image information contained in the second dataset, acquired by the data acquisition unit 103, into features. In other words, the feature transformation unit 104 obtains the features of each first image information and each second image information by extracting features from each of the multiple first image information contained in the first dataset and the multiple second image information contained in the second dataset, acquired by the data acquisition unit 103. For example, the feature transformation unit 104 performs dimensionality reduction on each of the multiple first image information contained in the first dataset and the multiple second image information contained in the second dataset, acquired by the data acquisition unit 103, and transforms them into features with lower dimensions than the original image information.

[0016] The correlation value calculation unit 105 calculates a correlation value for each first image information, indicating the correlation between each first image information included in the first dataset acquired by the data acquisition unit 103 and a plurality of second image information included in the second dataset acquired by the data acquisition unit 103. For example, the correlation value calculation unit 105 calculates a correlation value for each first image information, indicating the correlation between each first image information and the plurality of second image information, based on the feature quantities of each first image information and the feature quantities of each second image information acquired by the feature quantity conversion unit 104. For example, the correlation value calculation unit 105 calculates the Euclidean distance between the centroid of each feature quantity of the plurality of second image information and the feature quantity of each first image information as the correlation value between each first image information and the plurality of second image information. Specifically, the correlation value calculation unit 105 calculates the Euclidean distance between the centroid of a plurality of vectors representing each feature of the plurality of second image information and the vector representing the feature of each first image information, as a correlation value between each first image information and a plurality of second image information.

[0017] Furthermore, for example, the correlation value calculation unit 105 calculates a correlation value between each first image information and a plurality of second image information by dividing the Euclidean distance between the centroid of each feature of the plurality of second image information and each feature of the first image information by the variance or standard deviation of the set of features of the plurality of second image information. Specifically, the correlation value calculation unit 105 calculates a correlation value between each first image information and a plurality of second image information by dividing the Euclidean distance between the centroid of a plurality of vectors representing each feature of the plurality of second image information and each vector representing the feature of the first image information by the variance or standard deviation of the set of a plurality of vectors representing each feature of the plurality of second image information. In Embodiment 1, the feature of the first image information is also referred to as the "first feature," and the feature of the second image information is also referred to as the "second feature."

[0018] Furthermore, the correlation value calculation unit 105 calculates the correlation value between the new image information acquired by the image information acquisition unit 111 and the multiple second image information. The function of the correlation value calculation unit 105 to calculate the correlation value between the new image information acquired by the image information acquisition unit 111 and the multiple second image information is the same as the function to calculate the correlation value between each first image information and the multiple second image information, so a detailed explanation is omitted. Details about the new image information acquired by the image information acquisition unit 111 will be described later.

[0019] The correlation determination unit 106 determines for each piece of first image information whether the correlation value calculated by the correlation value calculation unit 105 is less than a preset first threshold. For example, if the Euclidean distance between any first feature and the centroid of a plurality of second features is less than a preset first threshold, the correlation determination unit 106 determines that the first image information relating to that first feature is specific image information. Also, for example, if the Euclidean distance between any first feature and the centroid of a plurality of second features is not less than a preset first threshold, the correlation determination unit 106 determines that the first image information relating to that first feature is not specific image information. In Embodiment 1, the image shown by the specific image information is also referred to as the "specific image".

[0020] Furthermore, for example, the correlation determination unit 106 determines whether the correlation value between the new image information acquired by the image information acquisition unit 111 and the plurality of second image information is less than a preset third threshold, based on the correlation value between the new image information and the plurality of second image information. The third threshold may be the same value as the first threshold, or it may be a different value from each other. For example, the correlation determination unit 106 may be configured to determine whether the correlation value between the new image information acquired by the image information acquisition unit 111 and the plurality of second image information is less than a preset third threshold that is smaller than the first threshold, or it may be configured to determine whether it is less than a preset third threshold that is larger than the first threshold.

[0021] The display object detection unit 107 detects specific display objects within a specific image, which is an image indicated by specific image information, whose display size is equal to or greater than a preset second threshold. In other words, for each first image information determined by the correlation determination unit 106 to be specific image information, the display object detection unit 107 detects specific display objects within the specific image indicated by each first image information, whose display size is equal to or greater than a preset second threshold. For example, from among the multiple first image information acquired by the data acquisition unit 103, each of which has segments set for each display object within the specific image, the display object detection unit 107 detects specific display objects from the first image information corresponding to the specific image information, whose area or number of pixels of each segment set for each display object within the specific image is equal to or greater than a preset second threshold.

[0022] Furthermore, for example, the display object detection unit 107 detects a specific display object from among the multiple first image information acquired by the data acquisition unit 103, each of which has a segment set for each display object in the image. The first image information corresponds to the specific image information, and the area or number of pixels of the circumscribing polygon of each segment set for each display object within the specific image is equal to or greater than a preset second threshold. Note that the display objects detected by the display object detection unit 107 are not limited to objects displayed in the image, but may also be planar objects such as characters, images, or reflections on reflective objects displayed in the image, spaces such as holes or depressions, or meteorological phenomena such as puddles, snow, or fog.

[0023] The image information acquisition unit 111 acquires new image information in which the display range of a specific image of a specific display object is replaced by another image, based on the detection of a specific display object by the display object detection unit 107. For example, the image information acquisition unit 111 acquires new image information in which the display range of a specific display object in a specific image is replaced by an image generated based on the surrounding partial image information of the specific display object in the specific image information, based on the detection of a specific display object by the display object detection unit 107. For example, the image information acquisition unit 111 has a trained model for performing image inpainting, and acquires new image information in which the display range of a specific image of a specific display object is replaced by another image by supplementing the display range of the specific image of the specific display object through image inpainting. Specifically, the image information acquisition unit 111 has a trained model for performing image inpainting using algorithms such as Stable Diffusion or GAN (Generative Adversarial Network). The image information acquisition unit 111 may be configured to acquire new image information, which is the output of a trained model, from an external device that has a trained model for performing image inpainting, by outputting specific image information to the external device. In Embodiment 1, the new image information acquired by the image information acquisition unit 111 is also simply referred to as "new image information".

[0024] The data determination unit 112 decides whether or not to exclude specific image information from the first dataset based on the determination result by the correlation determination unit 106. For example, the data determination unit 112 decides whether or not to exclude the first image information, which is specific image information related to the new image information, from the first dataset based on the correlation value between the new image information and the multiple second image information. Specifically, if the correlation value between the new image information and the multiple second image information is not less than the third threshold, the data determination unit 112 decides not to exclude the specific image information related to the new image information from the first dataset. If the correlation value between the new image information and the multiple second image information is less than the third threshold, the data determination unit 112 decides to exclude the specific image information related to the new image information from the first dataset.

[0025] In this way, the information processing device 100 acquires new image information in which a specific object in the image shown by the first image information is replaced by another image, based on the first image information whose correlation value with the plurality of second image information is less than a first threshold, and determines whether or not to exclude the first image information from the first dataset based on the correlation value between the new image information and the plurality of second image information.

[0026] For example, if the first image information includes objects whose display size is greater than or equal to the second threshold, depending on the content of those objects, the feature quantities between the first image information with those objects and the first image information without those objects may diverge. As a result, the correlation value between the first image information without those objects and multiple second image information may be greater than or equal to the first threshold, while the correlation value between the first image information with those objects and multiple second image information may be less than the first threshold. In such cases, depending on the content of the objects, it may be desirable to include both the first image information with and without those objects in the same dataset when generating a trained model.

[0027] The information processing device 100, when it detects that the image shown by the first image information contains a specific display object whose display size is equal to or greater than the second threshold, acquires new image information showing an image without that specific display object, calculates a correlation value between the new image information and the multiple second image information, and compares the calculated correlation value with the threshold. By doing so, it prevents the exclusion of first image information that would be excluded from the first dataset if judged solely based on the correlation value between the first image information and the multiple second image information, but which should not be excluded from the first dataset in the first place; in other words, it prevents the exclusion of first image information from the first dataset whose correlation value with the multiple second image information is less than the first threshold due to the inclusion of a display object in the image whose display size is equal to or greater than the second threshold.

[0028] Next, the hardware configuration of the information processing device 100 will be described with reference to Figures 2 and 3. Figure 2 is a diagram showing an example of the hardware configuration of the information processing device 100, and Figure 3 is a diagram showing an example of the hardware configuration of the information processing device 100 that is different from Figure 2. For example, as shown in Figure 2, the information processing device 100 is composed of a computer having a processor 100a, a memory 100b, and an I / O port 100c, and is configured so that the processor 100a reads and executes a program stored in the memory 100b.

[0029] Furthermore, as shown in Figure 3, for example, the information processing device 100 is composed of a computer having a processing circuit 100d, which is dedicated hardware, and an I / O port 100c. The processing circuit 100d is composed of, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. Each function of the information processing device 100 is realized by these processors 100a or the processing circuit 100d, which is dedicated hardware, executing a program. Note that the information processing device 100 may also have hardware other than that described above.

[0030] Next, with reference to Figures 1, 4 to 7, the details of the processing performed by the information processing device 100 will be described. Figure 4 is a flowchart showing an example of the processing performed by the information processing device 100 according to Embodiment 1. The processing performed by the information processing device 100 shown in Figure 4 is a process for determining whether or not to exclude first image information from the first dataset if the correlation value between it and a plurality of second image information is less than a first threshold.

[0031] As shown in Figure 4, when the information processing device 100 starts processing, it first acquires a first dataset and a second dataset (step ST01). In this process, the information processing device 100 acquires a first dataset containing a plurality of first image information, each representing a different image, and a second dataset containing a plurality of second image information, each representing a different image, through the data acquisition unit 103. For example, in this process, the information processing device 100 acquires a first dataset containing a plurality of first image information acquired by an imaging device installed on a vehicle that captures images of the outside of the vehicle, and a second dataset containing a plurality of second image information for causing a trained model generated based on the input of the first dataset to perform inference.

[0032] When the information processing device 100 performs the processing in step ST01, it converts each first image information and each second image information into feature quantities (step ST02). In this process, the information processing device 100 converts each of the multiple first image information contained in the first dataset acquired in the processing of step ST01, and the multiple second image information contained in the second dataset acquired in the processing of step ST01, into first feature quantities and second feature quantities, respectively, using the feature quantity conversion unit 104.

[0033] Figure 5 is a scatter plot in the feature space showing a plurality of first features and a plurality of second features acquired by the information processing device 100 according to Embodiment 1. As shown in Figure 5, for example, the information processing device 100 acquires in step ST01 a first dataset containing a plurality of first image information, which includes first image information in which the feature extracted in step ST02 is J1, and first image information in which the feature extracted in step ST02 is J2, and a second dataset containing a plurality of second image information.

[0034] When the information processing device 100 performs the processing in step ST02, it calculates a correlation value between the first image information and the plurality of second image information (step ST03). In this process, the information processing device 100 uses a correlation value calculation unit 105 to calculate a correlation value between each first image information included in the first dataset obtained in step ST01 and the plurality of second image information included in the second dataset obtained in step ST01, based on the plurality of first and plurality of second features obtained in step ST02.

[0035] When the information processing device 100 performs the processing in step ST03, it determines whether or not there is specific image information whose correlation value is less than the first threshold (step ST04). In this process, the information processing device 100 determines for each first feature whether or not the correlation value between each first image information calculated in the processing of step ST03 and the plurality of second image information is specific image information, and extracts specific image information from the plurality of first image information included in the first dataset. For example, in this process, the information processing device 100 determines that the first image information of features J1 and J2, which are plot data outside the range D1 that indicates the boundary of the range where the correlation value is the first threshold in Figure 5, are specific image information.

[0036] In step ST04, if there is specific image information for which the correlation value is less than the first threshold (YES in step ST04), the information processing device 100 determines whether or not a specific display object is included in the specific image information (step ST05). In this process, the information processing device 100 uses the display object detection unit 107 to detect specific display objects within each specific image shown by the specific image information extracted in step ST04 whose display size is equal to or greater than the second threshold, thereby determining for each specific image information whether or not a specific display object is included in the specific image shown by each specific image information.

[0037] Figure 6A is a diagram showing an example of a specific image indicated by specific image information acquired by the information processing device 100 according to Embodiment 1. As shown in Figure 6A, for example, in the processing of step ST05, if the information processing device 100 detects specific display objects B1, B2, and B3 among the display objects in the image whose display size is equal to or greater than the second threshold, it determines that the image contains specific display objects.

[0038] In step ST05, if the specific image information contains a specific display object (YES in step ST05), the information processing device 100 acquires new image information in which the specific display object in the specific image information is replaced by another image (step ST10). In this process, based on the detection of a specific display object in the image in step ST05, the information processing device 100 acquires new image information by the image information acquisition unit 111 in which the display range of the specific display object in the specific image in which the specific display object was detected is replaced by another image. For example, in this process, based on the detection of a specific display object in the image in step ST05, the information processing device 100 acquires new image information in which the display range of the specific display object in the specific image is replaced by an image generated based on the surrounding partial image information of the specific display object in the specific image information that shows the specific image. Specifically, in this process, the information processing device 100 generates new image information in which the display range of the specific display object in the specific image is replaced by another image by image inpainting. For example, in the process of step ST10, the information processing device 100 first performs a masking process to set the area in which image-in-painting will be performed.

[0039] Figure 6B shows a specific image in a state where masking processing has been performed by the information processing device 100 according to Embodiment 1. For example, the information processing device 100 sets the range for image inpainting by masking the display ranges H1, H2, and H3 of the specific display objects B1, B2, and B3 detected in step ST05 through masking processing. When the information processing device 100 performs masking processing, it generates a new image in which the image in the range set by the masking processing is replaced with other images by image inpainting.

[0040] Figure 6C shows an image represented by new image information acquired by the information processing device 100 according to Embodiment 1. For example, the information processing device 100 estimates the background image when a specific display object within the range set by the masking process is not included in the specific image, thereby generating a new image in which the specific display object is replaced by the background image. In this way, the information processing device 100 acquires new image information in which the specific display object of the specific image information is replaced by another image for each specific image information that was determined to contain a specific display object in the processing of step ST05.

[0041] When the information processing device 100 performs the processing in step ST10, it calculates a correlation value between the new image information and the multiple second image information (step ST11). In this process, the information processing device 100 uses a correlation value calculation unit 105 to calculate the correlation value between the new image information acquired in the processing of step ST10 and the multiple second image information included in the second dataset acquired in the processing of step ST01. Furthermore, if multiple new image information is acquired in the processing of step ST10, the information processing device 100 calculates a correlation value between the new image information and the second image information for each piece of new image information.

[0042] When the information processing device 100 performs the processing in step ST11, it determines whether the correlation value is less than the third threshold (step ST12). In this process, the information processing device 100 uses a correlation determination unit 106 to determine whether the correlation value calculated in the processing of step ST11 is less than the third threshold. Furthermore, if multiple new image information is acquired in the processing of step ST10, the information processing device 100 determines for each of the multiple second image information whether the correlation value between them is less than the third threshold.

[0043] In the process of step ST12, when the correlation value is not less than the third threshold value (NO in step ST12), the information processing apparatus 100 determines not to exclude the specific image information from the first data set (step ST13). In this process, based on the determination result in the process of step ST12, the data determination unit 112 of the information processing apparatus 100 determines not to exclude the first image information, which is the specific image information related to the new image information whose correlation value with the plurality of second image information is less than the third threshold value, from the first data set.

[0044] In the process of step ST05, when the specific display object is not included in the specific image information (NO in step ST05), and in the process of step ST12, when the correlation value is less than the third threshold value (YES in step ST12), the information processing apparatus 100 excludes the specific image information from the first data set (step ST14).

[0045] In this process, when the correlation value between the first image information and the plurality of second image information is less than the first threshold value even though the image indicated by the first image information does not include the specific display object, and when the correlation value between the new image information in which the specific display object included in the image indicated by the first image information is not included and the plurality of second image information is less than the third threshold value, the information processing apparatus 100 determines to exclude the first image information from the first data set on the grounds that the first image information should not be included in the first data set.

[0046] Figure 7 is a scatter plot in the feature space showing the feature quantities of new image information acquired by the information processing device 100 according to Embodiment 1. Figure 7 shows, for example, that when the feature quantity of new image information acquired based on the first image information whose feature quantity was J1 in the scatter plot of Figure 5 is J1', and the feature quantity of new image information acquired based on the first image information whose feature quantity was J2 in the scatter plot of Figure 5 is J2', the correlation value between the new image information with feature quantity J1' and the multiple second image information is greater than or equal to the third threshold, and the correlation value between the new image information with feature quantity J2' and the multiple second image information is less than the third threshold. In other words, Figure 7 shows that the first image information whose feature quantity was J1 in the scatter plot of Figure 5 is image information whose correlation value with multiple second image information is such that it should not be excluded from the first dataset when no specific object is included, and the first image information whose feature quantity was J2 in the scatter plot of Figure 5 is image information whose correlation value with multiple second image information is such that it should be excluded from the first dataset even when no specific object is included.

[0047] The information processing device 100 terminates processing if, in the processing of step ST04, there is no specific image information whose correlation value is less than the first threshold (NO in step ST04), if the processing of step ST13 is performed, or if the processing of step ST14 is performed. For example, the information processing device 100 outputs to the database DB1 information on whether or not to exclude each first image information determined based on the results of steps ST13 and ST14 from the first dataset.

[0048] As described above, the information processing apparatus 100 according to Embodiment 1 includes a first data set including a plurality of first image information items each indicating a different image, and a second data set including a plurality of second image information items each indicating a different image. The data acquisition unit 103 acquires the first data set and the second data set. Based on the feature amounts of each first image information item and the feature amounts of each second image information item, the correlation value calculation unit 105 calculates a correlation value indicating the correlation between each first image information item and the plurality of second image information items. The correlation determination unit 106 determines, for each first image information item, whether the specific image information has a correlation value calculated by the correlation value calculation unit 105 that is less than a preset first threshold value. The display object detection unit 107 detects a specific display object among the display objects in the specific image indicated by the specific image information, where the display size of the specific display object is equal to or greater than a preset second threshold value. Based on the detection of the specific display object by the display object detection unit 107, the image information acquisition unit 111 acquires new image information in which the display range of the specific display object in the specific image is replaced by another image. The data determination unit 112 determines whether to exclude the specific image information from the first data set based on the determination result by the correlation determination unit 106. The correlation value calculation unit 105 calculates a correlation value between the new image information and the plurality of second image information items, and the data determination unit 112 is configured to determine whether to exclude the specific image information from the first data set based on the correlation value between the new image information and the plurality of second image information items.

[0049] For example, the information processing apparatus 100 includes a correlation determination unit 106 that determines whether the correlation value between the new image information and the plurality of second image information items is less than a preset third threshold value. When the correlation value between the new image information and the plurality of second image information items is not less than the third threshold value, the data determination unit 112 determines not to exclude the specific image information from the first data set. When the correlation value between the new image information and the plurality of second image information items is less than the third threshold value, the data determination unit 112 determines to exclude the specific image information from the first data set.

[0050] Also, for example, the information processing apparatus 100 includes a display object detection unit 107 that detects a specific display object based on a segment preset in the specific image information.

[0051] Furthermore, for example, the information processing device 100 includes an image information acquisition unit 111 that, based on the detection of a specific display object by the display object detection unit 107, acquires new image information in which the display range in a specific image of the specific display object is replaced by an image generated based on the partial image information surrounding the specific display object in the specific image information.

[0052] With this configuration, the information processing device 100 decides not to exclude from the first dataset first image information whose correlation value with multiple second image information is less than the first threshold due to the inclusion of a display object with a display size greater than or equal to the second threshold in the image. Conversely, it decides to exclude from the first dataset first image information whose correlation value with multiple second image information is low despite the absence of a display object with a display size greater than or equal to the second threshold in the image, and first image information whose correlation value with multiple second image information is low despite the replacement of a display object with a display size greater than or equal to the second threshold in the image with another image. As a result, the information processing device 100 can suppress the exclusion from the first dataset of first image information that should not be excluded in principle, but whose correlation value with multiple second image information is low due to the inclusion of a display object with a display size greater than or equal to the second threshold in the image.

[0053] In Embodiment 1, the information processing device 100 includes a data acquisition unit 103 that acquires a first dataset containing a plurality of first image information, each representing a different image, and a second dataset containing a plurality of second image information, each representing a different image, from a database DB1 via an input unit 101, and a feature conversion unit 104 that converts each of the plurality of first image information contained in the first dataset and the plurality of second image information contained in the second dataset acquired by the data acquisition unit 103 into feature quantities, but is not limited to this. The information processing device is configured such that a correlation value calculation unit calculates a correlation value showing the correlation between each first image information and the plurality of second image information based on the information acquired by the data acquisition unit. For example, the information processing device may include a data acquisition unit that acquires a first dataset, which is a set of feature quantities for a plurality of first images. Furthermore, such an information processing device does not need to include a feature conversion unit.

[0054] Furthermore, in Embodiment 1, the information processing device includes a display object detection unit 107 that detects a specific display object from among a plurality of first image information acquired by the data acquisition unit 103, each of which has segments set for each display object within the specific image, such that the area or number of pixels of each segment set for each display object within the specific image is equal to or greater than a preset second threshold. However, the device is not limited to this. The first image information acquired by the data acquisition unit does not have segments set for each display object within the image. For example, the information processing device may include a display object detection unit that detects a display object using a trained model for detecting a display object from within the specific image indicated by the input specific image information, based on the input of specific image information. The display object detection unit may have such a trained model, or it may be configured to output specific image information to an external device having such a trained model, thereby detecting a display object within the specific image based on the output of the trained model possessed by the external device.

[0055] Embodiment 2. Next, the information processing system 1A according to Embodiment 2 will be described with reference to Figures 8 and 9. The information processing system 1A according to Embodiment 2 differs from the information processing system 1 according to Embodiment 1 in that the configuration related to the conditions for the image information acquisition unit to acquire new image information differs, but other configurations are the same, and the same names and reference numerals as in Embodiment 1 will be used and their descriptions will be omitted.

[0056] Figure 8 is a block diagram showing the schematic configuration of the information processing system 1A according to Embodiment 2. As shown in Figure 8, the information processing system 1A according to Embodiment 2 comprises a database DB1 and an information processing device 100A, which are connected wirelessly or via wired connections so that they can communicate with each other. The database DB1 and the information processing device 100A may also be connected to each other via devices or communication lines not shown, so that they can communicate information with each other.

[0057] The information processing device 100A includes an input unit 101, an output unit 102, a data acquisition unit 103, a feature quantity conversion unit 104, a correlation value calculation unit 105, a correlation determination unit 106, a display object detection unit 107, a similarity calculation unit 108, an image information acquisition unit 111A, and a data determination unit 112.

[0058] The similarity calculation unit 108 calculates the similarity between the feature quantities of the partial image information of the display range in the specific image of the specific display and the feature quantities of the partial image information surrounding the display range in the specific image of the specific display, based on the detection of the specific display by the display object detection unit 107. For example, the similarity calculation unit 108 first extracts partial image information from the specific image information that represents an image within a certain distance from the contour of the display range in the specific image of the specific display, as partial image information surrounding the display range in the specific image of the specific display. The similarity calculation unit 108 then calculates a numerical value that shows the result of comparing the feature quantities of the partial image information of the display range in the specific image of the specific display and the extracted partial image information surrounding the display range in the specific image of the specific display with one or more feature quantities from brightness, hue, and frequency characteristics, as the similarity between the feature quantities of the partial image information of the display range in the specific image of the specific display and the extracted partial image information surrounding the display range in the specific image of the specific display.

[0059] The image information acquisition unit 111A acquires new image information in which the display range of the specific display object in the specific image is replaced by another image, based on the fact that the specific display object is detected by the display object detection unit 107 from the specific image information and the similarity calculated by the similarity calculation unit 108 is less than a preset fourth threshold. For example, even if the image shown by the specific image information contains a specific display object whose display size is greater than or equal to the second threshold, if the similarity between the feature quantities of the specific display object and the feature quantities of the surrounding display objects is high, a large difference is unlikely to occur between the correlation value between the new image information in which the specific display object is replaced by another image and the multiple second image information, and the correlation value between the original first image information and the multiple second image information. For this reason, the information processing device 100A according to Embodiment 2 reduces the processing burden on the device that generates new image information by not acquiring new image information in which the specific display object is replaced by another image when the similarity between the feature quantities of the specific display object and the feature quantities of the surrounding display objects is high.

[0060] The hardware configuration of the information processing device 100A is the same as that of the information processing device 100 according to Embodiment 1, so its description will be omitted.

[0061] Next, with reference to Figures 8 and 9, the details of the processing performed by the information processing device 100A will be described. Figure 9 is a flowchart showing an example of the processing performed by the information processing device 100A according to Embodiment 2. Note that some of the processing performed by the information processing device 100A according to Embodiment 2 is the same as the processing performed by the information processing device 100 according to Embodiment 1, so the same processing as in Embodiment 1 is denoted by the same reference numerals as in Embodiment 1 and its description is omitted.

[0062] As shown in Figure 9, when the information processing device 100A starts processing, it first acquires a first dataset and a second dataset (step ST01). After performing the processing in step ST01, the information processing device 100A converts each first image information and each second image information into feature quantities (step ST02). After performing the processing in step ST02, the information processing device 100A calculates the correlation value between the first image information and the multiple second image information (step ST03). After performing the processing in step ST03, the information processing device 100A determines whether or not there is specific image information for which the correlation value is less than the first threshold (step ST04). If, in the processing of step ST04, there is specific image information for which the correlation value is less than the first threshold (YES in step ST04), the information processing device 100A determines whether or not the specific image information contains a specific display object (step ST05).

[0063] In step ST05, if the specific image information contains a specific display object (YES in step ST05), the information processing device 100A calculates the similarity between the feature quantities of the partial image information of the specific display object and the feature quantities of the partial image information surrounding the specific display object (step ST06). In this process, based on the fact that the specific image information contained a specific display object in step ST05, the information processing device 100A uses a similarity calculation unit 108 to calculate the similarity between the feature quantities of the partial image information showing the partial image of the display range of the specific display object within the specific image shown by the specific image information and the feature quantities of the partial image information showing the partial image surrounding the specific display object.

[0064] When the information processing device 100A performs the processing in step ST06, it determines whether the similarity is less than the fourth threshold (step ST07). In this process, the information processing device 100A compares the similarity calculated in step ST06 with a preset fourth threshold to determine whether the similarity between the feature quantities of the partial image information showing a partial image of a specific display object and the feature quantities of the partial image information showing a partial image of the area surrounding the specific display object is less than the fourth threshold.

[0065] In step ST07, if the similarity is less than the fourth threshold (YES in step ST07), the information processing device 100A acquires new image information in which the specific display of the specific image information is replaced by another image (step ST10). After performing the processing in step ST10, the information processing device 100A calculates a correlation value between the new image information and the multiple second image information (step ST11). After performing the processing in step ST11, the information processing device 100A determines whether the correlation value is less than the third threshold (step ST12). In the processing in step ST12, if the correlation value is not less than the third threshold (NO in step ST12), the information processing device 100A decides not to exclude the specific image information from the first dataset (step ST13).

[0066] If, in step ST05, the specific image information does not contain the specific display object (NO in step ST05), if, in step ST07, the similarity is not less than the fourth threshold (NO in step ST07), or if, in step ST12, the correlation value is less than the third threshold (YES in step ST12), the information processing device 100A removes the specific image information from the first dataset (step ST14).

[0067] If the similarity calculated in step ST06 is not below the fourth threshold, the information processing device 100A decides to exclude the first image information from the first dataset without acquiring new image information, thereby suppressing the acquisition of new image information by the image information acquisition unit 111A.

[0068] The information processing device 100A terminates processing if, in the processing of step ST04, there is no specific image information whose correlation value is less than the first threshold (NO in step ST04), if the processing of step ST13 has been performed, or if the processing of step ST14 has been performed.

[0069] As described above, the information processing device 100A according to Embodiment 2 includes a similarity calculation unit 108 that calculates the similarity between the feature quantity of partial image information of the display range in a specific image of the specific display and the feature quantity of partial image information surrounding the display range in the specific image of the specific display, based on the detection of a specific display by the display object detection unit 107, and an image information acquisition unit that acquires new image information in which the display range in the specific image of the specific display has been replaced by another image, based on the detection of a specific display by the display object detection unit 107 and the similarity calculated by the similarity calculation unit 108 being less than a preset fourth threshold.

[0070] With this configuration, the information processing device 100A can reduce the processing burden on the device that generates new image information by not acquiring new image information in which the specific display object is replaced by another image when the similarity between the feature quantities of the specific display object and the feature quantities of the surrounding display objects is high.

[0071] Embodiment 3. Next, the information processing system 1B according to Embodiment 3 will be described with reference to Figures 10 to 12. The information processing system 1B according to Embodiment 3 differs from the information processing system 1A according to Embodiment 2 in its configuration related to the conditions for the image information acquisition unit to acquire new image information, but other configurations are the same, and the same names and reference numerals as in Embodiment 2 will be used and their descriptions will be omitted.

[0072] Figure 10 is a block diagram showing the schematic configuration of the information processing system 1B according to Embodiment 3. As shown in Figure 10, the information processing system 1B according to Embodiment 3 comprises a database DB1 and an information processing device 100B, which are connected wirelessly or via wired connections so that they can communicate with each other. The database DB1 and the information processing device 100B may also be connected to each other via devices or communication lines not shown, so that they can communicate information with each other.

[0073] The information processing device 100B includes an input unit 101, an output unit 102, a data acquisition unit 103, a feature quantity conversion unit 104, a correlation value calculation unit 105, a correlation determination unit 106, a display object detection unit 107, a similarity calculation unit 108, a location information acquisition unit 109, a location determination unit 110, an image information acquisition unit 111B, and a data determination unit 112.

[0074] The location information acquisition unit 109 learns from a trained model that outputs location information indicating the position of a characteristic sub-image within a given image based on the input of multiple image information, each representing a different image, and information indicating the position of a characteristic sub-image contained in each image within that image. For example, the location information acquisition unit 109 learns from a trained model that outputs location information indicating the position of a characteristic sub-image within a given image based on the input of image information. Specifically, the location information acquisition unit 109 learns based on the input of multiple image information, each representing a different image, and information indicating the location within each image of a sub-image that humans perceive as unnatural or unnatural. Based on the input of image information representing an image, the trained model outputs location information indicating the location of a sub-image that humans perceive as unnatural within the input image. From this trained model, the unit acquires location information indicating the location of a sub-image that humans perceive as unnatural within a specific image.

[0075] For example, such a trained model is generated by collecting positional information for multiple images in response to an operator's operation of selecting the location of a partial image that they perceive as distinctive while viewing an image displayed on a display device, and then training the model based on the input of image information representing these multiple images and the collected positional information. For example, the positional information acquisition unit 109 acquires positional information indicating the location of a distinctive partial image within an image by acquiring a numerical value indicating the degree to which the image is distinctive for each pixel of the image. The positional information acquisition unit 109 may have a trained model that outputs such positional information, or it may be configured to acquire positional information which is the output of a trained model possessed by an external device by outputting specific image information to an external device that has a trained model that outputs such positional information. Furthermore, for example, a trained model that outputs such positional information is disclosed in Non-Patent Document 1 below.

[0076] Youwei Liang, Junfeng He, Gang Li, Peizhao, Arseniy Klimovskiy, Nicholas Carolan,Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, Junjie Ke, Krishnamurthy Dj Dvijotham, Katherine M. Collins, Yiwen Luo, Yang Li, Kai J Kohlhoff, Deepak Ramachandran, and Vidhya Navalpakkam, "Rich Human Feedback for Text-to-Image Generation", 9 Apr 2024, arXiv:2312.10240v2.

[0077] The position determination unit 110 determines whether the display range in a specific image of a specific display object overlaps with the position indicated by the position information acquired by the position information acquisition unit 109. For example, the position determination unit 110 determines whether the display range in a specific image of a specific display object overlaps with the position of a partial image that is determined to be a characteristic partial image within the image. Also, for example, the position determination unit 110 determines whether the display range in a specific image of a specific display object overlaps with the position of a pixel in the image whose numerical value indicating the degree of characteristicness is above a preset fifth threshold.

[0078] The image information acquisition unit 111B acquires new image information in which the display range of the specific image of the specific display object is replaced by another image, based on the detection of a specific display object by the position determination unit and the determination by the position determination unit that the display range of the specific image of the specific display object overlaps with the position indicated by the position information. For example, the image information acquisition unit 111B acquires new image information in which the display range of the specific image of the specific display object is replaced by another image, based on the detection of a specific display object by the position determination unit, the determination that the similarity calculated by the similarity calculation unit 108 is less than a preset fourth threshold, and the determination by the position determination unit that the display range of the specific image of the specific display object overlaps with the position indicated by the position information. In this way, the information processing device 100B reduces the processing burden on the device that generates new image information by not acquiring new image information in which the specific display object is replaced by another image when the display range of the specific image of the specific display object overlaps with a characteristic partial image within the specific image, in particular a partial image that a human would find unnatural.

[0079] The hardware configuration of the information processing device 100B is the same as that of the information processing device 100 according to Embodiment 1, so its description will be omitted.

[0080] Next, with reference to Figures 8 and 9, the details of the processing performed by the information processing device 100B will be described. Figure 9 is a flowchart showing an example of the processing performed by the information processing device 100B according to Embodiment 2. Note that some of the processing performed by the information processing device 100B according to Embodiment 2 is the same as the processing performed by the information processing device 100A according to Embodiment 2, so the same processing as in Embodiment 2 is denoted by the same reference numerals as in Embodiment 2 and its description is omitted.

[0081] As shown in Figure 9, when the information processing device 100B starts processing, it first acquires a first dataset and a second dataset (step ST01). After performing the processing in step ST01, the information processing device 100B converts each first image information and each second image information into feature quantities (step ST02). After performing the processing in step ST02, the information processing device 100B calculates the correlation value between the first image information and the multiple second image information (step ST03). After performing the processing in step ST03, the information processing device 100B determines whether or not there is specific image information for which the correlation value is less than the first threshold (step ST04). If, in the processing of step ST04, there is specific image information for which the correlation value is less than the first threshold (YES in step ST04), the information processing device 100B determines whether or not the specific image information contains a specific display object (step ST05).

[0082] In step ST05, if the specific image information contains a specific display object (YES in step ST05), the information processing device 100B calculates the similarity between the feature quantities of the partial image information of the specific display object and the feature quantities of the partial image information surrounding the specific display object (step ST06). After performing the processing in step ST06, the information processing device 100B determines whether the similarity is less than the fourth threshold (step ST07).

[0083] In step ST07, if the similarity is less than the fourth threshold (YES in step ST07), the information processing device 100B acquires location information of a characteristic partial image within the specific image (step ST08). In this process, the information processing device 100B acquires location information of a characteristic partial image within the specific image from a trained model, which is trained based on inputs of multiple image information showing different images and information indicating the position of a characteristic partial image contained in each image, and which outputs location information indicating the position of a characteristic partial image within the input image based on input image information showing an image. The location information acquisition unit 109 then acquires location information of a characteristic partial image within the specific image from this trained model.

[0084] Figure 12 shows location information acquired by the information processing device 100B according to Embodiment 3. Figure 12 shows a symbol P1 indicating the position of a characteristic partial image corresponding to the location information acquired by the location information acquisition unit 109.

[0085] When the information processing device 100B performs the processing in step ST08, it determines whether the display range in the specific image of the specific display object and the position indicated by the position information overlap (step ST09). In this process, the information processing device 100B uses the position determination unit 110 to determine whether the display range in the specific image of the specific display object and the position indicated by the position information overlap, based on the position information acquired in step ST08. For example, in the example shown in Figure 12, it is shown that the symbol P1, which indicates the position of a characteristic partial image corresponding to the position information acquired by the position information acquisition unit 109, overlaps with the specific display objects B1, B2, and B3.

[0086] In step ST09, if the display range of the specific display object in the specific image and the position indicated by the position information do not overlap (NO in step ST09), the information processing device 100B acquires new image information in which the specific display object in the specific image information is replaced by another image (step ST10). After performing the processing in step ST10, the information processing device 100B calculates a correlation value between the new image information and the multiple second image information (step ST11). After performing the processing in step ST11, the information processing device 100B determines whether the correlation value is less than the third threshold (step ST12). In the processing in step ST12, if the correlation value is not less than the third threshold (NO in step ST12), the information processing device 100B decides not to exclude the specific image information from the first dataset (step ST13).

[0087] In step ST05, if the specific image information does not contain the specific display object (step ST05 NO), in step ST07, if the similarity is not less than the fourth threshold (step ST07 NO), in step ST09, if the display range of the specific display object in the specific image overlaps with the position indicated by the position information (step ST09 YES), and in step ST12, if the correlation value is less than the third threshold (step ST12 YES), the information processing device 100B removes the specific image information from the first dataset (step ST14). For example, in step ST09, if the display range of all specific display objects in the specific image overlaps with the position indicated by the position information, the information processing device 100B removes the specific image information from the first dataset. Furthermore, for example, in the processing of step ST09, if there are specific display objects in a specific image whose display range and the position indicated by the position information overlap, and specific display objects whose display ranges do not overlap, the information processing device 100B acquires new image information in which only the display ranges of the specific display objects that do not overlap are replaced with other images. Also, for example, in the processing of step ST09, if the display range of the specific display object with the largest display size in a specific image overlaps with the position indicated by the position information, the information processing device 100B excludes the specific image information from the first dataset. If the display range of the specific display object with the largest display size in a specific image does not overlap with the position indicated by the position information, the information processing device 100B acquires new image information in which the display ranges of each specific display object are replaced with other images.

[0088] If the similarity calculated in step ST06 is not less than the fourth threshold, the information processing device 100B decides to exclude the first image information from the first dataset without acquiring new image information, thereby suppressing the acquisition of new image information by the image information acquisition unit 111A.

[0089] The information processing device 100B terminates processing if, in the processing of step ST04, there is no specific image information whose correlation value is less than the first threshold (NO in step ST04), if the processing of step ST13 has been performed, or if the processing of step ST14 has been performed.

[0090] As described above, the information processing device 100B according to Embodiment 3 includes: a location information acquisition unit 109 that acquires location information indicating the position of a characteristic partial image in a specific image from a trained model that learns based on inputs of a plurality of image information each showing different images and information indicating the position of a characteristic partial image contained in each image, and outputs location information indicating the position of a characteristic partial image in the input image based on input of image information showing an image; a location determination unit 110 that determines whether or not the display range in a specific image of a specific display object and the position indicated by the location information overlap; and an image information acquisition unit 111B that acquires new image information in which the display range in a specific image of a specific display object is replaced by another image, based on the detection of a specific display object by the display object detection unit 107 and the determination by the location determination unit 110 that the display range in a specific image of the specific display object and the position indicated by the location information overlap.

[0091] With this configuration, the information processing device 100B reduces the processing burden on the device that generates new image information by preventing the acquisition of new image information in which the specific display object is replaced by another image when the display range of the specific image of the specific display object and a characteristic partial image within the specific image, particularly a partial image that a human would find unnatural, overlap.

[0092] In any of the embodiments described above, the information processing device may include some or all of the functions of devices other than the information processing device described above, some of the components of the information processing device may be provided in other devices that are communicatively connected to the information processing device, or the functions of the information processing device may be performed by a plurality of devices that are formed independently of each other.

[0093] Furthermore, this disclosure allows for free combination of each embodiment, modification of any component of each embodiment, or omission of any component in each embodiment.

[0094] The information processing device described herein can be used in a system for removing information unsuitable for machine learning from a dataset.

[0095] 1 Information processing system, 1A Information processing system, 1B Information processing system, 100 Information processing device, 100A Information processing device, 100B Information processing device, 100a Processor, 100b Memory, 100c I / O port, 100d Processing circuit, 101 Input unit, 102 Output unit, 103 Data acquisition unit, 104 Feature quantity conversion unit, 105 Correlation value calculation unit, 106 Correlation determination unit, 107 Display object detection unit, 108 Similarity calculation unit, 109 Location information acquisition unit, 110 Location determination unit, 111 Image information acquisition unit, 111A Image information acquisition unit, 111B Image information acquisition unit, 112 Data determination unit, B1 Specific display object, B2 Specific display object, B3 Specific display object, D1 Range, DB1 Database, H1 Display range, H2 Display range, H3 Display range.

Claims

1. A data acquisition unit that acquires a first dataset containing a plurality of first image information sets, each showing a different image from the others, and a second dataset containing a plurality of second image information sets, each showing a different image from the others; a correlation value calculation unit that calculates a correlation value indicating the correlation between each first image information set and the plurality of second image information sets based on the feature quantities of each first image information set and the feature quantities of each second image information set; a correlation determination unit that determines for each first image information set whether or not the correlation value calculated by the correlation value calculation unit is less than a preset first threshold; a display object detection unit that detects a specific display object in a specific image shown by the specific image information, whose display size is greater than or equal to a preset second threshold; an image information acquisition unit that acquires new image information in which the display range of the specific display object in the specific image is replaced by another image, based on the detection of the specific display object by the display object detection unit; and a data determination unit that determines whether or not to exclude the specific image information from the first dataset based on the determination result by the correlation determination unit. The information processing apparatus is characterized in that the correlation value calculation unit calculates a correlation value between the new image information and the plurality of second image information, and the data determination unit determines whether or not to exclude the specific image information from the first dataset based on the correlation value between the new image information and the plurality of second image information.

2. The information processing apparatus according to claim 1, characterized in that the correlation determination unit determines whether the correlation value between the new image information and the plurality of second image information is less than a preset third threshold, the data determination unit determines not to exclude the specific image information from the first dataset if the correlation value between the new image information and the plurality of second image information is not less than the third threshold, and determines to exclude the specific image information from the first dataset if the correlation value between the new image information and the plurality of second image information is less than the third threshold.

3. The information processing apparatus according to claim 1 or 2, characterized in that the display object detection unit detects the specific display object based on a segment pre-set in the specific image information.

4. The information processing apparatus according to any one of claims 1 to 3, characterized in that the image information acquisition unit acquires new image information in which the display range of the specific display in the specific image of the specific display is replaced by an image generated based on the partial image information surrounding the specific display in the specific image information, based on the detection of the specific display by the display object detection unit.

5. The information processing apparatus according to any one of claims 1 to 4, comprising a similarity calculation unit that calculates the similarity between the feature quantity of partial image information of the display range in the specific image of the specific display and the feature quantity of partial image information surrounding the display range in the specific image of the specific display, based on the detection of the specific display by the display object detection unit, wherein the image information acquisition unit acquires new image information in which the display range in the specific image of the specific display is replaced by another image, based on the detection of the specific display by the display object detection unit and the similarity calculated by the similarity calculation unit being less than a preset fourth threshold.

6. An information processing apparatus according to any one of claims 1 to 5, comprising: a position information acquisition unit that learns based on the input of a plurality of image information each showing different images and information indicating the position of a characteristic partial image contained in each image within the image, and that acquires position information indicating the position of a characteristic partial image within the input image based on the input of image information showing an image; and a position determination unit that determines whether or not the display range of the specific display object in the specific image and the position indicated by the position information overlap, wherein the image information acquisition unit acquires new image information in which the display range of the specific display object in the specific image is replaced by another image, based on the detection of the specific display object by the display object detection unit and the determination by the position determination unit that the display range of the specific display object in the specific image and the position indicated by the position information overlap.

7. A program for causing a computer to function as: a data acquisition unit that acquires a first dataset containing a plurality of first image information sets, each showing a different image; a second dataset containing a plurality of second image information sets, each showing a different image; a correlation value calculation unit that calculates a correlation value indicating the correlation between each first image information set and the plurality of second image information sets based on the feature quantities of each first image information set and the feature quantities of each second image information set; a correlation determination unit that determines for each first image information set whether the correlation value calculated by the correlation value calculation unit is less than a preset first threshold; a display object detection unit that detects a specific display object in a specific image shown by the specific image information, whose display size is greater than or equal to a preset second threshold; an image information acquisition unit that acquires new image information in which the display range of the specific display object in the specific image is replaced by another image, based on the detection of the specific display object by the display object detection unit; and a data determination unit that determines whether or not to exclude the specific image information from the first dataset based on the determination result by the correlation determination unit. The program is characterized in that the correlation value calculation unit calculates a correlation value between the new image information and the plurality of second image information, and the data determination unit determines whether or not to exclude the specific image information from the first dataset based on the correlation value between the new image information and the plurality of second image information.

8. An information processing method performed by an apparatus comprising a data acquisition unit, a correlation value calculation unit, a correlation determination unit, a display object detection unit, an image information acquisition unit, and a data determination unit, the method comprising: the data acquisition unit acquiring a first dataset containing a plurality of first image information sets, each showing a different image; and a second dataset containing a plurality of second image information sets, each showing a different image; the correlation value calculation unit calculating a correlation value indicating the correlation between each of the first image information sets and the plurality of second image information sets, based on the feature quantities of each first image information set and the feature quantities of each second image information set; the correlation determination unit determining for each first image information set whether the correlation value calculated by the correlation value calculation unit is less than a preset first threshold; and the display object detection unit detecting a specific display object in a specific image shown by the specific image information, whose display size is greater than or equal to a preset second threshold; Information processing method comprising: an image information acquisition unit acquiring new image information in which the display range of the specific image of the specific display is replaced by another image, based on the detection of the specific display by the display object detection unit; a data determination unit determining whether or not to exclude the specific image information from the first dataset based on the determination result by the correlation determination unit; a correlation value calculation unit calculating a correlation value between the new image information and the plurality of second image information; and a data determination unit determining whether or not to exclude the specific image information from the first dataset based on the correlation value between the new image information and the plurality of second image information.

Images