Information processing system, information processing method, and recording medium

Abstract

An information processing system (10) comprises: an input means (110) for inputting a first data set and a second data set to a learning model; an extraction means (120) for extracting the output of an intermediate process of the learning model; and an output means (130) for outputting the results of a comparison between the first data set and the second data set, in a space in which the dimensions of the output of the intermediate process have been compressed. This information processing system makes it possible to use the results from comparing data sets to efficiently collect data which is effective for machine learning.

Description

[[Technical Field]]

[0001] The present disclosure relates to the technical field of information processing systems, information processing methods, and recording media. [[Background Art]]

[0002] Systems of this type that collect learning data and perform training of machine learning models are known in the art. Patent Document 1 discloses a technique of acquiring microscope image data of tissue specimens and training a model for determining tissue types. Patent Document 2 discloses a technique of inputting a feature vector obtained from a neural network model into a classifier, and determining the weights of the classifier based on a comparison result between the output of the classifier and comparison data. Patent Document 3 discloses a technique of acquiring training images, performing training of a neural network, and constructing a neural network that outputs feature amounts capable of reconstructing the training images. [[Prior Art Literature]] [[Patent Literature]]

[0003] [[Patent Document 1]] International Publication No. WO 2021 / 132633 [[Patent Document 2]] International Publication No. WO 2021 / 019748 [[Patent Document 3]] Japanese Unexamined Patent Publication No. 2019-215698 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0004] An object of the present disclosure is to improve the technology disclosed in the prior art documents. [[Means for Solving the Problem]]

[0005] One aspect of the information processing system disclosed herein includes input means for inputting a first dataset and a second dataset into a learning model, extraction means for extracting the output of an intermediate processing of the learning model, and output means for outputting a comparison result between the first dataset and the second dataset in a space in which the dimensions of the output of the intermediate processing have been compressed.

[0006] One aspect of the information processing method disclosed herein involves inputting a first dataset and a second dataset into a learning model using at least one computer, extracting the output of the intermediate processing of the learning model, and outputting a comparison result between the first dataset and the second dataset in a space obtained by compressing the dimensions of the output of the intermediate processing.

[0007] One aspect of the recording medium of this disclosure includes a computer program recorded on at least one computer that causes the computer to execute an information processing method which involves inputting a first dataset and a second dataset into a learning model, extracting the output of the intermediate processing of the learning model, and outputting a comparison result between the first dataset and the second dataset in a space in which the dimensions of the output of the intermediate processing have been compressed. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the hardware configuration of the information processing system according to the first embodiment. [Figure 2] This is a block diagram showing the functional configuration of the information processing system according to the first embodiment. [Figure 3] This is a flowchart showing the operation flow of the information processing system according to the first embodiment. [Figure 4] This is a conceptual diagram showing the first and second datasets in the information processing system according to the second embodiment. [Figure 5] This is a flowchart showing the operation flow of the information processing system according to the fourth embodiment. [Figure 6] This is a flowchart showing the operation flow of the information processing system according to the fifth embodiment. [Figure 7] This block diagram shows the functional configuration of the information processing system according to the sixth embodiment. [Figure 8] This is a flowchart showing the flow of update and setting operations in the information processing system according to the sixth embodiment. [Figure 9] This is a flowchart showing the flow of the update setting operation in the information processing system according to the seventh embodiment. [Figure 10] This is a block diagram showing the functional configuration of the information processing system according to the eighth embodiment. [Figure 11] This is a conceptual diagram showing an example of image display operation by the information processing system according to the eighth embodiment. [Figure 12] This is a block diagram showing the functional configuration of the information processing system according to the ninth embodiment. [Figure 13] This is a conceptual diagram showing an example of graph display operation by the information processing system according to the 9th embodiment. [Figure 14] This is a block diagram showing the functional configuration of the information processing system according to the 10th embodiment. [Figure 15] This is a flowchart showing the operation flow of the information processing system according to the 10th embodiment. [Figure 16] This is a conceptual diagram illustrating the comparative operation of the information processing system according to the 11th embodiment. [Figure 17] This is a block diagram showing the functional configuration of the information processing system according to the 12th embodiment. [Figure 18] This is a conceptual diagram illustrating the learning operation by the information processing system according to the 12th embodiment. [Figure 19] This is a block diagram showing the functional configuration of the information processing system according to the 13th embodiment. [Figure 20] This is a conceptual diagram showing an example of generating a first dataset in the information processing system according to the 14th embodiment. [Figure 21] This is a conceptual diagram illustrating the comparative operation of the information processing system according to the 15th embodiment. [Modes for carrying out the invention]

[0009] Hereinafter, embodiments of an information processing system, an information processing method, and a recording medium will be described with reference to the drawings.

[0010] <First Embodiment> An information processing system according to a first embodiment will be described with reference to FIGS. 1 to 3.

[0011] (Hardware Configuration) First, the hardware configuration of the information processing system according to the first embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the hardware configuration of the information processing system according to the first embodiment.

[0012] As shown in FIG. 1, an information processing system 10 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing system 10 may further include an input device 15 and an output device 16. The processor 11, the RAM 12, the ROM 13, the storage device 14, the input device 15, and the output device 16 described above are connected via a data bus 17.

[0013] The processor 11 reads a computer program. For example, the processor 11 is configured to read a computer program stored in at least one of the RAM 12, ROM 13, and storage device 14. Alternatively, the processor 11 may read a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may also obtain (i.e., read) a computer program from a device (not shown) located outside the information processing system 10 via a network interface. By executing the read computer program, the processor 11 controls the RAM 12, storage device 14, input device 15, and output device 16. In this embodiment in particular, when the processor 11 executes the read computer program, a functional block for outputting the comparison results of the dataset is realized within the processor 11. That is, the processor 11 may function as a controller that executes each control in the information processing system 10.

[0014] The processor 11 may be configured as, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (field-programmable gate array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or a quantum processor. The processor 11 may consist of one of these, or it may be configured to use multiple of them in parallel.

[0015] RAM12 temporarily stores computer programs executed by processor 11. RAM12 also temporarily stores data that processor 11 uses temporarily while executing computer programs. RAM12 may be, for example, D-RAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). Alternatively, other types of volatile memory may be used instead of RAM12.

[0016] ROM 13 stores computer programs executed by processor 11. ROM 13 may also store other static data. ROM 13 may be, for example, P-ROM (Programmable Read Only Memory) or EPROM (Erasable Read Only Memory). Alternatively, other types of non-volatile memory may be used instead of ROM 13.

[0017] The storage device 14 stores data that the information processing system 10 will save for the long term. The storage device 14 may also operate as a temporary storage device for the processor 11. The storage device 14 may include, for example, at least one of a hard disk drive, a magneto-optical disk drive, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from the user of the information processing system 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be configured as a mobile terminal such as a smartphone or tablet. The input device 15 may also be a device capable of voice input, for example, including a microphone.

[0019] The output device 16 is a device that outputs information related to the information processing system 10 to the outside. For example, the output device 16 may be a display device (e.g., a display) capable of displaying information related to the information processing system 10. Alternatively, the output device 16 may be a speaker or the like capable of outputting information related to the information processing system 10 as sound. The output device 16 may be configured as a mobile terminal such as a smartphone or tablet. Furthermore, the output device 16 may be a device that outputs information in a format other than an image. For example, the output device 16 may be a speaker that outputs information related to the information processing system 10 as sound.

[0020] Although Figure 1 shows an example of an information processing system 10 comprising multiple devices, all or some of these functions may be implemented by a single device (information processing device). In that case, the information processing device may be configured to include only the processor 11, RAM 12, and ROM 13 described above, and the other components (i.e., storage device 14, input device 15, output device 16, and camera 18) may be provided by external devices connected to the information processing device. Furthermore, the information processing device may implement some of its computational functions through external devices (e.g., external servers or cloud services).

[0021] (Functional configuration) Next, the functional configuration of the information processing system 10 according to the first embodiment will be described with reference to Figure 2. Figure 2 is a block diagram showing the functional configuration of the information processing system according to the first embodiment.

[0022] As shown in Figure 2, the information processing system 10 according to the first embodiment is configured to include an input unit 110, an extraction unit 120, and an output unit 130 as components for realizing its functions. Each of the input unit 110, the extraction unit 120, and the output unit 130 may be a processing block realized by, for example, the processor 11 (see Figure 1) described above.

[0023] The input unit 110 is configured to accept the first dataset and the second dataset as input to the learning model. Here, the "first dataset" and the "second dataset" are datasets containing multiple data points, respectively. The first dataset and the second dataset may include, for example, image data such as face images. Specific examples of the first dataset and the second dataset will be described in detail in other embodiments described later. The "learning model" is a model configured to perform machine learning using the training data dataset, and is configured to include, for example, a neural network. The learning model may be configured, for example, as a model that detects objects from images. More specifically, the learning model may be configured as a model that detects faces from face images and detects facial feature points.

[0024] The extraction unit 120 is configured to extract the output of the intermediate processing of the learning model that has been input with the first dataset and the second dataset. The output of the intermediate processing may be, for example, the output of an intermediate layer among the multiple layers that the learning model has. The multiple layers that the learning model has may be the input layer, intermediate layer, and output layer of a multi-layered neural network. The input layer is the layer into which information is first input. The intermediate layer is a layer that performs various calculations on the information input to the input layer, and there may be multiple intermediate layers. The output layer is a layer that outputs the calculation results from the intermediate layer. The output of the intermediate processing of the learning model is the result of performing various calculations on the input first dataset and the second dataset, and may be, for example, a multi-dimensional value that shows the relationship between the first dataset and the second dataset. Hereinafter, the output of the intermediate processing of the learning model may be referred to as "intermediate output" as appropriate. The intermediate output extracted by the extraction unit 120 is output to the output unit 130.

[0025] The output unit 130 is configured to compress the dimension of the intermediate output extracted by the extraction unit 120. Here, "dimensionality compression" refers to a process that reduces the dimension of the intermediate output to a level that can be understood by humans. The output unit 130 may, for example, compress the dimension of the intermediate output to two or three dimensions. The output unit 130 is further configured to output the comparison result between the first dataset and the second dataset based on the intermediate output extracted by the extraction unit 120. The output unit 130 outputs the comparison result between the first dataset and the second dataset in a dimension-compressed space. For example, if the dimension of the intermediate output is compressed to two dimensions, the output unit 130 may output the comparison result between the first dataset and the second dataset as a two-dimensional map. Alternatively, if the dimension of the intermediate output is compressed to three dimensions, the output unit 130 may output the comparison result between the first dataset and the second dataset as a three-dimensional map. The output unit 130 may output the comparison result between the first dataset and the second dataset using, for example, the output device 16 described above. More specifically, the output unit 130 may output the comparison result between the first dataset and the second dataset as an image using the display provided by the output device 16.

[0026] (Flow of operations) Next, the operation flow of the information processing system 10 according to the first embodiment will be described with reference to Figure 3. Figure 3 is a flowchart showing the operation flow of the information processing system according to the first embodiment.

[0027] As shown in Figure 3, when the operation of the information processing system 10 according to the first embodiment is started, the input unit 110 first inputs the first dataset and the second dataset into the learning model (step S101). This starts the processing in the learning model (i.e., processing using the first dataset and the second dataset as input).

[0028] Next, the extraction unit 120 extracts the output of the intermediate processing of the learning model (step S102). Then, the output unit 130 compresses the dimensionality of the intermediate output extracted by the extraction unit 120 (step S103). After that, the output unit 130 outputs the comparison result between the first dataset and the second dataset in the dimensionally compressed space (step S104).

[0029] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the first embodiment will be described.

[0030] As explained in Figures 1 to 3, in the information processing system 10 according to the first embodiment, the comparison result between the first dataset and the second dataset is output based on the output of the intermediate processing when the first dataset and the second dataset are input to the learning model. In this way, the differences between the first dataset and the second dataset can be easily grasped. Note that the output of the intermediate processing of the learning model is usually a multidimensional value that is difficult for humans to intuitively understand, but in this embodiment, the comparison result is output in a space with compressed dimensions. This makes it possible to output the specific differences between the data contained in the first dataset and the data contained in the second dataset in a manner that is easy for humans to understand. Specific examples of the output manner will be explained in other embodiments described later.

[0031] <Second Embodiment> The information processing system 10 according to the second embodiment will be described with reference to Figure 4. The second embodiment describes specific examples of the first and second datasets described above, and other parts may be the same as those of the first embodiment. For this reason, the following will describe in detail the parts that differ from the first embodiment already described, and will omit explanations of other overlapping parts as appropriate.

[0032] (Dataset) First, with reference to Figure 4, the first and second data sets input to the information processing system 10 according to the second embodiment will be described in detail. Figure 4 is a conceptual diagram showing the first and second data sets in the information processing system according to the second embodiment.

[0033] As shown in Figure 4, the first dataset in the information processing system 10 according to the second embodiment is the target dataset when operating the learning model. Here, "when operating" refers to the process of operating the trained learning model after it has been trained using machine learning. For example, if the learning model is used as a model to detect human faces from image data, the first dataset may contain facial image data of various people.

[0034] On the other hand, the second dataset is a dataset whose data distribution is intended to be closer to that of the target first dataset. The second dataset may, for example, have insufficient data compared to the first dataset. In this case, it is possible to bring the data distribution closer to that of the first dataset by adding the missing data to the second dataset. Alternatively, the second dataset may have an excess of data compared to the first dataset. In this case, it is possible to bring the data distribution closer to that of the first dataset by removing the excess data from the second dataset.

[0035] A more specific example of the first dataset is a publicly available dataset. For example, the first dataset could be an ideal dataset publicly available on the internet (i.e., a dataset containing a sufficient amount and variety of data when considering the operation of a learning model). On the other hand, a more specific example of the second dataset is a company-owned dataset. For example, the second dataset could be an incomplete dataset collected by the user themselves (i.e., a dataset with missing or excess data when considering the operation of a learning model).

[0036] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the second embodiment will be described.

[0037] As explained in Figure 4, in the information processing system 10 according to the second embodiment, the first dataset is the target dataset, and the second dataset is a dataset intended to approximate the target. By comparing the first and second datasets, it is possible to identify the data that is missing or excessive in the second dataset compared to the first dataset. Therefore, it becomes possible to specifically understand how to modify the second dataset to approximate the first dataset.

[0038] <Third Embodiment> The information processing system 10 according to the third embodiment will now be described. The third embodiment describes a specific example of a dimensionality reduction method for the output of intermediate processing, and other parts may be the same as those of the first and second embodiments. For this reason, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0039] (Dimensionality reduction techniques) In the information processing system 10 according to the third embodiment, the output unit 130 uses at least one of PCA (Principal Component Analysis), t-SNE (t-Distributed Stochastic Neighbor Embedding), and UMAP (Uniform Manifold Approximation and Projection) as a method for compressing the dimensionality of the intermediate output. That is, the output unit 130 may compress the dimensionality using any one of PCA, t-SNE, or UMAP, or it may compress the dimensionality using a combination of multiple methods from PCA, t-SNE, and UMAP. In addition to the above-mentioned PCA, t-SNE, and UMAP, the output unit 130 may also compress the dimensionality using other methods.

[0040] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the third embodiment will be described.

[0041] In the information processing system 10 according to the third embodiment, the dimensionality of the output of the intermediate processing is compressed using at least one of the following methods: PCA, t-SNE, and UMAP. Using these methods makes it possible to compress the dimensionality more effectively.

[0042] <Fourth Embodiment> The information processing system 10 according to the fourth embodiment will be described with reference to Figure 5. Note that the fourth embodiment differs from the first to third embodiments described above only in some operations; other parts may be identical to those of the first to third embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, while other overlapping parts will be omitted as appropriate.

[0043] (Flow of operations) First, the operation flow of the information processing system 10 according to the fourth embodiment will be explained with reference to Figure 5. Figure 5 is a flowchart showing the operation flow of the information processing system according to the fourth embodiment. Note that in Figure 5, the same reference numerals are used for the same processes as those described in Figure 3.

[0044] As shown in Figure 5, when the operation of the information processing system 10 according to the fourth embodiment begins, the input unit 110 first inputs the first dataset and the second dataset into the learning model (step S101). Then, the extraction unit 120 extracts the output of the intermediate processing of the learning model (step S102). After that, the output unit 130 compresses the dimensionality of the intermediate output extracted by the extraction unit 120 (step S103).

[0045] Next, the output unit 130 compares the first dataset and the second dataset and extracts at least one of the deficient area and the surplus area from the comparison result (step S401). Here, the "deficient area" is the area corresponding to the deficiency of the second dataset relative to the first dataset. In other words, the deficient area is the area that shows the data that is missing in the second dataset. The "surplus area" is the area corresponding to the surplus of the second dataset relative to the first dataset. In other words, the surplus area is the area that shows the data that is in excess in the second dataset.

[0046] Next, the output unit 130 outputs the comparison result between the first dataset and the second dataset, including information about at least one of the extracted deficient and surplus regions (step S402). The output unit 130 outputs the deficient and surplus regions in a dimensionally compressed space. For example, if the output unit 130 outputs the comparison result between the first dataset and the second dataset as a two-dimensional map, the deficient and surplus regions may be output as two-dimensional regions. Alternatively, if the output unit 130 outputs the comparison result between the first dataset and the second dataset as a three-dimensional map, the deficient and surplus regions may be output as three-dimensional regions.

[0047] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the fourth embodiment will be described.

[0048] As explained in Figure 5, the information processing system 10 according to the fourth embodiment outputs at least one of the missing or excess areas of the second dataset relative to the first dataset. This makes it easy to identify missing or excess data in the second dataset.

[0049] <Fifth Embodiment> The information processing system 10 according to the fifth embodiment will be described with reference to Figure 6. Note that the fifth embodiment differs from the fourth embodiment described above only in some operations; other parts may be the same as those of the first to fourth embodiments. Therefore, the following will describe in detail the parts that differ from each embodiment already described, and will omit explanations of other overlapping parts as appropriate.

[0050] (Flow of operations) First, the operation flow of the information processing system 10 according to the fifth embodiment will be explained with reference to Figure 6. Figure 6 is a flowchart showing the operation flow of the information processing system according to the fifth embodiment. Note that in Figure 6, the same reference numerals are used for the same processes as shown in Figure 5.

[0051] As shown in Figure 6, when the operation of the information processing system 10 according to the fifth embodiment begins, the input unit 110 first inputs the first dataset and the second dataset into the learning model (step S101). Then, the extraction unit 120 extracts the output of the intermediate processing of the learning model (step S102). After that, the output unit 130 compresses the dimensionality of the intermediate output extracted by the extraction unit 120 (step S103).

[0052] Next, the output unit 130 compares the first dataset and the second dataset and extracts at least one of the deficient region and the excess region from the comparison result (step S401).

[0053] Next, the output unit 130 outputs the comparison result between the first dataset and the second dataset, including information about at least one of the extracted deficient and surplus regions. In this embodiment, the output unit 130 outputs in a manner that emphasizes the deficient and surplus regions. For example, in a map showing the comparison result between the first dataset and the second dataset, the output unit 130 outputs the deficient and surplus regions so that they stand out more than the other parts. More specifically, the output unit 130 may display the deficient and surplus regions in a different color from the other regions. Alternatively, the output unit 130 may display the deficient and surplus regions surrounded by a frame of a conspicuous color. Alternatively, the output unit 130 may display the deficient and surplus regions by making them blink.

[0054] The above-described embodiment is merely an example, and the output unit 130 may be configured to emphasize the deficient and excess regions in other embodiments.

[0055] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the fifth embodiment will be described.

[0056] As explained in Figure 6, in the information processing system 10 according to the fifth embodiment, at least one of the missing or excess areas of the second dataset relative to the first dataset is highlighted and output. This makes it easier to identify missing or excess data in the second dataset.

[0057] <Sixth Embodiment> The information processing system 10 according to the sixth embodiment will be described with reference to Figures 7 and 8. Note that the sixth embodiment differs from the first to fifth embodiments described above only in some configurations and operations; other parts may be identical to the first to fifth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, while omitting explanations of other overlapping parts as appropriate.

[0058] (Functional configuration) First, the functional configuration of the information processing system 10 according to the sixth embodiment will be described with reference to Figure 7. Figure 7 is a block diagram showing the functional configuration of the information processing system according to the sixth embodiment. Note that in Figure 7, the same reference numerals are used for elements similar to those shown in Figure 2.

[0059] As shown in Figure 7, the information processing system 10 according to the sixth embodiment is configured to include an input unit 110, an extraction unit 120, an output unit 130, and an update unit 140 as components for realizing its functions. That is, the information processing system 10 according to the sixth embodiment further includes an update unit 140 in addition to the configuration of the first embodiment (see Figure 2). The update unit 140 may be, for example, a processing block realized by the processor 11 (see Figure 1) described above.

[0060] The update unit 140 is configured to add data to the second dataset based on the missing area output by the output unit 130. That is, the update unit 140 is configured to add data to the second dataset that corresponds to the missing portion of the second dataset relative to the first dataset. In addition, the update unit 140 is configured to delete data from the second dataset based on the excess area output by the output unit 130. That is, the update unit 140 is configured to delete data from the second dataset that corresponds to the excess portion of the second dataset relative to the first dataset.

[0061] (Update setting behavior) Next, the update setting operation (i.e., the operation of the update unit 140 described above) will be explained with reference to Figure 8. Figure 8 is a flowchart showing the flow of the update setting operation in the information processing system according to the sixth embodiment.

[0062] As shown in Figure 8, in the update setting operation by the information processing system 10 according to the sixth embodiment, the update unit 140 first determines whether or not there is insufficient space in the second dataset (step S601). That is, the update unit 140 determines whether or not there is insufficient data in the second dataset compared to the first dataset.

[0063] Subsequently, the update unit 140, if there is a shortage of space (step S601: YES), sets the system to add data corresponding to the shortage to the second dataset (step S602). When the system is set to add data to the second dataset, the update unit 140 determines, for example, whether the newly acquired data contains data corresponding to the shortage when new data is acquired. If the acquired data contains data corresponding to the shortage, it adds that data to the second dataset. Note that if there is no shortage of space in the second dataset (step S601: NO), the process in step S602 described above may be omitted.

[0064] Next, the update unit 140 determines whether or not there is excess space in the second dataset (step S603). That is, the update unit 140 determines whether or not there is excess data in the second dataset compared to the first dataset.

[0065] Subsequently, if there is excess space (step S603: YES), the update unit 140 sets the system to delete the data corresponding to the excess space from the second dataset (step S604). If the system is set to delete data from the second dataset, the update unit 140 may delete the data corresponding to the excess space immediately, or it may delete the data at a later time. For example, the update unit 140 may ask the user if it is OK to delete the data, and if the user agrees to delete the data, it may delete the data corresponding to the excess space from the second dataset. If there is no excess space in the second dataset (step S603: NO), the process in step S604 described above may be omitted.

[0066] In this example, the update unit 140 is shown as being capable of both adding and deleting data. However, the update unit 140 may be configured to perform only one of the two operations: adding or deleting data. For example, if the update unit 140 only adds data, only steps S601 and S602 described above may be executed. If the update unit 140 only deletes data, only steps S603 and S604 described above may be executed.

[0067] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the sixth embodiment will be described.

[0068] As explained in Figures 7 and 8, in the information processing system 10 according to the sixth embodiment, data is added to or deleted from the second dataset based on the outputted insufficient and excess areas. In this way, it is possible to efficiently bring the second dataset closer to the first dataset.

[0069] <Seventh Embodiment> The information processing system 10 according to the seventh embodiment will be described with reference to Figure 9. Note that the seventh embodiment differs from the sixth embodiment described above only in some operations; other parts may be the same as those of the first to sixth embodiments. Therefore, the following will describe in detail the parts that differ from each embodiment already described, and will omit explanations of other overlapping parts as appropriate.

[0070] (Update setting behavior) First, the update setting operation according to the seventh embodiment will be explained with reference to Figure 9. Figure 9 is a flowchart showing the flow of the update setting operation in the information processing system according to the seventh embodiment. Note that in Figure 9, the same reference numerals are used for the same processes as shown in Figure 8.

[0071] As shown in Figure 9, in the update setting operation by the information processing system 10 according to the seventh embodiment, the update unit 140 first determines whether or not there is insufficient space in the second data set (step S601). If there is no insufficient space in the second data set (step S601: NO), the subsequent steps S701 and S702 may be omitted.

[0072] If there are missing areas in the second dataset (step S601: YES), the update unit 140 obtains the density difference between the first dataset and the second dataset for each missing area (step S701). That is, the update unit 140 obtains the density difference between the first dataset and the second dataset for each of the multiple missing areas in the second dataset (in other words, how much data is missing). Note that if there is only one missing area in the second dataset, the density difference does not need to be obtained. In this case, the update unit 140 may be set to add the data of the missing area to the second dataset, as in the sixth embodiment described above (see step S602 in Figure 8).

[0073] The density difference in the missing region may be obtained, for example, based on a comparison between the first and second datasets. More specifically, the density difference may be, for example, the difference between the number of data points in the missing region of the first dataset and the number of data points in the second dataset. However, if there is a large difference between the total number of data points in the first dataset and the total number of data points in the second dataset, the difference may be calculated after performing a normalization process (i.e., after equalizing the total number of data points).

[0074] Subsequently, the update unit 140 is configured to add data to the region with the largest density difference among the multiple deficient regions (step S702). For example, the update unit 140 sets a priority order for each of the multiple deficient regions according to the density difference. Specifically, the update unit 140 sets the region with the largest density difference as priority 1, the region with the next largest density difference as priority 2, and so on. In this case, data will be added preferentially to the regions with higher priority (i.e., regions with the largest density differences).

[0075] Next, the update unit 140 determines whether or not there is excess space in the second dataset (step S603). If there is no excess space in the second dataset (step S603: NO), the subsequent steps S703 and S704 may be omitted.

[0076] If there is excess data in the second dataset (step S603: YES), the update unit 140 obtains the density difference between the second dataset and the first dataset for each excess data (step S703). That is, the update unit 140 obtains the density difference between the second dataset and the first dataset for each of the multiple excess data in the second dataset. Note that if there is only one excess data in the second dataset, the density difference does not need to be obtained. In this case, the update unit 140 may be set to delete the data of the excess data from the second dataset, similar to the sixth embodiment described above (see step S604 in Figure 8).

[0077] The density difference in the excess region may be obtained based on a comparison between the first and second datasets, similar to the density difference in the deficient region described above. More specifically, the density difference may be, for example, the difference between the number of data points in the first dataset in the excess region and the number of data points in the second dataset. However, if there is a large difference between the total number of data points in the first dataset and the total number of data points in the second dataset, normalization may be performed before calculating the difference.

[0078] Subsequently, the update unit 140 sets the data to be deleted prioritizing the region with the largest density difference among the multiple excess regions (step S704). For example, the update unit 140 sets a priority order for each of the multiple excess regions according to the density difference. Specifically, the update unit 140 sets the region with the largest density difference as priority 1, the region with the next largest density difference as priority 2, and so on. In this case, data will be deleted preferentially from the region with the highest priority (i.e., from the region with the largest density difference).

[0079] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the seventh embodiment will be described.

[0080] As explained in Figure 9, in the information processing system 10 according to the seventh embodiment, data is added to or deleted from the second dataset according to the difference in density between the first dataset and the second dataset. In this way, corrections are made starting from the parts where the data difference is largest, making it possible to bring the second dataset closer to the first dataset more efficiently.

[0081] <Eighth Embodiment> The information processing system 10 according to the eighth embodiment will be described with reference to Figures 10 and 11. Note that the eighth embodiment differs from the first to seventh embodiments described above only in some configurations and operations; other parts may be identical to the first to seventh embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, while omitting explanations of other overlapping parts as appropriate.

[0082] (Functional configuration) First, the functional configuration of the information processing system 10 according to the eighth embodiment will be described with reference to Figure 10. Figure 10 is a block diagram showing the functional configuration of the information processing system according to the eighth embodiment. Note that in Figure 10, the same reference numerals are used for elements similar to those shown in Figure 2.

[0083] As shown in Figure 10, the information processing system 10 according to the eighth embodiment is configured to include an input unit 110, an extraction unit 120, an output unit 130, and a first display unit 150 as components for realizing its functions. That is, the information processing system 10 according to the eighth embodiment further includes a first display unit 150 in addition to the configuration of the first embodiment (see Figure 2). The first display unit 150 may be, for example, the display provided in the output device 16 (see Figure 1) described above.

[0084] The first display unit 150 is configured to display the comparison results between the first dataset and the second dataset as multiple objects. For example, the first display unit 150 is configured to display each of the multiple data contained in the first dataset and the second dataset as multiple objects on a two-dimensional map. The multiple objects displayed by the first display unit 150 can be selected by the user. For example, when the user clicks on an object, the clicked object may be treated as the selected object. The first display unit 150 is configured to display the content of the data corresponding to the selected object. For example, if the first dataset and the second dataset contain image data, the first display unit 150 may display the image corresponding to the selected object.

[0085] (Image display operation) Next, with reference to Figure 11, the image display operation by the information processing system 10 according to the eighth embodiment (i.e., the operation by the first display unit 150) will be described. Figure 11 is a conceptual diagram showing an example of the image display operation by the information processing system according to the eighth embodiment.

[0086] In the example shown in Figure 11, the first display unit 150 displays the comparison results between the first dataset and the second dataset as a two-dimensional map. The multiple points (objects) displayed on the two-dimensional map correspond to each individual data point included in each dataset. Specifically, the lighter points in Figure 11 represent data included in the first dataset, while the darker points represent data included in the second dataset.

[0087] Multiple points displayed by the first display unit 150 are selectable by the user. When the user selects one point, the content of the data corresponding to the selected point is displayed. Figure 11 shows an example where an image corresponding to the selected object is displayed. Note that multiple objects may be selectable simultaneously. In this case, multiple images corresponding to the selected points may be displayed side by side.

[0088] Although this example describes a scenario where the first and second datasets contain image data, the first and second datasets may contain data other than image data. For example, if the first and second datasets contain text data, the text corresponding to the selected object may be displayed.

[0089] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the eighth embodiment will be described.

[0090] As explained in Figures 10 and 11, the information processing system 10 according to the eighth embodiment displays the data content corresponding to the selected object. In this way, the comparison results between the first dataset and the second dataset are displayed as multiple objects, and it becomes possible to confirm what kind of data each object specifically corresponds to. Therefore, for example, it is possible to check what kind of data is missing or excessive in the second dataset.

[0091] <Ninth Embodiment> The information processing system 10 according to the ninth embodiment will be described with reference to Figures 12 and 13. Note that the ninth embodiment differs from the first to eighth embodiments described above only in some configurations and operations; other parts may be identical to the first to eighth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, while omitting explanations of other overlapping parts as appropriate.

[0092] (Functional configuration) First, the functional configuration of the information processing system 10 according to the ninth embodiment will be described with reference to Figure 12. Figure 12 is a block diagram showing the functional configuration of the information processing system according to the ninth embodiment. Note that in Figure 12, the same reference numerals are used for elements similar to those shown in Figure 2.

[0093] As shown in Figure 15, the information processing system 10 according to the ninth embodiment is configured to include an input unit 110, an extraction unit 120, an output unit 130, and a second display unit 160 as components for realizing its functions. That is, the information processing system 10 according to the ninth embodiment further includes a second display unit 160 in addition to the configuration of the first embodiment (see Figure 2). The second display unit 160 may be, for example, the display provided in the output device 16 (see Figure 1) described above.

[0094] The second display unit 160 is configured to display the comparison results between the first dataset and the second dataset in a manner that allows for region selection. Here, "region selection" means selecting a desired region from the comparison results between the first dataset and the second dataset. For example, if the comparison results between the first dataset and the second dataset are output as a two-dimensional map, the region specified by the user operation may be treated as the selected region. More specifically, if the user performs an operation to click on multiple locations on the two-dimensional map, the region within the frame connecting the multiple clicked locations may be treated as the selected region.

[0095] The second display unit 160 is configured to graph and display the proportion of data included in the selected area by attribute. For example, if the first dataset and the second dataset include image data, the second display unit 160 graphs and displays the proportion of multiple image data included in the selected area by image attribute. The image attributes may be determined, for example, by the objects included in the image. Specifically, the image attributes may be attributes indicating that the image contains many human faces, attributes indicating that the image contains many animals, or attributes indicating that the image contains many buildings. Alternatively, the image attributes may indicate the orientation of the faces of the people included in the image. For example, the image attributes may be attributes indicating that the image includes faces facing right, attributes indicating that the image includes faces facing left, or attributes indicating that the image includes faces facing forward.

[0096] Furthermore, the second display unit 160 may be configured to have the same functions as the first display unit 150 in the eighth embodiment described above. That is, in addition to the function of graphing and displaying the proportion of data included in the selected area, the second display unit 160 may also have the function of displaying the contents of the selected object.

[0097] (Graph display operation) Next, with reference to Figure 13, the graph display operation by the information processing system 10 according to the ninth embodiment (i.e., the operation by the second display unit 160) will be described. Figure 13 is a conceptual diagram showing an example of the graph display operation by the information processing system according to the ninth embodiment.

[0098] In the example shown in Figure 13, the second display unit 160 displays the comparison results between the first dataset and the second dataset as a two-dimensional map. As with the eighth embodiment described above, the multiple points displayed on the two-dimensional map correspond to each individual data point included in each dataset. Specifically, the lighter points in Figure 13 represent data included in the first dataset, while the darker points represent data included in the second dataset.

[0099] The two-dimensional map displayed by the second display unit 160 allows the user to select an area. When the user selects an area, the proportion of data contained in the selected area (the area circled in the figure) is displayed as a graph by attribute. In the example in Figure 13, the proportion of data contained in the area is displayed as a pie chart. The second display unit 160 may also display the proportion of data by attribute using a graph other than a pie chart. Alternatively, the second display unit 160 may display the proportion of data numerically. For example, the second display unit 160 may display a score corresponding to the proportion of data for each attribute.

[0100] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the ninth embodiment will be described.

[0101] As explained in Figures 12 and 13, in the information processing system 10 according to the ninth embodiment, the proportion of data contained in the selected area is displayed graphically by attribute. This makes it easy to understand what attributes of data are included in the selected area. Therefore, for example, it is possible to check what attributes of data are missing or excessive in the second dataset.

[0102] <Tenth Embodiment> The information processing system 10 according to the 10th embodiment will be described with reference to Figures 14 and 15. Note that the 10th embodiment differs from the first to 9th embodiments described above only in some configurations and operations; other parts may be identical to the first to 9th embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, while omitting explanations of other overlapping parts as appropriate.

[0103] (Functional configuration) First, the functional configuration of the information processing system 10 according to the 10th embodiment will be described with reference to Figure 14. Figure 14 is a block diagram showing the functional configuration of the information processing system according to the 10th embodiment. Note that in Figure 14, the same reference numerals are used for elements as in the components shown in Figure 2.

[0104] As shown in Figure 14, the information processing system 10 according to the tenth embodiment is configured to include an input unit 110, an extraction unit 120, an output unit 130, and an adjustment unit 170 as components for realizing its functions. That is, the information processing system 10 according to the tenth embodiment further includes an adjustment unit 170 in addition to the configuration of the first embodiment (see Figure 2). The adjustment unit 170 may be, for example, a processing block realized by the processor 11 (see Figure 1) described above.

[0105] The input unit 110 according to the tenth embodiment is configured to accept a third data set in addition to the first and second data sets. The output unit 130 is configured to output the comparison results of the first, second, and third data sets.

[0106] The adjustment unit 170 is configured to perform adjustments to compensate for any excesses or deficiencies in the first, second, and third datasets based on the comparison results. Specifically, it may add missing data to datasets that are lacking, and delete excess data from datasets that are excessive. Alternatively, the adjustment unit 170 may move data between datasets. For example, the adjustment unit 170 may move data contained in the first dataset to the second or third dataset.

[0107] Furthermore, when setting a target dataset as described in the second embodiment, for example, the first dataset may be the target, and the second and third datasets may be datasets that aim to approach the target. In other words, multiple datasets that aim to approach the target may be set. Alternatively, the first and second datasets may be the target, and the third dataset may be a dataset that aims to approach the target. In other words, multiple target datasets may be set.

[0108] (Flow of operations) Next, the operation flow of the information processing system 10 according to the 10th embodiment will be described with reference to Figure 15. Figure 15 is a flowchart showing the operation flow of the information processing system according to the 10th embodiment.

[0109] As shown in Figure 15, when the operation of the information processing system 10 according to the 10th embodiment begins, the input unit 110 first inputs the first dataset, the second dataset, and the third dataset to the learning model (step S1001). Then, the extraction unit 120 extracts the output of the intermediate processing of the learning model (step S1002).

[0110] Next, the output unit 130 compresses the dimensionality of the intermediate output extracted by the extraction unit 120 (step S1003). After that, the output unit 130 outputs the comparison results of the first dataset, the second dataset, and the third dataset in the dimensionally compressed space (step S1004).

[0111] Next, the adjustment unit 170 adjusts the first to third datasets based on the comparison results so that they complement each other's excesses and deficiencies (step S1005). Note that the adjustment by the adjustment unit 170 may also be performed when new data is acquired. That is, when new data is acquired, the adjustment unit 170 may extract the data that is missing from each dataset from the acquired data and add the extracted data to the dataset being drawn.

[0112] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the tenth embodiment will be described.

[0113] As explained in Figures 14 and 15, in the information processing system 10 according to the 10th embodiment, adjustments are made to compensate for the excesses and deficiencies of each dataset based on the comparison results of the first dataset, the second dataset, and the third dataset. In this way, it is possible to adjust each of the three or more datasets to a more appropriate state. In this embodiment, an example was given in which three datasets (i.e., the first dataset, the second dataset, and the third dataset) are input to the learning model, but four or more datasets may also be input. In this case, adjustments may be made to compensate for the excesses and deficiencies of each dataset based on the comparison results of the four or more input datasets.

[0114] <Embodiment 11> The information processing system 10 according to the 11th embodiment will be described with reference to Figure 16. Note that the 11th embodiment differs from the 10th embodiment described above only in some operations; other parts may be the same as those of the 1st to 10th embodiments. Therefore, the following will describe in detail the parts that differ from each embodiment already described, and will omit explanations of other overlapping parts as appropriate.

[0115] (Dataset integration) First, we will describe the data set comparison operation by the information processing system 10 according to the 11th embodiment, referring to Figure 16. Figure 16 is a conceptual diagram showing the comparison operation by the information processing system according to the 11th embodiment.

[0116] As shown in Figure 16, the output unit 130 in the information processing system 10 according to the 11th embodiment first performs a process to integrate the first dataset and the second dataset (hereinafter referred to as "integration process" as appropriate). For example, the integration process may be a process of simply adding up the data contained in the first dataset and the data contained in the second dataset. Alternatively, the integration process may be a process of taking the average of the data contained in the first dataset and the data contained in the second dataset. Alternatively, the integration process may be a process of normalizing the first dataset and the second dataset by density comparison and then adding up the data contained in the first dataset and the data contained in the second dataset.

[0117] After performing the integration process, the output unit 130 outputs the integration result of the first and second datasets and the comparison result with the third dataset. That is, the output unit 130 outputs the integration result of two datasets and the comparison result with one dataset. If there are four or more datasets, the output unit 130 may output the comparison result of the integration results themselves. For example, the output unit 130 may output the comparison result of the integration result of the first and second datasets and the integration result of the third and fourth datasets.

[0118] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the 11th embodiment will be described.

[0119] As explained in Figure 16, in the information processing system 10 according to the 11th embodiment, the result of integrating the first and second datasets is compared with the third dataset. This makes it possible to efficiently compare three or more datasets. For example, the system can output the results of comparing the first and second datasets with the third dataset, which is intended to approximate them, using the first and second datasets as targets.

[0120] <Twelfth Embodiment> The information processing system 10 according to the 12th embodiment will be described with reference to Figures 17 and 18. Note that the 12th embodiment differs from the first to 11th embodiments described above only in some configurations and operations; other parts may be identical to the first to 11th embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, while omitting explanations of other overlapping parts as appropriate.

[0121] (Functional configuration) First, the functional configuration of the information processing system 10 according to the 12th embodiment will be described with reference to Figure 17. Figure 17 is a block diagram showing the functional configuration of the information processing system according to the 12th embodiment. Note that in Figure 17, the same reference numerals are used for elements similar to those shown in Figure 2.

[0122] As shown in Figure 17, the information processing system 10 according to the 12th embodiment is configured to include an input unit 110, an extraction unit 120, an output unit 130, and a learning unit 180 as components for realizing its functions. That is, the information processing system 10 according to the 12th embodiment further includes a learning unit 180 in addition to the configuration of the first embodiment (see Figure 2). The learning unit 180 may be, for example, a processing block realized by the processor 11 (see Figure 1) described above.

[0123] The first and second datasets in the 12th embodiment each contain 2D face images (i.e., two-dimensional face images). The second dataset is modified to approximate the first dataset based on the comparison results output by the output unit 130. That is, data is added to or deleted from the second dataset, targeting the first dataset. Such modification processing may be performed, for example, by the update unit 140 (see Figure 7) described above.

[0124] The learning unit 180 is configured to train a face recognition engine for use in face recognition using the modified second dataset as described above. More specifically, the learning unit 180 is configured to train a face recognition engine using multiple 2D face images included in the modified second dataset and 3D face images (i.e., three-dimensional face images) of people corresponding to the multiple 2D face images. The face recognition engine trained by the learning unit 180 may be an engine capable of performing face recognition using 2D face images and 3D face images. For example, the face recognition engine may be an engine that performs face recognition by matching 2D face images and 3D face images.

[0125] (Learning process) Next, with reference to Figure 18, the learning operation by the information processing system 10 according to the 12th embodiment (i.e., the operation of the learning unit 180) will be described. Figure 18 is a conceptual diagram showing the learning operation by the information processing system according to the 12th embodiment.

[0126] As shown in Figure 18, in the information processing system 10 according to the 12th embodiment, first, the second dataset is modified to approximate the first dataset based on the comparison result between the first dataset and the second dataset. For example, the missing data in the second dataset compared to the first dataset is added to the second dataset. Alternatively, the excess data in the second dataset compared to the first dataset is removed from the second dataset.

[0127] The corrected second dataset is input to the learning unit 180 as training data. In addition, the 3D face images corresponding to the 2D face images included in the second dataset are also input to the learning unit 180 as training data. The 2D face images and 3D face images may be input to the learning unit 180 linked to each other. For example, the 2D face image and its corresponding 3D face image may be input to the learning unit 180 in a way that makes it clear that they are images of the same person. The learning unit 180 uses the input 2D face images and 3D face images to train the face recognition engine.

[0128] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the 12th embodiment will be described.

[0129] As explained in Figures 17 and 18, in the information processing system 10 according to the 12th embodiment, the face recognition engine is trained using a second dataset containing 2D face images modified to approximate the first dataset, and 3D face images corresponding to the 2D face images included in the second dataset. In this way, a face recognition engine capable of authentication using both 2D and 3D face images can be appropriately trained.

[0130] <13th Embodiment> The information processing system 10 according to the 13th embodiment will be described with reference to Figure 19. Note that the 13th embodiment differs from the 12th embodiment described above only in some configurations and operations; other parts may be the same as those of the 1st to 12th embodiments. Therefore, the following will describe in detail the parts that differ from each embodiment already described, and will omit explanations of other overlapping parts as appropriate.

[0131] (Functional configuration) First, the functional configuration of the information processing system 10 according to the 13th embodiment will be described with reference to Figure 19. Figure 19 is a block diagram showing the functional configuration of the information processing system according to the 13th embodiment. Note that in Figure 19, the same reference numerals are used for elements similar to those shown in Figure 17.

[0132] As shown in Figure 19, the information processing system 10 according to the 13th embodiment is configured to include an input unit 110, an extraction unit 120, an output unit 130, a learning unit 180, and an authentication unit 190 as components for realizing its functions. That is, the information processing system 10 according to the 13th embodiment further includes an authentication unit 190 in addition to the configuration of the 12th embodiment (see Figure 17). The authentication unit 190 may be, for example, a processing block realized by the processor 11 (see Figure 1) described above.

[0133] The authentication unit 190 is configured to perform facial recognition using the facial recognition engine learned by the learning unit 180 described in the 12th embodiment. Specifically, the authentication unit 190 is configured to perform facial recognition by matching a 2D facial image obtained from the person to be authenticated with a pre-registered 3D facial image using the facial recognition engine. The 2D facial image may be obtained by taking a picture of the face of the person to be authenticated with a camera. The 3D facial image may be generated by taking pictures of the face of the person to be authenticated with a camera from multiple angles in advance. As existing technologies can be appropriately adopted for the authentication method using the 2D and 3D facial images, a detailed explanation is omitted here.

[0134] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the 13th embodiment will be described.

[0135] As explained in Figure 19, in the information processing system 10 according to the 13th embodiment, facial recognition is performed using a facial recognition engine trained with 2D facial images and 3D facial images. This makes it possible to achieve high recognition accuracy in facial recognition that matches 2D facial images with 3D facial images.

[0136] <14th Embodiment> The information processing system 10 according to the 14th embodiment will be described with reference to Figure 20. Note that the 14th embodiment differs from the 12th and 13th embodiments described above only in some operations; other parts may be identical to the 1st to 13th embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, while other overlapping parts will be omitted as appropriate.

[0137] (Generating the first dataset) First, the operation of generating the first dataset by the information processing system 10 according to the 14th embodiment will be explained with reference to Figure 20. Figure 20 is a conceptual diagram showing an example of generating the first dataset in the information processing system according to the 14th embodiment.

[0138] As shown in Figure 20, in the information processing system 10 according to the 14th embodiment, multiple 2D face images with different face orientation angles are generated by rotating a 3D face image. For example, a face image facing forward, a face image facing right, and a face image facing left are generated from a single 3D face image. In the 14th embodiment, the multiple 2D face images generated in this way are used as a first dataset. That is, the first dataset according to the 14th embodiment includes multiple 2D face images with different face orientation angles.

[0139] The first dataset described above is compared with a second dataset containing multiple 2D facial images. The output unit 130 then outputs information regarding facial orientation angles that are missing or excessive in the second dataset, as a result of the comparison between the first and second datasets.

[0140] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the 14th embodiment will be described.

[0141] As explained in Figure 20, in the information processing system 10 according to the 14th embodiment, the first dataset includes multiple 2D face images (i.e., 2D images with different face orientation angles) obtained by rotating a 3D face image. Using such a first dataset, it becomes possible to appropriately identify any missing or excessive face orientation angles in the second dataset.

[0142] <15th Embodiment> The information processing system 10 according to the 15th embodiment will be described with reference to Figure 21. Note that the 15th embodiment differs from the 12th to 14th embodiments described above only in some operations; other parts may be identical to the 1st to 14th embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, while omitting explanations of other overlapping parts as appropriate.

[0143] (Example of guide information output) First, the comparison operation of the data sets by the information processing system 10 according to the 15th embodiment will be explained with reference to Figure 28. Figure 21 is a conceptual diagram showing the comparison operation by the information processing system according to the 15th embodiment.

[0144] As shown in Figure 21, in the information processing system 10 according to the 15th embodiment, the first dataset includes multiple angle-specific datasets, each containing 2D face images categorized by face orientation angle. For example, the first dataset includes a right-facing image dataset containing right-facing face images, a front-facing image dataset containing front-facing face images, and a left-facing image dataset containing left-facing face images.

[0145] The first dataset described above is compared with a second dataset containing multiple 2D face images. In this case, the second dataset may be compared sequentially with each of the angle-specific datasets. For example, the process of comparing the right-facing image dataset with the second dataset, the process of comparing the front-facing image dataset with the second dataset, and the process of comparing the left-facing image dataset with the second dataset may be performed separately.

[0146] The output unit 130 then outputs information regarding face orientation angles that are missing or excessive in the second dataset, as a result of comparing the first dataset with the second dataset. For example, if a comparison of the right-facing image dataset with the second dataset reveals a deficiency in the second dataset, the output unit 130 may output a comparison result indicating that the second dataset is lacking right-facing face images. Similarly, if a comparison of the left-facing image dataset with the second dataset reveals an excess in the second dataset, the output unit 130 may output a comparison result indicating that the second dataset has an excess of left-facing face images.

[0147] (Technical effects) Next, the technical effects obtained by the information processing system 10 according to the 15th embodiment will be described.

[0148] As explained in Figure 21, in the information processing system 10 according to the 15th embodiment, the first dataset includes multiple angle-specific datasets, each containing 2D facial images collected according to facial orientation angle. Using such a first dataset, it becomes possible to appropriately identify any missing or excessive facial orientation angles in the second dataset.

[0149] The processing method of recording a program that operates the configuration of each embodiment in order to realize the functions of each embodiment described above on a recording medium, reading the program recorded on the recording medium as code, and executing it on a computer is also included in the scope of each embodiment. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, not only the recording medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment.

[0150] As recording media, for example, floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs can be used. Furthermore, the scope of each embodiment is not limited to programs that perform processing on the recording media alone, but also includes programs that operate on the OS and perform processing in cooperation with other software and the functions of expansion boards. In addition, the program itself may be stored on a server, and part or all of the program may be made available for download from the server to the user terminal. The program may be provided to the user in, for example, SaaS (Software as a Service) format.

[0151] <Note> The embodiments described above may also be described in the following appendix, but are not limited to these.

[0152] (Note 1) The information processing system described in Appendix 1 is an information processing system comprising: input means for inputting a first dataset and a second dataset into a learning model; extraction means for extracting the output of the intermediate processing of the learning model; and output means for outputting the comparison result between the first dataset and the second dataset in a space in which the dimensionality of the output of the intermediate processing has been compressed.

[0153] (Note 2) The information processing system described in Appendix 2 is the information processing system described in Appendix 1, wherein the first dataset is the target dataset when operating the learning model, and the second dataset is a dataset whose data distribution is intended to be similar to that of the first dataset.

[0154] (Note 3) The information processing system described in Appendix 3 is the information processing system described in Appendix 1 or 2, wherein the output means uses at least one of PCA, t-SNE, and UMAP to compress the dimensionality of the output of the intermediate processing.

[0155] (Note 4) The information processing system described in Appendix 4 is the information processing system described in any one of Appendix 1 to 3, wherein the output means compares the first dataset and the second dataset, and extracts and outputs at least one of the insufficient area and the excess area of ​​the second dataset relative to the first dataset.

[0156] (Note 5) The information processing system described in Appendix 5 is the information processing system described in Appendix 4, wherein the output means outputs the comparison result of the first dataset and the second dataset in a manner that highlights the portion corresponding to at least one of the deficient region and the excess region.

[0157] (Note 6) The information processing system described in Appendix 6 is the information processing system described in Appendix 4 or 5, further comprising data updating means for adding data to the second dataset based on the insufficient area and deleting data from the second dataset based on the excess area.

[0158] (Note 7) The information processing system described in Appendix 7 is the information processing system described in Appendix 6, wherein the data update means obtains the density difference between the first dataset and the second dataset, and adds or deletes data prioritizing the area with the largest difference.

[0159] (Note 8) The information processing system described in Appendix 8 is the information processing system described in any one of Appendix 1 to 7, further comprising a first display means that displays the comparison result between the first dataset and the second dataset as a plurality of objects, and displays the content of data corresponding to an object selected from the plurality of objects.

[0160] (Note 9) The information processing system described in Appendix 9 is the information processing system described in any one of Appendix 1 to 8, further comprising a second display means that displays the comparison results of the first dataset and the second dataset in a manner that allows for region selection, and displays the proportion of data included in the selected region in a graph by attribute.

[0161] (Note 10) The information processing system described in Appendix 10 is the information processing system described in any one of Appendix 1 to 9, wherein the input means inputs a third dataset in addition to the first dataset and the second dataset to the learning model, the output means outputs a comparison result of the first dataset, the second dataset and the third dataset in a space with compressed dimensions of the output of the intermediate processing, and the adjustment means further comprises adjusting the first dataset, the second dataset and the third dataset to complement each other's excesses and deficiencies based on the comparison result.

[0162] (Note 11) The information processing system described in Appendix 11 is the information processing system described in Appendix 10, wherein the output means outputs the excess or deficiency of the third dataset relative to the first dataset and the second dataset by comparing the result of integrating the first dataset and the second dataset with the third dataset.

[0163] (Note 12) The information processing system described in Appendix 12 is the information processing system described in any one of Appendix 2 to 11, wherein the first dataset and the second dataset each contain 2D facial images, and further comprises a learning means for learning a facial recognition engine to be used for facial recognition using the second dataset which has been modified to be closer to the first dataset based on the comparison results, and 3D facial images of people corresponding to the 2D facial images included in the second dataset.

[0164] (Note 13) The information processing system described in Appendix 13 is the same as the information processing system described in Appendix 12, further comprising an authentication means that uses the facial recognition engine learned by the learning means to compare a 2D facial image obtained from a target for authentication with a pre-registered 3D facial image to perform facial recognition.

[0165] (Note 14) The information processing system described in Appendix 14 is the information processing system described in Appendix 12 or 13, wherein the first dataset includes a plurality of 2D face images with different face orientation angles obtained by rotating a 3D face image, and the output means outputs the face orientation angle which is the excess or deficiency of the second dataset relative to the first dataset by comparing the first dataset and the second dataset.

[0166] (Note 15) The information processing system described in Appendix 15 is the information processing system described in Appendix 12 or 13, wherein the first dataset includes multiple angle-specific datasets that collect 2D face images according to face orientation angle, and the output means outputs the face orientation angle which is the excess or deficiency of the second dataset relative to the first dataset by comparing the first dataset and the second dataset.

[0167] (Note 16) The information processing method described in Appendix 16 is an information processing method which involves inputting a first dataset and a second dataset into a learning model using at least one computer, extracting the output of the learning model's intermediate processing, and outputting the comparison result between the first dataset and the second dataset in a space obtained by compressing the dimensionality of the output of the intermediate processing.

[0168] (Note 17) The recording medium described in Appendix 17 is a recording medium on which a computer program is recorded that causes at least one computer to execute an information processing method that inputs a first dataset and a second dataset into a learning model, extracts the output of the intermediate processing of the learning model, and outputs the comparison result between the first dataset and the second dataset in a space in which the dimensionality of the output of the intermediate processing has been compressed.

[0169] (Note 18) The computer program described in Appendix 18 is a computer program that causes at least one computer to execute an information processing method that inputs a first dataset and a second dataset into a learning model, extracts the output of the intermediate processing of the learning model, and outputs the comparison result between the first dataset and the second dataset in a space in which the dimensionality of the output of the intermediate processing has been compressed.

[0170] This disclosure may be modified as appropriate, without contradicting the gist or idea of ​​the invention as can be inferred from the claims and the specification as a whole, and information processing systems, information processing methods, and recording media that include such modifications are also included in the technical idea of ​​this disclosure. [Explanation of Symbols]

[0171] 10. Information Processing Systems 11 processors 110 Input Section 120 Extraction part 130 Output section 140 Update Department 150 1st display section 160 2nd display section 170 Adjustment section 180 Learning Department 190 Certification Department

Claims

1. An input means for inputting the first dataset and the second dataset into a learning model, Extraction means for extracting the output of the intermediate processing of the learning model, Output means for comparing the first dataset and the second dataset in a space obtained by compressing the dimensions of the output of the intermediate processing, and extracting and outputting at least one of the missing region and the excess region of the second dataset relative to the first dataset, A data update means that adds data to the second dataset based on the insufficient area and deletes data from the second dataset based on the excess area, An information processing system equipped with the following features.

2. The first dataset is the target dataset when operating the learning model, The second dataset is a dataset that attempts to approximate the data distribution of the first dataset. The information processing system according to claim 1.

3. The output means compresses the dimensionality of the output of the intermediate processing using at least one of PCA, t-SNE, and UMAP. The information processing system according to claim 1 or 2.

4. The output means outputs the comparison result of the first dataset and the second dataset in a manner that highlights the portion corresponding to at least one of the deficient region and the excess region. The information processing system according to claim 1 or 2.

5. The data update means obtains the difference in density between the first dataset and the second dataset, and adds or deletes data prioritizing the region with the largest difference. The information processing system according to claim 1 or 2.

6. The system further includes a first display means that displays the comparison results between the first dataset and the second dataset as a plurality of objects, and displays the data content corresponding to an object selected from the plurality of objects. The information processing system according to claim 1 or 2.

7. By at least one computer, The first and second datasets are input into the learning model. Extract the output of the intermediate processing of the aforementioned learning model, In the space obtained by compressing the dimensions of the output of the intermediate processing, the first dataset and the second dataset are compared, and at least one of the missing region and the excess region of the second dataset relative to the first dataset is extracted and output. Data is added to the second dataset based on the insufficient area, and data is deleted from the second dataset based on the excess area. Information processing methods.

8. On at least one computer, The first and second datasets are input into the learning model. Extract the output of the intermediate processing of the aforementioned learning model, In the space obtained by compressing the dimensions of the output of the intermediate processing, the first dataset and the second dataset are compared, and at least one of the missing region and the excess region of the second dataset relative to the first dataset is extracted and output. Data is added to the second dataset based on the insufficient area, and data is deleted from the second dataset based on the excess area. A computer program that executes information processing methods.

Images