IMAGE CLASSIFICATION DEVICE, IMAGE CLASSIFICATION METHOD, AND IMAGE CLASSIFICATION PROGRAM

Abstract

To provide an image classifying device capable of sufficiently classifying image data.SOLUTION: An arithmetic processing unit 14 repeatedly executes, until a predetermined completion condition is satisfied, a loop process A of: classifying multiple pieces of target image data into multiple clusters with the number of such clusters being limited to an upper limit that is a predetermined value; determining, for each cluster, whether or not the cluster is appropriate on the basis of the image data belonging to the cluster; and excluding, as classification-completed data, from the pieces of target image data, image data belonging to the appropriate cluster. When the completion condition is satisfied, a transmitting and receiving unit 11 outputs, as a classification result of classification of the target image data, a combination of the appropriate cluster and the classification completed data.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an image classification device, an image classification method, and an image classification program. [Background technology]

[0002] An algorithm called VBGMM (Variational Bayesian Gaussian Mixture) has been attracting attention as a clustering algorithm for classifying multiple image data into multiple clusters (groups) (see Patent Document 1). VBGMM has the characteristic that it is not necessary to specify the number of clusters in advance, since it can determine the number of clusters to be classified by itself according to the image data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-057236 A Summary of the Invention [Problem to be solved by the invention]

[0004] In VBGMM, the upper limit of the number of clusters can be specified, but even if the amount of image data is much larger than the upper limit of the number of clusters, the number of clusters actually classified may be significantly lower than the upper limit. For example, even if the upper limit of the number of clusters is specified as 1000 for a large amount of image data of more than 1 million items, the number of clusters actually classified is often about 100. This causes a problem that the number of clusters does not reach the desired number, and the image data cannot be classified sufficiently.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide an image classification device, an image classification method, and a program capable of sufficiently classifying image data. [Means for solving the problem]

[0006] An image classification device according to one aspect of the present disclosure is an image classification device that classifies image data, and has a calculation unit that classifies target image data, which is a plurality of image data to be classified, into a number of provisional clusters up to a predetermined value, and for each provisional cluster, determines whether the provisional cluster is an appropriate cluster based on the image data belonging to the provisional cluster, and removes the image data belonging to the appropriate cluster as classification completion data from the target image data, repeatedly performing this process until a predetermined termination condition is satisfied, and an output unit that, when the termination condition is satisfied, outputs a combination of the appropriate cluster and the classification completion data as a classification result for the target image data. Effect of the Invention

[0007] According to the present invention, it becomes possible to sufficiently classify image data. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an image classification system according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 illustrates an example of a functional configuration of an analysis server. [Diagram 3] 11 is a flowchart illustrating an example of a calculation process. [Figure 4] 11 is a flowchart illustrating an example of an image classification process. [Diagram 5] 13 is a flowchart illustrating an example of a classification result integration process. [Figure 6] FIG. 11 is a diagram illustrating an example of average image data. [Figure 7] FIG. 11 is a diagram illustrating an example of a process related to integration of clusters. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0010] Fig. 1 is a diagram showing a configuration of an image classification system according to an embodiment of the present disclosure. The image classification system 100 shown in Fig. 1 includes an analysis server 1, a data management server 2, and a user terminal 3. The analysis server 1 and the data management server 2 are connected to each other so as to be able to communicate with each other, and the data management server 2 and the user terminal 3 are connected to each other so as to be able to communicate with each other. The communication protocol connecting the analysis server 1, the data management server 2, and the user terminal 3 is not particularly limited, but may be, for example, HTTP (Hyper Text Transfer Protocol).

[0011] The analysis server 1 is an image classification device that classifies a plurality of image data into a plurality of clusters.

[0012] The data management server 2 is a server that exists between the analysis server 1 and the user terminal 3. The data management server 2 has, for example, a management function for managing the status of the analysis server 1, a transfer function for accepting various requests from the user terminal 3 and transferring them to the analysis server 1, and a transfer function for transferring response information to the requests from the analysis server 1 to the user terminal 3.

[0013] The user terminal 3 is a terminal device operated by a user who uses the image classification system 100. The user terminal 3 has an interface 31 for, for example, receiving requests from the user to the analysis server 1 and displaying response information from the analysis server 1. The interface 31 is, for example, a GUI (Graphical User Interface) or a CUI (Character User Interface).

[0014] 2 is a diagram showing an example of a functional configuration of the analysis server 1. The processes performed by each part of the analysis server 1 described below may be realized by a processor such as a CPU (Central Processing Unit) executing a computer program. The computer program may also be recorded on a recording medium 4 that stores data non-temporarily, such as a semiconductor memory, a magnetic disk, an optical disk, a magnetic tape, or a magneto-optical disk.

[0015] As shown in FIG. 1, the analysis server 1 includes a transmission / reception unit 11, a storage unit 12, a processing unit 13, and a calculation unit .

[0016] The transmitter / receiver 11 communicates with the user terminal 3 and has a function as an input unit that receives a classification request, which is a request to classify image data, from the user terminal 3, and a function as an output unit that transmits the classification results of the image data to the user terminal 3. In this embodiment, the classification request includes address information indicating the storage location of the target image data, which is the image data to be classified, and an upper limit value for the number of clusters into which the target image data is classified. The address information and the upper limit value can be specified by the user, for example, using the interface 31 of the user terminal 3.

[0017] The storage unit 12 stores various information and data used or generated in the processing performed by the analysis server 1. For example, the storage unit 12 stores target image data, classification results, and the like.

[0018] The processing unit 13 causes the calculation unit 14 to execute calculation processing in accordance with the classification request received by the transmission / reception unit 11, and outputs the classification result, which is the processing result of the calculation unit 14, from the transmission / reception unit 11.

[0019] The calculation unit 14 executes calculation processing according to the classification request received by the transmission / reception unit 11, and classifies the target image data into a plurality of clusters.

[0020] FIG. 3 is a flowchart for explaining an example of the calculation process performed by the calculation unit 14.

[0021] In the calculation process, the calculation unit 14 first acquires a plurality of target image data for which classification is requested in a classification request (step S101). For example, the calculation unit 14 acquires the target image data from a storage location indicated by address information included in the classification request. The storage location may be an external device such as a file server (not shown), or the data management server 2. The target image data may be transmitted from the user terminal 3 together with the classification request.

[0022] The calculation unit 14 calculates image feature amounts for each of the plurality of target image data (step S102). In this embodiment, the image feature amounts are vector amounts generated using a predetermined clustering model that classifies the target image data, which will be described later.

[0023] The calculation unit 14 executes an image classification process (see FIG. 4) for classifying each piece of target image data into a plurality of clusters based on the image feature amount of each piece of target image data (step S103).

[0024] The calculation unit 14 executes a classification result integration process (see FIG. 5) for organizing the classification results of the image classification process in step S103 (step S104).

[0025] The calculation unit 14 outputs the classification results organized in the classification result integration process (step S105), and ends the process. The classification results may be transmitted to the user terminal 3 by the transmission / reception unit 11, or may be stored in the storage unit 12 or the like.

[0026] FIG. 4 is a flowchart illustrating an example of the image classification process in step S103 of FIG.

[0027] In the image classification process, the calculation unit 14 executes loop process A in which the processes of steps S202 to S208 are repeated until a predetermined end condition is satisfied (step S201). In this embodiment, the end condition is that the ratio of the number of classified data, which is target image data for which classification has been completed, to the number of original target image data is equal to or greater than an end threshold. The end threshold may be determined in advance, or may be set by a user or administrator of the image classification system 100. The end threshold is not particularly limited, but is set to 90% here. The end condition may also be another condition, such as the number of times the loop has been performed (the number of times the processes of steps S202 to S208 have been performed) being equal to or greater than a threshold.

[0028] In loop process A, the calculation unit 14 classifies the target image data into a number of provisional clusters, the upper limit of which is a predetermined value, using a predetermined clustering model for classifying image data (step S202). At this time, the calculation unit 14 classifies the target image data, for example, until the number of target image data in any of the clusters becomes a number obtained by dividing the number of target image data by a specified upper limit value, which is the upper limit value of the number of clusters specified in the classification request.

[0029] The predetermined clustering model is a VBGMM model, but may be another machine learning model. The predetermined value is not particularly limited, but is preferably sufficiently smaller than the specified upper limit. For example, the specified upper limit is about 1000, and the predetermined value is 2 or 3.

[0030] The calculation unit 14 generates average image data of the target image data classified into each temporary cluster in step S302 (step S203). The average image data is image data having a pixel value obtained by dividing the sum of the pixel values ​​of each target image data by the number of target image data for each pixel position. In this embodiment, the target image data is all adjusted to the same resolution in step S102 of FIG. 3.

[0031] For each provisional cluster, the calculation unit 14 compares the minimum pixel value, which is the pixel value of the darkest pixel in the average image (the lowest pixel value in the average image), with the judgment threshold (step S204), and determines whether the minimum pixel value in all provisional clusters is less than or equal to the judgment threshold (step S205).

[0032] If there is a tentative cluster whose minimum pixel value exceeds the judgment threshold (step S205: No), the calculation unit 14 associates the tentative cluster whose minimum pixel value exceeds the judgment threshold with the target image data belonging to the tentative cluster, and adds them to the unclassified dictionary. The calculation unit 14 further determines the other tentative clusters as appropriate clusters, and adds the appropriate clusters to the classified dictionary, and associates them with the target image data belonging to the cluster (step S206).

[0033] On the other hand, if the minimum pixel value in all tentative clusters is below the threshold value (step S204: Yes), the calculation unit 14 determines that all tentative clusters are appropriate clusters, and associates the appropriate clusters with the target image data belonging to those clusters and adds them to the classification completed dictionary (step S207).

[0034] Thereafter, the calculation unit 14 adjusts the determination threshold (step S208). For example, the calculation unit 14 may increase the determination threshold each time the processes of steps S202 to S207 are repeated, or may increase the determination threshold as the ratio of the number of classified data, which is the target image data added to the classification completed dictionary (target image data for which classification has been completed), to the number of original target image data increases.

[0035] When the process of step S208 ends, if the termination condition is not satisfied, the calculation unit 14 returns to the process of step S202. In step S202, the calculation unit 14 removes the target image data from the classified data, and classifies the target image data added to the unclassified dictionary and the target image data that has not yet been classified. On the other hand, when the termination condition is satisfied, that is, when the ratio of the number of classified data to the number of original target image data becomes 90% or more, the calculation unit 14 exits from the loop process A (step S209).

[0036] Thereafter, the calculation unit 14 adds the target image data that is not registered in the classification complete dictionary to the classification complete dictionary (step S210) and ends the process. Note that in step S210, for the target image data registered in the classification incomplete dictionary, the calculation unit 14 adds the provisional cluster of the target image data to the classification complete dictionary as an appropriate cluster in association with the target image data. In addition, this classification complete dictionary becomes the classification result of the target image data.

[0037] FIG. 5 is a flowchart illustrating an example of the classification result integration process in step S104 of FIG.

[0038] In the classification result integration process, the calculation unit 14 executes a loop process B in which the processes of steps S302 to S304 are repeated for each cluster registered in the classification completed dictionary (step S301).

[0039] In loop process B, first, the calculation unit 14 selects one of the clusters registered in the classification completed dictionary that has not yet been selected. Then, the calculation unit 14 compares the selected average image data, which is the average image data of the target image data belonging to the selected cluster, with each of the reference average image data, which is the average image data of the target image data belonging to each of the other clusters, to calculate the similarity between the selected average image data and each average image data (step S302). The similarity is not particularly limited, but is, for example, a similarity based on a hash value (image hash) calculated from the average image using a predetermined hash function.

[0040] The calculation unit 14 determines whether or not there is a reference average image whose similarity is equal to or greater than the similarity threshold (step S303).

[0041] If there is a reference average image whose similarity is equal to or greater than the threshold (step S303: Yes), the calculation unit 14 generates a new cluster by integrating the selected cluster with the cluster corresponding to the reference average image whose similarity is equal to or greater than the threshold, and updates the classification completed dictionary (step S304).Then, the calculation unit 14 generates an average image of the target image data belonging to the new cluster (step S305).

[0042] On the other hand, if there is no reference average image whose similarity is equal to or greater than the similarity threshold value (step S303: No), the calculation unit 14 skips the processes of steps S304 and S305.

[0043] If all the clusters have not been selected, the calculation unit 14 returns to the process of step S302. In the process of step S302, the new cluster generated in step S304 is treated as a cluster that has not yet been selected. However, the new cluster may be treated as a cluster that has already been selected.

[0044] On the other hand, if all the clusters have been selected, the calculation unit 14 exits from the loop process B (step S306) and ends the classification result integration process.

[0045] FIG. 6 is a diagram for explaining an example of the average image data.

[0046] As shown in Fig. 6(a), when target image data 601-603 belonging to a provisional cluster are similar to each other, their average image 604 will have dark pixels due to overlapping pixels in the target image data 601-603. In contrast, as shown in Fig. 6(b), an average image 614 of dissimilar target images 611-613 has fewer overlapping areas, making it less likely to have dark pixels.

[0047] Therefore, the smaller the minimum pixel value of the average image data, the more similar the target image data 601 to 603 belonging to the provisional cluster are considered to be, and the provisional cluster into which the target image data has been classified is considered to be an appropriate cluster.

[0048] FIG. 7 is a diagram for explaining an example of the process relating to the integration of clusters in steps S304 and S305 in FIG.

[0049] 7(a), when average image data 701 and 702 of different clusters are similar to each other, it is considered preferable that these clusters are the same cluster, and therefore these clusters are integrated in step S304. Then, in step S305, average image data 703 of a new cluster generated by the integration is generated.

[0050] On the other hand, if the average image data 711 and 712 of different clusters are not similar to each other, the clusters are considered to be already appropriate and therefore the clusters are not integrated.

[0051] As described above, according to this embodiment, the calculation unit 14 classifies multiple target image data into a number of provisional clusters up to a predetermined value, and for each provisional cluster, determines whether the provisional cluster is an appropriate cluster based on the image data belonging to the provisional cluster, and removes the image data belonging to the appropriate cluster as classification completion data from the target image data, repeating this loop process A until a predetermined termination condition is satisfied. When the termination condition is satisfied, the transmission / reception unit 11 outputs a combination of the appropriate cluster and the classification completion data as the classification result of the target image data. Therefore, since the image data is repeatedly classified into an appropriate cluster, it is possible to sufficiently classify the image data.

[0052] In this embodiment, the calculation unit 14 classifies the target image data in one of the temporary clusters until the number of image data belonging to that temporary cluster becomes the number obtained by dividing the number of target image data by the specified upper limit of the number of clusters. In this case, it becomes possible to classify the number of clusters up to about the specified upper limit, so that it becomes possible to classify the image data up to a desired number of clusters.

[0053] In addition, in this embodiment, when the smallest pixel value of the average image data of the target image data belonging to a provisional cluster is equal to or smaller than the determination threshold, the calculation unit 14 determines that the provisional cluster is an appropriate cluster. Therefore, it is possible to accurately determine an appropriate cluster.

[0054] In this embodiment, the calculation unit 14 increases the determination threshold value each time the loop process is repeated, or increases the determination threshold value as the ratio of the number of classified data to the number of target image data increases. Therefore, the end condition is gradually relaxed, making it possible to classify image data with higher accuracy.

[0055] In the present embodiment, the end condition is that the ratio of the number of classified data items to the number of target image data items is equal to or greater than the end threshold value. In this case, it is possible to classify image data items with higher accuracy.

[0056] In addition, in this embodiment, when the termination condition is satisfied, the calculation unit 14 organizes the clusters based on the similarity between the average image data of the image data belonging to each cluster. Specifically, the calculation unit 14 integrates clusters whose similarity is equal to or greater than the similarity threshold. This makes it possible to prevent the clusters from being classified into smaller groups than necessary, thereby making it possible to more appropriately classify the image data.

[0057] The above-described embodiments of the present disclosure are illustrative examples of the present disclosure, and are not intended to limit the scope of the present disclosure to only these embodiments. A person skilled in the art can implement the present disclosure in various other forms without departing from the scope of the present disclosure. [Explanation of symbols]

[0058] 1: Analysis server 2: Data management server 3: User terminal 11: Transmitter / receiver 12: Storage unit 13: Processing unit 14: Calculation unit 100: Image classification system

Claims

1. An image classification device for classifying image data, comprising: a calculation unit that classifies target image data, which is a plurality of image data to be classified, into a number of provisional clusters up to a predetermined value, determines for each provisional cluster whether the provisional cluster is an appropriate cluster based on the image data belonging to the provisional cluster, and removes the image data belonging to the appropriate cluster as classification completed data from the target image data, and repeats this process until a predetermined termination condition is satisfied; an output unit that outputs, when the termination condition is satisfied, a combination of the appropriate cluster and the classified data as a classification result of the target image data; The calculation unit determines that a provisional cluster is the appropriate cluster when a smallest pixel value of average image data of target image data belonging to the provisional cluster is equal to or smaller than a determination threshold.

2. The image classification device according to claim 1 , wherein the calculation unit increases the determination threshold value each time the process is repeated.

3. The image classification device according to claim 1 , wherein the calculation unit sets the determination threshold higher as a ratio of the number of the classified data to the number of the original target image data increases.

4. An image classification device for classifying image data, comprising: a calculation unit that classifies target image data, which is a plurality of image data to be classified, into a number of provisional clusters up to a predetermined value, determines for each provisional cluster whether the provisional cluster is an appropriate cluster based on the image data belonging to the provisional cluster, and removes the image data belonging to the appropriate cluster as classification completed data from the target image data, and repeats this process until a predetermined termination condition is satisfied; an output unit that outputs, when the termination condition is satisfied, a combination of the appropriate cluster and the classified data as a classification result of the target image data; When the termination condition is satisfied, the calculation unit organizes the appropriate clusters based on a similarity between average image data of image data belonging to each of the appropriate clusters.

5. The image classification device according to claim 4 , wherein the calculation unit merges the appropriate clusters whose similarity is equal to or greater than a similarity threshold.

6. 5. The image classification device according to claim 1, wherein the calculation unit classifies the target image data in one of the tentative clusters until the number of image data belonging to the tentative cluster becomes a number obtained by dividing the number of original target image data by an upper limit value of the number of appropriate clusters included in the classification result.

7. The image classification device according to claim 1 , wherein the termination condition is that a ratio of the number of the classified data to the number of the original target image data is equal to or greater than a termination threshold.

8. An image classification method by an image classification device that classifies image data, comprising: classifying target image data, which is a plurality of image data to be classified, into a number of provisional clusters up to a predetermined value, determining for each provisional cluster whether the provisional cluster is an appropriate cluster based on the image data belonging to the provisional cluster, and removing the image data belonging to the appropriate cluster as classified data from the target image data, repeating this process until a predetermined termination condition is satisfied; If the termination condition is satisfied, outputting the combination of the appropriate cluster and the classified data as a classification result of the target image data; In determining whether the provisional cluster is an appropriate cluster, if the smallest pixel value of average image data of target image data belonging to the provisional cluster is equal to or smaller than a determination threshold, the provisional cluster is determined to be the appropriate cluster.

9. An image classification method using an image classification device that classifies image data, comprising: classifying target image data, which is a plurality of image data to be classified, into a number of provisional clusters up to a predetermined value, determining for each provisional cluster whether the provisional cluster is an appropriate cluster based on the image data belonging to the provisional cluster, and removing the image data belonging to the appropriate cluster as classified data from the target image data, repeating this process until a predetermined termination condition is satisfied; If the termination condition is satisfied, the appropriate clusters are arranged based on a similarity between average image data of image data belonging to each of the appropriate clusters; An image classification device that outputs a combination of the organized appropriate cluster and the classified data as a classification result for classifying the target image data.

10. A computer that classifies image data a calculation unit that classifies target image data, which is a plurality of image data to be classified, into a number of provisional clusters up to a predetermined value, determines for each provisional cluster whether the provisional cluster is an appropriate cluster based on the image data belonging to the provisional cluster, and removes the image data belonging to the appropriate cluster as classification completed data from the target image data, and repeats this process until a predetermined termination condition is satisfied; and an output unit that outputs, when the termination condition is satisfied, a combination of the appropriate cluster and the classified data as a classification result of the target image data. an image classification program for causing the calculation unit to determine that a provisional cluster is the appropriate cluster when a smallest pixel value of average image data of target image data belonging to the provisional cluster is equal to or smaller than a determination threshold;

11. A computer for classifying image data, comprising: a calculation unit that classifies target image data, which is a plurality of image data to be classified, into a number of provisional clusters up to a predetermined value, determines for each provisional cluster whether the provisional cluster is an appropriate cluster based on the image data belonging to the provisional cluster, and removes the image data belonging to the appropriate cluster as classification completed data from the target image data, and repeats this process until a predetermined termination condition is satisfied; and an output unit that outputs, when the termination condition is satisfied, a combination of the appropriate cluster and the classified data as a classification result of the target image data, An image classification program for causing the calculation unit to organize the appropriate clusters based on the similarity between average image data of image data belonging to each of the appropriate clusters when the termination condition is satisfied.

Images