Machine learning device, machine learning method, and machine learning program

The machine learning device addresses performance issues in models trained with images containing missing values or outliers by selecting and preprocessing image sets, enhancing model performance through a comparison and preprocessing mechanism.

WO2024236790A1PCT designated stage expired Publication Date: 2024-11-21MITSUBISHI ELECTRIC CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2023/018511
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-18
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Machine learning models perform poorly when images in the training set contain missing values or outliers, leading to performance deterioration.

Method used

A machine learning device that acquires and preprocesses image sets using a selection and comparison process to identify similar sets and apply suitable preprocessings, thereby enhancing model performance even with missing values or outliers.

Benefits of technology

The solution effectively suppresses performance deterioration of machine learning models by selecting and preprocessing image sets to improve model performance, even when images contain missing values or outliers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2023018511_21112024_PF_FP_ABST
    Figure JP2023018511_21112024_PF_FP_ABST
Patent Text Reader

Abstract

This machine learning device is configured to comprise: an image set acquisition unit (1) that acquires an image set including one or more images; and an image set selection unit (6) that selects an image set similar to the image set acquired by the image set acquisition unit (1) from among a plurality of image sets different from the image set acquired by the image set acquisition unit (1). In addition, the machine learning device is provided with: a performance comparison unit (10) that acquires, as the performances of a plurality of machine-trained learning models, the performances of learning models in which machine training has been performed using respective image sets obtained by performing a plurality of different preprocessings on the image set which has been selected by the image set selection unit (6), and compares, with each other, the performances of the plurality of machine-trained learning models; and a preprocessing acquisition unit (11) that, on the basis of the performance comparison results from the performance comparison unit (10), selects one of the plurality of machine-trained learning models, and acquires the preprocessing that has been performed on the image set used in machine training for the selected learning model. In addition, the machine learning device is provided with a model training unit (12) that performs the preprocessing acquired by the preprocessing acquisition unit (11) on an image set acquired by the image set acquisition unit (1), and uses the preprocessed image set to perform machine training on an untrained learning model.
Need to check novelty before this filing date? Find Prior Art

Description

Machine learning device, machine learning method, and machine learning program

[0001] The present disclosure relates to a machine learning device, a machine learning method, and a machine learning program.

[0002] There are machine learning devices that use a plurality of data sets to train a learning model. For example, Patent Literature 1 discloses a machine learning device including a learning unit as such a machine learning device. The learning unit trains a learning model using an image set including a plurality of images.

[0003] JP 2023-30540 A

[0004] Images included in an image set used for machine learning of a learning model may contain, for example, missing values ​​or outliers. The machine learning device disclosed in Patent Literature 1 has a problem in that the performance of the learning model after machine learning by the learning unit may be reduced because images included in the image set used for machine learning contain missing values ​​or the like.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a machine learning device that can suppress performance degradation of a learning model after machine learning even if one or more images included in an image set contain missing values, etc.

[0006] A machine learning device according to the present disclosure includes an image set acquisition unit that acquires an image set including one or more images, and an image set selection unit that selects, from among a plurality of image sets different from the image set acquired by the image set acquisition unit, an image set similar to the image set acquired by the image set acquisition unit. The machine learning device also includes a performance comparison unit that acquires, as performances of a plurality of machine-learned learning models, performances of learning models in which a plurality of different preprocessing processes have been performed on the image sets selected by the image set selection unit and machine learning has been performed using the image sets after each preprocessing, and compares the performances of the plurality of machine-learned learning models with each other, and a preprocessing acquisition unit that selects one of the plurality of machine-learned learning models based on the performance comparison results by the performance comparison unit, and acquires the preprocessing performed on the image set used in the machine learning of the selected learning model. The machine learning device also includes a model training unit that performs preprocessing acquired by the preprocessing acquisition unit on the image set acquired by the image set acquisition unit, and causes an untrained learning model to perform machine learning using the preprocessed image set.

[0007] According to the present disclosure, even if one or more images included in an image set contain missing values, etc., it is possible to suppress performance degradation of a learning model after machine learning.

[0008] FIG. 1 is a configuration diagram showing a machine learning device according to a first embodiment. FIG. 2 is a hardware configuration diagram showing hardware of the machine learning device according to the first embodiment. FIG. 3 is a hardware configuration diagram of a computer when the machine learning device is realized by software, firmware, or the like. FIG. 4 is a flowchart showing a machine learning method, which is a processing procedure of the machine learning device. FIG. 5 is a configuration diagram showing a machine learning device according to a second embodiment. FIG. 6 is a hardware configuration diagram showing hardware of the machine learning device according to the second embodiment. FIG. 7 is an explanatory diagram showing an example of a second learning model to which an image set GS″ after data dimension compression is given.

[0009] In order to explain the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0010] Embodiment 1. Fig. 1 is a configuration diagram showing a machine learning device according to embodiment 1. Fig. 1 is a hardware configuration diagram showing hardware of the machine learning device according to embodiment 1. The machine learning device shown in Fig. 1 includes an image set acquisition unit 1, a data storage unit 2, an image set selection unit 6, a preprocessing selection unit 9, a model learning unit 12, a model evaluation unit 13, and a model output unit 14.

[0011] The image set acquisition unit 1 is realized by, for example, the image set acquisition circuit 21 shown in Fig. 2. The image set acquisition unit 1 acquires an image set GS including one or more images from the outside. The image set acquisition unit 1 outputs the image set GS to each of the image set selection unit 6 and the model learning unit 12.

[0012] The data storage unit 2 is realized by, for example, a data storage circuit 22 shown in FIG. 2. The data storage unit 2 includes a feature amount recording unit 3, a performance recording unit 4, and a preprocessing recording unit 5. The feature amount recording unit 3 stores N image sets GS different from the image set acquired by the image set acquisition unit 1. 1 ~GS N The feature Fv extracted from each of 1 ~Fv N N is an integer of 1 or more.

[0013] The performance recording unit 4 records M pre-processing P 1 ~P M Any of the pre-processing P m (m=1, . . . , M) is performed on an N×M image set GS 1,1 ~GS N,M N × M learning models MDL in which machine learning was performed using each of 1,1 ~MDL N,M Performance S 1,1 ~S N,M M is an integer equal to or greater than 2. The preprocessing recording unit 5 stores M preprocessing P 1 ~P M It has been recorded.

[0014] The image set selection unit 6 is realized by, for example, the image set selection circuit 23 shown in Fig. 2. The image set selection unit 6 includes a feature extraction unit 7 and a feature comparison unit 8. The image set selection unit 6 acquires an image set GS from the image set acquisition unit 1. The image set selection unit 6 selects N image sets GS. 1 ~GS N Among them, image sets GS similar to image set GS are selected. SEL The image set selection unit 6 selects the selected image set GS SEL is output to the preprocessing selection unit 9.

[0015] The feature extraction unit 7 acquires the image set GS from the image set acquisition unit 1. The feature extraction unit 7 extracts feature quantities Fv of the image set GS and outputs the feature quantities Fv to the feature comparison unit 8 and the model evaluation unit 13. The feature comparison unit 8 acquires the feature quantities Fv of the image set GS from the feature extraction unit 7. The feature comparison unit 8 compares the feature quantities Fv of the image set GS with the N image sets GS recorded in the feature recording unit 3. 1 ~GS N The feature Fv extracted from each of 1 ~Fv N The feature comparison unit 8 compares the feature Fv of the image set GS with the N feature Fv 1 ~Fv N Based on the comparison result, N image sets GS 1 ~GS N image set GS similar to the image set GS acquired by the image set acquisition unit 1 SEL The feature comparison unit 8 selects the selected image set GS SEL is output to the performance comparison unit 10.

[0016] The preprocessing selection unit 9 is realized by, for example, the preprocessing selection circuit 24 shown in FIG. 2. The preprocessing selection unit 9 includes a performance comparison unit 10 and a preprocessing acquisition unit 11. The preprocessing selection unit 9 selects M preprocessing processes P recorded in the preprocessing recording unit 5. 1 ~P M Among these, a pre-processing P suitable for the image set GS acquired by the image set acquisition unit 1 is selected. SELThe pre-processing selection unit 9 obtains the pre-processing P SEL is output to the model learning unit 12.

[0017] The performance comparison unit 10 compares the image set GS selected by the image set selection unit 6 SEL The performance comparison unit 10 acquires M machine-learned learning models MDL from the performance recording unit 4. SEL,1 ~MDL SEL,M Performance S SEL,1 ~S SEL,M The image set GS selected by the image set selection unit 6 is SEL M different preprocessing P 1 ~P M are performed, and the image set GS after each preprocessing is SEL,m A learning model MDL in which machine learning was performed using (m = 1, ..., M) SEL,m Performance S SEL,m The performance comparison unit 10 obtains the M performance S SEL,1 ~Performance S SEL,M The performance comparison results are output to the preprocessing acquisition unit 11.

[0018] The preprocessing acquisition unit 11 acquires the performance comparison results from the performance comparison unit 10. Based on the performance comparison results from the performance comparison unit 10, the preprocessing acquisition unit 11 generates M learning models MDL SEL,1 ~MDL SEL,M Among them, one of the learning models MDL SEL The preprocessing acquisition unit 11 selects M preprocessing Ps recorded in the preprocessing recording unit 5. 1 ~P M Among them, pre-processing P SEL As a learning model MDL SEL Image set GS used for machine learning SEL,m Pretreatment P performed m The preprocessing acquisition unit 11 acquires the acquired preprocessing P SEL are output to the model learning unit 12 and the model evaluation unit 13, respectively.

[0019] The model learning unit 12 is realized by, for example, a model learning circuit 25 shown in Fig. 2. The model learning unit 12 acquires the image set GS from the image set acquisition unit 1, and the preprocessing P SEL The model learning unit 12 performs preprocessing P SEL The model training unit 12 outputs the machine-learned training model MDL' to the model evaluation unit 13. The model training unit 12 selects a plurality of preprocessing processes P by the preprocessing selection unit 9. SEL is selected, for the image set GS, each preprocessing P SEL The model training unit 12 performs machine learning on the untrained learning model MDL' using each preprocessed image set GS'. The model training unit 12 outputs the multiple machine-learned learning models MDL' to the model evaluation unit 13.

[0020] The model evaluation unit 13 is realized by, for example, the model evaluation circuit 26 shown in FIG. 2. The model evaluation unit 13 acquires the machine-learned learning model MDL' from the model learning unit 12. The model evaluation unit 13 outputs the machine-learned learning model MDL' to the model output unit 14. The model evaluation unit 13 evaluates the performance of the machine-learned learning model MDL' and causes the performance recording unit 4 to record the performance of the learning model MDL'. In addition, the model evaluation unit 13 records the feature Fv extracted by the feature extraction unit 7 in the feature recording unit 3, and the preprocessing P acquired by the preprocessing acquisition unit 11. SEL are recorded in the preprocessing recording unit 5.

[0021] When the model evaluation unit 13 acquires multiple machine-learned learning models MDL' from the model learning unit 12, it evaluates the performance of each of the multiple machine-learned learning models MDL', and selects one or more learning models MDL' from the multiple machine-learned learning models MDL' based on the performance evaluation results. The model evaluation unit 13 outputs the selected learning models MDL' to the model output unit 14. The model evaluation unit 13 records the performance of the selected learning models MDL' in the performance recording unit 4. In addition, the model evaluation unit 13 records the feature Fv extracted by the feature extraction unit 7 in the feature recording unit 3, and the preprocessing P acquired by the preprocessing acquisition unit 11. SEL are recorded in the preprocessing recording unit 5.

[0022] The model output unit 14 is realized by, for example, the model output circuit 27 shown in Fig. 2. The model output unit 14 acquires the machine-learned learning model MDL' from the model evaluation unit 13. The model output unit 14 outputs the machine-learned learning model MDL' to, for example, a device that uses the learning model MDL'.

[0023] 1, it is assumed that each of the components of the machine learning device, namely, an image set acquisition unit 1, a data storage unit 2, an image set selection unit 6, a preprocessing selection unit 9, a model learning unit 12, a model evaluation unit 13, and a model output unit 14, is realized by dedicated hardware such as that shown in Fig. 2. In other words, it is assumed that the machine learning device is realized by an image set acquisition circuit 21, a data storage circuit 22, an image set selection circuit 23, a preprocessing selection circuit 24, a model learning circuit 25, a model evaluation circuit 26, and a model output circuit 27.

[0024] Here, the data storage circuit 22 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read Only Memory), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc). Furthermore, each of the image set acquisition circuit 21, the image set selection circuit 23, the preprocessing selection circuit 24, the model learning circuit 25, the model evaluation circuit 26, and the model output circuit 27 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0025] The components of the machine learning device are not limited to those realized by dedicated hardware, and the machine learning device may be realized by software, firmware, or a combination of software and firmware. The software or firmware is stored as a program in the memory of a computer. The computer refers to hardware that executes a program, and includes, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor).

[0026] 3 is a hardware configuration diagram of a computer when the machine learning device is realized by software, firmware, or the like. When the machine learning device is realized by software, firmware, or the like, the data storage unit 2 is configured on the computer's memory 31. The memory 31 stores machine learning programs for causing the computer to execute each of the processing procedures in the image set acquisition unit 1, image set selection unit 6, performance comparison unit 10, preprocessing acquisition unit 11, model learning unit 12, model evaluation unit 13, and model output unit 14, including an image set acquisition procedure, an image set selection procedure, a performance comparison procedure, a preprocessing acquisition procedure, a model learning procedure, a model evaluation procedure, and a model output procedure. The computer's processor 32 then executes the machine learning program stored in the memory 31.

[0027] 2 shows an example in which each of the components of the machine learning device is realized by dedicated hardware, while Fig. 3 shows an example in which the machine learning device is realized by software, firmware, etc. However, this is merely an example, and some of the components in the machine learning device may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0028] Next, the operation of the machine learning device shown in Figure 1 will be described. Figure 4 is a flowchart showing a machine learning method, which is a processing procedure of the machine learning device. The image set acquisition unit 1 acquires an image set GS containing one or more images from the outside (step ST1 in Figure 4). The image set GS may be acquired by any method, for example, by reading it from an external storage device (not shown), or by reading it from a communication device (not shown) via a network. The image set acquisition unit 1 outputs the image set GS to each of the image set selection unit 6 and the model learning unit 12.

[0029] The feature extraction unit 7 of the image set selection unit 6 acquires the image set GS from the image set acquisition unit 1. The feature extraction unit 7 extracts a feature Fv of the image set GS (step ST2 in FIG. 4 ). Examples of the feature Fv extracted by the feature extraction unit 7 include the average pixel values ​​of the N images included in the image set GS and the variance of the pixel values ​​of the N images, expressed in the form of a multidimensional vector. Examples of the feature Fv include the N images included in the image set GS embedded in an arbitrary vector space, each represented as a set of vectors. Examples of the feature Fv include the N images included in the image set GS embedded in an arbitrary vector space, each represented as a distribution of the image set GS in the vector space. The feature extraction unit 7 outputs the feature Fv of the image set GS to the feature comparison unit 8 and the model evaluation unit 13.

[0030] The feature comparison unit 8 acquires the feature Fv of the image set GS from the feature extraction unit 7. The feature comparison unit 8 acquires the feature Fv of the N image sets GS from the feature recording unit 3. 1 ~GS N The feature Fv extracted from each of 1 ~Fv N The feature comparison unit 8 compares the feature Fv of the image set GS with the feature Fv of the image set GS. n (n=1, ..., N) feature Fv n and compare the feature amount Fv with the feature amount Fv n Based on the comparison result, the image set GS and the image set GS n The degree of similarity between the image set GS and the image set GS is determined. 1 ~GS N As a method for determining the similarity between the feature Fv and the feature Fv 1 ~Fv N The Euclidean distance between the feature Fv and the feature Fv 1 ~Fv N or the Mahalanobis distance between the feature Fv and the feature Fv 1 ~Fv NThe feature comparison unit 8 can use a method of comparing the N Euclidean distances, the N Mahalanobis distances, or the N Fréchet inception distances with each other, and making a determination based on the comparison result of the Euclidean distances, the comparison result of the Mahalanobis distances, or the comparison result of the Fréchet inception distances. 1 ~GS N As a method for determining the similarity between the feature Fv and the feature Fv 1 ~Fv N Cosine similarity between feature Fv and feature Fv 1 ~Fv N The feature comparison unit 8 can use a method of comparing the N cosine similarities or the N Dice coefficients with each other and making a determination based on the comparison result of the cosine similarities or the comparison result of the Dice coefficients.

[0031] The feature comparison unit 8 compares the feature Fv of the image set GS with the N feature Fv 1 ~Fv N Based on the comparison result, N image sets GS 1 ~GS N Among them, image sets GS similar to image set GS are SEL Specifically, the feature comparison unit 8 selects the feature Fv of the image set GS and the N feature Fv 1 ~Fv N Based on the comparison result, N image sets GS 1 ~GS N Among them, the image set GS that is most similar to the image set GS SEL Here, the feature comparison unit 8 selects N image sets GS 1 ~GS N Among them, the image set GS that is most similar to the image set GS SEL However, this is only an example, and the feature comparison unit 8 selects N image sets GS 1 ~GS N From the image set GS, the similarity is greater than or equal to the threshold.SEL For example, N image sets GS 1 ~GS N Among them, the image set with similarity above the threshold is called GS 2 and G.S. 5 If so, image set GS SEL As the image set GS 2 , G.S. 5 The threshold value may be stored in an internal memory of the feature comparison unit 8, or may be provided from outside the machine learning device. The feature comparison unit 8 selects the selected image set GS SEL is output to the performance comparison unit 10.

[0032] The performance comparison unit 10 receives the image set GS from the feature comparison unit 8. SEL Furthermore, the performance comparison unit 10 acquires M machine-learned learning models MDL from the performance recording unit 4. SEL,1 ~MDL SEL,M Performance S SEL,1 ~S SEL,M Obtain the machine-learned learning model MDL SEL,m (m=1, . . . , M) is the image set GS selected by the image set selection unit 6. SEL For the pre-processing P m The image set GS SEL,m This is a learning model that has been machine-learned using the preprocessing P m Examples of preprocessing include missing value processing, outlier processing, resizing, and filtering. m If the number of images is M, the performance recording unit 4 stores the image set GS selected by the image set selection unit 6. SEL For preprocessing P m (m = 1, ..., M) is performed, and machine learning is performed using the preprocessed image set. SEL,m Performance S SEL,m As a result, the M performance S SEL,1 ~Performance S SEL,M The performance comparison unit 10 records M performance S SEL,1 ~S SEL,M The performance comparison results are output to the preprocessing acquisition unit 11 (step ST4 in FIG. 4).

[0033] The preprocessing acquisition unit 11 acquires the performance comparison results from the performance comparison unit 10. Based on the performance comparison results from the performance comparison unit 10, the preprocessing acquisition unit 11 generates M learning models MDL SEL,1 ~MDL SEL,M Among these, one or more learning models MDL SEL (Step ST5 in FIG. 4). Specifically, the preprocessing acquisition unit 11 selects M learning models MDL SEL,1 ~MDL SEL,M From among these, the learning model MDL SEL For example, the learning model with the highest performance may be selected, or several learning models with the highest performance may be selected, or a learning model with performance equal to or greater than a threshold may be selected. 1 ~P M Among them, the preprocessing P suitable for the image set GS is SEL As a learning model MDL SEL Image set GS used for machine learning SEL,m Pretreatment P performed m (Step ST6 in FIG. 4). SEL are output to the model learning unit 12 and the model evaluation unit 13, respectively.

[0034] The model learning unit 12 acquires the image set GS from the image set acquisition unit 1, and the preprocessing P SEL The model learning unit 12 performs preprocessing P SEL (Step ST7 in FIG. 4). The model learning unit 12 performs a plurality of preprocessing P SEL When the image set GS is obtained, each preprocessing P SEL Do the following.

[0035] The model learning unit 12 uses the preprocessed image set GS' to make the untrained learning model MDL perform machine learning (step ST8 in FIG. 4). The untrained learning model MDL may be stored in an internal memory of the model learning unit 12, or may be stored in a storage device or the like provided outside the model learning unit 12. The model learning unit 12 receives a plurality of preprocessing P SEL When the preprocessed image sets GS′ are acquired, the model training unit 12 causes the untrained training model MDL to perform machine learning using the preprocessed image sets GS′. The model training unit 12 outputs the multiple machine-learned training models MDL′ to the model evaluation unit 13.

[0036] When the model evaluation unit 13 acquires one machine-learned learning model MDL' from the model training unit 12, it evaluates the performance S of the one machine-learned learning model MDL' (step ST9 in FIG. 4). When the model evaluation unit 13 acquires multiple machine-learned learning models MDL' from the model training unit 12, it evaluates the performance S of each machine-learned learning model MDL' (step ST9 in FIG. 4). Methods for evaluating the performance S include, for example, a method based on the magnitude of loss when machine-learning the learning model MDL', a method based on the accuracy rate using an evaluation image set, or a method using the AUC (Area Under the Curve) for the evaluation image set as an index.

[0037] When the model evaluation unit 13 acquires one machine-learned learning model MDL' from the model learning unit 12, it outputs the one machine-learned learning model MDL' to the model output unit 14. When the model evaluation unit 13 acquires multiple machine-learned learning models MDL' from the model learning unit 12, it outputs the learning model MDL' with the highest evaluation in performance S among the multiple machine-learned learning models MDL' to the model output unit 14. Here, the model evaluation unit 13 outputs the learning model MDL' with the highest evaluation in performance S among the multiple machine-learned learning models MDL' to the model output unit 14. However, this is merely an example, and the model evaluation unit 13 may be configured to output the top few learning models MDL' with the highest evaluation in performance S among the multiple machine-learned learning models MDL' to the model output unit 14. Furthermore, the model evaluation unit 13 may be configured to output the learning model MDL' with an evaluation in performance S equal to or greater than a threshold among the multiple machine-learned learning models MDL' to the model output unit 14.

[0038] The model evaluation unit 13 records the performance S of the learning model MDL′ output to the model output unit 14 in the performance recording unit 4. The model evaluation unit 13 also records the feature Fv extracted by the feature extraction unit 7 in the feature recording unit 3, and records the preprocessing P acquired by the preprocessing acquisition unit 11. SEL are recorded in the preprocessing recording unit 5.

[0039] The model output unit 14 acquires the machine-learned learning model MDL' from the model evaluation unit 13. The model output unit 14 outputs the machine-learned learning model MDL' to, for example, a device that uses the machine-learned learning model MDL'.

[0040] In the first embodiment described above, the machine learning device is configured to include an image set acquisition unit 1 that acquires an image set including one or more images, and an image set selection unit 6 that selects an image set similar to the image set acquired by the image set acquisition unit 1 from a plurality of image sets different from the image set acquired by the image set acquisition unit 1. The machine learning device also includes a performance comparison unit 10 that acquires the performance of a plurality of machine-learned learning models in which a plurality of different preprocessing processes are performed on the image sets selected by the image set selection unit 6 and machine learning is performed using the image sets after each preprocessing, and compares the performance of the plurality of machine-learned learning models with each other, and a preprocessing acquisition unit 11 that selects one of the plurality of machine-learned learning models based on the performance comparison result by the performance comparison unit 10 and acquires the preprocessing performed on the image set used for machine learning of the selected learning model. The machine learning device also includes a model learning unit 12 that performs the preprocessing acquired by the preprocessing acquisition unit 11 on the image set acquired by the image set acquisition unit 1, and causes an untrained learning model to perform machine learning using the preprocessed image set. Therefore, even if one or more images included in the image set contain missing values ​​or the like, the machine learning device can suppress a decrease in the performance of the learning model after machine learning.

[0041] In the machine learning device shown in FIG. 1 , the model learning unit 12 causes an untrained learning model MDL to perform machine learning using the preprocessed image set GS' selected by the preprocessing selection unit 9. However, this is merely an example, and the model learning unit 12 may cause an untrained learning model MDL to perform machine learning using the preprocessed image set GS' selected by the preprocessing selection unit 9, or may cause an untrained learning model MDL to perform machine learning using the image set GS acquired by the image set acquisition unit 1 without performing preprocessing on the image set GS. In this case, the model evaluation unit 13 evaluates the performance S of the learning model MDL' after machine learning using the preprocessed image set GS' selected by the preprocessing selection unit 9, and evaluates the performance S of the learning model MDL' after machine learning using the image set GS acquired by the image set acquisition unit 1. Then, the model evaluation unit 13 selects one or more learning models MDL' from among these learning models MDL' after machine learning, based on the evaluation results of the performance S of these learning models MDL' after machine learning.

[0042] Second Embodiment In a second embodiment, a machine learning device including a data dimension compression unit 15 that compresses the data dimension of the image set GS acquired by the image set acquisition unit 1 will be described.

[0043] FIG. 5 is a configuration diagram showing a machine learning device according to embodiment 2. In FIG. 5, the same reference numerals as in FIG. 1 indicate the same or corresponding parts, and detailed description thereof will be omitted. FIG. 6 is a hardware configuration diagram showing hardware of the machine learning device according to embodiment 2. In FIG. 6, the same reference numerals as in FIG. 2 indicate the same or corresponding parts, and detailed description thereof will be omitted. The machine learning device shown in FIG. 5 includes an image set acquisition unit 1, a data dimension compression unit 15, a data storage unit 2, an image set selection unit 6, a preprocessing selection unit 9, a model learning unit 12, a model evaluation unit 13, and a model output unit 14.

[0044] The data dimension compression unit 15 is realized by, for example, the data dimension compression circuit 28 shown in FIG. 6. The data dimension compression unit 15 acquires the image set GS from the image set acquisition unit 1. The data dimension compression unit 15 compresses the data dimensions of the image set GS. The data dimension compression unit 15 outputs the image set GS″ after data dimension compression to the feature extraction unit 7a of the image set selection unit 6.

[0045] In the machine learning device shown in FIG. 5 , the image set selection unit 6 includes a feature extraction unit 7a and a feature comparison unit 8. The feature extraction unit 7a obtains the image set GS″ after data dimension compression from the data dimension compression unit 15. The feature extraction unit 7a provides the image set GS″ after data dimension compression to, for example, a second learning model as shown in FIG. 7 . The feature extraction unit 7a obtains a vector set output from either the intermediate layer of the second learning model or the output layer of the second learning model as a feature Fv to be extracted. The feature extraction unit 7a outputs the feature Fv to the feature comparison unit 8. FIG. 7 is an explanatory diagram showing an example of a second learning model to which an image set GS" after data dimension compression is given. The second learning model is a learning model that, during learning, is given, for example, an image set after data dimension compression and training data indicating the features of the image set, and machine-learns the features of the image set. During inference, when the image set GS" after data dimension compression is given to the input layer, the second learning model outputs the feature Fv of the image set GS" from the intermediate layer or the output layer.

[0046] 5 assumes that each of the components of the machine learning device, namely, the image set acquisition unit 1, the data dimension compression unit 15, the data storage unit 2, the image set selection unit 6, the preprocessing selection unit 9, the model learning unit 12, the model evaluation unit 13, and the model output unit 14, is realized by dedicated hardware such as that shown in FIG. 6. That is, it assumes that the machine learning device is realized by an image set acquisition circuit 21, a data dimension compression circuit 28, a data storage circuit 22, an image set selection circuit 23, a preprocessing selection circuit 24, a model learning circuit 25, a model evaluation circuit 26, and a model output circuit 27. Each of the image set acquisition circuit 21, the data dimension compression circuit 28, the image set selection circuit 23, the preprocessing selection circuit 24, the model learning circuit 25, the model evaluation circuit 26, and the model output circuit 27 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0047] The components of the machine learning device are not limited to those realized by dedicated hardware, and the machine learning device may be realized by software, firmware, or a combination of software and firmware. When the machine learning device is realized by software, firmware, or the like, the data storage unit 2 is configured on a memory 31 shown in FIG. 3. A machine learning program is stored in the memory 31 shown in FIG. 3 to cause a computer to execute each of the processing procedures in the image set acquisition unit 1, the data dimension compression unit 15, the image set selection unit 6, the performance comparison unit 10, the preprocessing acquisition unit 11, the model learning unit 12, the model evaluation unit 13, and the model output unit 14, including an image set acquisition procedure, a data dimension compression procedure, an image set selection procedure, a performance comparison procedure, a preprocessing acquisition procedure, a model learning procedure, a model evaluation procedure, and a model output procedure. The processor 32 shown in FIG. 3 then executes the machine learning program stored in the memory 31.

[0048] 6 shows an example in which each of the components of the machine learning device is realized by dedicated hardware, while Fig. 3 shows an example in which the machine learning device is realized by software, firmware, etc. However, this is merely an example, and some of the components in the machine learning device may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0049] Next, the operation of the machine learning device shown in Fig. 5 will be described. However, apart from the data dimension compression unit 15 and the feature extraction unit 7a, the machine learning device is the same as that shown in Fig. 1. Therefore, only the operation of the data dimension compression unit 15 and the feature extraction unit 7a will be described here.

[0050] The data dimension compression unit 15 acquires the image set GS from the image set acquisition unit 1. The data dimension compression unit 15 compresses the data dimensions of the image set GS. One method of compressing the data dimensions is to provide the image set GS to the input layer of a convolutional neural network (hereinafter referred to as "CNN") trained with an image set for an ImageNet classification task, and output a vector set output from either the intermediate layer of the CNN or the output layer of the CNN as an image set GS" after data dimension compression. In the machine learning device shown in FIG. 5 , the image set GS" after data dimension compression by the data dimension compression unit 15 is an image set whose data size has been reduced by any method. The data dimension compression unit 15 outputs the image set GS" after data dimension compression to the feature extraction unit 7a of the image set selection unit 6.

[0051] The feature extraction unit 7a acquires the image set GS" after data dimension compression from the data dimension compression unit 15. The feature extraction unit 7a provides each image included in the image set GS" after data dimension compression to, for example, any one of the input layers in a second learning model as shown in FIG. 7. The feature extraction unit 7a acquires a vector set output from either the intermediate layer of the second learning model or the output layer of the second learning model as the feature Fv to be extracted. In the example of FIG. 7, the second learning model has three intermediate layers. However, this is merely an example, and the second learning model may have two or fewer intermediate layers, or four or more intermediate layers. For example, if the second learning model has three intermediate layers, the feature extraction unit 7a acquires a vector set output from any one of the three intermediate layers as the feature Fv to be extracted. Furthermore, the feature extraction unit 7a may calculate a probability mass function of a set of vectors output from the intermediate layer of the second learning model or the output layer of the second learning model, and use the probability mass function as the feature Fv.

[0052] In the second embodiment described above, the machine learning device shown in FIG. 5 is configured to include a data dimension compression unit 15 that compresses the data dimensions of the image set acquired by the image set acquisition unit 1. Therefore, similar to the machine learning device shown in FIG. 1, the machine learning device shown in FIG. 5 can suppress performance degradation of the learning model after machine learning and reduce unnecessary information even if one or more images included in the image set contain missing values, etc. Therefore, the machine learning device shown in FIG. 5 can achieve higher accuracy in calculating feature quantities and faster feature comparison processing than the machine learning device shown in FIG. 1.

[0053] In the machine learning device shown in FIG. 5, the feature extraction unit 7a obtains the feature Fv to be extracted by providing the image set GS″ after data dimension compression to the input layer of the second learning model. However, this is merely an example, and the feature extraction unit 7a may be configured to extract the feature Fv of the image set GS″ in the same way as the feature extraction unit 7 shown in FIG. 1.

[0054] In the machine learning device shown in FIG. 5 , the feature extraction unit 7 a obtains the feature Fv to be extracted by providing the image set GS″ after data dimension compression to the input layer of the second learning model. However, this is merely an example, and the feature extraction unit 7 a may provide the image set GS obtained by the image set obtaining unit 1 to the input layer of the second learning model, and obtain a vector set output from either the intermediate layer of the second learning model or the output layer of the second learning model as the feature Fv to be extracted. In this case, the second learning model is a learning model that, during learning, is given, for example, an image set and training data indicating the feature of the image set, and machine-learns the feature of the image set. During inference, when the image set GS is provided to the input layer, the second learning model outputs the feature Fv of the image set GS from the intermediate layer or the output layer.

[0055] In addition, the present disclosure allows for free combination of the respective embodiments, modification of any of the components of the respective embodiments, or omission of any of the components of the respective embodiments.

[0056] The present disclosure is suitable for a machine learning device, a machine learning method, and a machine learning program.

[0057] 1 Image set acquisition unit, 2 Data storage unit, 3 Feature recording unit, 4 Performance recording unit, 5 Preprocessing recording unit, 6 Image set selection unit, 7 Feature extraction unit, 7a Feature extraction unit, 8 Feature comparison unit, 9 Preprocessing selection unit, 10 Performance comparison unit, 11 Preprocessing acquisition unit, 12 Model learning unit, 13 Model evaluation unit, 14 Model output unit, 15 Data dimension compression unit, 21 Image set acquisition circuit, 22 Data storage circuit, 23 Image set selection circuit, 24 Preprocessing selection circuit, 25 Model learning circuit, 26 Model evaluation circuit, 27 Model output circuit, 28 Data dimension compression circuit, 31 Memory, 32 Processor.

Claims

1. A machine learning device comprising: an image set acquisition unit that acquires an image set including one or more images; an image set selection unit that selects an image set similar to the image set acquired by the image set acquisition unit from among a plurality of image sets different from the image set acquired by the image set acquisition unit; a performance comparison unit that acquires performance of a learning model in which a plurality of different preprocessing processes are performed on the image sets selected by the image set selection unit and machine learning is performed using the image sets after each preprocessing as performance of a plurality of machine-learned learning models, and compares the performance of the plurality of machine-learned learning models with each other; a preprocessing acquisition unit that selects a learning model from the plurality of machine-learned learning models based on a performance comparison result by the performance comparison unit, and acquires the preprocessing performed on the image set used for machine learning of the selected learning model; and a model learning unit that performs the preprocessing acquired by the preprocessing acquisition unit on the image set acquired by the image set acquisition unit, and causes an untrained learning model to perform machine learning using the preprocessed image set.

2. The machine learning device described in claim 1, characterized in that the image set selection unit includes a feature extraction unit that extracts features of the image set acquired by the image set acquisition unit, and a feature comparison unit that compares the features extracted from each of a plurality of image sets different from the image set acquired by the image set acquisition unit with the features extracted by the feature extraction unit, and selects an image set similar to the image set acquired by the image set acquisition unit from among the plurality of image sets based on the feature comparison result.

3. The machine learning device of claim 1, further comprising a model evaluation unit that evaluates the performance of each of the learning models after machine learning by the model learning unit and selects one or more learning models from among the multiple learning models after machine learning based on the performance evaluation results, wherein the preprocessing acquisition unit acquires multiple preprocessing processes, the model learning unit performs each of the preprocessing processes acquired by the preprocessing acquisition unit on the image set acquired by the image set acquisition unit, and the model evaluation unit performs machine learning on an untrained learning model using each of the preprocessed image sets.

4. The machine learning device described in claim 3, characterized in that the model learning unit, in addition to performing machine learning on an untrained learning model using each preprocessed image set, performs machine learning on an untrained learning model using the image set acquired by the image set acquisition unit without performing preprocessing on the image set, and the model evaluation unit evaluates the performance of each of the multiple learning models after machine learning by the model learning unit, and selects one or more learning models from the multiple learning models after machine learning based on the performance evaluation results.

5. The machine learning device according to claim 1, further comprising a data dimension compression unit that compresses the data dimension of the image set acquired by the image set acquisition unit and outputs the image set after data dimension compression to the image set selection unit.

6. The machine learning device described in claim 2, characterized in that the feature extraction unit provides the image set acquired by the image set acquisition unit to a second learning model and acquires a set of vectors output from either the intermediate layer of the second learning model or the output layer of the second learning model as the feature to be extracted.

7. The machine learning device according to claim 2, further comprising a data dimension compression unit that compresses the data dimensions of the image set acquired by the image set acquisition unit, and the feature extraction unit provides the image set after data dimension compression by the data dimension compression unit to a second learning model, and acquires a set of vectors output from either an intermediate layer of the second learning model or an output layer of the second learning model as the feature to be extracted.

8. The machine learning device described in claim 2, characterized in that the feature comparison unit calculates a Fréchet initiation distance between the feature extracted from each of a plurality of image sets different from the image set acquired by the image set acquisition unit and the feature extracted by the feature extraction unit, compares the plurality of Fréchet initiation distances with each other, and selects an image set similar to the image set acquired by the image set acquisition unit from among the plurality of image sets based on a comparison result of the Fréchet initiation distances.

9. A machine learning method in which an image set acquisition unit acquires an image set including one or more images; an image set selection unit selects an image set similar to the image set acquired by the image set acquisition unit from a plurality of image sets different from the image set acquired by the image set acquisition unit; a performance comparison unit acquires, as performance of a plurality of machine-learned learning models, performance of a learning model in which a plurality of different preprocessing processes are performed on the image set selected by the image set selection unit and machine learning is performed using the image sets after each preprocessing, and compares the performances of the plurality of machine-learned learning models with each other; a preprocessing acquisition unit selects one of the plurality of machine-learned learning models based on a performance comparison result by the performance comparison unit, and acquires the preprocessing performed on the image set used for machine learning of the selected learning model; and a model learning unit performs the preprocessing acquired by the preprocessing acquisition unit on the image set acquired by the image set acquisition unit, and causes an untrained learning model to perform machine learning using the preprocessed image set.

10. A machine learning program for causing a computer to execute the following steps: an image set acquisition procedure in which an image set acquisition unit acquires an image set including one or more images; an image set selection procedure in which an image set selection unit selects an image set similar to the image set acquired by the image set acquisition unit from among a plurality of image sets different from the image set acquired by the image set acquisition unit; a performance comparison procedure in which a performance comparison unit acquires performance of a learning model in which a plurality of different preprocessing processes are performed on the image set selected by the image set selection unit and machine learning is performed using the image sets after each preprocessing, as performance of a plurality of machine-learned learning models, and compares the performances of the plurality of machine-learned learning models with each other; a preprocessing acquisition procedure in which a preprocessing acquisition unit selects one of the plurality of machine-learned learning models based on a performance comparison result by the performance comparison unit, and acquires the preprocessing performed on the image set used for machine learning of the selected learning model; and a model learning procedure in which a model learning unit performs the preprocessing acquired by the preprocessing acquisition unit on the image set acquired by the image set acquisition unit, and causes an untrained learning model to perform machine learning using the preprocessed image set.

Citation Information

Patent Citations

  • Image processor, image processing method, and image processing program

    JP2010257256A

  • Machine learning apparatus and method

    JP2019032782A

  • Learning data generating device, discrimination model generating device, and program

    JP2020027424A