Machine learning method in granular material determination

Abstract

To provide a machine learning method in granular material determination capable of substantially decreasing time and labor required for machine learning by performing the machine learning while decreasing learning images to be an origin of teacher data.SOLUTION: A machine learning method includes: a region dividing step of marking inside of a specific kind of granular material to a learning image obtained by photographing a plurality of kinds of granular materials, and dividing image elements other than the specific kind of granular material into regions; an identification label adding step of adding an identification label to the specific kind of granular material extracted by the region dividing step; a teacher data creation step of creating teacher data for the plurality of kinds of granular materials by executing the region dividing step and the identification label adding step for each kind of granular materials other than the specific kind of granular material; and a learning step of performing machine learning of the teacher data.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a machine learning method for granular object discrimination that is capable of discriminating granular objects contained in image data. [Background technology]

[0002] Machine learning methods have been used in various fields for image analysis, for example, Patent Document 1 discloses various inspection devices for supporting diagnosis in the medical field. Patent Document 2 discloses a method for creating quality discrimination standards that support discrimination of the appearance quality of granular objects.

[0003] In the above-mentioned Patent Document 1, the purpose is to automatically extract lesion areas from image data of human tissue, in which a user fills in the lesion areas in the image data without omitting any areas, converting the data into a large amount of annotation data, which is then used as training data to improve the accuracy of lesion extraction.

[0004] In addition, in the above-mentioned Patent Document 2, as shown in Figure 3 of the document, images of granular objects separated by type are used to create a quality discrimination standard for the granular objects using a predetermined machine learning algorithm. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2021 / 261323 [Patent Document 2] JP 2015-230265 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, the above-mentioned conventional techniques have the problem that they require a huge amount of time and manpower, from collecting the image data necessary for machine learning to creating identification labels for the objects in each image.

[0007] In view of the above-mentioned problems, the present invention aims to provide a machine learning method for granular matter discrimination that can perform machine learning using a small number of learning images that serve as the source of training data, thereby significantly reducing the time and effort required for the machine learning. [Means for solving the problem]

[0008] The invention related to (1) is a machine learning method for granular material discrimination, characterized by comprising: an area division step of marking the inside of a specific type of granular material in a learning image in which multiple types of granular material are captured, and dividing the image elements other than the specific type of granular material into areas; an identification label assignment step of assigning an identification label to the specific type of granular material extracted by the area division step; a teacher data creation step of executing the area division step and the identification label assignment step for each type of granular material other than the specific type of granular material to create teacher data for the multiple types of granular material; and a learning step of machine learning the teacher data.

[0009] The invention related to (2) is the machine learning method for granular matter discrimination described in (1) above, further comprising an inference evaluation step of performing an inference evaluation of granular matter discrimination on an evaluation image in which the granular matter is captured, and an additional learning step of, if the inference evaluation result of the inference evaluation step does not satisfy a predetermined discrimination accuracy, creating teacher data by assigning an identification label to an additional learning image in which the granular matter is captured, and further performing machine learning.

[0010] The invention related to (3) is a machine learning method for granular matter discrimination described in (2) above, characterized in that the additional learning image is an image of a granular matter that does not meet a predetermined discrimination accuracy in the inference evaluation result.

[0011] The invention related to (4) is a machine learning method for granular material discrimination described in any of (1) to (3) above, characterized in that the multiple types of granular materials include at least one of grains, seeds, or granular crops. Effect of the Invention

[0012] According to the present invention, even from a single learning image in which multiple types of granular objects are captured, it is possible to assign identification labels to each type of granular object by dividing the image elements into regions, and it is possible to easily create training data in a short time, which makes it possible to effectively reduce the time and effort required for machine learning. [Brief description of the drawings]

[0013] [Figure 1] 1 is an overall flow of machine learning in one embodiment of the present invention. [Diagram 2] 1 is a preprocessing flow for machine learning in one embodiment of the present invention. [Diagram 3] FIG. 11 is a diagram illustrating a procedure according to a pre-processing flow in one embodiment of the present invention. [Figure 4] FIG. 11 is a diagram illustrating a procedure according to a pre-processing flow in one embodiment of the present invention. [Diagram 5] 1 is a learning flow in machine learning according to an embodiment of the present invention. [Figure 6] 1 is a diagram showing an inference evaluation flow in machine learning according to an embodiment of the present invention. [Figure 7] 1 is a diagram illustrating a procedure according to a learning flow and an inference evaluation flow in one embodiment of the present invention. [Figure 8] FIG. 11 is a diagram illustrating an additional learning procedure in one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] An embodiment of the machine learning method for granular object discrimination according to the present invention will be described below with reference to the drawings. Note that the present invention uses a multi-layer neural network created using a well-known deep learning technique as a machine learning device, and can utilize necessary devices (not shown), such as a keyboard, a mouse, a display, a personal computer, or a server.

[0015] An overall flow of machine learning in one embodiment of the present invention is shown in Fig. 1. That is, preprocessing (S1) corresponding to so-called annotation performed in machine learning is performed to create teacher data. Then, learning (S2) is performed using the teacher data, and inference evaluation (S3) is performed using evaluation images to confirm the accuracy of discrimination of granular objects.

[0016] If the result of the inference evaluation (S3) does not meet the specified accuracy and the expected result is not obtained, an image for additional learning is prepared, and preprocessing (S1) and learning (S2) are performed again to confirm the discrimination accuracy by inference evaluation (S3).

[0017] FIG. 2 shows a flow of the above-mentioned pre-treatment (S1), and FIGS. 3 and 4 show specific work procedures according to the pre-treatment flow.

[0018] More specifically, when inputting data (S11), learning images of multiple types of granular matter are prepared as shown in the upper diagram of Fig. 3. Then, a file selection operation is performed, and in label selection (S12), a predetermined identification label corresponding to the type of granular matter is selected (corresponding to label 1 in Fig. 3).

[0019] In the illustrated example, cartons, unhulled rice, and polished rice are captured together with a background image as learning images. A configuration for attaching an identification label to a carton as label 1 will be described below.

[0020] Next, in region drawing (S13), the inside of the carton to which an identification label is to be added is marked, for example, in red, as shown in the lower diagram of Figure 3. Image elements other than the carton, such as the unhulled rice, white rice, and background, are roughly marked, for example, in blue, and the regions are divided using graph cut technology that has been used for image region extraction (region division step). Note that in addition to graph cut, other region division methods such as machine learning, differential calculus, and color subtraction processing can also be used for region division.

[0021] If necessary, an area correction process (S14) is performed, and the area result display (S15) is performed, whereby the identification label of the carton is displayed as shown in the upper diagram of FIG. 4, and if there is no problem with the label area, the identification label is assigned as is (identification label assignment step).

[0022] After assigning identification labels to the cartons extracted by the above-mentioned area division step, the above-mentioned area division step and identification label assignment step are performed for each type of unhulled rice and polished rice other than the cartons to create learning identification labels (teacher data) (teacher data creation step).

[0023] Fig. 5 shows the learning flow in the above-mentioned learning (S2), and Fig. 6 shows the inference evaluation flow in the above-mentioned inference evaluation (S3). In addition, Fig. 7 and Fig. 8 show specific work procedures according to the learning flow and the inference evaluation flow.

[0024] When the learning identification labels (teaching data) are created by the above-mentioned teaching data creation step, the learning operation is executed in the reading of the learning data (S21) as shown in the upper diagram of Fig. 7. This starts the learning (S22) and the trained model is output (S23).

[0025] As described above, the present invention makes it possible to create learning identification labels (teacher data) using one or a small number of learning images. In particular, when the granular material contains grains, multiple types of granular material such as cartons and rice grains are often mixed in one image, making machine learning possible with a small number of learning images. Therefore, it can be said that the present invention is more suitable for application to discrimination of granular materials containing grains.

[0026] Next, when the trained model is output (S23), in order to verify the training result, the trained model is read (S31) as shown in Fig. 6. Next, image data for inference evaluation in which a granular object is captured is selected (S32) as shown in the lower diagram of Fig. 7, and an inference evaluation process is executed (S33) to confirm the inference evaluation accuracy of the granular object discrimination (inference evaluation step).

[0027] The result of the inference evaluation process is displayed in the inference evaluation process result display (S34), and the inference evaluation result is displayed, for example, in a form as shown in the lower diagram of Fig. 7. The example shown is a case where white rice was not correctly identified. Note that a configuration may be added in which granular objects that could not be correctly identified are selected, and the degree of correct match is displayed as a percentage.

[0028] In this way, when the inference evaluation result does not satisfy the specified discrimination accuracy, as shown in the upper diagram of Figure 8, additional learning images of white rice are prepared, and additional learning identification labels are added to create training data, and further machine learning is performed (additional learning step).

[0029] After the additional learning step, if the inference evaluation step described above is carried out again and the accuracy of discriminating white rice is improved as shown in the lower diagram of Figure 8, it will be possible to actually discriminate granular objects with high accuracy.

[0030] According to the machine learning method for granular matter discrimination of the present invention described above, it is possible to calculate the degree of agreement between the inference evaluation result in the inference evaluation image data and the correct answer, and visualize the result as shown in the lower diagram of Fig. 7. This makes it possible to easily grasp areas where learning has not gone well from the numerical values ​​and the visualized image.

[0031] In addition, as mentioned above, it is possible to create learning labels (teacher data) using one or a small number of learning images, making it possible to perform machine learning while quickly visualizing the assignment of identification labels and confirmation of inference evaluation results. This makes it possible to significantly reduce the time and effort required for machine learning.

[0032] (Other embodiments) Although an embodiment of the present invention has been described above with reference to the drawings, the machine learning method for discriminating granular matter according to the present invention is not necessarily limited to the above-described embodiment.

[0033] For example, in the above-mentioned embodiment, examples of discriminating carton, unhulled rice, and polished rice are shown, but the discrimination target is not necessarily limited to these granular objects, and the discrimination target can be granular objects including at least one of grains, seeds, and granular crops. That is, as cereals, for example, rice, wheat, corn, etc., miscellaneous grains (foxtail millet, barnyard millet, sorghum) and beans (edamame, soybeans) are also discrimination targets. Furthermore, as seeds, vegetable seeds such as carrots and spinach, and as granular crops, for example, fruits (raisins, blueberries, etc.) and vegetables (cherry tomatoes, etc.) are also discrimination targets. Granular objects including resin pieces such as pellets and beads, medicines, ores, and fine articles such as whitebait are also discrimination targets. Furthermore, the present invention is not limited to the above-mentioned granular objects themselves, and can be applied to cases where a part of a granular object and the rest of it are discriminated, such as a defective part in a granular object.

[0034] Although one embodiment of the present invention and other embodiments have been described above, the above-mentioned embodiments of the invention are intended to facilitate understanding of the present invention and do not limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and the present invention includes equivalents thereof. Furthermore, the components described in the claims and specification may be combined or omitted within the scope of solving at least part of the above-mentioned problems or achieving at least part of the effects.

Claims

1. a region dividing step of marking the inside of a specific type of granular object in a learning image in which a plurality of types of granular objects are captured, and dividing image elements other than the specific type of granular object into regions; an identification labeling step of assigning an identification label to the specific type of granular matter extracted by the area dividing step; a training data creation step of performing the region division step and the identification label assignment step for each type of granular material other than the specific type of granular material to create training data for the plurality of types of granular materials; A learning step of machine learning the teacher data. A machine learning method for discriminating granular matter, comprising:

2. Further, an inference evaluation step of performing an inference evaluation of granular object discrimination on an evaluation image in which the granular object is captured; and an additional learning step of generating training data by adding an identification label to an additional learning image in which a granular object is captured, and further performing machine learning when the inference evaluation result of the inference evaluation step does not satisfy a predetermined discrimination accuracy. The machine learning method for determining granular matter according to claim 1 .

3. The additional learning image is an image of a granular object that does not satisfy a predetermined discrimination accuracy in the inference evaluation result. The machine learning method for determining granular matter according to claim 2 .

4. The plurality of types of particulate matter includes at least one of grains, seeds, and particulate crops. The machine learning method for determining granular matter according to any one of claims 1 to 3.

Images