Processing apparatus and processing system

Abstract

To provide a processing apparatus capable of grasping the determination result of a class to which an object article belongs and the likelihood of the determination result.SOLUTION: According to an embodiment, the processing apparatus includes an algorithm generation unit, a determination unit, and a notification control unit. The algorithm generation unit generates a classification algorithm for classifying a category to which an object belongs into a plurality of classes based on color data relating to the category to which the object belongs. The determination unit determines which class the object belongs to among the plurality of classes by using a photographed image of the object and the classification algorithm. The notification control unit notifies the information on the determination result of the determination unit about the class to which the object article belongs and the likelihood of the determination result derived using the classification algorithm.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] SUMMARY OF THE DISCLOSURE 1. Field of the Invention Embodiments of the invention relate to processing devices and processing systems. [Background technology]

[0002] Processing devices are used to classify items such as agricultural products, food products, and manufactured products based on color. In this case, the category to which the target item to be judged belongs is divided into a plurality of classes corresponding to the color, and the processing device judges to which of the plurality of divided classes the target item belongs. The judgment result regarding the class to which the target item belongs is then notified to the user of the processing device, etc. As described above, when classifying items based on color using a processing device, etc., there is a demand from the user of the processing device, etc. to be able to grasp the judgment result regarding the class to which the target item belongs, as well as the likelihood (degree of appropriateness) of the judgment result regarding the class to which the target item belongs. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 4171806 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to provide a processing device and a processing system that make it possible to grasp the judgment result regarding the class to which the target product belongs, as well as the likelihood of the judgment result. [Means for solving the problem]

[0005] According to an embodiment, the processing device includes an algorithm generating unit, a determining unit, and a notification control unit. The algorithm generating unit generates a classification algorithm for classifying a category to which the target item belongs into a plurality of classes based on color data related to the category to which the target item belongs. The determining unit determines which of the plurality of classes the target item belongs to using a photographed image of the target item and the classification algorithm generated by the algorithm generating unit. The notification control unit notifies information related to the determination result by the determining unit regarding the class to which the target item belongs and the likelihood of the determination result by the determining unit derived using the classification algorithm. Effect of the Invention

[0006] According to the present invention, it is possible to provide a processing device and a processing system that make it possible to grasp the judgment result regarding the class to which the target product belongs, as well as the likelihood of the judgment result. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram illustrating a processing system according to a first embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing an example of color data related to a category to which a target product belongs, which is used in processing by the processing execution unit in the first embodiment. [Diagram 3] FIG. 3 is a functional block diagram illustrating an example of processing in the processing execution unit in the first embodiment. [Figure 4] FIG. 4 is a schematic diagram illustrating an example of notification information displayed on a display unit in the first embodiment. [Diagram 5] FIG. 5 is a schematic diagram showing an example of notification information displayed on a display unit in a different display manner from the example shown in FIG. 4 in the first embodiment. [Figure 6] FIG. 6 is a schematic diagram showing an example of notification information displayed on a display unit in a different display manner from the examples of FIG. 4 and FIG. 5 in the first embodiment. [Figure 7]FIG. 7 is a schematic diagram showing an example of a usage mode of the processing apparatus according to the first embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an example of a process performed by the process execution unit in the first embodiment. [Figure 9] FIG. 9 is a schematic diagram showing an example of time changes after the start of a shooting operation by the shooting unit with respect to the determination result by the determination unit, the fluctuation state of the derived likelihood, and the update state of the notification information in the first embodiment. [Figure 10] FIG. 10 is a flowchart illustrating an example of processing performed by the processing execution unit when the processing device operates in the update-off mode in the first modified example. [Figure 11] FIG. 11 is a schematic diagram showing an example of a usage mode of the processing device according to the first modified example when the processing device is used to sequentially classify a plurality of target articles. [Figure 12] FIG. 12 is a schematic diagram showing an example of the change over time in the judgment result by the judgment unit, the fluctuation state of the derived likelihood, and the update state of the notification information after the start of the shooting operation by the shooting unit in the first variant example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The processing device (10) of the embodiment includes an algorithm generating unit (21), a determining unit (22), and a notification control unit (23). The algorithm generating unit (21) generates a classification algorithm for classifying a category to which the target item belongs into a plurality of classes based on color data related to the category to which the target item belongs. The determining unit (22) determines which of the plurality of classes the target item belongs to using a photographed image of the target item and the classification algorithm generated by the algorithm generating unit. The notification control unit (23) notifies information on the determination result of the determining unit for the class to which the target item belongs and the likelihood of the determination result of the determining unit derived using the classification algorithm. This allows a user of the processing device (10) to know the likelihood of the determination result together with the determination result for the class to which the target item belongs.

[0009] In the processing device (10) of the embodiment, the determination unit (22) uses a captured image of the target item to output information about the class to which the target item belongs from the classification algorithm in determining the class to which the target item belongs. The processing device (10) further includes a likelihood derivation unit (25), which derives the likelihood of the determination result by the determination unit (22) based on the information output from the classification algorithm in determining the class to which the target item belongs. This makes it possible to appropriately derive the likelihood of the determination result for the class to which the target item belongs by, for example, outputting information about the class to which the target item belongs from each of a plurality of classification algorithms that perform classification using different methods.

[0010] In the processing device (10) of the embodiment, the determination unit (22), in determining the class to which the target item belongs, uses a captured image of the target item to cause the classification algorithm to output information about the class to which the target item belongs. In the process of causing the classification algorithm to output information about the class to which the target item belongs, the determination unit (22) causes the classification algorithm to derive the likelihood of the determination result or an index related to the likelihood. This makes it possible to appropriately derive the likelihood of the determination result even when only one classification algorithm is used to determine the class to which the target item belongs. In addition, it becomes possible to appropriately derive the likelihood of the determination result without generating an algorithm for deriving the likelihood separately from the classification algorithm.

[0011] In the processing device (10) of the embodiment, the determination unit (22) sequentially performs a determination of the class to which the target item belongs while the photographed images of the target item are sequentially acquired. The notification control unit (23) updates notification information indicating the determination result and the likelihood of the determination result each time the determination unit determines the class to which the target item belongs until the determination result and the likelihood of the determination result become stable while the determination of the class to which the target item belongs is sequentially performed. The notification control unit (23) stops updating the determination result and the notification information indicating the likelihood of the determination result at least in response to the determination result and the likelihood of the determination result becoming stable while the determination of the class to which the target item belongs is sequentially performed. By stopping the update of the notification information at the timing when the determination result and the likelihood of the determination result become stable to a certain extent, the user of the processing device (10) can easily check the notification information to be notified.

[0012] The processing device (10) of the embodiment further includes an image capturing unit (15) that captures an image of the target product. This allows a user of the processing device (10) equipped with the image capturing unit (15) to grasp the likelihood of the determination result as well as the determination result regarding the class to which the target product belongs, by using the processing device (10) equipped with the image capturing unit (15).

[0013] The processing system (1) of the embodiment includes an image capturing unit (15), an algorithm generating unit (21), a determination unit (22), and a notification control unit (23). The algorithm generating unit (21) generates a classification algorithm for classifying a category to which the target item belongs into a plurality of classes based on color data related to the category to which the target item belongs. The determination unit (22) determines which of a plurality of classes the target item belongs to using the image of the target item captured by the image capturing unit (15) and the classification algorithm generated by the algorithm generating unit (21). The notification control unit (23) notifies information related to the determination result of the determination unit (22) for the class to which the target item belongs and the likelihood of the determination result of the determination unit (22) derived using the classification algorithm. This allows a user of the processing system (1) including the image capturing unit (15) to know the likelihood of the determination result together with the determination result for the class to which the target item belongs.

[0014] Hereinafter, an embodiment will be described with reference to the drawings.

[0015] (First embodiment) First, a first embodiment will be described as an example of an embodiment. FIG. 1 shows a processing system 1 according to the first embodiment. As shown in FIG. 1, the processing system 1 includes a processing device (classification device) 10. The processing device 10 includes a processing execution unit 11, a data storage unit 12, a communication interface 13, an imaging unit 15, and a display unit (notification unit) 16. The processing execution unit 11 includes an algorithm generation unit 21, a determination unit 22, and a notification control unit 23, and in the example of FIG. 1, the processing execution unit 11 includes a likelihood derivation unit 25. Each of the algorithm generation unit 21, the determination unit 22, the notification control unit 23, and the likelihood derivation unit 25 performs a part of the processing performed by the processing execution unit 11.

[0016] In one example, the processing device 10 is, for example, a mobile terminal such as a smartphone or a computer, and includes a processor or an integrated circuit, and a storage medium such as a memory. In the processing device 10, the processor or the integrated circuit functions as a processing execution unit 11, and the storage medium functions as a data storage unit 12. The processor or the integrated circuit of the processing device 10 includes any of a central processing unit (CPU), an application specific integrated circuit (ASIC), a microcomputer, a field programmable gate array (FPGA), and a digital signal processor (DSP). The processing device 10 may include only one integrated circuit or the like, or may include multiple integrated circuits or the like. The processing device 10 may also include only one storage medium or may include multiple storage media.

[0017] In the processing device 10, the processor executes a program stored in a storage medium or the like, whereby the processing execution unit 11 performs the processing described below. In one example, the program executed by the processor in the processing device 10 may be stored in a computer (server) connected via a network such as the Internet, or in a server in a cloud environment. In this case, the processor of the processing device 10 downloads the program via the network. For example, when the processing device 10 is a terminal device such as a smartphone, the processor of the processing device 10 can execute the processing described below by the processing execution unit 11 by downloading a dedicated application via the network.

[0018] In one example, the processing device 10 is configured as a server in a cloud environment. The infrastructure of the cloud environment is configured by a virtual processor such as a virtual CPU and a cloud memory. In the server in the cloud environment that serves as the processing device 10, the virtual processor or the like functions as a processing execution unit 11, and the processing described below is performed by the processing execution unit 11 through the virtual processor or the like. And the cloud memory functions as a data storage unit 12.

[0019] In the processing system 1, the processing device 10 may communicate with other processing devices via the communication interface 13. This allows the processing device 10 to receive information from other processing devices and transmit information to other processing devices. The photographing unit 15 is composed of a camera, a video camera, or the like. In the processing device 10, the photographing unit 15 photographs a subject. The photographing unit 15 may be provided separately from the processing device 10.

[0020] In the processing device 10, a part of the user interface is configured by the display unit 16. The display unit 16 can notify the user of the processing system 1 and the processing device 10 of information by displaying the information on a screen or the like. In one example, the display unit 16 is configured by a touch panel or the like. Information can be displayed on the display unit 16, and the user of the processing device 10 can input operations on the display unit 16. In this case, the display unit 16 is configured by, for example, a display screen of a mobile terminal.

[0021] In the processing device 10, notification of information is not limited to a screen display, and information may be notified by voice transmission or the like. In addition, in the processing device 10, operation members such as buttons and switches may be provided separately from the display unit 16, and operations may be input by the user of the processing device 10 or the like through the operation members. In this case, the operation members form a part of the user interface. In one example, a user interface including the display unit 16 may be provided separately from the processing device 10. In this case, the display unit 16 is formed from a monitor or the like separate from the processing device 10.

[0022] In the processing system 1 of this embodiment, the processing device 10 classifies items such as agricultural products, foods, and products based on color. Items classified using the processing device 10 include vegetables and fruits, as well as meat, processed foods, alcoholic beverages, and dairy products. Vegetables classified using the processing device 10 include tomatoes, cabbages, and green peppers, and fruits classified using the processing device 10 include mandarin oranges, apples, and grapes. Items classified using the processing device 10 also include non-food products such as painted products and pottery. Classification here refers to dividing an object into at least two or more categories according to a certain criterion. The certain criterion may be the variety of the object, the size or weight of the object, the quality or grade of the object, or the freshness of the object.

[0023] In the classification process of items based on color, the judgment unit 22 makes a judgment on the target item, which is the item to be judged. Here, "category" and "class" are defined as divisions for classification. A "category" is a larger division than a "class", and for example, "items" are divided into multiple categories. And one category is divided into multiple classes. Hereinafter, it is assumed that the target item to be judged belongs to one category, and the category to which the target item belongs is divided into multiple classes corresponding to the color. Also, it is possible to use a "major division" instead of a "category", and a "minor division" instead of a "class".

[0024] In the classification process of items based on color, the determination unit 22 determines to which of a plurality of classes the target item belongs in the category to which the target item belongs. The category to which the target item belongs is set, for example, based on an operation on a user interface by a user of the processing device 10. For example, a type of fruit such as mandarin oranges and apples, or a type of vegetable such as tomatoes and cabbage, may be set as the category to which the target item belongs. In one example, a type of meat food such as beef and chicken, a type of painted product such as painted walls, or a type of pottery such as ceramics may be set as the category to which the target item belongs.

[0025] Furthermore, the setting of multiple classes in the category to which the target product belongs is performed, for example, based on an operation on a user interface by a user of the processing device 10. In one example, mandarins are set as the category to which the target product belongs, and multiple classes are set for mandarins. As a result, mandarins, which are the category to which the target product belongs, are divided into multiple classes corresponding to colors. The determination unit 22 then determines to which of the multiple classes divided for mandarins the target product belongs.

[0026] In the classification process of items based on color, the process execution unit 11 acquires color data related to the category to which the target item belongs. For the category to which the target item belongs, for example, a variable color range is defined as a color range within which items belonging to that category can change. The color data related to the category to which the target item belongs indicates a plurality of pieces of color information included in the variable color range of that category. The color data related to the category to which the target item belongs indicates which of a plurality of divided classes each of the plurality of pieces of color information belongs to. Therefore, the color data related to the category to which the target item belongs indicates each of the plurality of pieces of color information included in the variable color range of that category in association with belonging class information, which is information about the class to which the item belongs.

[0027] Furthermore, the color data relating to the category to which the target item belongs indicates one or more pieces of color information for each of the multiple classes divided in that category. That is, the color data relating to the category to which the target item belongs indicates one or more pieces of color information for each of the multiple classes divided in that category. Therefore, in the color data relating to the category to which the target item belongs, there is no class among the multiple classes divided in that category for which color information is not indicated.

[0028] FIG. 2 shows an example of color data related to the category to which the target product belongs, which is used for processing by the processing execution unit 11 in this embodiment. In the example of FIG. 2, the category to which the target product belongs is divided into three classes A1 to A3. In the example of color data of FIG. 2, a plurality of pieces of color information included in the variable color range of the category to which the target product belongs are shown, and each of the plurality of pieces of color information is shown by RGB values. In the example of color data of FIG. 2, it is shown that the color information of RGB(200;160;100), RGB(100;220;50), RGB(130;130;130), RGB(140;130;135), and RGB(220;160;90) belong to classes A1, A2, A3, A3, and A1, respectively. Therefore, in the example of color data of FIG. 2, one or more pieces of color information are shown for each of the three classes A1 to A3 divided in the category to which the target product belongs.

[0029] The method of generating color data related to the category to which the target item belongs is not particularly limited. In one example, a plurality of items in the category to which the target item belongs and different from the target item are photographed as subjects by the photographing unit 15. Then, the processing execution unit 11 calculates color information such as RGB values ​​for each of the plurality of items photographed by the photographing unit 15 by performing image processing on the photographed image. Also, for each of the photographed plurality of items, belonging class information, which is information on the class to which the photographed plurality of items belongs, is input by an operation on the user interface by a user of the processing device 10 or the like. Then, the processing execution unit 11 generates color data related to the category to which the target item belongs by associating the calculated color information for each of the photographed plurality of items with the belonging class information. Note that, in this example, the photographed item is selected in a state in which color information based on the photographed image is calculated for each of the plurality of classes divided in the category to which the target item belongs.

[0030] In another example, instead of photographing a plurality of items, a plurality of colors shown on a color chart are photographed as subjects by the photographing unit 15. At this time, a plurality of colors included in the variable color range of the category to which the target item belongs are photographed. In this example, the processing execution unit 11 calculates color information for each of the plurality of colors photographed by the photographing unit 15 by performing image processing on the photographed image. Also, for each of the photographed plurality of colors, belonging class information is input by an operation on the user interface by a user of the processing device 10, etc. Then, the processing execution unit 11 generates color data related to the category to which the target item belongs by associating the calculated color information with the belonging class information for each of the photographed plurality of colors. Note that, in this example, for each of the plurality of classes divided in the category to which the target item belongs, the color photographed on the color chart is selected in a state in which color information based on the photographed image is calculated.

[0031] In another example, in generating color data related to the category to which the target item belongs, no photography is performed using the photography unit 15 or the like. In this case, color information for each of the multiple classes divided in the category to which the target item belongs is input by an operation on a user interface by a user of the processing device 10 or the like. In this example, for example, RGB values ​​are input as color information, and one or more pieces of color information for each of the multiple divided classes are input.

[0032] Furthermore, when multiple categories of items can be classified using the processing device 10, color data relating to each category is generated for each category. In one example, mandarins and apples can be classified using the processing device 10. In this case, color data is generated for mandarins in which color information is associated with class information to which the mandarins belong, and color data is generated for apples in which color information is associated with class information to which the mandarins belong.

[0033] FIG. 3 explains an example of processing in the processing execution unit 11 in this embodiment. As shown in FIG. 3, the algorithm generation unit 21 of the processing execution unit 11 acquires the above-mentioned color data related to the category to which the target item belongs. In the example of FIG. 3, the color data related to the category to which the target item belongs is stored in the data storage unit 12, and the algorithm generation unit 21 reads out the color data from the data storage unit 12. The algorithm generation unit 21 generates a classification algorithm based on the color data related to the category to which the target item belongs. The classification algorithm is generated as an algorithm for classifying the category to which the target item belongs into a plurality of classes.

[0034] In the generated algorithm, the category to which the target item belongs is classified into a plurality of classes by an appropriate method used in classification, for example, by any of a decision tree, SVM (support vector machine), etc. The algorithm generation unit 21 may generate only one classification algorithm, or may generate a plurality of classification algorithms. When a plurality of classification algorithms are generated, the plurality of classification algorithms classify the category to which the target item belongs into a plurality of classes by mutually different methods. In one example, a plurality of classification algorithms are generated, including a classification algorithm that performs classification by a decision tree and a classification algorithm that performs classification by SVM. The algorithm generation unit 21 stores the generated classification algorithm in the data storage unit 12.

[0035] The determination unit 22 acquires a captured image of the target item to be determined. The captured image of the target item is captured by the photographing unit 15. The determination unit 22 also acquires one or more classification algorithms generated by the algorithm generating unit 21. The determination unit 22 acquires the classification algorithm by, for example, reading out the classification algorithm stored in the data storage unit 12. The determination unit 22 uses the captured image of the target item and the classification algorithm to determine which of multiple classes the target item belongs to.

[0036] In determining the target item, the determination unit 22 extracts color information of the target item, such as RGB values, from the captured image of the target item. Then, the determination unit 22 inputs the extracted color information of the target item to each of one or more classification algorithms. As a result, information on the class to which the target item belongs is output from each of the one or more classification algorithms. Therefore, the determination unit 22 uses the captured image of the target item to cause the classification algorithm to output information on the class to which the target item belongs. The information output from the classification algorithm, i.e., the information on the class to which the target item belongs, may indicate the class itself to which the target item belongs, or may indicate an index or the like related to the class to which the target item belongs.

[0037] In one example, a photographed image of the target item may be input to each of the classification algorithms. In this case, in each of the classification algorithms, color information of the target item is extracted from the input photographed image, and processing is performed based on the extracted color information of the target item. In this example, information regarding the class to which the target item belongs is output from each of the one or more classification algorithms. Therefore, in this example, the determination unit 22 uses the photographed image of the target item to cause the classification algorithm to output information regarding the class to which the target item belongs.

[0038] In determining the target item, the determination unit 22 determines the class to which the target item belongs based on the information output from the classification algorithm. When only one classification algorithm is used, the determination unit 22 determines, for example, the class corresponding to the information output from the classification algorithm as the class to which the target item belongs. When multiple classification algorithms are used, the determination unit 22 calculates, for each of multiple classes divided in the category to which the target item belongs, the number of classification algorithms that have output that the class corresponds to the class to which the target item belongs, based on the information output from the multiple classification algorithms. Then, the determination unit 22 determines, among the multiple classes, the class with the largest number of classification algorithms that have output that the class corresponds to the class to which the target item belongs, as the class to which the target item belongs.

[0039] 3, the likelihood derivation unit 25 derives the likelihood (degree of suitability) of the judgment result for the class to which the target item belongs. The likelihood derivation unit 25 acquires information output from the classification algorithm in the judgment by the judgment unit 22 for the class to which the target item belongs. Then, the likelihood derivation unit 25 derives the likelihood of the judgment result by the judgment unit 22 based on the information output from the classification algorithm. The likelihood derivation unit 25 also derives information on the likelihood including the likelihood of the judgment result. The information on the likelihood, for example, indicates the probability (likelihood) that the target item belongs to the class indicated by the judgment result as the likelihood of the judgment result, and indicates the probability (likelihood) that the target item belongs to a class other than the class indicated by the judgment result.

[0040] When deriving the likelihood of the judgment result as in the example of Fig. 3, multiple classification algorithms are used in the judgment by the judgment unit 22, and information on the class to which the target item belongs is output from each of the multiple algorithms. Then, based on the information output from the multiple classification algorithms, the likelihood derivation unit 25 calculates the number of classification algorithms that output that the class corresponds to the class to which the target item belongs, for each of the multiple classes divided in the category to which the target item belongs. Then, the likelihood derivation unit 25 calculates the ratio between the multiple classes for the number of classification algorithms that output that the class corresponds to the class to which the target item belongs, and derives the likelihood of the judgment result by the judgment unit 22 based on the calculated ratio.

[0041] For example, assume that the category to which the target item belongs is divided into three classes A1 to A3, and four classification algorithms are used to determine the class to which the target item belongs. When color information of the target item is input to each of the four classification algorithms, three of the four classification algorithms output that class A1 corresponds to the class to which the target item belongs, and one classification algorithm outputs that class A2 corresponds to the class to which the target item belongs. In this case, the determination unit 22 determines that class A1, which has the largest number of classification algorithms that output that the class to which the target item belongs, is the class to which the target item belongs.

[0042] Furthermore, when information is output from four classification algorithms as described above, the ratio of the number of classification algorithms that output that the target product belongs to the class to which it belongs among the three classes A1 to A3 is A1:A2:A3=3:1:0. Therefore, the likelihood derivation unit 25 derives that the likelihood of the judgment result by the judgment unit 22, that is, the appropriateness of the judgment result that judges that the target product belongs to class A1, is 75%. Furthermore, the likelihood derivation unit 25 derives, as information on the likelihood, the probability (likelihood) that the target product belongs to classes A2 and A3 other than class A1 indicated by the judgment result. When information is output from four classification algorithms as described above, the likelihood derivation unit 25 derives the probability (likelihood) that the target product belongs to class A2 as 25% and the probability (likelihood) that the target product belongs to class A3 as 0%.

[0043] It should be noted that information output from the classification algorithm does not have to be used in deriving the likelihood of a judgment result for the class to which the target item belongs by the judgment unit 22. In one example, the judgment unit 22 inputs color information of the target item to the classification algorithm, and in the process of causing the classification algorithm to output information about the class to which the target item belongs, the classification algorithm derives the likelihood of a judgment result for the class to which the target item belongs, or an index related to the likelihood.

[0044] Here, it is assumed that the determination unit 22 performs a determination using a classification algorithm that classifies the category to which the target item belongs into three classes A1 to A3, and in the process of outputting information on the class to which the target item belongs from the classification algorithm, the likelihood of the determination result by the determination unit 22 is derived by the classification algorithm. In this case, for example, when color information of the target item is input to the classification algorithm, a probability α1 that the target item belongs to class A1 is calculated, and when the target item does not belong to class A1, a probability α2 that the target item belongs to class A2 is calculated. As a result, α1, (1-α1)×α2, and (1-α1)×(1-α2) are calculated as the probabilities (likelihoods) that the target item belongs to classes A1, A2, and A3, respectively.

[0045] In the above example, the class with the highest calculated probability among classes A1 to A3 is output from the classification algorithm as information regarding the class to which the target item belongs. Then, the determination unit 22 determines that the class with the highest calculated probability among classes A1 to A3 is the class to which the target item belongs. In addition, the probability calculated for the class determined to be the class to which the target item belongs is derived as the likelihood of the determination result. For example, if the determination unit 22 determines that the target item belongs to class A2, the calculated value of the probability (1-α1)×α2 is derived as the likelihood of the determination result.

[0046] In one example, in a classification algorithm, a category to which the target item belongs is classified into a plurality of classes using a decision tree. Then, by inputting color information of the target item into a classification algorithm using the decision tree, in the process of causing the classification algorithm to output information on the class to which the target item belongs, a Gini coefficient indicating the impurity of data is calculated for each node of the decision tree. In this case, the likelihood derivation unit 25 etc. derives the likelihood of the determination result by the determination unit 22 using one or more of the calculated Gini coefficients. Therefore, in this example, in the process of causing the classification algorithm to output information on the class to which the target item belongs, the classification algorithm calculates the Gini coefficient as an index related to the likelihood of the determination result for the class to which the target item belongs.

[0047] In any of the above-mentioned examples, one or more classification algorithms are used in deriving the likelihood of the determination result by the determination unit 22. That is, in both cases where the likelihood of the determination result by the determination unit 22 is derived based on information output from the classification algorithm, and where the likelihood of the determination result, etc. is derived by the classification algorithm in the process of outputting information about the class to which the target product belongs from the classification algorithm, the classification algorithm is used in deriving information about the likelihood, including the likelihood.

[0048] The notification control unit 23 acquires the judgment result by the judgment unit 22 for the class to which the target item belongs, and information on the likelihood of the judgment result by the judgment unit 22. Then, the notification control unit 23 generates notification information indicating the judgment result for the class to which the target item belongs and the likelihood of the judgment result, and causes the display unit 16, which serves as the notification unit, to notify the generated notification information. In this way, the user of the processing device 10 is notified of the judgment result for the class to which the target item belongs, and the likelihood of the judgment result. The notification information also indicates, as information on the likelihood of the judgment result, the probability (likelihood) that the target item belongs to a class other than the class indicated in the judgment result, etc.

[0049] 4 to 6 show examples of notification information displayed on the display unit 16 in this embodiment. Here, FIG. 5 is an example in which the display mode of the notification information is different from the example in FIG. 4, and FIG. 6 is an example in which the display mode of the notification information is different from the example in FIG. 4 and the example in FIG. 5. In each of the examples in FIG. 4 to FIG. 6, a determination result regarding the class to which the target product belongs is displayed. Then, in each of the examples in FIG. 4 to FIG. 6, class A1 is determined to be the class to which the target product belongs. In the example in FIG. 4, the likelihood of the determination result is shown as a numerical value, and the probability that the target product belongs to each of classes A2 and A3 other than class A1 is shown as a numerical value.

[0050] In the example of FIG. 5, the likelihood of the judgment result is shown by a valve chart, which is a kind of graph. In this case, the size of the valve corresponding to the class A1 shown in the judgment result changes in response to the change in the likelihood. Also, for each of the classes A2 and A3 other than the class A1 shown in the judgment result, the size of the valve changes in response to the change in the probability (likelihood) that the target product belongs to. In the example of FIG. 6, the likelihood of the judgment result is shown in an analog meter format, and the cursor 31 moves in response to the change in the likelihood of the judgment result. In the example of FIG. 6, the cursor 31 approaches the class A1 shown in the judgment result as the likelihood of the judgment result increases. Note that the method of notifying the notification information is not limited to displaying it on the display unit 16, and may be notified by voice or the like.

[0051] FIG. 7 shows an example of a usage mode of the processing device 10 according to this embodiment. In the example of FIG. 7, the processing device 10 is a mobile terminal such as a smartphone, and is equipped with a photographing unit 15 such as a camera and a display unit 16 such as a touch panel. A user 32 of the processing device 10 photographs the target item X to be judged by the photographing unit 15, and the processing execution unit 11 judges the class to which the target item X belongs as described above using the photographed image of the target item X. The processing execution unit 11 also derives the likelihood of the judgment result for the class to which the target item X belongs as described above. Then, the judgment result for the class to which the target item X belongs and notification information indicating the likelihood of the judgment result are displayed on the display unit 16 as described above.

[0052] When using the processing device 10 as in the example of FIG. 7, for example, a dedicated application is started in the processing device 10. Then, if the above-mentioned classification algorithm has already been stored in the data storage unit 12, the photographing operation by the photographing unit 15 is started, and the target item X is photographed. On the other hand, if the classification algorithm has not been stored in the data storage unit 12, color data related to the category to which the target item X belongs is generated by any of the above-mentioned methods, and the processing execution unit 11 generates a classification algorithm for classifying the category to which the target item X belongs into a plurality of classes based on the generated color data, as described above. Then, when the classification algorithm is generated, the photographing operation by the photographing unit 15 is started, and the target item X is photographed.

[0053] In one example, the process execution unit 11 automatically starts the photographing operation by the photographing unit 15 in response to the classification algorithm being stored or generated. In another example, an operation button or the like for inputting an operation to start the photographing operation is displayed on the display unit 16. Then, the photographing operation by the photographing unit 15 is started when a user of the processing device 10 or the like inputs an operation using the operation button or the like. When the photographing operation is started, multiple frames are photographed continuously, and photographed images of the target item T, i.e., photographed images by the photographing unit 15 are obtained sequentially.

[0054] FIG. 8 is a flowchart showing an example of processing performed by the processing execution unit 11 in this embodiment. The processing of the example of FIG. 8 is performed in a state in which the photographed images are sequentially acquired by the photographing unit 15. When the processing of the example of FIG. 8 is started, the processing execution unit 11 acquires the photographed image of the target item photographed by the photographing unit 15 (S101). Then, the determination unit 22 determines the class to which the target item belongs using the photographed image and a classification algorithm (S102). At this time, the determination of the class to which the target item belongs is performed in the same manner as in any of the above-mentioned examples. In addition, the processing execution unit 11 derives the likelihood of the determination result for the class to which the target item belongs (S103). At this time, the likelihood of the determination result is derived in the same manner as in any of the above-mentioned examples, and information on the likelihood including the likelihood is derived.

[0055] Then, the notification control unit 23 etc. judges whether the judgment result for the class to which the target item belongs has stabilized (S104). At this time, for example, it is judged that the judgment result for the class to which the target item belongs has stabilized in correspondence with the fact that the class indicated by the judgment result is the same for N consecutive frames (N is, for example, any natural number equal to or greater than 2) in the photographed images acquired sequentially. Note that the fact that the judgment result has stabilized means that the judgment result for the class to which the target item belongs no longer varies from frame to frame.

[0056] When the judgment result for the class to which the target item belongs has stabilized (S104-Yes), the notification control unit 23 and the like judge whether the likelihood of the judgment result for the class to which the target item belongs has stabilized (S105). At this time, for example, it is judged that the likelihood of the judgment result for the class to which the target item belongs has stabilized in correspondence with the fact that the fluctuation in the likelihood has fallen below a specified level for N consecutive frames in sequentially acquired photographic images. Note that the stabilization of the likelihood means that the likelihood of the judgment result for the class to which the target item belongs no longer fluctuates from frame to frame, or has barely fluctuated at all.

[0057] When the judgment result is not stable (S104-No) and when the likelihood is not stable (S105-No), the notification control unit 23 etc. updates the notification information indicating the judgment result in S102 and the likelihood of the judgment result derived in S103 to the latest information (S106). As a result, in the notification information to be notified, the judgment result for the class to which the target product belongs is updated to the latest judgment result, and the information on the likelihood including the likelihood of the judgment result is also updated to the latest information. Then, the process returns to S101, and the captured image captured in the next frame in S101 is obtained. Then, the processes from S101 onwards are performed sequentially.

[0058] On the other hand, if the judgment result is stable (S104-Yes) and the likelihood is stable (S105-Yes), the notification control unit 23 stops updating the notification information indicating the judgment result in S102 and the likelihood of the judgment result derived in S103 (S107). As a result, the judgment result for the class to which the target item belongs and information regarding the likelihood including the likelihood of the judgment result are held in the notification information without being updated. Then, the example process of FIG. 8 ends.

[0059] If the judgment result is stable (S104-Yes) and the likelihood is stable (S105-Yes), and the update of the notification information is stopped, the photographing operation by the photographing unit 15 may be stopped or continued. If the judgment result is stable (S104-Yes) and the likelihood is stable (S105-Yes), and the update of the notification information is stopped, the judgment by the judgment unit 22 for the class to which the target item belongs may be ended or continued. However, if the judgment result is stable (S104-Yes) and the likelihood is stable (S105-Yes), even if the judgment by the judgment unit 22 is made, the judgment result, likelihood, etc. in the notification information are not updated to the latest information.

[0060] In this embodiment, since the process shown in Fig. 8 is performed, in a state where photographed images are sequentially acquired, the determination unit 22 uses the acquired photographed image to determine the class to which the target item belongs each time the photographed image is acquired. Therefore, in a state where photographed images of the target item are sequentially acquired, the determination unit 22 sequentially performs determination of the class to which the target item belongs.

[0061] Also, since the process of the example of Fig. 8 is performed, in a state where the determination unit 22 is sequentially making a determination on the class to which the target item belongs, notification information indicating the determination result and the likelihood of the determination result is updated until the determination result by the determination unit 22 and the likelihood of the determination result become stable. At this time, the notification control unit 23 etc. updates the notification information every time the determination unit 22 determines the class to which the target item belongs. Also, since the process of the example of Fig. 8 is performed, in a state where the determination is sequentially made on the class to which the target item belongs, the notification control unit 23 etc. stops updating the notification information in response to the determination result on the class to which the target item belongs and the likelihood of the determination result becoming stable.

[0062] The processing device 10 in which the process of the example of FIG. 8 is performed by the process execution unit 11 can operate in an update-on mode in which the notification information is updated, and an update-off mode in which the notification information is not updated. The processing device 10 in which the process of the example of FIG. 8 is performed operates in the update-on mode when the shooting operation is started. Therefore, the start of the shooting operation corresponds to the start of the operation in the update-on mode. In addition, in a state in which the process of the example of FIG. 8 is performed, the processing device 10 continues to operate in the update-on mode until the determination result by the determination unit 22 and the likelihood of the determination result become stable. Then, in a state in which the process of the example of FIG. 8 is performed, the processing device 10 switches from the update-on mode to the update-off mode in response to the determination result by the determination unit 22 and the likelihood of the determination result becoming stable.

[0063] Therefore, the example of Fig. 8 shows the processing performed by the processing execution unit 11 in the update on mode, and the processing of the example of Fig. 8 ends by switching from the update on mode to the update off mode. Note that in the update off mode, if the notification information is not updated, the photographing operation may or may not be performed. Similarly, in the update off mode, if the notification information is not updated, the determination by the determination unit 22 regarding the class to which the target item belongs may or may not be performed.

[0064] FIG. 9 shows an example of time changes after the start of the photographing operation by the photographing unit 15 in the present embodiment, regarding the judgment result by the judgment unit 22, the variation state of the derived likelihood, and the update state of the notification information. In FIG. 9, the judgment result by the judgment unit 22 is shown for each frame in the photographing operation. In the example of FIG. 9, the photographing operation is started at time ts. Then, immediately after the start of the photographing operation, the judgment result for the class to which the target item belongs varies from frame to frame and is in an unstable state. Also, immediately after the start of the photographing operation, the likelihood of the judgment result also varies beyond a specified level. That is, immediately after the start of the photographing operation, the likelihood of the judgment result varies greatly from frame to frame and is in an unstable state. Therefore, immediately after the start of the photographing operation, the notification information is updated every time a judgment is made by the judgment unit 22.

[0065] In the example of Fig. 9, in the judgment using the captured image in frame Fa, the class to which the target item belongs is judged to be class A2, and the class indicated in the judgment result is the same for four consecutive frames. As a result, it is judged that the judgment result for the class to which the target item belongs is stable. Also, in the example of Fig. 9, in the judgment using the captured image in frame Fa, the fluctuation in the likelihood of the judgment result falls within a specified level for four consecutive frames. As a result, it is judged that the likelihood of the judgment result is stable.

[0066] In the example of Fig. 9, immediately after the class to which the target item belongs is determined using the captured image in frame Fa, it is determined that the determination result by the determination unit 22 and the likelihood of the determination result are stable. Therefore, at the time ta immediately after the class to which the target item belongs is determined using the captured image in frame Fa, the update of the notification information is stopped and a transition is made to the update off mode. Note that, in the example of Fig. 9, in synchronization with the stopping of the update of the notification information, the photographing operation by the photographing unit 15 is also stopped, and the determination of the class to which the target item belongs and the derivation of the likelihood of the determination result are also terminated.

[0067] As described above, in this embodiment, in classifying items by the processing device 10, in addition to the determination result of the class to which the target item belongs, information on the likelihood of the determination result derived using the classification algorithm is notified. This allows users of the processing device 10 and the processing system 1 to grasp the likelihood of the determination result as well as the determination result of the class to which the target item belongs.

[0068] By grasping the likelihood of the judgment result, the user of the processing device 10 can quickly evaluate the appropriateness of the setting in real time for the class that is the division of the category to which the target item belongs, i.e., for the class that is the division of the classification. In fact, when the judgment result and the likelihood of the judgment result are notified, it becomes possible to evaluate the appropriateness of the class (division) that is set in real time even if the number of target items to be judged is reduced, compared to when only the judgment result of the class to which the target item belongs is notified.

[0069] Furthermore, by early evaluation of the appropriateness of the classes set in real time, if the class settings are inappropriate, the class settings can be corrected early. For example, in the case where, in the information on the likelihood of the judgment result, the probability (likelihood) that the target product belongs to two or more classes is equal, which frequently occurs for a plurality of target products, the user of the processing device 10 or the like evaluates that the classes set in real time are inappropriate.

[0070] Here, suppose that the category to which the target item belongs is divided into three classes A1 to A3, and in the classification of the items, it frequently occurs that the probability (likelihood) that the target item belongs to class A1 is equal to class A2 for multiple target items. In this case, the user of the processing device 10 adjusts the balance between classes A1 to A3 by, for example, bringing the standard of class A2 closer to class A3, or sets a new class (division) in the middle between the standard of class A1 and the standard of class A2, thereby correcting the class setting.

[0071] If the probability (likelihood) that a target item belongs to two or more classes is equal frequently for multiple target items, the process execution unit 11 may notify, via the display unit 16, that it is recommended to modify the class settings. If the probability (likelihood) that a target item belongs to two or more classes is equal frequently for multiple target items, the process execution unit 11 may automatically modify the class settings from the real-time settings.

[0072] In one example of this embodiment, when determining the class to which the target item belongs, information about the class to which the target item belongs is output from the classification algorithm. Then, based on the information output from the classification algorithm when determining the class to which the target item belongs, the likelihood of the determination result in the determination unit is derived. Therefore, by outputting information about the class to which the target item belongs from each of a plurality of classification algorithms that perform classification in different ways, it becomes possible to appropriately derive the likelihood of the determination result for the class to which the target item belongs.

[0073] In addition, in one example of this embodiment, in determining the class to which the target item belongs, information about the class to which the target item belongs is output from the classification algorithm. Then, in the process of outputting information about the class to which the target item belongs from the classification algorithm, the classification algorithm derives the likelihood of the determination result or an index related to the likelihood. Therefore, even if only one classification algorithm is used to determine the class to which the target item belongs, it is possible to appropriately derive the likelihood of the determination result. Also, it is possible to appropriately derive the likelihood of the determination result without generating an algorithm for deriving the likelihood separately from the classification algorithm.

[0074] In addition, in this embodiment, in a state where photographed images of the target item are sequentially acquired, a determination is sequentially made as to the class to which the target item belongs. Then, in a state where a determination is sequentially made as to the class to which the target item belongs, updating of the notification information is stopped at least in response to the determination result for the class to which the target item belongs and the likelihood of the determination result becoming stable. By stopping the update of the notification information at a timing when the determination result and the likelihood of the determination result become stable to a certain extent, it becomes easier for a user of the processing device 10 to check the notification information notified by display on the display unit 16, etc.

[0075] Furthermore, in this embodiment, in a state where the judgment of the class to which the target item belongs is being performed sequentially, notification information indicating the judgment result and the likelihood of the judgment result are updated each time the class to which the target item belongs is judged until the judgment result and the likelihood of the judgment result become stable. Therefore, until the judgment result and the likelihood of the judgment result become stable, the judgment result of the class to which the target item belongs and the likelihood of the judgment result are updated, and the judgment result of the class to which the target item belongs and the likelihood of the judgment result are appropriately notified to the user of the processing device 10, etc.

[0076] In addition, in one example of this embodiment, the photographing operation by the photographing unit 15 is stopped in response to switching to the update-off mode in which the notification information is not updated. This eliminates the need for the user of the processing device 10 to perform an operation to stop the photographing operation, thereby reducing the effort of the user of the processing device 10. In addition, the occurrence of blurring due to an operation to stop the photographing operation is appropriately suppressed.

[0077] (Modification) In the above-mentioned embodiment, the process of the example of Fig. 8 is performed, and the update of the notification information is stopped in response to the determination result and the likelihood of the determination result for the class to which the target item belongs becoming stable, and the processing device 10 switches to the update-off mode. However, in a modified example, the update of the notification information is stopped in response to the determination result and the likelihood of the determination result for the class to which the target item belongs becoming stable, and a specified time has elapsed since the start of operation in the update-on mode. In this case, the condition for switching from the update-on mode to the update-off mode is that the determination result and the likelihood of the determination result for the class to which the target item belongs becoming stable, and a specified time has elapsed since the start of operation in the update-on mode.

[0078] In addition, in one modified example, in the update-off mode, the photographing operation by the photographing unit 15 is stopped, and no judgment is made regarding the class to which the target item belongs. Then, the update of the notification information is stopped in response to the fact that the judgment result regarding the class to which the target item belongs and the likelihood of the judgment result are stable, and a specified time has elapsed since the start of the photographing operation being performed in real time. In this case, the condition for switching from the update-on mode to the update-off mode is that the judgment result regarding the class to which the target item belongs and the likelihood of the judgment result are stable, and a specified time has elapsed since the start of the photographing operation being performed in real time. These modified examples also provide the same actions and effects as the above-mentioned embodiment and the like.

[0079] FIG. 10 is a flowchart showing an example of processing performed by the processing execution unit 11 when the processing device 10 operates in the update-off mode in the first modified example. In this modified example, even in the update-off mode, the photographing unit 15 performs a photographing operation, and a determination is made as to the class to which the target product belongs. For this reason, the processing of the example of FIG. 10 is performed in a state in which photographed images are sequentially acquired by the photographing unit 15. When the processing of the example of FIG. 10 is started, the processing execution unit 11 acquires a photographed image of the target product photographed by the photographing unit 15 (S111). Then, the determination unit 22 uses the photographed photographed image and a classification algorithm to determine the class to which the target product belongs (S112). At this time, the class to which the target product belongs is determined in the same manner as in any of the above-mentioned examples.

[0080] Then, the process execution unit 11 judges whether or not the transition condition for transitioning from the update-off mode to the update-on mode is satisfied (S113). Here, the variable color range of the category to which the target item belongs is defined as described above. In the judgment of S113, for example, it is judged that the transition condition is satisfied in correspondence with M frames (M is, for example, any natural number equal to or greater than 2) of consecutive captured images in which the majority of the captured images are outside the variable color range. The color outside the variable color range varies depending on the type of the target item. In one example, the color of human skin and the color of a conveyor belt are colors outside the variable color range.

[0081] In addition, in one example of this modified example, an acceleration sensor may be installed in the processing device 10. In this example, in S113, if the majority of captured images are colors outside the variable color range and M frames are consecutively captured, it is determined that the transition condition is satisfied, and also if the acceleration detected by the acceleration sensor is equal to or greater than a threshold value.

[0082] If the transition condition is not satisfied (S113-No), the process execution unit 11 maintains the operation of the processing device 10 in a mode in which the notification information is not updated (update-off mode) (S114). Then, the process returns to S111, and a captured image captured in the next frame is obtained in S111. Then, the processes from S111 onwards are performed sequentially. On the other hand, if the transition condition is satisfied (S113-Yes), the process execution unit 11 transitions the operation of the processing device 10 to a mode in which the notification information is updated (S115). By switching to the update-on mode, the process of the example of FIG. 10 ends.

[0083] In this modification, in the update-on mode, the processing device 10 operates in the same manner as in the above-described embodiment, and the processing execution unit 11 performs the same processing as the example processing in FIG. 8. Therefore, in the update-on mode, notification information indicating the judgment result and the likelihood of the judgment result is updated every time the class to which the target item belongs is judged until the judgment result and the likelihood of the judgment result for the class to which the target item belongs become stable. Then, the update of the notification information is stopped at least in response to the judgment result and the likelihood of the judgment result for the class to which the target item belongs becoming stable, and the processing device 10 switches to the update-off mode. At this time, the update of the notification information may be stopped in response to the judgment result and the likelihood of the judgment result for the class to which the target item belongs becoming stable and a specified time has elapsed since the start of operation in the update-on mode.

[0084] When using the processing device 10 to determine the class to which a target item belongs, multiple target items to be determined may be photographed in sequence, and the classes to which the multiple target items belong may be determined in sequence. That is, multiple target items may be classified in sequence using the processing device 10. Figure 11 shows an example of a usage mode of the processing device 10 when multiple target items X are classified in sequence using the processing device 10 according to this modified example.

[0085] In the example of FIG. 11, a plurality of target items X to be judged are placed on a belt conveyor 33. The plurality of target items X move from the upstream side to the downstream side by the belt conveyor 33 (arrow P). The plurality of target items X belong to the same category with respect to each other. Furthermore, the plurality of target items X are placed on the belt conveyor 33 with a gap between them. The processing device 10 sequentially photographs the plurality of target items X moving by the belt conveyor 33. The processing execution unit 11 of the processing device 10 sequentially judges the class to which the plurality of target items X belong, and sequentially derives the likelihood of the judgment results.

[0086] FIG. 12 shows an example of time changes after the start of the photographing operation by the photographing unit 15 in the present modified example, regarding the judgment result by the judgment unit 22, the fluctuation state of the derived likelihood, and the update state of the notification information. In FIG. 12, the judgment result by the judgment unit 22 is shown for each frame in the photographing operation. In addition, in the judgment result by the judgment unit, "Z" indicates that colors outside the variable color range of the category to which the target product belongs occupy the majority of the photographed image. In the example of FIG. 12, the processing device 10 operates in the update on mode until time tb. Therefore, until time tb, the notification information is updated every time a judgment is made by the judgment unit 22.

[0087] In the example of FIG. 12, in the determination using the captured image in frame Fb, the class to which the target item belongs is determined to be class A2, and the class indicated in the determination result is the same for four consecutive frames. As a result, it is determined that the determination result for the class to which the target item belongs is stable. Also, in the example of FIG. 12, in the determination using the captured image in frame Fb, the fluctuation in the likelihood of the determination result is within a specified level for four consecutive frames. As a result, it is determined that the likelihood of the determination result is stable. Therefore, in the example of FIG. 12, at time tb immediately after the determination of the class to which the target item belongs using the captured image in frame Fb, the update of the notification information is stopped, and a transition is made to the update off mode.

[0088] In this modified example, even in the update-off mode, the photographing operation by the photographing unit 15 is performed, and the judgment by the judgment unit 22 continues. Therefore, in the example of FIG. 12, the judgment using the photographed image continues even after the transition to the update-off mode at time tb. In the example of FIG. 12, in the judgment using the photographed image in frame Fc after time tb, three consecutive frames of photographed images show a majority of colors outside the variable color range. At this time, for example, three consecutive frames of photographed images show a majority of the color of a person's hand or the color of a conveyor belt. As a result, it is judged that the transition condition for transitioning from the update-off mode to the update-on mode is satisfied.

[0089] Therefore, in the example of Fig. 12, at time tc immediately after the captured image in frame Fc is used to determine the class to which the target item belongs, the mode transitions to update-on mode. By transitioning to update-on mode, the class to which the next target item belongs is determined, and the likelihood of the determination result is derived. Then, after time tc when the mode transitions to update-on mode, the notification information is updated every time a determination is made by the determination unit 22. This modified example also provides the same actions and effects as the above-mentioned embodiment and the like.

[0090] In addition, in this modified example, the update-off mode is switched to the update-on mode by having the photographing unit 15 photograph a person's hand, a conveyor belt, or the like. Therefore, after the processing device 10 is switched to the update-off mode in response to the completion of the determination of the class to which a certain target item belongs, the processing device 10 is put into a state in which notification information indicating the determination result for the next target item is notified by having the photographing unit 15 photograph a person's hand or a conveyor belt, or the like. This reduces the effort of the user of the processing device 10 and improves work efficiency when the classes to which multiple target items belong are determined sequentially, that is, when multiple target items are classified sequentially.

[0091] In addition, in this modified example, when the mode transitions to the update-off mode, the processing execution unit 11 may store the retained notification information in the data storage unit 12 or the like, or output it to an external processing device, during the time until the mode transitions back to the update-on mode. This makes it possible to properly store, for each target item, notification information indicating the judgment result for the class to which the target item belongs and the likelihood of the judgment result, even when multiple target items are classified sequentially. Therefore, even when multiple target items are classified sequentially, the user of the processing device 10 or the like can properly grasp, for each target item, the judgment result for the class to which the target item belongs and the likelihood of the judgment result.

[0092] According to at least one of these embodiments, the determination result of the determination unit for the class to which the target product belongs and information on the likelihood of the determination result of the determination unit derived using a classification algorithm are notified, thereby making it possible to provide a processing device and a processing system that make it possible to grasp the likelihood of the determination result together with the determination result for the class to which the target product belongs.

[0093] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0094] 1... processing system, 10... processing device, 11... processing execution unit, 12... data storage unit, 15... imaging unit, 16... display unit, 21... algorithm generation unit, 22... judgment unit, 23... notification control unit, 25... likelihood derivation unit.

Claims

1. an algorithm generating unit that generates a classification algorithm for classifying a category to which a target item belongs into a plurality of classes based on color data related to the category to which the target item belongs; a determination unit that determines to which of the multiple classes the target item belongs, using the captured image of the target item and the classification algorithm generated by the algorithm generation unit; a notification control unit that notifies the determination result by the determination unit regarding the class to which the target product belongs, and information regarding the likelihood of the determination result by the determination unit derived using the classification algorithm; A processing device comprising:

2. the determination unit, in determining the class to which the target item belongs, causes the classification algorithm to output information regarding the class to which the target item belongs by using the captured image of the target item; The method further includes a likelihood derivation unit that derives the likelihood of the judgment result in the judgment unit based on the information output from the classification algorithm in the judgment of the class to which the target item belongs. The processing device of claim 1.

3. the determination unit, in determining the class to which the target item belongs, causes the classification algorithm to output information regarding the class to which the target item belongs by using the captured image of the target item; The determination unit causes the classification algorithm to derive the likelihood of the determination result or an index related to the likelihood in a process of outputting the information on the class to which the target item belongs from the classification algorithm. The processing device of claim 1.

4. The determination unit sequentially determines a class to which the target item belongs while the captured images of the target item are being sequentially acquired, the notification control unit, in a state in which the determination of the class to which the target item belongs is being performed sequentially, updates notification information indicating the determination result and the likelihood of the determination result each time the determination unit determines the class to which the target item belongs, until the determination result and the likelihood of the determination result for the class to which the target item belongs become stable; the notification control unit, in a state in which the determination for the class to which the target item belongs is being performed sequentially, stops updating the notification information indicating the determination result and the likelihood of the determination result at least in response to the determination result for the class to which the target item belongs and the likelihood of the determination result becoming stable. The processing device according to any one of claims 1 to 3.

5. The processing device according to claim 1 , further comprising an image capturing unit that captures the image of the target item.

6. With the photography department; an algorithm generating unit that generates a classification algorithm for classifying a category to which a target item belongs into a plurality of classes based on color data related to the category to which the target item belongs; a determination unit that determines to which of the multiple classes the target item belongs, using the image of the target item captured by the imaging unit and the classification algorithm generated by the algorithm generation unit; a notification control unit that notifies information regarding a determination result by the determination unit regarding a class to which the target product belongs and a likelihood of the determination result by the determination unit derived using the classification algorithm; A processing system comprising:

Images