Sensory evaluation method and sensory evaluation system

Abstract

To provide a sensory evaluation having high generality, and facilitating the stability of an evaluation result.SOLUTION: An evaluation group comprising a plurality of evaluation objects is divided into two subgroups based on a result of having an evaluator compare each evaluation object and evaluation reference with a predetermined standard. Then, each subgroup is further divided into two subgroups based on a comparison between each evaluation object and evaluation reference in each subgroup. The division of subgroups is repeated until the number of evaluation objects in each subgroup is equal to or less than a specified value. Within each of the final subgroups, the evaluator is asked to rank the evaluation objects with respect to the standard.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to a sensory evaluation method and a sensory evaluation system. [Background technology]

[0002] Conventionally, sensory evaluation has been performed in which an object or information to be evaluated is evaluated by the five human senses. For example, Non-Patent Documents 1 and 2 evaluate the ripeness of tomatoes through human vision. In these studies, multiple categories and category labels are set according to the ripeness of tomatoes. Then, the evaluator categorizes the tomatoes to be evaluated based on the set categories. For each category, the evaluator assigns an evaluation object that is appropriate to the content of the label attached to that category. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Wan, P., Toudeshki, A., Tan, H., & Ehsani, R. (2018). Original papers A methodology for fresh tomato maturity detection using computer vision. Computers and Electronics in Agriculture, 146, 43-50. https: / / doi.org / 10.1016 / j.compag.2018.01.011 [Non-Patent Document 2] El-Bendary, N., El Hariri, E., Hassanien, AE, & Badr, A. (2015). Using machine learning techniques for evaluating tomato ripeness. Expert Systems with Applications, 42(4), 1892-1905. https: / / doi.org / 10.1016 / j.eswa.2014.09.057 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the above-mentioned evaluation method, since there is no general standard for setting categories, the labeling of categories and the number of categories are different for each evaluation. For example, in Non-Patent Document 1, three categories labeled "Green", "Orange" and "Red" are set, whereas in Non-Patent Document 2, five categories labeled "Green and Breaker", "Turning", "Pink", "Light Red" and "Red" are set. Since evaluators classify evaluation targets according to the labels and the number of categories, as described above, if the category settings are different, the results are likely to be different even for evaluation of the same evaluation target. In addition, since there is no objective standard for label naming, when the contents of the labels are abstract contents such as color names, the results of categorization are likely to differ depending on the evaluator. In other words, the evaluation results are likely to lack stability. In addition, the evaluation results depend on the category settings. For this reason, it is difficult to compare evaluation results with different category settings, and the generality of the evaluation results is low.

[0005] An object of the present invention is to provide a sensory evaluation method and a sensory evaluation system that are highly versatile and can easily ensure the stability of evaluation results. [Means for solving the problem]

[0006] The sensory evaluation method according to the present invention is a method for evaluating evaluation objects that generate stimuli that can be perceived by humans, and includes a comparison step in which an evaluator compares the stimuli generated by each of the evaluation objects belonging to an evaluation group consisting of a plurality of the evaluation objects with a reference stimulus on a predetermined scale based on the perception of the stimuli; a division step in which the evaluation group is divided into two subgroups on the predetermined scale based on the comparison result of the comparison step; a repeating step in which the comparison step and the division step are repeated one or more times to generate a plurality of subgroups such that the number of elements in each subgroup is within a predetermined range, and a ranking step in which the evaluator assigns a rank on the predetermined scale to all of the evaluation objects belonging to the subgroup in each of the plurality of subgroups generated by the repeating step based on the perception of the stimuli.

[0007] According to the sensory evaluation method of the present invention, as shown in the examples described later, it is easy to ensure the stability of the evaluation results, that is, the rankings given to the evaluation objects. Furthermore, since the rankings are not based on the setting of categories, the evaluation results are easy to compare, unlike the evaluation results of the prior art that depend on the setting of categories. For example, when comparing the evaluation results, a category that serves as a comparison standard may be set anew, and the evaluation results may be classified using the category. In this case, since each evaluation result has already been ranked with respect to the entire evaluation object, it is not time-consuming to categorize them. In other words, the evaluation results are highly generalizable. As a result, the stability of the evaluation results is easily ensured, and a highly versatile sensory evaluation method is realized.

[0008] The "predetermined range" of the present invention may be any range in which the number of elements in each subgroup is relatively small. For example, it may be a range in which the number of elements in each subgroup is equal to or less than a predetermined value, or a range in which the number of elements in more than half of the subgroups is equal to or less than a predetermined value.

[0009] In the sensory evaluation method of the present invention, it is preferable that the evaluation object is an image or a sound, the predetermined scale is a scale related to the appearance of the image or the sound, and the perception of the stimulus is visual or auditory. In this way, the stability of the evaluation result for the image or the sound is easily ensured, and it is highly generalizable.

[0010] In the sensory evaluation method of the present invention, it is preferable that in the comparison step, the evaluation object and a reference image or a reference sound are outputted from an output device, and the output evaluation object and the reference image or the reference sound are compared by an evaluator. In this way, the comparison can be smoothly performed between the evaluation object and the reference image or sound by making the comparison according to the output contents of the output device.

[0011] In the sensory evaluation method of the present invention, it is preferable that the evaluation object generates a stimulus that can be perceived by any one of the senses of smell, taste, and touch, and the predetermined scale is a scale related to any one of the smell, taste, and touch corresponding to the stimulus. This makes it easier to ensure the stability of the evaluation results regarding the smell, taste, or touch, and is highly generalizable.

[0012] In the sensory evaluation method of the present invention, it is preferable that identification information for distinguishing each of the plurality of evaluation objects from the other evaluation objects is set for each of the plurality of evaluation objects, and in the comparison step, the identification information of one of the plurality of evaluation objects is displayed on a display device, and the evaluator compares the stimulus generated by the evaluation object corresponding to the displayed identification information with the reference stimulus. In this way, the comparison can be made smoothly between the evaluation object and the reference stimulus by making the comparison according to the display content on the display device.

[0013] In addition, in the sensory evaluation method of the present invention, the step of acquiring the reference stimulus generator to be used in the comparison step preferably includes a reference extraction step of extracting the evaluation objects as reference evaluation objects from the entire evaluation objects, the number of which corresponds to a predetermined number of the division steps, or more, from the entire evaluation objects, a reference ranking step of assigning a rank to the entire reference evaluation objects with respect to the predetermined scale, and a reference assignment step of assigning the reference evaluation objects to each of the predetermined number of division steps based on the rank assigned in the reference ranking step. This makes it possible to appropriately acquire the reference stimulus generator to be used in the comparison step.

[0014] In addition, a sensory evaluation system according to another aspect of the present invention is a system for evaluating a plurality of evaluation targets, the system comprising one or more output devices for outputting images or sounds and one or more computers, the one or more computers comprising an input unit and a control unit for receiving user input, the control unit executing an output process for causing the output device to display related information related to two or more of the plurality of evaluation targets, a ranking acquisition process for acquiring a ranking for the evaluation targets related to the related information output by the output device in the output process based on the user input received by the input unit, and a determination process for determining a ranking of the entire plurality of evaluation targets on the predetermined scale by repeatedly executing the ranking acquisition process while changing the combination of the two or more evaluation targets for which the output device outputs the related information.

[0015] According to the sensory evaluation system of the present invention, as shown in the examples described later, the stability of the evaluation results, i.e., the rankings given to the evaluation objects, is high. Furthermore, since the rankings are not based on the setting of categories, the evaluation results are easy to compare, unlike the evaluation results of the prior art that depend on the setting of categories. For example, when comparing the evaluation results, a category that serves as a comparison standard may be set anew, and the evaluation results may be classified using the category. In this case, since each evaluation result has already been ranked with respect to the entire evaluation object, it is not time-consuming to categorize them. In other words, the evaluation results are highly generalizable. As a result, a sensory evaluation method that provides highly stable and versatile evaluation results is realized.

[0016] The related information according to the present invention may be the evaluation target itself. This corresponds to the case where the evaluation target is, for example, an image or a sound. Furthermore, the related information according to the present invention may not be the evaluation target itself, but may be information representing the evaluation target, such as identification information.

[0017] In the sensory evaluation system of the present invention, it is preferable that the number of pieces of related information simultaneously output in the output process is 2. With this, the ranking of the evaluation targets can be easily obtained by repeating the comparison of two evaluation targets.

[0018] In the sensory evaluation system of the present invention, it is preferable that the sensory evaluation system includes a plurality of the output devices and a plurality of the computers that can communicate with each other, the plurality of output devices correspond to the plurality of computers, respectively, and the output process and the ranking acquisition process are executed in each of the plurality of computers, and the determination process is executed by the cooperation of all of the plurality of computers. In this way, the output process and the ranking acquisition process are executed in each of the plurality of computers, and the determination process is executed by the cooperation of all of the plurality of computers. Therefore, the time required to assign rankings to evaluation objects is likely to be shortened.

[0019] According to a neural network learning method according to yet another aspect of the present invention, processing is carried out by the sensory evaluation method or sensory evaluation system of the present invention, and data indicating the evaluation objects ranked thereby and the rankings are used as teacher data. Learning is performed based on highly stable and general evaluation results obtained by the sensory evaluation method or sensory evaluation system of the present invention, which allows for highly versatile learning. [Brief description of the drawings]

[0020] [Figure 1] 1 is a block diagram showing a configuration of an evaluation support device according to a first embodiment which is one embodiment of the present invention. [Diagram 2] 2 is an example of a display screen on the display of FIG. 1 showing an image showing an evaluation target and a reference image. [Diagram 3] 2 is an example of a display screen on the display of FIG. 1 which displays a display for assigning a rank to images to be evaluated. [Figure 4] 2 is a flowchart showing the flow of a sensory evaluation method using the evaluation support device of FIG. 1. [Diagram 5] 5 is a flowchart showing a detailed flow of the criterion extraction step in FIG. 4. [Figure 6] 6 is a graph showing a specific example of candidate extraction performed according to the flowchart of FIG. 5 when the sample is a vegetable (or an image thereof). [Figure 7] 6 is a graph showing another specific example of candidate extraction performed according to the flowchart of FIG. 5 when the sample is a vegetable (or an image thereof). [Figure 8] 8 is a graph showing a specific example of further extracting a portion from the samples extracted in the candidate extraction shown in FIGS. 6 and 7 . [Figure 9] 5 is a flowchart showing a detailed flow of the division and ranking step in FIG. 4. [Figure 10] FIG. 9 is a conceptual diagram showing a situation in which the evaluation criteria extracted as in FIG. 8 are assigned to each division step. [Figure 11] FIG. 13 is a distribution diagram showing the evaluation results of an image of cherry tomatoes according to an embodiment of the present invention. [Figure 12] FIG. 2 is a block diagram showing a configuration of an evaluation support system according to a second embodiment which is one embodiment of the present invention. [Figure 13] 13 is an example of a display screen on the display of FIG. 12 showing an image showing an evaluation target and a reference image. [Figure 14] 11 is a display screen of a display according to another embodiment of the present invention. [Figure 15] 13 is a display screen of a display according to yet another embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] [First embodiment] A sensory evaluation method according to a first embodiment of the present invention will be described below. The sensory evaluation method according to this embodiment evaluates an evaluation object by utilizing the perception of a person (evaluator) to a stimulus generated by the evaluation object. The main senses assumed as perception are sight, hearing, smell, taste, and touch. Any one of these is used for one evaluation.

[0022] The evaluation object of the sensory evaluation method of this embodiment is an object that generates a stimulus to the above senses. For example, the evaluation object may be an object that generates a stimulus to the senses of smell, taste, or touch, such as an odor, taste, or touch, in addition to an object that generates a stimulus to the senses of sight or hearing, such as an image or sound. In this embodiment, an evaluation group consisting of a plurality of evaluation objects is evaluated.

[0023] In this embodiment, as an example, it is assumed that the sensory evaluation method is executed using an evaluation support device 1. The evaluation support device 1 is a device used in the sensory evaluation method according to this embodiment, and has various input devices and output devices (hereinafter, these may be collectively referred to as input / output devices) and a control unit 40, as shown in Fig. 1 as an example. The input / output devices are a display 10 (corresponding to the display device of the present invention), a speaker 20, and an input unit 30 shown in Fig. 1. The input unit 30 includes a pointing device such as a mouse, a touch panel, or a touch pad, and a keyboard.

[0024] The evaluation support device 1 is configured by a computer equipped with or connected to the above-mentioned input / output devices. The computer is constructed by hardware such as a CPU (Central Processing Unit), a memory device (hereinafter referred to as memory), various interfaces, and software such as program data stored in the memory. This software can be distributed by downloading via the Internet or by various recording media. The control unit 40 is constructed by the functioning of these hardware based on the software.

[0025] The control unit 40 holds target data, which is data related to the evaluation target, and reference data, which is the standard for evaluation. The target data and reference data are input to the evaluation support device 1 in advance. There are two types of target data and reference data. The first type of data is data for directly generating a stimulus to the senses by output from the display 10 and the speaker 20 (hereinafter, these may be collectively referred to as output devices). This corresponds to image data and sound data. Image data is data for outputting an image related to the evaluation on the display 10. This image generates a visual stimulus related to the evaluation. Sound data is data for outputting a sound from the speaker 20. This sound generates a auditory stimulus related to the evaluation.

[0026] The second type of data does not directly generate a stimulus for perception, but is data indicating identification information (hereinafter referred to as a target ID or a reference ID) given to an evaluation target or evaluation standard. The target ID and the reference ID are each composed of letters, numbers, and various symbols, and are output by the display 10. The actual stimulus is generated from a real sample to which the target ID or the reference ID is given.

[0027] The control unit 40 outputs to an output device (corresponding to the output device of the present invention) an image, sound, object ID, or reference ID indicated by the object data and reference data for each of a plurality of evaluation objects included in an evaluation group. Fig. 2(a) is an example in which an image indicating an evaluation object (hereinafter referred to as object image) and a reference image (hereinafter referred to as reference image) are displayed on the display 10. Fig. 2(b) is an example in which an object ID and a reference ID are displayed on the display 10.

[0028] The evaluator compares the evaluation object and the evaluation criteria on a predetermined scale based on his / her perception of the stimuli generated by the evaluation object and the evaluation criteria, respectively, in accordance with the output from the output device (corresponding to the comparison step of the present invention). When the output device outputs images or sounds, these images and sounds directly generate the stimuli related to the evaluation. When the output device outputs an object ID and a criteria ID, the stimuli related to the evaluation are generated from the actual samples indicated by the object ID and the criteria ID, respectively.

[0029] The predetermined scale is a scale related to perception and is used to rank a plurality of evaluation targets. For example, the scale can classify any two evaluation targets among the plurality of evaluation targets into a high one and a low one in the above-mentioned ranking. The scale may be a relatively objective one such as size, color, or pitch, or a relatively subjective one. For example, the quality or maturity of the evaluation target, "suitability for lunch", etc. may be used as the scale.

[0030] Based on the output of the output device, the evaluator inputs the ranking of the above scale to the evaluation support device 1 through the input unit 30. The user input is performed, for example, by touching the touch panel with a finger or the like, or by moving the cursor image and operating a button using a mouse as shown in FIG.

[0031] The control unit 40 performs a process for all evaluation objects by having the user input whether each evaluation object in the evaluation group is ranked high or low with respect to the evaluation criterion. As a result, the control unit 40 divides the evaluation group into a subgroup consisting of objects ranked higher than the evaluation criterion and a subgroup consisting of objects ranked lower (corresponding to the division step of the present invention). One of the two subgroups obtained by this division has a higher rank with respect to the above-mentioned scale than the other. The control unit 40 performs the same process on each of the divided subgroups, thereby dividing each subgroup into smaller subgroups. The control unit 40 repeats this division until the number of elements of the evaluation objects belonging to each subgroup becomes equal to or less than a predetermined number (corresponding to the repeat step of the present invention). The predetermined number may be, for example, around 10. As a result, a plurality of subgroups with mutual ranks determined with respect to the above-mentioned scale are obtained.

[0032] FIG. 10 shows an example of how an evaluation group is divided into subgroups as described above. The two subgroups resulting from each division are arranged in the figure with the higher quality one on the left and the lower quality one on the right. This shows the state in which division proceeds from top to bottom in a tree structure. In this example, it is assumed that an evaluation group consisting of 100 tomato images is divided based on quality. Criteria A to G indicate the evaluation criteria used in each division step. By dividing the original evaluation group into three stages using these criteria, the original evaluation group is finally divided into eight subgroups (1) to (8). The subgroups (1) to (8) are ranked higher on the left and lower on the right.

[0033] Next, the control unit 40 ranks the evaluation targets in each subgroup in which the number of elements is equal to or less than a predetermined number based on a user input through an input / output device (corresponding to a ranking step of the present invention). For example, the control unit 40 causes the display 10 to display images or target IDs of the evaluation targets indicated by the target data for each subgroup. FIG. 3 shows a screen of the display 10 displaying images of the evaluation targets as an example. The evaluator selects the images or target IDs of the evaluation targets in an order according to the above-mentioned scale by a user input through the input unit 30. The control unit 40 acquires the ranking of the evaluation targets based on the order in which the evaluator selected the images or target IDs of the evaluation targets through such user input. The control unit 40 generates evaluation result data indicating the ranking of the acquired evaluation targets. The evaluation result data is output to the display 10 in the form of numerical values, graphs, etc., written to various recording media, or output to the outside through a communication network such as the Internet.

[0034] As a result, the rankings between the subgroups and the rankings of the evaluation objects within each subgroup are all determined for the above scale. This determines the rankings for all evaluation objects within the original evaluation group. FIG. 10 shows, as an example, a situation in which rankings are assigned to each of the subgroups (1) to (8). It also shows a situation in which the rankings within all the subgroups are combined according to the subgroup rankings to assign a final ranking to all 100 images.

[0035] A specific example of the sensory evaluation method according to this embodiment will be described below with reference to Fig. 4 to Fig. 7. In this method, some steps are performed by a person, and other steps are performed by the evaluation support device 1. All steps may be performed by a person.

[0036] In this sensory evaluation method, it is assumed that the sample group to be evaluated is divided into evaluation objects and evaluation criteria, and the evaluation objects are ranked based on the evaluation criteria. For this reason, this sensory evaluation method has two steps, criterion extraction (S1) and division and ranking (S2), as shown in Figure 4.

[0037] The criterion extraction in S1 is a step of acquiring evaluation criteria to be used in the division and ranking step from the original sample group. In this step, as shown in FIG. 5, the evaluator extracts a predetermined number of samples (e.g., 20) from the original sample group as candidates for extracting evaluation criteria (S11). It is preferable that the extraction of candidates is performed as uniformly as possible with respect to the scale used in the division and ranking step in S2. As an example, a plurality of categories according to the evaluation scale are prepared, and a predetermined number of samples are extracted from the original sample group so that the number of elements in each category is the same. FIG. 6 shows a specific example of candidate extraction when the sample is a vegetable (or an image thereof). The four categories in FIG. 6 are "no scratches", "slight scratches", "moderate scratches", and "large scratches". Five samples are extracted for each category. FIG. 7 shows another example of candidate extraction when the sample is a vegetable (or an image thereof). In this example, it is assumed that when the evaluation scale is the quality of the sample, candidates are extracted based on the size of scratches present on the sample. Then, a predetermined number of samples are taken so that the frequency of the flaws in the sample group is as close as possible to the original sample group. The step S11 corresponds to the criterion extraction step of the present invention.

[0038] Next, the evaluator ranks the samples taken in S11 (S12). The rank is a rank on the evaluation scale used in the division and ranking in S2. The evaluator ranks the samples based on their perception of the stimuli generated by the samples. Note that the step of S12 corresponds to the criterion ranking step of the present invention.

[0039] Next, the evaluator extracts evaluation criteria from the samples ranked in S12 (S13). The number of criteria to be extracted is the number of times the samples are divided into subgroups in the division and ranking in S2. Of the original sample group, the samples remaining after the criteria have been extracted in S13 become the evaluation targets for the division and ranking in S2, that is, the evaluation groups.

[0040] The extraction in S13 is performed so that the evaluation criteria are evenly distributed for the rankings given in S12. Figure 8 shows a specific example of extraction performed when the samples are images of vegetables. In this example, it is assumed that 20 tomato images are ranked according to quality. The number of images to be extracted is 7. In the figure, the numbers 1 to 20 indicate the rankings of the images.

[0041] In the example of FIG. 8, the extraction of the criteria is performed in the following order. In the figure, the alphabets A to G indicate the criteria. First, the criterion A used for the first stage division performed in S2 is extracted. As the criterion A, the 10th ranked image corresponding to the rank near the center of the 20 images is extracted. Next, the criteria B and C used for the second stage division performed in S2 are extracted. As the criterion B, the 5th ranked image corresponding to the rank near the center of the images on the higher quality side than A is extracted. As the criterion C, the 16th ranked image corresponding to the rank near the center of the images on the lower quality side than A is extracted. Next, the criteria D to G used for the third stage division performed in S2 are extracted. As the criterion D, the 3rd ranked image corresponding to the rank near the center of the images on the higher quality side than B is extracted. As the criterion E, the 8th ranked image corresponding to the rank near the center of the images from A to B is extracted. As the criterion F, the 13th ranked image corresponding to the rank near the center of the images from A to C is extracted. As the standard G, from among the images on the lower quality side than C, the image ranked 18th, which corresponds to a ranking near the middle, is extracted.

[0042] The division and ranking step S2 based on the evaluation criteria extracted in the above-mentioned criterion extraction step S1 is a step for ranking the evaluation targets in the evaluation group. In this step, first, as shown in Fig. 9, the evaluator inputs the evaluation criteria extracted in the criterion extraction step S1 to the evaluation support device 1 so that they are assigned to the division into subgroups that is performed in the subsequent steps (S21).

[0043] FIG. 10 shows a state where the evaluation criteria A to G extracted as in FIG. 8 are assigned to each division step. The first extracted criterion A is assigned to the first division step, the next extracted criteria B and C are assigned to the second division step, and the next extracted criteria D to G are assigned to the third division step. In addition, in the second step, criterion B is assigned to the division of the subgroup with higher quality, and criterion C, which corresponds to a subgroup with a lower quality than criterion B, is assigned to the division of the subgroup with lower quality. Furthermore, in the third step, criteria D to G are assigned to the division of the subgroups according to these qualities, respectively. Note that the step of S21 corresponds to the criterion assignment step of the present invention.

[0044] The steps from S22 onwards are executed by using the evaluation support device 1. The control unit 40 divides the evaluation group into two subgroups using the evaluation criteria input in S21 as the first stage of division (S22). As described above, this process is performed by repeatedly having the evaluator compare each evaluation target in the evaluation group with the evaluation criteria and having the evaluator input which one has a higher ranking. Next, the control unit 40 judges whether the number of elements in each subgroup divided immediately before is equal to or less than a predetermined number (S23). If it is judged that the number of elements in at least one of the subgroups is not equal to or less than the predetermined number (S23, No), the control unit 40 performs the next stage of division using the evaluation criteria input in S21, thereby dividing each subgroup into two subgroups (S24). As described above, this process is performed by performing the same process as the division in S22 in each subgroup. Then, the control unit 40 executes the step of S23.

[0045] In S23, when it is determined that the number of elements in each subgroup is equal to or less than a predetermined number (S23, Yes), the control unit 40 ranks the evaluation targets in each subgroup (S25). This process is performed based on the user input via the input / output device, as described above.

[0046] (First embodiment) Hereinafter, a first example, which is an example related to the first embodiment, will be described. Images of cherry tomatoes with a diameter of about 3 cm printed on A5 paper were used as the evaluation object and evaluation standard. Each of the 12 evaluators performed the sensory evaluation method shown in the flowcharts of Figs. 4, 5, and 9. In this example, the evaluation support device 1 was not used, and the evaluators themselves performed all steps, including the steps shown in the flowchart of Fig. 9. As a result, rankings of 100 cherry tomato images were obtained. Here, the evaluation scale was the quality of the cherry tomatoes. When the Kendall's consistency coefficient was calculated for the rankings by the 12 evaluators, the calculation result was 0.84. In other words, the degree of agreement between the evaluation results between the evaluators was high, indicating high stability of the evaluation method.

[0047] In addition, the average ranking given by the 12 evaluators was calculated for each of the above images. Meanwhile, the area of ​​the scratch in each image was derived, and the image was ranked based on the derived area. The larger the area, the higher the ranking. Figure 11(a) is a distribution diagram showing the results. Each circle in Figure 11(a) corresponds to one image. In the figure, R 2 is the Spearman rank correlation coefficient between the rank of the images for scar area and the mean rank of the images assigned by the assessors.

[0048] Furthermore, the ratio of the blemish area to the area of ​​the cherry tomato body in each image was calculated, and the images were ranked based on the magnitude of the derived ratio. Figure 11(b) shows the results and a distribution diagram based on the average rankings given by the 12 evaluators. Each circle in Figure 11(b) corresponds to one image. In the figure, R 2 is the Spearman rank correlation coefficient between the rank of the images based on the proportion of scar area and the mean rank by the assessors.

[0049] (Second Example) Another example related to the first embodiment will be described. A convolutional neural network was trained using 100 cherry tomato images and the rankings given to them in the first example by 12 evaluators. Specifically, 5-fold cross-validation was performed as follows. VGG16 was used as the convolutional neural network. 100 cherry tomato images were divided into 5 groups of 20 images each, one of the groups was used as validation data, and the remaining 4 groups were used as training data. Next, VGG16 was trained by deep learning using 80 images in the 4 groups as example data and the rankings for these as correct answer data. The trained VGG16 was then verified using the 20 images and their rankings that were validation data, and the R 2 The learning and validation were repeated five times, changing the group used as validation data among the five groups. As a result, R 2 The average value was 0.77.

[0050] (Effects of the first embodiment) According to the sensory evaluation method of the first embodiment described above, as shown in the above-mentioned examples, it is easy to ensure the stability of the evaluation results, that is, the rankings given to the evaluation objects. In addition, since the setting of categories is not a prerequisite for the assignment of rankings, the evaluation results are easy to compare, unlike the evaluation results of the conventional technology that depend on the setting of categories. For example, when comparing the evaluation results, a category that serves as a comparison standard may be set anew, and the evaluation results may be classified using the category. In this case, since each evaluation result has already been ranked for the entire evaluation object, it is not time-consuming to categorize them. In other words, the generality of the evaluation results is high. As described above, a sensory evaluation method that is easy to ensure the stability of the evaluation results and is highly general is realized.

[0051] In this embodiment, based on a comparison between each evaluation target and the evaluation criteria by the evaluator, the evaluation group is divided into subgroups in which the number of evaluation target elements is a predetermined number or less. Then, ranking is performed for each divided subgroup. In this way, ranking is performed for a relatively small number of elements, so that ranking by the evaluator is smooth and quick. In particular, it is known that a human being can only short-term remember about 10 pieces of information, and cannot process more information than that at one time. Therefore, by setting the predetermined number to about 10, ranking of each subgroup is easily processed at once based on short-term memory. Therefore, the ranking process is smoother and quicker.

[0052] The evaluation targets of the sensory evaluation method of the present embodiment include images and sounds that generate visual and auditory stimuli, as well as various samples that generate odor, taste, or tactile stimuli for the senses of smell, taste, or touch. Therefore, the stability of the evaluation results is easily ensured for each of these images, sounds, odors, tastes, and tactile sensations, and the method is highly generalizable.

[0053] In this embodiment, the evaluator can compare the output contents of the evaluation target from the output device with the output contents of the evaluation criteria by using the evaluation support device 1. Therefore, the comparison of the evaluation target with the evaluation criteria can be smoothly performed.

[0054] [Second embodiment] An evaluation support system 100 (corresponding to the sensory evaluation system of the present invention) according to a second embodiment, which is another embodiment of the present invention, will be described below. As shown in Fig. 12, the evaluation support system 100 has one or more mobile terminals 110 and a server 120. These devices can communicate with each other via a communication network N such as the Internet.

[0055] The mobile terminal 110 is, for example, a portable computer such as a notebook PC, a tablet PC, or a smartphone. In this embodiment, it is assumed that a plurality of evaluators are involved in the evaluation, and each evaluator uses one mobile terminal 110. The mobile terminal 110 has a display 111, a speaker 112, an input unit 113, and a control unit 115. The output device consisting of the display 111 (corresponding to the output device of the present invention) and the speaker 112 is capable of outputting images and sounds. The input unit 113 includes a pointing device such as a mouse or a touch panel, and a keyboard. The display 111 and the input unit 113 may be configured by different devices, or may be configured by a device integrated with input and output functions such as a touch panel display.

[0056] The control unit 115 is constructed by hardware such as a CPU, memory, various interfaces, etc., and software such as program data stored in the memory. The program data includes data related to applications dedicated to the evaluation support system 100. These software can be distributed by downloading via the Internet or by various recording media. The control unit 115 is constructed by these hardware functions based on the software. In this way, the control unit 115 controls processing such as data communication with the server 120, acceptance of user input via the input unit 113, and output of images or sounds from the output device.

[0057] The server 120 is a computer configured with hardware such as a CPU, memory, and various interfaces, and software such as program data stored in memory. This software can be distributed by downloading via the Internet or by various recording media. The server 120 is constructed by these hardware functions based on the software. The server 120 holds target data and evaluation result data. The target data is the same as in the first embodiment. The evaluation result data is data that indicates an evaluation value in association with an evaluation target. Before evaluation, the evaluation value is set to a predetermined default value for all evaluation targets. The predetermined default value is, for example, zero.

[0058] A sensory evaluation method using the evaluation support system 100 and a process flow by the evaluation support system 100 will be described below. The server 120 extracts data related to two evaluation targets from the target data and transmits it to the mobile terminal 110. The control unit 115 of the mobile terminal 110 causes an output device to output an image, sound, or target ID (corresponding to related information of the present invention) corresponding to the two evaluation targets based on the target data related to the two evaluation targets from the server 120 (corresponding to output processing of the present invention). Figure 13 is an example of an image showing two evaluation targets displayed on the display 10.

[0059] The evaluator compares two evaluation objects on a predetermined scale based on his / her perception of the stimuli generated by the evaluation object and the evaluation criteria, respectively, according to the output from the output device. When the output device outputs images or sounds, these images or sounds directly generate the stimuli related to the evaluation. When the output device outputs object IDs, the stimuli related to the evaluation are generated from the actual samples indicated by the object IDs. The scale is the same as in the first embodiment.

[0060] Based on the output of the output device, the evaluator inputs the ranking of the above scale to the mobile terminal 110 through the input unit 113. The user input is performed, for example, by touching the touch panel with a finger or the like, or by moving the cursor image shown in FIG. 2 using a mouse and operating a button. The control unit 115 transmits the result of this user input to the server 120. The server 120 obtains the ranking of the two evaluation objects indicated by the result of the user input transmitted from the mobile terminal 110, and updates the evaluation result data based on the obtained ranking (corresponding to the ranking acquisition process of the present invention). Specifically, a predetermined value is added to the evaluation value of the higher ranking indicated by the result of the user input of the two evaluation objects compared by the evaluator. The predetermined value is, for example, 1.

[0061] The evaluation support system 100 repeats the above process of having the evaluator compare two evaluation targets based on the output contents from the output device of the mobile terminal 110 for all combinations of two evaluation targets. In this repetition, the server 120 transmits target data related to the two evaluation targets to each of the multiple mobile terminals 110 so that no combinations overlap. Note that when the evaluator performs a user input requesting transmission of target data through the input unit 113 of the mobile terminal 110, the request may be transmitted from the mobile terminal 110 to the server 120, and the server 120 may transmit the target data related to the two evaluation targets to the mobile terminal 110 in response.

[0062] After the above process is repeated for all combinations of two evaluation targets, the server 120 determines the rankings of all evaluation targets based on the evaluation result data (corresponding to the determination process of the present invention). The rankings are given, for example, in descending order of evaluation value. That is, the evaluation target with the highest evaluation value has the highest ranking, and the lower the evaluation value, the lower the ranking. The determined rankings are output to the display 10 in the form of numerical values, graphs, etc., written to various recording media, or output to the outside via a communication network such as the Internet.

[0063] (Third Example) An example related to the second embodiment will be described. The same 100 cherry tomato images as in the first embodiment were used as the evaluation targets, and a sensory evaluation method using the evaluation support system 100 according to the second embodiment was performed. The Kendall's consistency coefficient was calculated between the rankings of the 100 cherry tomato images obtained as a result and the average value of the rankings given by the 12 evaluators in the first embodiment. The calculation result was 0.97, indicating a high consistency with the first embodiment.

[0064] (Fourth Example) Environmental sounds recorded at nine indoor and outdoor locations were used as evaluation targets, and three evaluators were each made to perform a sensory evaluation method using the evaluation support system 100 according to the second embodiment. The titles of the environmental sounds were "cafe," "Traffic city," "Ticket Gate," and the like. These titles express the recording environments in English. The server 120 was made to hold data for reproduction of the nine environmental sounds as target data. Next, the server 120 was made to transmit target data relating to two of these environmental sounds to the mobile terminal 110. In response to this, a screen was displayed on the display 111 of the mobile terminal 110 to allow the user to select one of the two environmental sounds. FIG. 14 shows an overview of the screen. The evaluator was made to select button images B1 and B2 using the input unit 113, and the two environmental sounds were played on the mobile terminal 110 in response to this. This allowed the evaluator to compare the two environmental sounds. Based on the result, the evaluator was made to select button image B3 or B4 using the input unit 113. The evaluators were asked to select from a scale of two environmental sounds in which they would prefer to have lunch. Each evaluator evaluated all combinations of two environmental sounds in this way. As a result, a ranking of nine environmental sounds was obtained as the evaluation results by each evaluator. When Kendall's consistency coefficient was calculated for the evaluation results of the three evaluators, the calculated result for the evaluation results of two was 0.88, indicating a high degree of consistency between the evaluation results. The evaluation result of the remaining person was completely different from that of the other two.

[0065] (Fifth Example) The aromas of ten kinds of aromas were evaluated, and three evaluators performed the sensory evaluation method using the evaluation support system 100 according to the second embodiment. The aromas were "eucalyptus", "ylang-ylang", "lavender", etc. The server 120 was made to hold target data related to the identification information of the ten kinds of aromas. Next, the server 120 was made to transmit target data related to two of these aromas to the mobile terminal 110. In response to this, a screen was displayed on the display 111 of the mobile terminal 110, which allows the user to select the identification information of one of these two aromas. FIG. 15 shows an outline of the screen. The evaluators were made to compare the two aroma samples, each of which had a label with the identification information displayed on the screen attached. Based on the result, the evaluators were made to select the button image B5 or B6 using the input unit 113. The evaluators were made to make a selection based on a scale of which of the two aromas stimulates appetite. Such evaluation of the two aromas by each evaluator was performed for all combinations of aromas. As a result, the ranking of the 10 aromas by each evaluator was obtained. The Kendall's coefficient of agreement between the evaluation results by the three evaluators was calculated to be 0.75, which indicates a high degree of agreement between the evaluation results.

[0066] (Effects of the second embodiment) According to the sensory evaluation method using the evaluation support system 100 according to the second embodiment described above, as shown in the above-mentioned examples, it is easy to ensure the stability of the evaluation results, that is, the rankings given to the evaluation objects. In addition, since the setting of categories is not a prerequisite for the assignment of rankings, the evaluation results are easy to compare, unlike the evaluation results of the conventional technology that depend on the setting of categories. For example, when comparing the evaluation results, a category that serves as a comparison standard may be set anew, and the evaluation results may be classified using the category. In this case, since each evaluation result has already been ranked for the entire evaluation object, it is not time-consuming to categorize them. In other words, the generality of the evaluation results is high. As described above, a sensory evaluation method that is easy to ensure the stability of the evaluation results and is highly general is realized.

[0067] In this embodiment, the evaluator can compare the output contents of the evaluation target from the output device with the output contents of the evaluation standard by using the evaluation support system 100. Therefore, the comparison of the evaluation target with the evaluation standard can be smoothly performed.

[0068] In this embodiment, outputs related to two evaluation targets are generated on multiple mobile terminals 110, and each evaluator compares the two evaluation targets based on the outputs. Then, the results input by the user on each mobile terminal 110 are transmitted to the server 120, and the server 120 tallys them up as evaluation results related to all evaluation targets. In other words, each process related to the evaluation support system 100 is executed by the cooperation of the multiple mobile terminals 110 and the server 120. This makes it easy to shorten the time it takes to rank the evaluation targets.

[0069] <Modification> The above is a description of a preferred embodiment of the present invention, but the present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the means for solving the problems.

[0070] According to the first embodiment described above, the evaluation group is divided into subgroups in which the number of elements to be evaluated is a predetermined number (for example, about 10) or less. In contrast, other conditions may be adopted as long as the number of elements is within a predetermined range or less. For example, the evaluation group may be divided so that the number of elements in some of the subgroups after division, for example, more than half of the subgroups, is equal to or less than the predetermined number.

[0071] According to the second embodiment described above, the mobile terminal 110 outputs data based on two evaluation targets, and the two evaluation targets are compared by the evaluator. Alternatively, the mobile terminal 110 may output data based on three or more evaluation targets, and the evaluator may compare the data based on three or more evaluation targets. [Explanation of symbols]

[0072] 1. Evaluation Support Device 10. Display 20 Speaker 30 Input section 40 Control section 100 Evaluation Support System 110 Mobile Devices 111 Display 112 Speaker 113 Input section 115 Control Unit 120 Servers

Claims

1. A method for evaluating an evaluation target that generates a stimulus perceptible by a human, comprising the steps of: A comparison step of having an evaluator compare a stimulus generated by each of the evaluation subjects belonging to an evaluation group consisting of a plurality of the evaluation subjects with a reference stimulus on a predetermined scale based on perception of the stimulus; A division step of dividing the evaluation group into two subgroups with respect to the predetermined scale based on the comparison result of the comparison step; a repeating step of performing the comparing step and the dividing step one or more times with each of the two subgroups generated by the dividing step as an evaluation group, thereby generating a plurality of the subgroups such that the number of elements in each subgroup falls within a predetermined range; a ranking step of having an evaluator assign a rank on the predetermined scale to all of the evaluation objects belonging to each of the plurality of subgroups generated by the repeating step, based on perception of a stimulus to the entire evaluation objects belonging to the subgroup.

2. The evaluation target is an image or a sound, The predetermined scale is a scale related to an image or a sound, 2. The method for sensory evaluation according to claim 1, wherein the sense of the stimulus is visual or auditory.

3. 3. The sensory evaluation method according to claim 2, wherein in the comparison step, the evaluation object and a reference image or a reference sound are output to an output device, and the output evaluation object and the reference image or the reference sound are compared by an evaluator.

4. The evaluation subject generates a stimulus that can be perceived by any one of the senses of smell, taste, and touch, 2. The sensory evaluation method according to claim 1, wherein the predetermined scale is a scale relating to any one of smell, taste, and touch corresponding to the stimulus.

5. Identification information is set for each of the plurality of evaluation targets to distinguish them from the other evaluation targets, 2. The sensory evaluation method according to claim 1, wherein in the comparison step, the identification information of any one of the plurality of evaluation objects is displayed on a display device, and an evaluator is made to compare the stimulus generated by the evaluation object corresponding to the displayed identification information with the reference stimulus.

6. As a step of obtaining a stimulus generating the reference stimulus used in the comparison step, a reference extraction step of extracting, from the entire evaluation objects, a number of the evaluation objects equal to or greater than a predetermined number of the division steps as reference evaluation objects; a criterion ranking step of ranking all of the criterion evaluation objects with respect to the predetermined scale; 2. The sensory evaluation method according to claim 1, further comprising a criterion assignment step of assigning the reference evaluation objects to each of the predetermined number of division steps based on the rankings assigned in the criterion ranking step.

7. A system for evaluating a plurality of evaluation targets, comprising one or more output devices for outputting images or sounds and one or more computers, the one or more computers include an input unit and a control unit for accepting user input; The control unit: an output process for displaying, on the output device, related information related to two or more of the plurality of evaluation targets; a ranking acquisition process for acquiring a ranking of the evaluation object related to the related information outputted to the output device in the output process based on a user input received by the input unit; and a determination process for determining a ranking of the entire plurality of evaluation objects with respect to the specified scale by repeatedly executing the ranking acquisition process while changing the combination of the two or more evaluation objects for which the output device outputs the related information.

8. 8. The sensory evaluation system according to claim 7, wherein the number of pieces of related information that are simultaneously output in the output process is two.

9. a plurality of said output devices and a plurality of said computers capable of communicating with each other; the plurality of output devices correspond to the plurality of computers, respectively; 8. The sensory evaluation system according to claim 7, wherein the output process and the ranking acquisition process are executed in each of the plurality of computers, and the determination process is executed by cooperation of all of the plurality of computers.

10. A neural network learning method comprising: executing a process by the sensory evaluation method according to any one of claims 1 to 6 or the sensory evaluation system according to any one of claims 7 to 9; and using data indicating the evaluation objects ranked by this process and the rankings as teacher data.

Images