Sensory evaluation method, sensory evaluation system, sensory evaluation program, and learning method for neural networks using these.

Abstract

In sensory evaluation, the stability and generality of the evaluation results are high. [Solution] An evaluation group consisting of multiple evaluation targets is divided into subgroups with 3 to 16 elements each (S1). Next, evaluators are asked to rank the evaluation targets belonging to each subgroup according to a predetermined scale (S2). Then, from the group of evaluation targets consisting of the highest-ranked evaluation targets in each subgroup, evaluators are asked to extract the evaluation target with the highest evaluation according to the predetermined scale, and this extracted evaluation target is removed from the original subgroup. This process is repeated until all evaluation targets have been extracted from all subgroups. The order in which the evaluation targets are extracted in this way is then used as the ranking of the evaluation group as a whole (S3).

Description

Technical Field

[0001] The present invention relates to a sensory evaluation method, a sensory evaluation system, a sensory evaluation program, and a learning method for a neural network using these.

Background Art

[0002] Conventionally, sensory evaluation has been performed to evaluate objects, information, etc. to be evaluated by the five senses of a person. For example, Non-Patent Document 1 and Non-Patent Document 2 evaluate the ripeness of tomatoes through human vision. In these studies, a plurality of categories and category labels corresponding to the ripeness of tomatoes are set. Then, the tomatoes to be evaluated are classified by an evaluator based on the set categories. For each category, the evaluator assigns an evaluation target suitable for the content of the label attached to the category.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

[0004] According to the evaluation method described above, there are no general criteria for setting categories, so the labeling of categories and the number of categories will differ each time an evaluation is performed. For example, Non-Patent Document 1 sets three categories labeled "Green," "Orange," and "Red," while Non-Patent Document 2 sets five categories labeled "Green and Breaker," "Turning," "Pink," "Light Red," and "Red." Since evaluators classify the evaluation subjects according to the category labels and the number of categories, if the category settings differ as described above, the results are likely to differ even when evaluating the same evaluation subject. In addition, because there are no objective criteria for how to name the labels, if the content of the labels is abstract, such as the names of colors, the categorization results are likely to differ depending on the evaluator. In other words, the stability of the evaluation results is likely to be lacking. Furthermore, the evaluation results depend on the category settings. For this reason, it is difficult to compare evaluation results with different category settings, and the generality of the evaluation results is low.

[0005] The object of the present invention is to provide a sensory evaluation method, a sensory evaluation system, and a sensory evaluation program that exhibit high stability and generality of evaluation results, as well as a neural network learning method using these. [Means for solving the problem]

[0006] The sensory evaluation method of the present invention is a method for evaluating an evaluation target that generates a stimulus perceptible to a person, comprising: a division step of dividing an evaluation group consisting of a plurality of evaluation targets into a plurality of subgroups such that the number of elements in each of the plurality of subgroups is in the range of 3 to 16; a subgroup ranking step of having the evaluator rank the evaluation targets according to a predetermined scale based on the perception of the stimulus generated by the evaluation target for each of the plurality of subgroups generated by the division step; and an evaluation extraction step of having the evaluator extract the evaluation target with the highest or lowest evaluation according to the predetermined scale from the group of evaluation targets consisting of the evaluation targets with the highest or lowest ranking in each of the plurality of subgroups based on the ranking made in the subgroup ranking step, based on the perception of the stimulus generated by the evaluation target, and removing the extracted evaluation target from the original subgroup, repeating this until all evaluation targets have been extracted from the plurality of subgroups, wherein the order of extraction in the evaluation extraction step is the ranking relative to the entire evaluation group.

[0007] Furthermore, the sensory evaluation system of the present invention is a system for evaluating an evaluation target that generates a stimulus perceptible to a person, and comprises an output device, an input device and a calculation device, wherein the calculation device performs an output process that causes the output device to output the evaluation target, and a subgroup ranking process that ranks the evaluation targets according to a predetermined scale for each of the multiple subgroups that constitute an evaluation group consisting of multiple evaluation targets, each subgroup having 3 to 16 elements, based on the content received by the input device from user input made by an evaluator who has perceived the evaluation target output by the output device, and the subgroup ranking An evaluation extraction process is performed, in which, from a group of evaluation targets consisting of the highest or lowest ranked evaluation targets in each of the multiple subgroups in the ranking process, the evaluation target with the highest or lowest evaluation according to the predetermined scale is extracted based on the user input made by the evaluator who perceived the evaluation target output by the output device, and the extracted evaluation target is removed from the original subgroup, and this process is repeated until all evaluation targets have been extracted from the multiple subgroups. A ranking determination process is also performed, in which the order of extraction by the evaluation extraction process is determined as the ranking of the entire evaluation group.

[0008] The present invention is a program for evaluating an evaluation target that generates a stimulus perceptible to a person, and the program causes a computer to execute the following: a subgroup ranking process which ranks the evaluation targets according to a predetermined scale based on user input received by an input device from an evaluator who has perceived the evaluation target output by an output device, for each of the multiple subgroups which constitute an evaluation group consisting of a plurality of evaluation targets, each having 3 to 16 elements; an evaluation extraction process which extracts the evaluation target with the highest or lowest evaluation according to the predetermined scale from the group of evaluation targets consisting of the evaluation targets with the highest or lowest ranking in each of the multiple subgroups based on the ranking made by the subgroup ranking process, based on user input received by the input device from an evaluator who has perceived the evaluation target output by an output device, and removes the extracted evaluation target from the original subgroup, repeating this process until all evaluation targets have been extracted from the multiple subgroups; and a ranking determination process which determines the order of extraction by the evaluation extraction process as the ranking of the entire evaluation group.

[0009] According to the sensory evaluation method, sensory evaluation system, or program of the present invention, the assignment of rankings does not presuppose the setting of categories. Therefore, the problem of unstable evaluation results due to different category settings by evaluators is avoided. Furthermore, unlike the evaluation results of the conventional technology, which depend on the setting of categories, it is easy to compare evaluation results with each other. For example, when comparing evaluation results, a new category to be used as the basis for comparison can be set, and the evaluation results can be classified using that category. In this case, since each evaluation result has already been ranked relative to the entire evaluation target, there is no extra effort required for categorization. In other words, the generality of the evaluation results is high. Thus, a sensory evaluation method or sensory evaluation system with high stability and generality of evaluation results is realized. Moreover, as shown in the embodiments described later, the effort required for ranking and evaluation is reduced.

[0010] Furthermore, in the present invention, it is preferable that the object of evaluation is an image or sound, the predetermined scale is a scale relating to the appearance of the image or the sound, and the perception of the stimulus is visual or auditory. This ensures high stability and generality of the evaluation results for images or sounds.

[0011] Furthermore, in the present invention, it is preferable to have the evaluation targets output to an output device in both the subgroup ranking step and the evaluation extraction step, and to have the evaluator rank or evaluate the outputted evaluation targets. Ranking and evaluation can be carried out smoothly according to the output content of the output device.

[0012] Furthermore, in the present invention, it is preferable that the object to be evaluated generates a stimulus that can be perceived by either the sense of smell, taste, or touch, and that the predetermined scale is a scale relating to one of the odor, taste, or touch sensations corresponding to the stimulus. This ensures high stability and generality of the evaluation results for odor, taste, or touch sensation.

[0013] Furthermore, in the present invention, identification information is set for each of the multiple evaluation targets to distinguish it from other evaluation targets, and in each of the subgroup ranking step and the evaluation extraction step, the identification information of one of the multiple evaluation targets is displayed on the display device, and the evaluator is asked to rank or evaluate the stimuli generated by the evaluation target corresponding to the displayed identification information. This allows for a smooth comparison between the evaluation targets and the reference stimuli by making comparisons according to the content displayed on the display device.

[0014] Furthermore, the reliability evaluation method of the present invention is a method for evaluating the reliability of rankings made by the sensory evaluation method described above, comprising: a sample extraction step of extracting 3 to 16 samples for reliability evaluation from the plurality of evaluation targets; a re-ranking step of having the evaluator rank all the samples extracted in the sample extraction step according to a predetermined scale based on the perception of stimuli generated by each of the evaluation targets; and a sample ranking comparison step of comparing the ranking of the samples made in the re-ranking step with the ranking of the samples based on the order of extraction in the evaluation extraction step.

[0015] According to the reliability evaluation method of the present invention, it becomes possible to evaluate the reliability of the results ranked by the sensory evaluation method of the present invention.

[0016] Furthermore, according to the neural network learning method of the present invention, the processing by the above-mentioned sensory evaluation method, sensory evaluation system, or sensory evaluation program is performed, and the data indicating the evaluation target to which a rank has been assigned, along with the rank, are used as training data.

[0017] According to the neural network learning method of the present invention, learning is performed based on evaluation results with high stability and generality obtained by the sensory evaluation method, sensory evaluation system, or sensory evaluation program of the present invention. Therefore, highly versatile learning becomes possible. [Brief explanation of the drawing]

[0018] [Figure 1] This is a block diagram showing the configuration of an evaluation support device according to a first embodiment, which is one embodiment of the present invention. [Figure 2] Figure 1 shows an example of a display screen on a display that assigns rankings to the images being evaluated. [Figure 3] Figure 1 is a flowchart showing the flow of the sensory evaluation method using the evaluation support device. [Figure 4]It is a conceptual diagram showing a situation where rankings are assigned to the entire evaluation target by the process shown in the flowchart of FIG. 3. [Figure 5] An example of a screen for selecting an image of a tomato, which is an evaluation target belonging to one subgroup, on the display screen in the sorting step of FIG. 3 is shown. [Figure 6] It is a conceptual diagram showing a situation where evaluation targets belonging to each subgroup are integrated into one integrated group in the integration step of FIG. 3. [Figure 7] An example of a screen for selecting an image of a tomato, which is an evaluation target, on the display screen in the integration step according to FIG. 6 is shown. [Figure 8] An example of a screen for selecting an image of a tomato, which is an evaluation target, on the display screen in the consistency evaluation step of FIG. 3 is shown. [Figure 9] It is a graph showing the results of two rankings by two evaluators in one embodiment of the present invention. [Figure 10] It is a block diagram showing the configuration of an evaluation support system according to a second embodiment, which is another embodiment of the present invention. [Figure 11] It is a block diagram showing the configuration of a neural network learning system according to yet another embodiment of the present invention. [Figure 12] It is a block diagram showing an example of a screen displayed on a display in the evaluation of environmental sounds according to yet another embodiment of the present invention. [Figure 13] It is a block diagram showing an example of a screen displayed on a display in the evaluation of scents according to yet another embodiment of the present invention.

Mode for Carrying Out the Invention

[0019] [First Embodiment] The following describes a sensory evaluation method according to a first embodiment, which is one embodiment of the present invention. The sensory evaluation method according to this embodiment evaluates an object to be evaluated by utilizing the perception of a person (evaluator) to the stimuli generated by the object to be evaluated. The perceptions assumed are mainly sight, hearing, smell, taste, and touch. One of these is used for each evaluation.

[0020] The objects of evaluation in the sensory evaluation method of this embodiment are those that generate stimuli to the above senses. For example, in addition to those that generate stimuli to sight and hearing, such as images and sounds, the objects of evaluation include those that generate stimuli to smell, taste, and touch, such as smell, taste, and touch. In this embodiment, an evaluation group consisting of multiple objects of evaluation is evaluated.

[0021] In this embodiment, as an example, it is assumed that a sensory evaluation method is performed using an evaluation support device 1. As shown in Figure 1 as an example, the evaluation support device 1 has various input devices and output devices (hereinafter, these may be collectively referred to as input / output devices) and a control unit 40. The input / output devices are the display 10 (corresponding to the display device of the present invention), speaker 20, and input unit 30 shown in Figure 1. The input unit 30 includes pointing devices such as a mouse, touch panel, touchpad, and keyboard.

[0022] The evaluation support device 1 is comprised of a computer on which the above-mentioned input / output devices are mounted or connected. The computer is constructed from hardware such as a CPU (Central Processing Unit), memory devices (hereinafter referred to as memory), and various interfaces, and software such as program data stored in memory. This software can be distributed via internet download or various recording media. The control unit 40 is constructed by the functioning of this hardware based on the software. As described above, the program, which is the software that makes the hardware function as the control unit 40, corresponds to the sensory evaluation program of the present invention.

[0023] The control unit 40 holds target data, which is data related to the evaluation target. The target data is input to the evaluation support device 1 in advance. There are two types of target data. The first type of data is data that directly generates perceptual stimuli through output from the display 10 and speaker 20 (hereinafter, these may be collectively referred to as output devices). This includes image data and sound data. Image data is data that outputs an image related to the evaluation to the display 10. This image generates visual stimuli related to the evaluation. Sound data is data that outputs sound to the speaker 20. This sound generates auditory stimuli related to the evaluation.

[0024] The second type of data does not directly generate perceptual stimuli, but rather represents identification information (hereinafter referred to as the target ID) assigned to the object being evaluated. The target ID consists of letters, numbers, and various symbols, and is output by display 10. The actual stimuli are generated from the physical sample to which the target ID has been assigned.

[0025] The control unit 40 causes the output device (corresponding to the output device of the present invention) to output the image, sound, or target ID represented by each of the target data for each of the multiple evaluation targets included in the evaluation group. Figure 2(a) is an example of displaying an image representing an evaluation target (hereinafter referred to as a target image) on the display 10. Figure 2(b) is an example of displaying a target ID on the display 10.

[0026] The evaluator compares the subjects against each other on a predetermined scale, based on their perception of the stimuli they generate, according to the output from the output device. If the output device outputs images or sounds, these images or sounds directly generate the stimuli related to the evaluation. If the output device outputs subject IDs, the stimuli related to the evaluation are generated from the real-world samples indicated by each subject ID.

[0027] The specified scale is a perceptual scale used to rank multiple evaluation targets. With respect to this scale, for example, any two evaluation targets can be classified as high and low in the above ranking. The scale may include relatively objective elements such as size, color, and pitch, as well as relatively subjective elements. For example, the quality or maturity of an evaluation target, or "suitability for lunch," may be used as scale elements.

[0028] The evaluator inputs the ranking (high or low) for the above scale to the evaluation support device 1 via the input unit 30, based on the output of the output device. User input is performed, for example, by touching the touch panel with a finger, or by moving the cursor image shown in Figure 2 using a mouse and operating buttons.

[0029] The control unit 40 ranks the evaluation subjects within the evaluation group based on the sensory evaluations performed by the evaluators. The method of this ranking will be explained below with reference to Figures 3 to 7. Figure 3 is a flowchart showing the ranking process. Figure 4 is a conceptual diagram showing an example of ranking. Figure 4 shows the case where 16 evaluation subjects are ranked. In Figure 4, each block with a numerical value corresponds to an evaluation subject. The numerical value indicates the final rank assigned to the entire evaluation group based on a certain scale. The column of evaluation subjects placed at the top represents the initial state of the evaluation group, reflecting a situation where the evaluation subjects are not in rank but are arranged randomly.

[0030] First, the control unit 40 divides the evaluation targets belonging to the evaluation group into multiple subgroups, each consisting of N elements (S1; division step). The number of elements N in a subgroup is a natural number between 3 and 16. The upper limit of N, 16, corresponds to the maximum number that can be efficiently ranked, based on human characteristics related to short-term memory (Meilgaard, MC, Civille, GV, & Carr, BT (2007). Sensory evaluation techniques (4th ed.). Boca Raton: CRC Press.). The lower limit of N, 3, corresponds to the minimum number of elements that are worth assigning a ranking to. Furthermore, it is preferable that the number of evaluation targets belonging to a subgroup be between 5 and 9, based on the number of information units that can be stored in short-term memory, which is 7±2 (Miller, GA (1994). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 101(2), 343-352. https: / / doi.org / 10.1037 / 0033-295x.101.2.343). The division can be done by any delimiter. For example, based on the order in which the filenames of the images representing the evaluation targets are listed, the evaluation targets can be divided into subgroups by N (N: any natural number between 3 and 16) from the top of that order. In Figure 4, the second column from the top of the evaluation targets shows the subgroups after division. In the example in Figure 4, each subgroup contains 8 evaluation targets.

[0031] Next, the control unit 40 sorts the evaluation targets into those belonging to each subgroup (S2; corresponding to the subgroup ranking step and subgroup ranking process). Specifically, for each evaluation target belonging to each subgroup, the control unit 40 outputs the image, sound, or target ID represented by each target data to the output device. The evaluator then selects the image or target ID in order from the highest ranking according to a predetermined scale, and the selection results are input to the evaluation support device 1 via user input.

[0032] For example, Figure 5 shows an example display where images of tomatoes, which are the subject of evaluation, are shown on the screen of display 10. In this example, the display screen includes images corresponding to N=8 subjects of evaluation belonging to one subgroup. The evaluator selects these images in order from highest to lowest ranking according to a predetermined scale through user input using the input unit 30. User input is performed, for example, by touching the touch panel with a finger, or by moving the cursor image shown in Figure 5 and operating buttons using a mouse. The control unit 40 rearranges the subjects of evaluation in each subgroup in the order indicated by the user input. This process corresponds to ranking the subjects of evaluation within each subgroup. In Figure 4, the third column from the top of the evaluation subjects shows each subgroup after rearrangement.

[0033] Next, the control unit 40 integrates the evaluation targets of all subgroups (S3; corresponding to the evaluation extraction step and evaluation extraction process). The integration is performed in the steps shown in Figure 6 as an example. First, the control unit 40 extracts the highest-ranking evaluation target from each subgroup according to the ranking performed in S2, and forms a high-evaluation extraction group (corresponding to the evaluation target group of the present invention) consisting of the extracted evaluation targets. The high-evaluation extraction group corresponds to the evaluation targets enclosed by the dashed line in Figure 6. Then, for each of these evaluation targets, the image, sound, or target ID output device is used to output the image, sound, or target ID that each of the target data represents. From these, the evaluator is asked to select the image or target ID with the highest ranking according to a predetermined scale. The selection result is input to the evaluation support device 1 through user input.

[0034] For example, Figure 7 shows an example display where images of tomatoes from the high-evaluation extraction group are shown on the screen of display 10. In this example, the display screen includes images corresponding to two evaluation targets extracted from two subgroups. The evaluator selects the image with the highest ranking according to a predetermined scale. The control unit 40 extracts the selected evaluation target from the high-evaluation extraction group and adds the extracted evaluation target to the integrated group. Then, it removes the evaluation target extracted from the high-evaluation extraction group from its original subgroup. This operation is repeated until all evaluation targets from all subgroups are integrated into the integrated group. In the integrated group, the evaluation targets are arranged in the order in which they were extracted from the high-evaluation extraction group. This order corresponds to the final ranking assigned to the entire evaluation group.

[0035] S3-1 in Figure 6 shows the process where evaluation subjects for the high-evaluation extraction group are extracted from each subgroup, and then the highest-ranking subjects from that group are further extracted and merged into the integrated group. The evaluation subjects extracted into the high-evaluation extraction group are removed from their original subgroups. In the next S3-2, similarly, evaluation subjects for the high-evaluation extraction group are extracted from each subgroup, and then the highest-ranking subjects from that group are further extracted and merged into the integrated group. The evaluation subjects extracted into the high-evaluation extraction group are removed from their original subgroups. The same process is carried out in the next S3-3. This is repeated until all evaluation subjects are finally arranged in the integrated group with appropriate rankings.

[0036] Next, the control unit 40 evaluates the consistency of the ranking of the evaluation targets in the integrated group obtained in S3 (S4 in Figure 3). Here, consistency refers to the consistency of the evaluators' evaluations of the evaluation targets. Specifically, the control unit 40 extracts M evaluation targets from the integrated group obtained in S3 as samples (corresponding to the reliability evaluation samples of the present invention) (corresponding to the sample extraction step of the present invention). M is a natural number between 3 and 16. The reason for setting M to this range is the same as for N. Also, for the same reasons as N, it is preferable that M is in the range of 5 to 9.

[0037] Next, the control unit 40 outputs the image, sound, or target ID represented by each of the M extracted evaluation targets to the output device. The same evaluators who participated in the evaluations of S2 and S3 then select the images or target IDs in order from those with the highest ranking according to a predetermined scale, and input the selection results to the evaluation support device 1 via user input (corresponding to the re-ranking step of the present invention).

[0038] For example, Figure 8 shows an example display where images of tomatoes, which are the subjects of evaluation and extracted from the integrated group, are displayed on the screen of display 10. In this example, the display screen includes images corresponding to the eight extracted subjects of evaluation. The evaluator selects these images in order from highest to lowest ranking according to a predetermined scale through user input using the input unit 30. User input is performed, for example, by touching the touch panel with a finger, or by moving the cursor image shown in Figure 5 and operating buttons using a mouse.

[0039] The control unit 40 compares the rankings assigned to the evaluation targets extracted from the integrated group by user input with the rankings assigned to those evaluation targets based on their order within the integrated group (corresponding to the sample ranking comparison step of the present invention). The control unit 40 then evaluates only data where the degree of agreement between the former ranking and the latter ranking is sufficiently high as highly consistent data. For example, the control unit 40 calculates Kendall's consistency coefficient between the former ranking and the latter ranking. If the consistency coefficient is above a threshold (e.g., 0.80), the evaluation results by that evaluator are evaluated as consistent and highly reliable. On the other hand, if the consistency coefficient is below the threshold, the evaluation results by that evaluator are evaluated as inconsistent and unreliable. By selectively using data with high consistency and reliability in this way, the stability and generality of the evaluation results can be ensured. The method performed in S4 of Figure 3 corresponds to the reliability evaluation method of the present invention.

[0040] The following describes one embodiment according to this embodiment. Damaged mini tomatoes were selected from a group of mini tomatoes harvested at a certain time, and an evaluation group consisting of 100 selected mini tomatoes was prepared. Six evaluators, A to F, then each performed ranking and consistency evaluation on the prepared evaluation group, as shown in the flowchart in Figure 3. In the consistency evaluation in S4, the Kendall consistency coefficient threshold was set to 0.80. Figure 9 shows the rankings obtained in S3 (first ranking) and S4 (second ranking) for evaluators D and F among the six evaluators. The eight points in Figure 9 indicate the positions of the eight evaluation targets extracted from the integrated group in S4. The Kendall consistency coefficients calculated from the two rankings assigned by each of the six evaluators were 0.79 (evaluator A), 0.86 (evaluator B), 0.93 (evaluator C), 0.93 (evaluator D), 1.00 (evaluator E), and 0.64 (evaluator F). Therefore, the data related to evaluators A and F was deemed inconsistent and unreliable, while the data related to evaluators B through E was evaluated as highly consistent and reliable.

[0041] According to the embodiment described above, the assignment of rankings does not presuppose the setting of categories. Therefore, the problem of unstable evaluation results due to different category settings by evaluators does not occur. Furthermore, unlike the evaluation results of conventional technologies which depend on category settings, it is easy to compare evaluation results with each other. For example, when comparing evaluation results, one can set a new category to be used as the basis for comparison and classify the evaluation results using that category. In this case, since each evaluation result has already been ranked relative to the entire evaluation target, there is no extra effort required for categorization. In other words, the generality of the evaluation results is high. Thus, a sensory evaluation method with high stability and generality of evaluation results is realized.

[0042] Furthermore, the ranking method according to this embodiment reduces the number of times the evaluator selects an evaluation target through user input. Table 1 below shows the number of selections compared to other ranking methods. The other method is a brute-force method in which the evaluator selects which of the two evaluation targets has a higher rank for all combinations, based on a predetermined scale, by comparing any two evaluation targets included in the evaluation group. As shown in Table 1, as the number of evaluation targets increases from 16, 32, 64, and 512, the number of selections in this embodiment increases from 32, 64, 128, and 1536, while in the other method the number of selections is 120, 496, 2016, and 130816. In other words, this embodiment requires less effort for ranking and evaluation.

[0043] [Table 1]

[0044] Furthermore, the embodiments described above mainly describe an example in which an image representing the object to be evaluated is displayed on the screen of the display 10. However, instead of an image, an identification ID corresponding to the object to be evaluated may be displayed on the screen of the display 10. In either case, whether an image or an identification ID is displayed, the evaluation of the object to be evaluated can be carried out smoothly by making a comparison according to the content displayed on the display 10.

[0045] [Second Embodiment] Hereinafter, an evaluation support system 100 (corresponding to the sensory evaluation system of the present invention), which is another embodiment of the present invention, will be described. As shown in Figure 10, the evaluation support system 100 has one or more mobile terminals 110 and a server 120. These devices can communicate with each other via a communication network N such as the Internet.

[0046] The mobile terminal 110 is comprised of, for example, a portable computer such as a notebook PC, tablet PC, or smartphone. This embodiment assumes that multiple evaluators are involved in the evaluation, with each evaluator using one mobile terminal 110. The mobile terminal 110 has a display 111, a speaker 112, an input unit 113, and a control unit 115. The output device, consisting of the display 111 (corresponding to the output device of the present invention) and the speaker 112, is capable of outputting images and sounds. The input unit 113 includes a pointing device such as a mouse or touch panel, or a keyboard. The display 111 and the input unit 113 may be composed of different devices, or they may be composed of a device with integrated input / output functions, such as a touch panel display.

[0047] The control unit 115 is constructed from hardware such as a CPU, memory, and various interfaces, and software such as program data stored in memory. The program data includes data related to an application specifically for the evaluation support system 100. This software can be downloaded via the Internet or distributed on various recording media. The control unit 115 is constructed by the functioning of this hardware based on the software. As a result, the control unit 115 controls processing such as data communication with the server 120, acceptance of user input through the input unit 113, and output of images or sounds from the output device. The program, which is the software that makes the hardware function as the control unit 115 as described above, corresponds to the sensory evaluation program of the present invention.

[0048] Server 120 is a computer composed of hardware such as a CPU, memory, and various interfaces, and software such as program data stored in memory. This software can be distributed via internet download or various recording media. Server 120 is constructed by the functioning of this hardware based on the software. Server 120 holds target data and evaluation result data. The target data is the same as in the first embodiment, but unlike the first embodiment, the evaluation target is divided in advance into subgroups of 3 to 16 elements. The evaluation result data is data that shows the ranking in relation to the evaluation target.

[0049] The following describes the sensory evaluation method using the evaluation support system 100 and an overview of the processing performed by the evaluation support system 100. The server 120 extracts data related to N evaluation targets belonging to each subgroup from the target data and transmits it to the mobile terminal 110. The control unit 115 of the mobile terminal 110 outputs images, sounds, or target IDs corresponding to the N evaluation targets to the output device based on the target data related to the evaluation targets from the server 120 (corresponding to the output processing of the present invention). Subsequently, the same processing as S2 to S4 shown in Figure 3 of the above embodiment is performed by the mobile terminal 110 instead of the evaluation support device 1. In this processing, data related to the evaluation targets is transmitted from the server 120 to the mobile terminal 110 as appropriate. In addition, the ranking results, etc., are transmitted from the mobile terminal 110 to the server 120 and are held as evaluation result data in the server 120.

[0050] [Other embodiments] (1) Use of evaluation results in neural networks The ranks assigned to the evaluation targets according to the first or second embodiment described above may be used in the neural network learning method as follows. As shown in Figure 11, the ranks and target data assigned by the evaluation support device 1 or evaluation support system 100 are input to the neural network learning device 200. The neural network learning device 200 uses the target data as problem data and the ranks assigned to the evaluation targets indicated by the target data as correct answer data to perform learning on a neural network such as a convolutional neural network. This neural network takes data of the same type as the target data as input and outputs evaluation results corresponding to the input data. The evaluation results correspond to the results of evaluating the evaluation targets corresponding to the target data using the same scale as the scale used when acquiring the learning data.

[0051] According to this embodiment, learning is performed based on evaluation results with high stability and generality obtained by the evaluation support device 1 or the evaluation support system 100. Therefore, highly versatile learning becomes possible.

[0052] (2) Evaluation of ambient sounds Environmental sounds recorded indoors and outdoors may also be used as evaluation targets. For example, an evaluator may perform a sensory evaluation using the evaluation support system 100 according to the second embodiment. The titles of the environmental sounds are "cafe," "Traffic city," "Ticket Gate," etc. These titles are English expressions of the environment in which the environmental sounds were recorded. The server 120 holds playback data for multiple environmental sounds as target data. The target data is pre-divided into subgroups with 3 to 16 elements each. Next, the server 120 transmits the target data related to the environmental sounds belonging to each subgroup to the mobile terminal 110. In response, the display 111 of the mobile terminal 110 displays a screen for selecting these environmental sounds. Figure 12 shows an example of this screen. The evaluator selects button images B1 to B8 using the input unit 113. Each time a button image is selected, the mobile terminal 110 plays the environmental sound corresponding to the selected button. This allows the evaluator to evaluate the environmental sounds belonging to the subgroups on the scale of whether they would want to have lunch in that environment. Based on the evaluation results, the evaluator is instructed to sequentially select button images B9 to B16 using the input unit 113. The evaluator is then instructed to rank the ambient sounds in this manner for all subgroups. This process corresponds to the process S2 in Figure 3. Then, the processes S3 and S4 based on the ranked results are performed in the same manner as in the second embodiment. Note that the evaluation of ambient sounds may also be performed using the evaluation support device 1 of the first embodiment instead of the second embodiment.

[0053] (3) Evaluation of scent Aroma may also be used as the evaluation target. For example, ten types of aromas may be used as the evaluation target, and the evaluator may perform a sensory evaluation using the evaluation support system 100 according to the second embodiment. The aromas may be "eucalyptus," "ylang-ylang," "lavender," etc. The server 120 stores target data related to the identification information of multiple types of aromas. Next, the server 120 transmits the target data related to the aromas belonging to each subgroup to the mobile terminal 110. In response, the display 111 of the mobile terminal 110 displays a screen for selecting the identification information of these aromas. Figure 13 shows an example of this screen. The evaluator evaluates the aroma samples with labels of the identification information displayed on the screen. This allows the evaluator to evaluate the aromas belonging to the subgroups on the scale of whether or not they stimulate appetite. Based on the results, the evaluator selects button images B17 to B24 in order using the input unit 113. The evaluator performs this ranking of aromas for all subgroups. The processing up to this point corresponds to the processing in S2 of Figure 3. Then, processes S3 and S4 based on the ranked results are performed in the same manner as in the second embodiment. Alternatively, the aroma evaluation may be performed using the evaluation support device 1 of the first embodiment instead of the second embodiment.

[0054] <Variation> The above describes preferred embodiments of the present invention, but the present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the means for solving the problem.

[0055] According to each of the embodiments described above, each subgroup has N elements. The number of elements in each subgroup may be different from one another, within the range of 3 to 16.

[0056] Furthermore, in S2 of Figure 3 according to the first embodiment described above, the evaluator is asked to select images or target IDs in order from those with the highest ranking according to a predetermined scale. Alternatively, the evaluator may be asked to select them in order from those with the lowest ranking according to the predetermined scale. Also, in S3, the highest-ranking target from each subgroup is extracted to the high-evaluation extraction group, and the highest-ranking target from the high-evaluation extraction group is extracted to the integrated group. Alternatively, the lowest-ranking target from each subgroup is extracted to the low-evaluation extraction group, and the lowest-ranking target from the low-evaluation extraction group is extracted to the integrated group.

[0057] Furthermore, in the first embodiment described above, as shown in S1 of Figure 3, the evaluation support device 1 performs the process of dividing the evaluation group into subgroups. Alternatively, the target data after the division from the evaluation group to subgroups may be stored in the evaluation support device 1 after the division into subgroups has been performed in advance. [Explanation of Symbols]

[0058] 1. Evaluation support device 10 displays 20 speakers 30 Input section 40 Control Unit 100 Evaluation Support System 110 Mobile devices 111 displays 112 speakers 113 Input section 115 Control Unit 120 servers

Claims

1. A method for evaluating an object that generates stimuli perceptible to humans, A division step of dividing an evaluation group consisting of multiple evaluation targets into multiple subgroups, such that the number of elements in each of the multiple subgroups is in the range of 3 to 16, A subgroup ranking step is performed in which, for each of the plurality of subgroups generated by the division step, the evaluator ranks the subjects of evaluation according to a predetermined scale based on their perception of the stimuli they generate. The system includes an evaluation extraction step in which, from a group of evaluation subjects consisting of the highest or lowest ranked evaluation subjects in each of the multiple subgroups, the evaluator extracts the evaluation subject with the highest or lowest evaluation according to the predetermined scale, based on the perception of stimuli generated by the evaluation subject, and the extracted evaluation subject is removed from the original subgroup, and this process is repeated until all evaluation subjects have been extracted from the multiple subgroups. A sensory evaluation method characterized in that the order of extraction in the evaluation extraction step is given as a ranking relative to the entire evaluation group.

2. The subject of evaluation is an image or sound, The aforementioned predetermined scale is a scale relating to the appearance or sound of an image, The sensory evaluation method according to claim 1, characterized in that the perception of the stimulus is visual or auditory.

3. The sensory evaluation method according to claim 2, characterized in that, in each of the subgroup ranking step and the evaluation extraction step, the evaluation targets are output to an output device, and the evaluator is instructed to rank or evaluate the output evaluation targets.

4. The subject of evaluation generates stimuli that can be perceived by any of the senses of smell, taste, and touch. The sensory evaluation method according to claim 1, characterized in that the predetermined scale is a scale relating to any of the odor, taste, and touch sensations corresponding to the stimulus.

5. For each of the multiple evaluation targets, identification information is set to distinguish it from other evaluation targets. The sensory evaluation method according to claim 1, characterized in that, in each of the subgroup ranking step and the evaluation extraction step, the identification information of one of the plurality of evaluation targets is displayed on a display device, and the evaluator ranks or evaluates the stimuli generated by the evaluation target corresponding to the displayed identification information.

6. A method for evaluating the reliability of a ranking made by a sensory evaluation method according to any one of claims 1 to 5, A sample extraction step of extracting 3 to 16 samples for reliability evaluation from the aforementioned multiple evaluation targets, A re-ranking step is performed in which the evaluators rank all the samples extracted in the sample extraction step according to a predetermined scale based on their perception of the stimuli generated by each of the subjects to be evaluated. A reliability evaluation method characterized by comprising a sample ranking comparison step, which compares the ranking of the samples made by the re-ranking step with the ranking of the samples based on the order of extraction by the evaluation extraction step.

7. A system for evaluating objects that generate stimuli perceptible to humans, It is equipped with an output device, an input device, and a processing unit. The aforementioned computing device is Output processing to cause the output device to output the evaluation target, A subgroup ranking process is performed for each of the multiple subgroups that constitute an evaluation group consisting of multiple evaluation targets, each subgroup having 3 to 16 elements, based on the content received by the input device from user input made by an evaluator who has perceived the evaluation targets output by the output device, in order to rank the evaluation targets according to a predetermined scale. An evaluation extraction process is performed which, from a group of evaluation targets consisting of the highest or lowest ranked evaluation targets in each of the multiple subgroups, based on the ranking performed by the subgroup ranking process, the evaluation target with the highest or lowest evaluation according to the predetermined scale is extracted based on the user input made by the evaluator who perceived the evaluation target output by the output device, and the extracted evaluation target is removed from the original subgroup, and this process is repeated until all evaluation targets are extracted from the multiple subgroups. A sensory evaluation system characterized by performing a ranking determination process that determines the order of extraction by the evaluation extraction process as the ranking of the entire evaluation group.

8. A program for evaluating an object that generates stimuli perceptible to humans, A subgroup ranking process is performed for each of the multiple subgroups that constitute an evaluation group consisting of multiple evaluation targets, each subgroup having 3 to 16 elements, based on the content received by the input device from user input made by an evaluator who has perceived the evaluation targets output by the output device, in which the evaluation targets are ranked according to a predetermined scale. An evaluation extraction process is performed which, from a group of evaluation targets consisting of the highest or lowest ranked evaluation targets in each of the multiple subgroups, based on the ranking performed by the subgroup ranking process, the evaluation target with the highest or lowest evaluation according to the predetermined scale is extracted based on the user input received by the input device from an evaluator who perceived the evaluation target output by the output device, and the extracted evaluation target is removed from the original subgroup, and this process is repeated until all evaluation targets are extracted from the multiple subgroups. A sensory evaluation program characterized by causing a computer to perform a ranking determination process that determines the order of extraction by the evaluation extraction process as the ranking of the entire evaluation group.

9. A method for learning a neural network, characterized in that processing is performed using the sensory evaluation method described in any one of claims 1 to 5, the sensory evaluation system described in claim 7, or the sensory evaluation program described in claim 8, and the data indicating the evaluation target to which a rank has been assigned and the rank are used as training data.

Images